In March 2008, I started comparing OpenStreetMap in England to the Ordnance Survey Meridian 2, as a way to evaluate the completeness of OpenStreetMap coverage. The rational behind the comparison is that Meridian 2 represents a generalised geographic dataset that is widely use in national scale spatial analysis. At the time that the study started, it was not clear that OpenStreetMap volunteers can create highly detailed maps as can be seen on the ‘Best of OpenStreetMap‘ site. Yet even today, Meridian 2 provides a minimum threshold for OpenStreetMap when the question of completeness is asked.
So far, I have carried out 6 evaluations, comparing the two datasets in March 2008, March 2009, October 2009, March 2010, September 2010 and March 2011. While the work on the statistical analysis and verification of the results continues, Oliver O’Brien helped me in taking the results of the analysis for Britain and turn them into an interactive online map that can help in exploring the progression of the coverage over the various time period.
Notice that the visualisation shows the total length of all road objects in OpenStreetMap, so does not discriminate between roads, footpaths and other types of objects. This is the most basic level of completeness evaluation and it is fairly coarse.
The application will allow you to browse the results and to zoom to a specific location, and as Oliver integrated the Ordnance Survey Street View layer, it will allow you to see what information is missing from OpenStreetMap.
Finally, note that for the periods before September 2010, the coverage is for England only.
Some details on the development of the map are available on Oliver’s blog.
22 October, 2009
While Google wasn’t the first website to implement slippy maps – maps that are based on tiles, download progressively and allow fairly smooth user interaction – it does deserve the credit for popularising them. The first version of Google Maps was a giant leap in terms of public web mapping applications, as described in our paper about Web Mapping 2.0.
In terms of usability, the slippy map increased the affordability of the map with direct manipulation functionality for panning, clear zoom operating through predefined scales, the use of as much screen assets for the map as possible, and the iconic and simple search box at the top. Though the search wasn’t perfect (see the post about the British Museum test), overall it offered a huge improvement in usability. It is not surprising that it became the most popular web mapping site and the principles of the slippy map are the de facto standard for web mapping interaction.
However, in recent months I couldn’t avoid noticing that the quality of the interface has deteriorated. In an effort to cram more and more functionality (such as the visualisation of the terrain, pictures, or StreetView), ease of use has been scarificed. For example, StreetView uses the icon of a person on top of the zoom scale, which the user is supposed to drag and drop on the map. It is the only such object on the interface, and appears on the zoom scale regardless of whether it is relevant or available. When you see the whole of the UK for example, you are surely not interested in StreetView, and if you are zooming to a place that wasn’t surveyed, the icon greys out after a while. There is some blue tinge to indicate where there is some coverage, but the whole interaction with it is very confusing. It’s not difficult to learn, though.
Even more annoying is that when you zoom to street level on the map, it switches automatically to StreetView, which I found distracting and disorientating.
There are similar issues with Google Earth – compare versions 4 and 5 in terms of ease of use for novice users, and my guess is that most of them will find 4 easier to use. The navigation both above the surface and at surface level is anything but intuitive in version 5. While in version 4 it was clear how to tilt the map, this is not the case in 5.
So maybe I should qualify what I wrote previously. There seems to be a range here, so it is not universally correct to say that the new generation of geographical applications are very usable just because they belong to the class of ‘neogeography’. Maybe, as ‘neogeography’ providers are getting more experienced, they are falling into the trap of adding functionality for the sake of it, and are slowly, but surely, destroying the advantages of their easy-to-use interfaces… I hope not!
18 November, 2008
In October 2007, Francis Harvey commissioned me to write a review article for Geography Compass on Neogeography. The paper was written in collaboration with Alex Singleton at UCL and Chris Parker from the Ordnance Survey.
The paper covers several issues. Firstly, it provides an overview of the developments in Web mapping from the early 1990s to today. Secondly, in a similar way to my Nestoria interview, it explains the reasons for the changes that enabled the explosion of geography on the Web in 2005: GPS availability, Web standards, increased spread of broadband, and a new paradigm in programming APIs. These changes affected the usability of geographic technologies and started a new era in Web mapping. Thirdly, we describe several applications that demonstrate the new wave – the London Profiler, OS OpenSpace and OpenStreetMap. The description of OSM is somewhat truncated, so my IEEE Pervasive Computing paper provides a better discussion.
The abstract of the paper is:
‘The landscape of Internet mapping technologies has changed dramatically since 2005. New techniques are being used and new terms have been invented and entered the lexicon such as: mash-ups, crowdsourcing, neogeography and geostack. A whole range of websites and communities from the commercial Google Maps to the grassroots OpenStreetMap, and applications such as Platial, also have emerged. In their totality, these new applications represent a step change in the evolution of the area of Internet geographic applications (which some have termed the GeoWeb). The nature of this change warrants an explanation and an overview, as it has implications both for geographers and the public notion of Geography. This article provides a critical review of this newly emerging landscape, starting with an introduction to the concepts, technologies and structures that have emerged over the short period of intense innovation. It introduces the non-technical reader to them, suggests reasons for the neologism, explains the terminology, and provides a perspective on the current trends. Case studies are used to demonstrate this Web Mapping 2.0 era, and differentiate it from the previous generation of Internet mapping. Finally, the implications of these new techniques and the challenges they pose to geographic information science, geography and society at large are considered.’
15 August, 2008
The Metropolitan Police Authority has released a beta version of their new Crime Mapping application, showing the generalised level of crime (burglary, robbery and vehicle) for Lower-level Super Output Areas (LSOAs). The application uses generalised boundaries of LSOAs , and use a clear classification of the level of crime. Interestingly, the Show Us a Better Way website includes several suggestions for crime mapping – so there is an ongoing public interest.
This is not surprising, based on my own experience with CamStats, which was developed in collaboration between me, Kate Jones and Dave Ashby for Camden Police in late 2003, with the website operating from early 2004 until late 2007.
As you can see from the slideshow above, the information that CamStats provided is richer than what is available today. CamStats was based on static maps, and was very easy to produce – we designed it so a team administrator (with no GIS skills) could compile monthly and annual statistics simply by copying a data file to a processing machine, and then clicking one button in Mapinfo Professional which called Mapbasic, Perl scripts and other utilities to create, process and map the data and compile the HTML pages for the website into one zip file. All the user had to do was transfer the zip file to the Met web team who easily updated the webserver by unzipping the files. The fact that it was running for three years without any request for support is something that Kate and I are justifiably proud of.
Notice that CamStats provided options to see different geographical units, different forms of visualisation and to view the information in tabular and chart forms. Users could even download the aggregate counts for each area to compile there own reports. This was particularly useful for a number of community groups in Camden.
There is no question that the use of Google Maps, which provide context for the statistics is a huge usability improvement over our implementation. However, it will be interesting to see how long it will take the Met team to reach the functionality and ease of use CamStats provided …
23 November, 2007
Back in 2005, when I worked with Artemis Skarlatidou on an evaluation of public mapping websites, we came up with a simple test to check how well these search sites perform: Can a tourist find a famous landmark easily?
The reasoning behind raising this question was that tourists are an obvious group of users of public mapping sites such as Multimap, MapQuest, Yahoo! Maps, Microsoft’s Virtual Earth or Google Maps. Market research information presented by Vincent Tao from Microsoft in a seminar a year ago confirmed this assumption.
During the usability evaluation, we gave the participants the instruction ‘Locate the following place on the map: British Museum: Great Russell Street, London, WC1B 3DG’. Not surprising, those participants who started with the postcode found the information quickly, but about a third typed ‘British Museum, London’. While our participants were London’s residents and were used to postcodes as a means of stating an address precisely, a more realistic expectation from tourists is that they would not use postcodes when searching for a landmark.
In the summer of 2005 when we ran the test, the new generation of public mapping websites (such as Google Maps and Microsoft Virtual Earth) performed especially bad.
The most amusing result came from Google Maps, pointing to Crewe as the location of the British Museum (!).
The most simple usability test for a public mapping site that came out of this experiment is the ‘British Museum Test’: find the 10 top tourist attractions in a city/country and check if the search engine can find them. Here is how it works for London:
The official Visit London site suggests the following top attractions: Tate Modern, British Museum, National Gallery, Natural History Museum, the British Airways London Eye, Science Museum, the Victoria & Albert Museum (V&A Museum), the Tower of London, St Paul’s Cathedral and the National Portrait Gallery.
Now, we can run the test by typing the name of the attraction in the search box of public mapping sites. As an example, here I’ve used Yahoo! Maps, Google Maps, Microsoft’s Virtual Earth and Multimap. With all these sites I’ve imitated a potential tourist – I’ve accessed the international site (e.g. maps.google.com) and panned the map to the UK, and then typed the query. The results are:
|Attraction (search term used)||Yahoo!||Microsoft||Multimap|
|Tate Modern||Found and zoomed||Found and zoomed||Found and zoomed||Found and zoomed|
|British Museum||Found and zoomed||Found as part of a list||Found and zoomed||Found and zoomed|
|National Gallery||Found and zoomed||Found as part of a list||Found and zoomed||Found as part of a list (twice!)|
|Natural History Museum||Failed||Found as part of a list||Found and zoomed||Found and zoomed|
|British Airways London Eye (commonly abbreviated to London Eye)||Failed on the full name, found and zoomed on the common abbreviation||Found as part of a list, failed on the common abbreviation||Failed on the full name, found and zoomed on the common abbreviation||Failed on the full name, found and zoomed on the common abbreviation|
|Science Museum||Found and zoomed||Found as part of a list||Found and zoomed||Found and zoomed|
|The Victoria & Albert Museum (commonly abbreviated to V&A Museum)||Found and zoomed on both||Found and zoomed, but failed on the common abbreviation||Found and zoomed, but failed on the common abbreviation||Found and zoomed, but the common abbreviation zoomed on Slough (!)|
|The Tower of London||Found and zoomed||Found and zoomed||Found and zoomed (failed if ‘the’ included in the search)||Found and zoomed|
|St Paul’s Cathedral||Found and zoomed||Found and zoomed||Found as part of a list||Failed|
|National Portrait Gallery||Failed (zoomed to the one in Washington DC)||Found and zoomed||Found and zoomed||Found and zoomed|
Notice that none of these search engines managed to pass the test on all the top ten attractions, which are visited by millions every year. There is a good reason for this – geographical search is not a trivial matter and the semantics of place names can be quite tricky (for example, if you look at a map of Ireland and the UK, there are two National Galleries).
On the plus side, I can note that search engines are improving. At the end of 2005 and for most of 2006 the failure rate was much higher. I used the image above in several presentations and have run the ‘British Museum Test’ several times since then, with improved results in every run.
The natural caveat is that I don’t have access to the server logs of the search engines and, therefore, can’t say that the test really reflects the patterns of use. It would be very interesting to have Google Maps Hot Trends or to see it for other search engines. Even without access to the search logs though, the test reveals certain aspects in the way that information is searched and presented and is useful in understanding how good the search engines are in running geographical queries.
By a simple variation of the test you can see how tolerant an engine is for spelling errors, and which one you should use when guests visit your city and you’d like to help them in finding their way around. It is also an indication of the general ability of the search engine to find places. You can run your own test on your city fairly quickly – it will be interesting to compare the results!
For me, Microsoft Virtual Earth is, today, the best one for tourists, though it should improve the handling of spelling errors…
31 October, 2007
As part of the Mapping Change for Sustainable Communities project, we organised the first workshop in the Royal Docks area, at the Sunborn Yacht Hotel last Saturday (27/10). The workshop was very successful and, as I usually do in these workshops, I start with ‘what mapping information can we find on the WWW about your locality’. I’ve been doing it now for about 7 or 8 years, but during the period, the Environment Agency’s Pollution Inventory maps never failed me as an example for technocratic dissemination of information which is not helping the people on the ground.
I find that, in all these workshops with people from many communities across London, very few knew about the Environment Agency information, let alone ever accessed it independently had. As the participants are usually from community or environmental interest groups, they are interested in the information – they just don’t know where to find it. I associate this lack of awareness with the fact that users find the information unfriendly and unhelpful, so there is no ‘word of mouth’ effect that leads to more use of the site. As someone in our Saturday workshop declared, ‘these maps are not written in community language or for community use’ – yet, they tick all the boxes of the Aarhus convention…
To understand what’s wrong, see the image below, which provides a view of the site on an average monitor (1024×768):
The header area is so big that all that is left is a fairly small area for the map.
Furthermore, as the full image of the page shows, the map is supposed to offer several layers of pollution data (on the right-hand side) but, as many of the layers include point data about the same site – all using the same symbols which overlap one another – the user can’t see if, in a given location, there is information from multiple categories.
The area of the map is very small (less than 400×400 pixels), and people find it very difficult to locate where they are or where the postcode is that they have selected in relation to the information on the map. Zooming in to the largest scale, or at any stage during the process, the system will run a query and provide information about the specific location only if the ‘learn more’ option is selected. Even as a more frequent user, I fail to click on this option and find the interaction with the system frustrating.
This site demonstrates that the Aarhus model of access to information, which is ‘we’ll build it and they’ll come’, is not sufficient and that a more user-centred approach is required to achieve public access to environmental information.