OpenStreetMap in Athens – as accurate as London

Most of the work that we carried out at UCL in evaluating the quality of OpenStreetMap is focused on England, and particularly on London. This is mainly due to the accessibility of comparative datasets. The reason for this was the availability of data, as the Ordnance Survey research unit kindly provided me with the full Meridian 2 dataset for comparison. More detailed comparison, for which we used MasterMap, came from the wonderful Digimap service, though because of the time that it takes to process it we were limited in the size of the area that was used for comparison.

One of the open questions that remained was the accuracy of data collection in other parts of the world. Luckily, Ourania (Rania) Kounadi, who studied our MSc in GIS at UCL, had access to detailed maps of Athens. She used a 1:10,000 map from the Hellenic Military Geographic Service (HGMS) and focused on an area of 25 square kilometres at the centre of the city. The roads were digitised from the HGMS map, and then the Goodchild-Hunter procedure was used to evaluate the positional accuracy.

The results show that for most of the roads in the evaluation area there was an overlap of 69% to 100% between OSM and HGMS datasets. The average overlap was very close to 90%. Her analysis also included attribute and completeness evaluation, showing that the quality is high on these aspects too.

OSM positional accuracy for Athens

So a pattern is starting to emerge showing that the quality of OSM data is indeed good in terms of positional accuracy. This is surprising at first glance – how come people who are not necessarily trained in geographical data collection and do not use rigorous quality assurance processes produce data that is as good as the authoritative data?

My explanation for this, as I’ve written in my paper about OSM quality, is that it ‘demonstrates the importance of the infrastructure, which is funded by the private and public sector and which allows the volunteers to do their work without significant personal investment. The GPS system and the receivers allow untrained users to automatically acquire their position accurately, and thus simplify the process of gathering geographical information. This is, in a way, the culmination of the process in which highly trained surveyors were replaced by technicians, with the introduction of high-accuracy GPS receivers in the construction and mapping industries over the last decade. The imagery also provides such an infrastructure function – the images were processed, rectified and georeferenced by experts and thus, an OSM volunteer who uses this imagery for digitising benefits from the good positional accuracy which is inherent in the image. So the issue here is not to compare the work of professionals and amateurs, but to understand that the amateurs are actually supported by the codified professionalised infrastructure and develop their skills through engagement with the project.’
Rania’s dissertation is available to download from here.

Delivering environmental information clearly – Sustainable Energy without the Hot Air

One of the best read that I had over the summer was David MacKay’s Sustainable Energy – without the hot air. The book (which you can download for free from his website) is easy to follow, clear and a very interesting analysis of the options open to the UK in terms of energy provision in a way that is sustainable and without reliance on fossil fuel.

David MacKay’s analysis covers the issue of energy both on the consumption and generation sides. It runs through a whole series of options by using lots of very intelligent and elegant ‘back of envelope’ calculations that show what the reasonable assumptions are for each source of energy and for its use.

What is especially fantastic with this book is the way in which a fairly complex environmental issue is made accessible through several means.

Firstly, the whole book is based on a single measure (kWh/day per person), which is explained clearly up front and then used throughout the book. This makes it easy to compare the different options.

Secondly, the book uses a clear visualisation of stack-bars to show how the different options of consumption and production add up.

Thirdly, the book is made of two parts – an easy–to-access first part, without the detailed scientific references and backing material that would make it difficult to understand, but with enough information to understand how each assertion is made. For interested readers there are technical chapters that provide all the scientific details towards the end of the book.

Altogether, it is a masterpiece of environmental information communication, which is very rare, unfortunately.

OpenStreetMap and Ordnance Survey Meridian 2 – Progress maps

As part of an update of the work that I published in August 2008, I re-ran the comparison between the OpenStreetMap and Ordnance Survey Meridian 2 datasets. In a future post, I will provide a full report of this assessment. As I have now completed the evaluation for October 2009 and a re-evaluation of the data from March 2008, I decided to publish some outputs. The map below shows the completeness of OpenStreetMap across England for the two periods. Click on the map to enlarge.

Meridian 2 - OSM Comparison - Mar '08 / Oct '09

The second set of maps show the estimation of completeness when attributes are considered. For this purpose, the calculation takes into account only line objects that are comparable to those in Meridian 2; thus not including features such as footpaths. The following types of roads were used: motorway, motorway_link, primary, primary_link, secondary, secondary_link, trunk, trunk_link, tertiary, tertiary_link, minor, unclassified and residential.

In addition, a test verified that the ‘name’ field is not empty. This is an indication that a street name or road number is included in the attributes of the objects, and thus it can be considered to be complete with basic attributes. In order to make the comparison appropriate, only objects that contain a road name or number in Meridian 2 were included.

Meridian 2 - OSM with Attributes Comparison - Mar '08 / Oct '09

The growth within just over a year and a half  is very impressive – rising from 27% in March 2008 to 65% in October 2009. When attributes are considered, it has risen from 7% to 25%. Notice that the criteria that I have set for this comparison are stringent than the one in the previous study, so the numbers – especially for the attribute completeness – are lower than those published in August 2008.

Linus’ Law and OpenStreetMap

One of the interesting questions that emerged from the work on the quality of OpenStreetMap (OSM) in particular, and Volunteered Geographical Information (VGI) in general, is the validity of the ‘Linus’ Law’ for this type of information.

The law came from Open Source software development and states that ‘Given enough eyeballs, all bugs are shallow’ (Raymond, 2001, p.19). For mapping, I suggest that this can be translated into the number of contributors that have worked on a given area. The rationale behind it is that if there is only one contributor in an area he or she might inadvertently introduce some errors. For example, they might forget to survey a street or might position a feature in the wrong location. If there are several contributors, they might notice inaccuracies or ‘bugs’ and therefore the more users, the less ‘bugs’.

In my original analysis, I looked only at the number of contributors per square kilometre as a proxy for accuracy, and provided a visualisation of the difference across England.

During the past year, Aamer Ather and Sofia Basiouka looked at this issue, by comparing the positional accuracy of OSM in 125 sq km of London. Aamer carried out a detailed comparison of OSM and the Ordnance Survey MasterMap Integrated Transport Network (ITN) layer. Sofia took the results from his study and divided them for each grid square, so it was possible to calculate an overall value for every cell. The value is the average of the overlap between OSM and OS objects, weighted by the length of the ITN object. The next step was to compare the results to the number of users at each grid square, as calculated from the nodes in the area.

The results show that, above 5 users, there is no clear pattern of improved quality. The graph below provide the details – but the pattern is that the quality, while generally very high, is not dependent on the number of users – so Linus’ Law does not apply to OSM (and probably not to VGI in general).

From looking at OSM data, my hypothesis is that, due to the participation inequality in OSM contribution (some users contribute a lot while others don’t contribute very much), the quality is actually linked to a specific user, and not to the number of users.
Yet, I will qualify the conclusion with the statement that further research is necessary. Firstly, the analysis was carried out in London, so checking what is happening in other parts of the country where different users collected the data is necessary. Secondly, the analysis did not include the interesting range of 1 to 5 users, so it might be the case that there is rapid improvement in quality from 1 to 5 and then it doesn’t matter. Maybe the big change is from 1 to 3? Finally, the analysis focused on positional accuracy, and it is worth exploring the impact of the number of users on completeness.

ICA Commission on Use and User Issues workshop – April 2010 – Call for papers

This is call for papers for a workshop on methods and research techniques that are suitable for geospatial technologies. The workshop is planned for the day before GISRUK 2010, and we are aware of the clashes with the AAG 2010 annual meeting, CHI 2010 and the Ergonomics Society Annual Conference. However, if you would like to contribute to the book that the commission is developing but can’t attend the workshop, please send an abstract and inform us that you can’t attend.

In the near future I’ll publish information about another workshop in March 2010 about the usability and Human-Computer Interaction aspects of geographical information itself – see the report from the Ordnance Survey workshop earlier in 2009.

So here is the full call:

Workshop on Methods and Techniques of Use, User and Usability Research in Geo-information Processing and Dissemination

Tuesday 13 April 2010 at University College London

The Commission on Use and User Issues of the International Cartographic Association (ICA) is currently working on a new handbook specifically addressing the application of user research methods and techniques in the geodomain.

In order to share experiences and interesting case studies a workshop is organized by the Commission, in collaboration with UCL, on the day preceding GISRUK 2010.

CALL FOR PAPERS

While there is growing awareness within the research community on the need to develop usability engineering and use and user research methods that are suitable for geographical and spatial information and systems, to date there is a lack of organized and documented experience in this area.

We therefore invite researchers with recent experience with use, user and usability research in the broad geodomain (cartography, GIS, geovisualization, Location Based Services, geographical information, GeoWeb etc.) to present a paper specifically focusing on the research methods and techniques applied, with an aim to develop the body of knowledge for the domain.

To participate, please send an abstract of 1 page A4 at maximum containing:

• A description of the research method(s) and technique(s) applied
• A short description of the case in which they have been applied
• The overall research framework
• Contact details and affiliation of the author(s)

We are also encouraging PhD researchers to submit paper proposals and share experiences from their research. At the workshop there will be ample time for discussing the application of user research methods and techniques. Good papers may be the basis for contributions to the handbook that is planned for publication in 2011.

Abstracts should be submitted on or before 1 December 2009 to the Chairman of the Commission Corné van Elzakker ( elzakker@itc.nl )

Also see:
the website of the ICA Commission on Use and User Issues and  the GISRUK2010 website

The end of the ‘ARC/INFO driving licence’ era

The discussion about the future of the GIS ‘profession’ has flared up in recent days – see the comments from Sean Gorman, Steven Feldman (well, citing me) and Don Meltz among others. My personal perspective is about the educational aspect of this debate.

I’ve been teaching GIS since 1995, and been involved in the MSc in GIS at UCL since 1998 – teaching on it since 2001. Around 1994 I was contemplating the excellent MSc in GIS programme in Edinburgh, though I opted to continue with my own mix of geography and computer science, which turned out to be great in the end – but I can say that I have been following the trends in GIS education for quite a while.

Based on this experience, I would argue that the motivation for studying an MSc in GIS over the past 20 years was to get the ‘ARC/INFO driving licence’. I use ARC/INFO as a metaphor – you can replace it with any other package, but ARC/INFO was the de facto package for teaching GIS (and its predecessor ArcGIS is today), so it is suitable shorthand. What I mean by that is that for a long time GIS packages were hard to use and required a significant amount of training in order to operate successfully. Even if a fairly simple map was needed, the level of technical knowledge and the number of steps required were quite significant. So employers, who mostly wanted someone who could make them maps, recruited people who gained skills in operating the complex packages that allow the production of maps.

The ‘ARC/INFO driving licence’ era included an interesting dissonance – the universities were telling themselves that they were teaching the principles of GIScience but the students were mostly interested in learning how to operate a GIS at a proficient level to get a job. I’ve seen and talked with enough students to recognise that many of them, in their daily jobs, rarely used the spatial statistical analysis that we were teaching and they mostly worked at ‘taming the beast’, which GIS was.

As expected, at UCL there was always a group that was especially interested in the principles of GIScience and that continued their studies beyond the MSc. But they are never the majority of the cohort.

The model worked well for everyone – universities were teaching GIS by a combination of principles and training of specific packages and the students found jobs at the end and joined GIS departments in different organisations.

The disruption that changed this arrangement started in the late 1990s, with Oracle Spatial starting to show that GIS can be integrated in mainstream products. The whole process accelerated around 2005 with the emergence of GeoWeb, Free and Open Source GIS (FOSS GIS) and the whole range of applications that come with it. Basically, you don’t need a licence any more. More and more employers (even GIS consultancies) are not recruiting from GIS education programmes – they are taking computing professionals and teaching them the GIS skills. Going through an MSc in GIS to be proficient with a tool is not necessary.

So in an era in which you don’t need a licence to join the party, what is the MSc in GIS for?

The answer is that it can be the time when you focus on principles and on improving specific skills. Personally, that was my route to education. I started working in GIS software development without much more than high school education in 1988. After hearing people around me talking about registers, bugs, polygons and databases I was convinced that I must understand these principles properly. So I went for a degree that provided me with the knowledge. In the same way, I would expect that MSc programmes cater for the needs of people who gain some practical experience with operating geospatial technologies and want to learn the principles or become specialists in specific aspects of these systems.

We already see people doing the MSc while working with GIS – currently studying an MSc by distance learning or in the evening is very popular and I expect that this will continue. However, the definition of what is covered by GIS must be extended – it should include everything from Bing Maps API to PostGIS to ArcGIS.

I can also see the need for specialised courses – maybe to focus on the technical development of geospatial technologies or maybe on spatial statistical analysis for those who want to become geographical information analysts. I would also expect much more integration of GIS with other fields of study where it is taught as a tool – just look at the many MSc programmes that currently include GIS. I’m already finding myself teaching students of urban design, development planning or asset management.
All in all, I’m not going to feel sorry that the ‘ARC/INFO driving licence’ era is coming to its end.

Google Geo applications – deteriorating interfaces?

While Google wasn’t the first website to implement slippy maps – maps that are based on tiles, download progressively and allow fairly smooth user interaction – it does deserve the credit for popularising them. The first version of Google Maps was a giant leap in terms of public web mapping applications, as described in our paper about Web Mapping 2.0.

In terms of usability, the slippy map increased the affordability of the map with direct manipulation functionality for panning, clear zoom operating through predefined scales, the use of as much screen assets for the map as possible, and the iconic and simple search box at the top. Though the search wasn’t perfect (see the post about the British Museum test), overall it offered a huge improvement in usability. It is not surprising that it became the most popular web mapping site and the principles of the slippy map are the de facto standard for web mapping interaction.

However, in recent months I couldn’t avoid noticing that the quality of the interface has deteriorated. In an effort to cram more and more functionality (such as the visualisation of the terrain, pictures, or StreetView), ease of use has been scarificed. For example, StreetView uses the icon of a person on top of the zoom scale, which the user is supposed to drag and drop on the map. It is the only such object on the interface, and appears on the zoom scale regardless of whether it is relevant or available. When you see the whole of the UK for example, you are surely not interested in StreetView, and if you are zooming to a place that wasn’t surveyed, the icon greys out after a while. There is some blue tinge to indicate where there is some coverage, but the whole interaction with it is very confusing. It’s not difficult to learn, though.

Even more annoying is that when you zoom to street level on the map, it switches automatically to StreetView, which I found distracting and disorientating.

There are similar issues with Google Earth – compare versions 4 and 5 in terms of ease of use for novice users, and my guess is that most of them will find 4 easier to use. The navigation both above the surface and at surface level is anything but intuitive in version 5. While in version 4 it was clear how to tilt the map, this is not the case in 5.

So maybe I should qualify what I wrote previously. There seems to be a range here, so it is not universally correct to say that the new generation of geographical applications are very usable just because they belong to the class of ‘neogeography’. Maybe, as ‘neogeography’ providers are getting more experienced, they are falling into the trap of adding functionality for the sake of it, and are slowly, but surely, destroying the advantages of their easy-to-use interfaces… I hope not!

GISRUK 2010 at UCL – Call for papers

Geographical Information Science Research UK (GISRUK) is a research conference that has been taking place in different university campuses around the UK (and once in Ireland) since 1993. Despite the name, it is open not just to researchers from the UK, but also to international participants, who are very welcome.

For me, GISRUK was the first international conference in which I presented a paper eleven years ago, so I have a soft spot for it. It was very friendly and welcoming for a starting research student (which I was at the time). It was especially useful to discover that all the famous academics who attended it were friendly and open to questions.

The conference will be held at UCL in April 2010, and the call for papers is now out, so consider submitting a paper.

The papers are rather short, about 1500 words, so there is plenty of time to write one in time for the deadline of the end of November.

Interacting with Geospatial Technologies

At the end of September, the manuscript of ‘Interacting with Geospatial Technologies’ was submitted to John Wiley & Sons. This is the reason for the silence on this blog since July while the final chapters were written.

The book, which is an introduction to usability and Human-Computer Interaction aspects of GIS and other geospatial technologies, was written because there is no other recent book that covers these aspects while taking into account the special characteristics of geographical information and the extensive use of maps.

There were several books in the early 1990s dedicated to human factors of GIS or to cognitive aspects of these systems. Since then, there have been many published articles, but no easy-to-access summary of the outcomes in a way that is useful for developers or people who want to understand how to design more usable systems. So, while working on a paper that called for developing ‘usability engineering for GIS’ in 2005, I figured out that, actually, it was time to write an introductory text in this area. In the end, this is an edited textbook written by me together with a group of excellent collaborators: Jochen Albrecht, Clare Davies, Catherine Emma Jones, Robert Laurini, Chao Li, Aaron M. Marcus, Stephanie L. Marsh, Annu-Maaria Nivala, Artemis Skarlatidou, Carolina Tobón, Jessica Wardlaw and Antigoni Zafiri.

So the book provides an introduction to user-centred design and usability engineering from a geospatial technologies perspective, theoretical aspects of human understanding of space and collaborative systems, practical aspects of cartography and map design that are useful for developers and application designers, guidance for evaluating geospatial systems and some tips for designing desktop, Web and mobile based systems. Each chapter includes case studies and examples that make the material more concrete.

The book is scheduled to be out by March 2010. A lot of work went into writing the various chapters and ensuring that the content is covering all the needed elements to create a usable GIS – I hope that it will be useful!

Volunteered Geographical Information Research Network

Chris Parker, a PhD student at Loughborough University, organised a dedicated Volunteered Geographical Information research group site on ResearchGate. While I dislike the term – I usually interpret it as the version of ‘volunteered’ as in ‘mum volunteered me to help the old lady cross the street’ – there is no point in trying to change it. When Mike Goodchild coins an acronym, it will stick; it’s sort of a GIScience law!

If you are interested in user-generated geographical content, crowdsourced geographical information, commons-based peer-produced geographical information, or any other way to call this phenomena (for example VGI) – join the group. It will be good to keep in touch, share information and discuss research aspects.
If you are researching in this area you are also welcome to submit a paper to GISRUK 2010 which will be hosted at UCL – we are keen to have a VGI element in the programme, considering that UCL is the host of OpenStreetMap .