Linus’ Law and OpenStreetMap

One of the interesting questions that emerged from the work on the quality of OpenStreetMap (OSM) in particular, and Volunteered Geographical Information (VGI) in general, is the validity of the ‘Linus’ Law’ for this type of information.

The law came from Open Source software development and states that ‘Given enough eyeballs, all bugs are shallow’ (Raymond, 2001, p.19). For mapping, I suggest that this can be translated into the number of contributors that worked on a given area. The rationale behind it is that if there is only one contributor in an area he or she might inadvertently introduce some errors. For example, they might forget to survey a street or might position a feature in the wrong location. If there are several contributors, they might notice inaccuracies or ‘bugs’ and therefore the more users, the less ‘bugs’.

In my original analysis I looked only at the number of contributors per square kilometre as a proxy for accuracy, and provided a visualisation of the difference across England.

During the past year, Aamer Ather and Sofia Basiouka looked at this issue, by comparing the positional accuracy of OSM in 125 sq km of London. Aamer carried out a detailed comparison of OSM and the Ordnance Survey MasterMap Integrated Transport Network (ITN) layer. Sofia took the results from his study and divided them for each grid square, so it was possible to calculate an overall value for every cell. The value is the average of the overlap between OSM and OS objects, weighted by the length of the ITN object. The next step was to compare the results to the number of users at each grid square, as calculated from the nodes in the area.

The results show that, above 5 users, there is no clear pattern of improved quality. The graph below provide the details – but the pattern is that the quality, while generally very high, is not dependent on the number of users – so Linus’ Law doesn’t apply to OSM (and probably not to VGI in general).

From looking at OSM data, my hypothesis is that, due to the participation inequality in OSM contribution (some users contribute a lot while others don’t contribute very much), the quality is actually linked to a specific user, and not to the number of users.
Yet, I will qualify the conclusion with the statement that further research is necessary. Firstly, the analysis was carried out in London – so checking what is happening in other parts of the country where different users collected the data is necessary. Secondly, the analysis did not include the interesting range of 1 to 5 users, so it might be the case that there is rapid improvement in quality from 1 to 5 and then it doesn’t matter. Maybe the big change is from 1 to 3? Finally, the analysis focused on positional accuracy, and it is worth exploring the impact of the number of users on completeness.

ICA Commission on Use and User Issues workshop – April 2010 – Call for papers

This is call for papers for a workshop on methods and research techniques that are suitable for geospatial technologies. The workshop is planned for the day before GISRUK 2010, and we are aware of the clashes with the AAG 2010 annual meeting, CHI 2010 and the Ergonomics Society Annual Conference. However, if you would like to contribute to the book that the commission is developing but can’t attend the workshop, please send an abstract and inform us that you can’t attend.

In the near future I’ll publish information about another workshop in March 2010 about the usability and Human-Computer Interaction aspects of geographical information itself – see the report from the Ordnance Survey workshop earlier in 2009.

So here is the full call:

Workshop on Methods and Techniques of Use, User and Usability Research in Geo-information Processing and Dissemination

Tuesday 13 April 2010 at University College London

The Commission on Use and User Issues of the International Cartographic Association (ICA) is currently working on a new handbook specifically addressing the application of user research methods and techniques in the geodomain.

In order to share experiences and interesting case studies a workshop is organized by the Commission, in collaboration with UCL, on the day preceding GISRUK 2010.

CALL FOR PAPERS

While there is growing awareness within the research community on the need to develop usability engineering and use and user research methods that are suitable for geographical and spatial information and systems, to date there is a lack of organized and documented experience in this area.

We therefore invite researchers with recent experience with use, user and usability research in the broad geodomain (cartography, GIS, geovisualization, Location Based Services, geographical information, GeoWeb etc.) to present a paper specifically focusing on the research methods and techniques applied, with an aim to develop the body of knowledge for the domain.

To participate, please send an abstract of 1 page A4 at maximum containing:

• A description of the research method(s) and technique(s) applied
• A short description of the case in which they have been applied
• The overall research framework
• Contact details and affiliation of the author(s)

We are also encouraging PhD researchers to submit paper proposals and share experiences from their research. At the workshop there will be ample time for discussing the application of user research methods and techniques. Good papers may be the basis for contributions to the handbook that is planned for publication in 2011.

Abstracts should be submitted on or before 1 December 2009 to the Chairman of the Commission Corné van Elzakker ( elzakker@itc.nl )

Also see:
the website of the ICA Commission on Use and User Issues and  the GISRUK2010 website

The end of the ‘ARC/INFO driving licence’ era

The discussion about the future of the GIS ‘profession’ has flared up in recent days – see the comments from Sean Gorman, Steven Feldman (well, citing me) and Don Meltz among others. My personal perspective is about the educational aspect of this debate.

I’ve been teaching GIS since 1995, and been involved in the MSc in GIS at UCL since 1998 – teaching on it since 2001. Around 1994 I was contemplating the excellent MSc in GIS programme in Edinburgh, though I opted to continue with my own mix of geography and computer science, which turned out to be great in the end – but I can say that I have been following the trends in GIS education for quite a while.

Based on this experience, I would argue that the motivation for studying an MSc in GIS over the past 20 years was to get the ‘ARC/INFO driving licence’. I use ARC/INFO as a metaphor – you can replace it with any other package, but ARC/INFO was the de facto package for teaching GIS (and its predecessor ArcGIS is today), so it is suitable shorthand. What I mean by that is that for a long time GIS packages were hard to use and required a significant amount of training in order to operate successfully. Even if a fairly simple map was needed, the level of technical knowledge and the number of steps required were quite significant. So employers, who mostly wanted someone who could make them maps, recruited people who gained skills in operating the complex packages that allow the production of maps.

The ‘ARC/INFO driving licence’ era included an interesting dissonance – the universities were telling themselves that they were teaching the principles of GIScience but the students were mostly interested in learning how to operate a GIS at a proficient level to get a job. I’ve seen and talked with enough students to recognise that many of them, in their daily jobs, rarely used the spatial statistical analysis that we were teaching and they mostly worked at ‘taming the beast’, which GIS was.

As expected, at UCL there was always a group that was especially interested in the principles of GIScience and that continued their studies beyond the MSc. But they are never the majority of the cohort.

The model worked well for everyone – universities were teaching GIS by a combination of principles and training of specific packages and the students found jobs at the end and joined GIS departments in different organisations.

The disruption that changed this arrangement started in the late 1990s, with Oracle Spatial starting to show that GIS can be integrated in mainstream products. The whole process accelerated around 2005 with the emergence of GeoWeb, Free and Open Source GIS (FOSS GIS) and the whole range of applications that come with it. Basically, you don’t need a licence any more. More and more employers (even GIS consultancies) are not recruiting from GIS education programmes – they are taking computing professionals and teaching them the GIS skills. Going through an MSc in GIS to be proficient with a tool is not necessary.

So in an era in which you don’t need a licence to join the party, what is the MSc in GIS for?

The answer is that it can be the time when you focus on principles and on improving specific skills. Personally, that was my route to education. I started working in GIS software development without much more than high school education in 1988. After hearing people around me talking about registers, bugs, polygons and databases I was convinced that I must understand these principles properly. So I went for a degree that provided me with the knowledge. In the same way, I would expect that MSc programmes cater for the needs of people who gain some practical experience with operating geospatial technologies and want to learn the principles or become specialists in specific aspects of these systems.

We already see people doing the MSc while working with GIS – currently studying an MSc by distance learning or in the evening is very popular and I expect that this will continue. However, the definition of what is covered by GIS must be extended – it should include everything from Bing Maps API to PostGIS to ArcGIS.

I can also see the need for specialised courses – maybe to focus on the technical development of geospatial technologies or maybe on spatial statistical analysis for those who want to become geographical information analysts. I would also expect much more integration of GIS with other fields of study where it is taught as a tool – just look at the many MSc programmes that currently include GIS. I’m already finding myself teaching students of urban design, development planning or asset management.
All in all, I’m not going to feel sorry that the ‘ARC/INFO driving licence’ era is coming to its end.

Google Geo applications – deteriorating interfaces?

While Google wasn’t the first website to implement slippy maps – maps that are based on tiles, download progressively and allow fairly smooth user interaction – it does deserve the credit for popularising them. The first version of Google Maps was a giant leap in terms of public web mapping applications, as described in our paper about Web Mapping 2.0.

In terms of usability, the slippy map increased the affordability of the map with direct manipulation functionality for panning, clear zoom operating through predefined scales, the use of as much screen assets for the map as possible, and the iconic and simple search box at the top. Though the search wasn’t perfect (see the post about the British Museum test), overall it offered a huge improvement in usability. It is not surprising that it became the most popular web mapping site and the principles of the slippy map are the de facto standard for web mapping interaction.

However, in recent months I couldn’t avoid noticing that the quality of the interface has deteriorated. In an effort to cram more and more functionality (such as the visualisation of the terrain, pictures, or StreetView), ease of use has been scarificed. For example, StreetView uses the icon of a person on top of the zoom scale, which the user is supposed to drag and drop on the map. It is the only such object on the interface, and appears on the zoom scale regardless of whether it is relevant or available. When you see the whole of the UK for example, you are surely not interested in StreetView, and if you are zooming to a place that wasn’t surveyed, the icon greys out after a while. There is some blue tinge to indicate where there is some coverage, but the whole interaction with it is very confusing. It’s not difficult to learn, though.

Even more annoying is that when you zoom to street level on the map, it switches automatically to StreetView, which I found distracting and disorientating.

There are similar issues with Google Earth – compare versions 4 and 5 in terms of ease of use for novice users, and my guess is that most of them will find 4 easier to use. The navigation both above the surface and at surface level is anything but intuitive in version 5. While in version 4 it was clear how to tilt the map, this is not the case in 5.

So maybe I should qualify what I wrote previously. There seems to be a range here, so it is not universally correct to say that the new generation of geographical applications are very usable just because they belong to the class of ‘neogeography’. Maybe, as ‘neogeography’ providers are getting more experienced, they are falling into the trap of adding functionality for the sake of it, and are slowly, but surely, destroying the advantages of their easy-to-use interfaces… I hope not!

GISRUK 2010 at UCL – Call for papers

Geographical Information Science Research UK (GISRUK) is a research conference that has been taking place in different university campuses around the UK (and once in Ireland) since 1993. Despite the name, it is open not just to researchers from the UK, but also to international participants, who are very welcome.

For me, GISRUK was the first international conference in which I presented a paper eleven years ago, so I have a soft spot for it. It was very friendly and welcoming for a starting research student (which I was at the time). It was especially useful to discover that all the famous academics who attended it were friendly and open to questions.

The conference will be held at UCL in April 2010, and the call for papers is now out, so consider submitting a paper.

The papers are rather short, about 1500 words, so there is plenty of time to write one in time for the deadline of the end of November.

Interacting with Geospatial Technologies

At the end of September, the manuscript of ‘Interacting with Geospatial Technologies’ was submitted to John Wiley & Sons. This is the reason for the silence on this blog since July while the final chapters were written.

The book, which is an introduction to usability and Human-Computer Interaction aspects of GIS and other geospatial technologies, was written because there is no other recent book that covers these aspects while taking into account the special characteristics of geographical information and the extensive use of maps.

There were several books in the early 1990s dedicated to human factors of GIS or to cognitive aspects of these systems. Since then, there have been many published articles, but no easy-to-access summary of the outcomes in a way that is useful for developers or people who want to understand how to design more usable systems. So, while working on a paper that called for developing ‘usability engineering for GIS’ in 2005, I figured out that, actually, it was time to write an introductory text in this area. In the end, this is an edited textbook written by me together with a group of excellent collaborators: Jochen Albrecht, Clare Davies, Catherine Emma Jones, Robert Laurini, Chao Li, Aaron M. Marcus, Stephanie L. Marsh, Annu-Maaria Nivala, Artemis Skarlatidou, Carolina Tobón, Jessica Wardlaw and Antigoni Zafiri.

So the book provides an introduction to user-centred design and usability engineering from a geospatial technologies perspective, theoretical aspects of human understanding of space and collaborative systems, practical aspects of cartography and map design that are useful for developers and application designers, guidance for evaluating geospatial systems and some tips for designing desktop, Web and mobile based systems. Each chapter includes case studies and examples that make the material more concrete.

The book is scheduled to be out by March 2010. A lot of work went into writing the various chapters and ensuring that the content is covering all the needed elements to create a usable GIS – I hope that it will be useful!

Volunteered Geographical Information Research Network

Chris Parker, a PhD student at Loughborough University, organised a dedicated Volunteered Geographical Information research group site on ResearchGate. While I dislike the term – I usually interpret it as the version of ‘volunteered’ as in ‘mum volunteered me to help the old lady cross the street’ – there is no point in trying to change it. When Mike Goodchild coins an acronym, it will stick; it’s sort of a GIScience law!

If you are interested in user-generated geographical content, crowdsourced geographical information, commons-based peer-produced geographical information, or any other way to call this phenomena (for example VGI) – join the group. It will be good to keep in touch, share information and discuss research aspects.
If you are researching in this area you are also welcome to submit a paper to GISRUK 2010 which will be hosted at UCL – we are keen to have a VGI element in the programme, considering that UCL is the host of OpenStreetMap .

OpenStreetMap and Ordnance Survey Master Map – Beyond good enough

OSM overlap with Master Map ITN for A and B roads

In June, Aamer Ather, an M.Eng. student at the department, completed his research comparing OpenStreetMap (OSM) to Ordnance Survey Master Map Integrated Transport Layer (ITN). This was based on the previous piece of research in which another M.Eng. student, Naureen Zulfiqar, compared OSM to Meridian 2.

There are really surprising results. The analysis shows that when A-roads, B-roads and a motorway from ITN are compared to OSM data, the overlap can reach values that are over 95%. When the comparison with Master Map was completed, it became clear that OSM is of better quality than Meridian 2. It is also interesting to note that the results of higher overlap with ITN were achieved under stricter criteria for the buffering procedure that is used for comparison.

As noted, in the original analysis, Meridian 2 was used as the reference dataset, the ground truth. However, comparing Meridian 2 and OSM is not like with like, because OSM is not generalised and Meridian 2 is. The justification for treating Meridian 2 as the reference dataset was that the nodes are derived from high-accuracy datasets and it was expected that the 20 metres filter would not change positions significantly. It turns out that the generalisation impacts the quality of Meridian more than I anticipated. Yet, the advantage of Meridian 2 is that it allows comparisons for the whole of England, since the file size is still manageable, while the complexity of ITN would make an extensive comparison difficult, time-consuming and lengthy.

The results show that for the 4 Ordnance Survey London tiles that we’ve compared, the results put OSM only 10-30% from the ITN centre line. Rather impressive when you consider the knowledge, skills and backgrounds of the participants. My presentation from the State of the Map conference, below, provides more details of this analysis – and the excellent dissertation by Aamer Ather, which is the basis for this analysis, is available to download here.

The one caveat that will need to be explored in future projects is that the comparison in London means that OSM mappers had access to very high-resolution imagery from Yahoo! which have been georeferenced and rectified. Therefore, the high precision might be a result of tracing these images, and the question is what happens in places where high resolution images are not available. Thus, we need to test more tiles and in other places to validate the results in other areas of the UK.

Another student is currently comparing OSM to 1:10,000 map of Athens, so by the end of the summer I hope that it will be possible to estimate quality in other countries. The comparison to ITN in other areas of the UK will wait for a future student who will be interested in this topic!

Neo and Paleo GIS – is the difference in the usability culture?

At several recent GIS industry and academic conferences, I was not very surprised to see GIS presentations in which the presenter started by talking about ‘usability enhancements’ and ‘we took usability very seriously in this application’ but failed to deliver. In contrast to such statements, the application itself was breaking basic usability guidelines such as not giving any feedback to the user about some activity of the system, or grouping related elements together in the interface, among other problems.

Then I came across a report from 1991, which talks about User-Centred Graphical User Interface for GIS and notes that ‘It is not unusual for more than 60% of the code in a complex software system to be dedicated purely to the user interface. This stands in sharp contrast to the 35% dedicated to the user interface in early GISs’. This is still true in spirit, if not in percentage. GIS applications require sophisticated data manipulation, and most of the development effort of GIS vendors or Open Source GIS projects is focused on the information itself and its manipulation. The interface is probably seen as an add-on – the ‘fun’ bit of the development that you leave to the end after cracking all the engineering challenges that make the application work.

What I would argue is that, as a result, GIS as an industry doesn’t have a ‘usability culture’. Compare that to Apple, where usability and interaction with users has been at the centre of what they are doing since they started. Or with e-commerce which also shows a ‘usability culture’ because, if you fail on usability, there is a direct link to loss of sales. These are examples of organisations and sectors who know that usability is important and commit resources to ensuring that their products are usable.

In contrast, in the GIS industry there is a feeling that usability is a ‘nice to have’ element of the development process, so there is no practice of involving usability experts in software development projects. There are relatively few examples of user-centred design in GIS, and they are mostly in research papers, very rarely in practice.

Neogeography is changing it somewhat, since parts of it are coming from companies and developers who see the value in understanding the users. Maybe the competition between the existing developers of GIS and neogeography companies will cause the former to change and they will become more serious about usability.

Terra Future 2009 – OpenStreetMap and Ordnance Survey

I have checked on Twitter to see how the follow-up meeting to Terra Future 2009, last Friday, went. It was a very pleasant surprise to see that the idea that I have put forward in February, that the Ordnance Survey should consider hosting OpenStreetMap and donate some data to it, was voted the best idea that came out of Terra Future 2009. With this sort of peer-review of the idea, and with the added benefit of 2 months for rethinking, I still think that it is quite a good idea.

The most important aspect of this idea is to understand that OpenStreetMap and Ordnance Survey can both thrive in the GeoWeb era. Despite the imaginary competition, each has a clear value to certain parts of the marketplace. There are a very clear benefits that the OpenStreetMap community can gain from working closely with the Ordnance Survey – such as some aspects of mapping that the Ordnance Survey are highly knowledgeable about, and vice versa, such as how to innovate in delivery of geographical information. A collaborative model might work after all…

I wonder how this idea will evolve now?