Levels of participation in citizen science and scientific knowledge production

The previous post focused on citizen science as participatory science. This post is discussing the meaning of this differentiation. It is the final part of the chapter that will appear next year in the book:

Sui, D.Z., Elwood, S. and M.F. Goodchild (eds.), 2012. Volunteered Geographic Information, Public Participation, and Crowdsourced Production of Geographic Knowledge. Berlin: Springer.

The typology of participation can be used across the range of citizen science activities, and one project should not be classified only in one category. For example, in volunteer computing projects most of the participants will be at the bottom level, while participants that become committed to the project might move to the second level and assist other volunteers when they encounter technical problems. Highly committed participants might move to a higher level and communicate with the scientist who coordinates the project to discuss the results of the analysis and suggest new research directions.

This typology exposes how citizen science integrates and challenges the way in which science discovers and produces knowledge. Questions about the way in which knowledge is produced and truths are discovered are part of the epistemology of science. As noted above, throughout the 20th century, as science became more specialised, it also became professionalised. While certain people were employed as scientists in government, industry and research institutes, the rest of the population – even if they graduated from a top university with top marks in a scientific discipline – were not regarded as scientists or as participants in the scientific endeavour unless they were employed professionally to do so. In rare cases, and following the tradition of ‘gentlemen/women scientists’, wealthy individuals could participate in this work by becoming an ‘honorary fellow’ or affiliated to a research institute that, inherently, brought them into the fold. This separation of ‘scientists’ and ‘public’ was justified by the need to access specialist equipment, knowledge and other privileges such as a well-stocked library. It might be the case that the need to maintain this separation is a third reason that practising scientists shy away from explicitly mentioning the contribution of citizen scientists to their work in addition to those identified by Silvertown (2009).

However, similarly to other knowledge professionals who operate in the public sphere, such as medical experts or journalists, scientists need to adjust to a new environment that is fostered by the Web. Recent changes in communication technologies, combined with the increased availability of open access information and the factors that were noted above, mean that processes of knowledge production and dissemination are opening up in many areas of social and cultural activities (Shirky 2008). Therefore, some of the elitist aspects of scientific practice are being challenged by citizen science, such as the notion that only dedicated, full-time researchers can produce scientific knowledge. For example, surely it should be professional scientists who can solve complex scientific problems such as long-standing protein-structure prediction of viruses. Yet, this exact problem was recently solved through a collaboration of scientists working with amateurs who were playing the computer game Foldit (Khatib et al. 2011). Another aspect of the elitist view of science can be witnessed in interaction between scientists and the public, where the assumption is of unidirectional ‘transfer of knowledge’ from the expert to lay people. Of course, as in the other areas mentioned above, it is a grave mistake to argue that experts are unnecessary and can be replaced by amateurs, as Keen (2007) eloquently argued. Nor is it suggested that, because of citizen science, the need for professionalised science will diminish, as, in citizen science projects, the participants accept the difference in knowledge and expertise of the scientists who are involved in these projects. At the same time, the scientists need to develop respect towards those who help them beyond the realisation that they provide free labour, which was noted above.

Given this tension, the participation hierarchy can be seen to be moving from a ‘business as usual’ scientific epistemology at the bottom, to a more egalitarian approach to scientific knowledge production at the top. The bottom level, where the participants are contributing resources without cognitive engagement, keeps the hierarchical division of scientists and the public. The public is volunteering its time or resources to help scientists while the scientists explain the work that is to be done but without expectation that any participant will contribute intellectually to the project. Arguably, even at this level, the scientists will be challenged by questions and suggestions from the participants and, if they do not respond to them in a sensitive manner, they will risk alienating participants. Intermediaries such as the IBM World Community Grid, where a dedicated team is in touch with scientists who want to run projects and a community of volunteered computing providers, are cases of ‘outsourcing’ the community management and thus allowing, to an extent, the maintenance of the separation of scientists and the public.

As we move up the ladder to a higher level of participation, the need for direct engagement between the scientist and the public increases. At the highest level, the participants are assumed to be on equal footing with the scientists in terms of scientific knowledge production. This requires a different epistemological understanding of the process, in which it is accepted that the production of scientific insights is open to any participant while maintaining scientific standards and practices such as systematic observations or rigorous statistical analysis to verify that the results are significant. The belief that, given suitable tools, many lay people are capable of such endeavours is challenging to some scientists who view their skills as unique. As the case of the computer game that helped in the discovery of new protein formations (Khatib et al. 2011) demonstrated, such collaboration can be fruitful even in cutting-edge areas of science. However, it can be expected that the more mundane and applied areas of science will lend themselves more easily to the fuller sense of collaborative science in which participants and scientists identify problems and develop solutions together. This is because the level of knowledge required in cutting-edge areas of science is so demanding.

Another aspect in which the ‘extreme’ level challenges scientific culture is that it requires scientists to become citizen scientists in the sense that Irwin (1995), Wilsdon, Wynne and Stilgoe (2005) and Stilgoe (2009) advocated (Notice Stilgoe’s title: Citizen Scientists). In this interpretation of the phrase, the emphasis is not on the citizen as a scientist, but on the scientist as a citizen. It requires the scientists to engage with the social and ethical aspects of their work at a very deep level. Stilgoe (2009, p.7) suggested that, in some cases, it will not be possible to draw the line between the professional scientific activities, the responsibilities towards society and a fuller consideration of how a scientific project integrates with wider ethical and societal concerns. However, as all these authors noted, this way of conceptualising and practising science is not widely accepted in the current culture of science.

Therefore, we can conclude that this form of participatory and collaborative science will be challenging in many areas of science. This will not be because of technical or intellectual difficulties, but mostly because of the cultural aspects. This might end up being the most important outcome of citizen science as a whole, as it might eventually catalyse the education of scientists to engage more fully with society.

GISRUK 2011 talk – Participatory GIS, Volunteered Geographic Information and Citizen Science

GIS Research UK (GISRUK) is a long running conference series, and the 2011 instalment was hosted by the University of Portsmouth at the end of April.

During the conference, I was asked to give a keynote talk about Participatory GIS. I decided to cover the background of Participatory GIS in the mid-1990s, and the transition to more advanced Web Mapping applications from the mid-2000s. Of special importance are the systems that allow user-generated content, and the geographical types of systems that are now leading to the generation of Volunteer Geographic Information (VGI).

The next part of the talk focused on Citizen Science, culminating with the ideas that are the basis for Extreme Citizen Science.

Interestingly, as in previous presentations, one of the common questions about Citizen Science came up. Professional scientists seem to have a problem with the suggestion that citizens are as capable as scientists in data collection and analysis. While there is an acceptance about the concept, the idea that participants can suggest problems, collect data rigorously and analyse it seems to be too radical – or worrying.

What is important to understand is that the ideas of Extreme Citizen Science are not about replacing the role of scientists, but are a call to rethink the role of the participants and the scientists in cases where Citizen Science is used. It is a way to consider science as a collaborative process of learning and exploration of issues. My own experience is that participants have a lot of respect for the knowledge of the scientists, as long as the scientists have a lot of respect for the knowledge and ability of the participants. The participants would like to learn more about the topic that they are exploring and are keen to know: ‘what does the data that I collected mean?’ At the same time, some of the participants can become very serious in terms of data collection, reading about the specific issues and using the resources that are available online today to learn more. At some point, they are becoming knowledgeable participants and it is worth seeing them as such.

The slides below were used for this talk, and include links to the relevant literature.

Public engagement with water conservation in London

In 2009, Ud Doron, who studied on our MSc in Environmental Systems Engineering developed a research project together with Tse-Hui Teh, who is doing her PhD on urban water issues. The project was co-supervised by Sarah Bell.

The focus of the project was on a series of participatory workshops to understand the relationships between urban residents and water technology. The workshops explored the perceptions and actions of environmentally aware citizens. Ud also explored the use of environmental information by the participants of the workshops.  The output of this work is now written and published in the Water and Environment Journal.

The paper is titled Public engagement with water conservation in London

The abstract is:

Understanding water demand and consumers’ capacity for change is essential in underpinning water demand management and water efficiency programmes. This paper presents the outcomes of a qualitative study, which used discussion groups relating to water infrastructure with environmentally aware citizens in five London boroughs in the Lower Lea River Basin. The results showed a subtle interaction between users, water and technology. Users are generally unaware of their own water consumption. Individual perceptions of changes in water behaviour are constrained by habit and lack of knowledge about what changes can be made and how. Knowledge of environmental information was described as the inspiration behind making any changes. The paper concludes that access to information about water resources, infrastructure and conservation measures should be enhanced because although information sources are abundant, participants claimed they were inaccessible without considerable effort. Finally, an emphasis should also be put on helping the public form a more substantial part in environmental decisions.

and the paper is accessible in the early view section of the  Water and Environment Journal http://onlinelibrary.wiley.com/doi/10.1111/j.1747-6593.2011.00256.x/full

Completeness in volunteered geographical information – the evolution of OpenStreetMap coverage (2008-2009)

The Journal of Spatial Information Science (JOSIS) is a new open access journal in GIScience, edited by Matt Duckham, Jörg-Rüdiger Sack, and Michael Worboys. In addition, the journal adopted an open peer review process, so readers are invited to comment on a paper while it goes through the formal peer review process. So this seem to be the most natural outlet for a new paper that analyses the completeness of OpenStreetMap over 18 months – March 2008 to October 2009. The paper was written in collaboration with Claire Ellul.  The abstract of the paper provided below, and you are very welcome to comment on the paper on JOSIS forum that is dedicated to it, where you can also download it.

Abstract: The ability of lay people to collect and share geographical information has increased markedly over the past 5 years as results of the maturation of web and location technologies. This ability has led to a rapid growth in Volunteered Geographical Information (VGI) applications. One of the leading examples of this phenomenon is the OpenStreetMap project, which started in the summer of 2004 in London, England. This paper reports on the development of the project over the period March 2008 to October 2009 by focusing on the completeness of coverage in England. The methodology that is used to evaluate the completeness is comparison of the OpenStreetMap dataset to the Ordnance Survey dataset Meridian 2. The analysis evaluates the coverage in terms of physical coverage (how much area is covered), followed by estimation of the percentage of England population which is covered by completed OpenStreetMap data and finally by using the Index of Deprivation 2007 to gauge socio-economic aspects of OpenStreetMap activity. The analysis shows that within 5 years of project initiation, OpenStreetMap already covers 65% of the area of England, although when details such as street names are taken into consideration, the coverage is closer to 25%. Significantly, this 25% of England’s area covers 45% of its population. There is also a clear bias in data collection practices – more affluent areas and urban locations are better covered than deprived or rural locations. The implications of these outcomes to studies of volunteered geographical information are discussed towards the end of the paper.

“How good is VGI? A comparative study of OpenStreetMap and Ordnance Survey datasets” – published

The process of academic publication takes a long time, so only now my paper from 2008 is finally in print.

So the paper, which should be cited as:
“Haklay, M., 2010, How good is volunteered geographical information? A comparative study of OpenStreetMap and Ordnance Survey datasets” Environment and Planning B: Planning and Design 37(4) 682 – 703″

It’s abstract is:
Within the framework of Web 2.0 mapping applications, the most striking example of a geographical application is the OpenStreetMap (OSM) project. OSM aims to create a free digital map of the world and is implemented through the engagement of participants in a mode similar to software development in Open Source projects. The information is collected by many participants, collated on a central database, and distributed in multiple digital formats through the World Wide Web. This type of information was termed ‘Volunteered Geographical Information’ (VGI) by Goodchild, 2007. However, to date there has been no systematic analysis of the quality of VGI. This study aims to fill this gap by analysing OSM information. The examination focuses on analysis of its quality through a comparison with Ordnance Survey (OS) datasets. The analysis focuses on London and England, since OSM started in London in August 2004 and therefore the study of these geographies provides the best understanding of the achievements and difficulties of VGI. The analysis shows that OSM information can be fairly accurate: on average within about 6 m of the position recorded by the OS, and with approximately 80% overlap of motorway objects between the two datasets. In the space of four years, OSM has captured about 29% of the area of England, of which approximately 24% are digitised lines without a complete set of attributes. The paper concludes with a discussion of the implications of the findings to the study of VGI as well as suggesting future research directions.

The paper can be found here. If you are interest in a copy of the published version, email me.

Mapping for Change activities covered in GIS Development

In its February issue, the magazine GIS development published an article on the activities of the social enterprise Mapping for Change. Please note that the image at the start of the published article is not from our activities and that the article was truncated for publication – we are currently working on a more comprehensive version that we aim to publish later in the year. Dr Hanif Rahemtulla contributed significantly to organising the writing of this article.

The article describes how we utilised community mapping, participatory sensing and mashups technologies to deal with a range of environmental issues with communities across London. It also provides information about our recent projects, including the North Dorset Climate Action map.

The article can be accessed here.

OpenStreetMap in Athens – as accurate as London

Most of the work that we carried out at UCL in evaluating the quality of OpenStreetMap is focused on England, and particularly on London. This is mainly due to the accessibility of comparative datasets. The reason for this was the availability of data, as the Ordnance Survey research unit kindly provided me with the full Meridian 2 dataset for comparison. More detailed comparison, for which we used MasterMap, came from the wonderful Digimap service, though because of the time that it takes to process it we were limited in the size of the area that was used for comparison.

One of the open questions that remained was the accuracy of data collection in other parts of the world. Luckily, Ourania (Rania) Kounadi, who studied our MSc in GIS at UCL, had access to detailed maps of Athens. She used a 1:10,000 map from the Hellenic Military Geographic Service (HGMS) and focused on an area of 25 square kilometres at the centre of the city. The roads were digitised from the HGMS map, and then the Goodchild-Hunter procedure was used to evaluate the positional accuracy.

The results show that for most of the roads in the evaluation area there was an overlap of 69% to 100% between OSM and HGMS datasets. The average overlap was very close to 90%. Her analysis also included attribute and completeness evaluation, showing that the quality is high on these aspects too.

OSM positional accuracy for Athens

So a pattern is starting to emerge showing that the quality of OSM data is indeed good in terms of positional accuracy. This is surprising at first glance – how come people who are not necessarily trained in geographical data collection and do not use rigorous quality assurance processes produce data that is as good as the authoritative data?

My explanation for this, as I’ve written in my paper about OSM quality, is that it ‘demonstrates the importance of the infrastructure, which is funded by the private and public sector and which allows the volunteers to do their work without significant personal investment. The GPS system and the receivers allow untrained users to automatically acquire their position accurately, and thus simplify the process of gathering geographical information. This is, in a way, the culmination of the process in which highly trained surveyors were replaced by technicians, with the introduction of high-accuracy GPS receivers in the construction and mapping industries over the last decade. The imagery also provides such an infrastructure function – the images were processed, rectified and georeferenced by experts and thus, an OSM volunteer who uses this imagery for digitising benefits from the good positional accuracy which is inherent in the image. So the issue here is not to compare the work of professionals and amateurs, but to understand that the amateurs are actually supported by the codified professionalised infrastructure and develop their skills through engagement with the project.’
Rania’s dissertation is available to download from here.

ICA Commission on Use and User Issues workshop – April 2010 – Call for papers

This is call for papers for a workshop on methods and research techniques that are suitable for geospatial technologies. The workshop is planned for the day before GISRUK 2010, and we are aware of the clashes with the AAG 2010 annual meeting, CHI 2010 and the Ergonomics Society Annual Conference. However, if you would like to contribute to the book that the commission is developing but can’t attend the workshop, please send an abstract and inform us that you can’t attend.

In the near future I’ll publish information about another workshop in March 2010 about the usability and Human-Computer Interaction aspects of geographical information itself – see the report from the Ordnance Survey workshop earlier in 2009.

So here is the full call:

Workshop on Methods and Techniques of Use, User and Usability Research in Geo-information Processing and Dissemination

Tuesday 13 April 2010 at University College London

The Commission on Use and User Issues of the International Cartographic Association (ICA) is currently working on a new handbook specifically addressing the application of user research methods and techniques in the geodomain.

In order to share experiences and interesting case studies a workshop is organized by the Commission, in collaboration with UCL, on the day preceding GISRUK 2010.

CALL FOR PAPERS

While there is growing awareness within the research community on the need to develop usability engineering and use and user research methods that are suitable for geographical and spatial information and systems, to date there is a lack of organized and documented experience in this area.

We therefore invite researchers with recent experience with use, user and usability research in the broad geodomain (cartography, GIS, geovisualization, Location Based Services, geographical information, GeoWeb etc.) to present a paper specifically focusing on the research methods and techniques applied, with an aim to develop the body of knowledge for the domain.

To participate, please send an abstract of 1 page A4 at maximum containing:

• A description of the research method(s) and technique(s) applied
• A short description of the case in which they have been applied
• The overall research framework
• Contact details and affiliation of the author(s)

We are also encouraging PhD researchers to submit paper proposals and share experiences from their research. At the workshop there will be ample time for discussing the application of user research methods and techniques. Good papers may be the basis for contributions to the handbook that is planned for publication in 2011.

Abstracts should be submitted on or before 1 December 2009 to the Chairman of the Commission Corné van Elzakker ( elzakker@itc.nl )

Also see:
the website of the ICA Commission on Use and User Issues and  the GISRUK2010 website

GISRUK 2010 at UCL – Call for papers

Geographical Information Science Research UK (GISRUK) is a research conference that has been taking place in different university campuses around the UK (and once in Ireland) since 1993. Despite the name, it is open not just to researchers from the UK, but also to international participants, who are very welcome.

For me, GISRUK was the first international conference in which I presented a paper eleven years ago, so I have a soft spot for it. It was very friendly and welcoming for a starting research student (which I was at the time). It was especially useful to discover that all the famous academics who attended it were friendly and open to questions.

The conference will be held at UCL in April 2010, and the call for papers is now out, so consider submitting a paper.

The papers are rather short, about 1500 words, so there is plenty of time to write one in time for the deadline of the end of November.

Interacting with Geospatial Technologies

At the end of September, the manuscript of ‘Interacting with Geospatial Technologies’ was submitted to John Wiley & Sons. This is the reason for the silence on this blog since July while the final chapters were written.

The book, which is an introduction to usability and Human-Computer Interaction aspects of GIS and other geospatial technologies, was written because there is no other recent book that covers these aspects while taking into account the special characteristics of geographical information and the extensive use of maps.

There were several books in the early 1990s dedicated to human factors of GIS or to cognitive aspects of these systems. Since then, there have been many published articles, but no easy-to-access summary of the outcomes in a way that is useful for developers or people who want to understand how to design more usable systems. So, while working on a paper that called for developing ‘usability engineering for GIS’ in 2005, I figured out that, actually, it was time to write an introductory text in this area. In the end, this is an edited textbook written by me together with a group of excellent collaborators: Jochen Albrecht, Clare Davies, Catherine Emma Jones, Robert Laurini, Chao Li, Aaron M. Marcus, Stephanie L. Marsh, Annu-Maaria Nivala, Artemis Skarlatidou, Carolina Tobón, Jessica Wardlaw and Antigoni Zafiri.

So the book provides an introduction to user-centred design and usability engineering from a geospatial technologies perspective, theoretical aspects of human understanding of space and collaborative systems, practical aspects of cartography and map design that are useful for developers and application designers, guidance for evaluating geospatial systems and some tips for designing desktop, Web and mobile based systems. Each chapter includes case studies and examples that make the material more concrete.

The book is scheduled to be out by March 2010. A lot of work went into writing the various chapters and ensuring that the content is covering all the needed elements to create a usable GIS – I hope that it will be useful!