The last day of the BES/Sfé meeting was in the mood of celebration, so a session dedicated to celebrating citizen science was in place.  My notes from first day and the second day are in previous posts. These notes are long…

Before the session, in a symposium on tree health, Michael Pocock (CEH) presented ‘Monitoring to assess the impacts of tree diseases: integrating citizen science with professional monitoring‘. Ash die-back is important, and in the rest of Europe, (e.g. Denmark, Lithuania or Poland) there are losses of 60-90% but there was very little work done on monitoring the biodiversity impact of the disease in general. There is a clear lack of knowledge on the impacts on biodiversity in general – how suitable are existing surveys, how they can enhance? In a work that he done with Shelley Hinsley they reviewed 79 relevant studies, from volunteers to national professional survey and local studies. They tried to answer questions such as: What kind of things can be impacted? they identified all sort of impacts - trophic networks, structural, cascading, and ecosystem functions. They looked at different receptors in different contexts – from animals and plants on the receptors, to where they are located as context – woodland, or hedgerow. They found that woods are fairly well monitored, but how much professionals will continue to monitor it with budget cuts is an issue. Ecosystem function is very poorly monitored. The recommendations of the report are that current ongoing activities are suitable and maybe should be modified a bit to make them better (e.g. asking another question in a survey) – they didn’t recommend brand new surveys. The report is available here . If we want future proof monitoring that deal with the range of tree disease and other issues – we need a better ‘spine’ of monitoring work (in the report on page 5), but improve the integration of information and synthesis between survey. Co-location of monitoring site can be great, but actually, there are specific reasons for the locations of places in each scheme so it’s not easy to do so. In addition, volunteers based monitoring require investment in maintenance. He completed his talk with more general citizen science issue that we can learn from this work – the national plant monitoring scheme is to be launched in 2015, and there are some specific focused on lichens and other issues that require specialist knowledge in survey programmes like Splash. Mass participation is useful in some cases, but there is an issue how much recording effort is quantified – there is a big differentiation in ability to monitor species across the country and the ability of participants to record information. The retention of volunteers in mass projects is an issue – only 10% continue after a year. In enthusiasts recruitment you get higher numbers 20% that continue to be involved. The most exciting opportunity that he see is in  hypothesis-led citizen science, like the Concker Tree Science project.

The ‘Celebrating Citizen Science’ session was at the  final group of sessions of the conference, but was very well attended. Chaired by  Michael Pocock, who, together with Helen Roy, runs the BES Citizen Science SIG.

Romain Julliard (Muséum national d’Histoire naturelle)  provided an overview of citizen science activities in France in his talk ‘Biodiversity monitoring through citizen science: a case study from France’. The starting statement was that unskilled amateurs from the general public can provide good information. The museum have a role in monitoring biodiversity at the national – common species are good indicators, the appropriate for studying global changes and the general public is interested in ‘ordinary Nature’ – the things that we see every day. Scientists alone cannot monitor biodiversity over a big space such as a country, so citizens can help to collect data on a country scale and they are already spread across the country. The trade-offs of using citizens as observers include skills vs. numbers of participants – there are only few experts and enthusiasts. Another issue is sampling design: are you aiming for representativeness of where people are or do you send observers to specific locations to do the survey. There is a need for a simple protocol for volunteers. Much simpler than procedures in a research station professionals. They started with French Bird Breeding Survey in coordination with NGOs like LPO and others – with over 2000 squared that are being observed since 1989 and over 1000 provide long-term monitoring. Now they have skilled amateur schemes – monitoring bats, butterflies and much more. They started their programmes in 2005 with butterfly programme, pollinating insect survey from photographs (Spipoll) in 2010 and garden bird watch in 2012 among others – new programmes especially in the past 5 years . Spipoll provides a good example of the work that they are doing. Pollinators are useful to raise awareness and explain multi-factor pressures on the environment. 2014-12-12 13.14.25The are many sampling sites and thousands of flowers dwelling insects in France. They Spipoll protocol starts with 20 minutes ‘safari-photo’ which mean that you select a flower and take photos of each visiting insects. Second step is to select the best single photo for each insect that was sampled. Third step to name each insect from 630 possibilities – and they create an online tool that helps the identification. Final step – share the collection with other people. Once photos are shared, there are plenty of comments from other participants. The participants are encouraged to help each other observations and there is also expert participation in identification. By now, they have over 600 regular participants, 18,000 collections, and 155,000 photos. Many of the participants are not experts in biological recording but have interest in photography. in terms of data quality they looked for precision, repeatability (how close the process was to the protocol). The social control help in improving quality, and the representativeness can be done in explicit sampling design but also in post-study statistical analysis. Beginners tend not to follow the protocol, but other people are helping them and within 3-4 iterations, people are learning the protocol and follow it.

Helen Roy (CEH) talk (with Harding, Preston, Pocock and Roy) ‘Celebrating 50 years of the Biological Records Centre. She gave some key achievements that also appear in a booklet on the 50 years of BRC. The BRC was established in the 1960s to support volunteer recording in the UK – they have now a team of 14 permanent staff. 85 different recording schemes from flee to bees, ladybirds and many other groups. Recording schemes are running by volunteers coordinators – so support is provided by printing newsletters, publishing atlases, etc. They cover a lot of taxa – plants and animals. Over the decades, they have long-term datasets which lead to distribution atlases. Over 80m records. UK biodiversity indicators for the UK government are collected by volunteers and used in decision-making – they are now growing from 24 indicators to include pollinators and other elements. Another area of importance is biological invasions as it cost the UK over 12 billion EUR a year – and not only to look at existing species but also to look forward about the threats – and because volunteers are so knowledgeable, they contributed to horizon scanning work. Work on surveillance and monitoring extend to the general public with publicity – this way they for example got information that Raccoons are being seen in the UK. Another important aspect of BRC data is the ability to use it to understand the decline of native species – for example understanding changes in native ladybird species. Finally, the information is very important in climate change scenarios and use the information about habitats can help in interpreting data and predict future directions.

In the work of the BRC, technology is becoming an important driver – they share it through the NBN gateway, and also apps and websites such as iSpot, iRecord and other bits are helping in developing new sources of information. In summary, to deal with environmental challenges that we’re currently facing cannot be done without this information and interpretation by volunteers. She finished with a big thank you to the many volunteers recorders.

In ‘How to use data generated by general public of a citizen science program for conservation purpose’ Nathalie Machon (Muséum national d’Histoire naturelle) explored another successful French study. They see importance in preserving biodiversity in cities – regulate city climate, dealing with air pollution, contributing to public health etc. In cities, most of the biodiversity is in parks and gardens but the urban matrix is permeable to many animal species such as pollinators. The potential of connection between green spaces is important to create a network in the city. How the structure and management of cities influence biodiversity? was a research question that the programme ‘sauvages de ma rue‘ was set to explore. Since 2011 participants share information about wild-flowers in their own streets. When the programme started, they wanted people to learn to recognise species near them and collect information about the distribution of plants in their area . The protocol is fairly simple – identify street, collect data about plants in different habitats (cracks, walls) and send the information. They created  a guide to help people identify species and also created a smartphone app. Usually people start by providing data about their street, but the programme grew and now they have groups and organisations that deal with naturalist activity and they send a lot of data from many streets in the same place. The organisations can be about sustainability, schools university or nature enthusiasts. They receives 40,660 data points by 2014 which provided the basis for her analysis.

After correction, they had reliable 20,000 data points in 38 cities and 2500 pavements – they check the richness of pavements and the obvious factor is the length (of course) but in about 100m there is a levelling in terms of species. They found that the structure of the street is important – if it is only in cracks, there are less species. The richness is not correlated to population density, but in large urban area (Paris) there is a significant decline toward the centre. They also look at pollination – and found that the number of pollinators is correlated to the human density of the city but not correlated to the distance to the centre of the city, apart from the case in Paris. They also seen increase with habitat types in a pavement. In terms of cities, they discovered that Nantes, Brest and Angers are doing well. However, they are aware that there is an observer effect on the results. Observers were shown to be good as botanists. In summary, they’ve learned that insect pollinated species are easy to recognise and it’s possible to carry out such studies effectively with lightly trained volunteers.

Anne-Caroline Prévot (CESCO – Muséum nationa l’Histoire Naturelle) reviewed her research on ‘Short and long-term individual consequences of participation to citizen-science projects’ in an approach that combines environmental psychology and ecology. There is growing concern on separation between people and nature: extinction of experience (Pyle 2003, Miller 2005) or environmental generational amnesia (Kahn 2002). There is a need engagement of majority of citizens to change their approach. In the psychology field  , there is Stern influential piece from 2000 on environmentally significant behaviour, linking individual to different aspects of pro-environmental behaviour. Identifying social and personal factors . On the other hand, in citizen science programme there are multiple goals – contribute to ecological science ; educate people to acquire knowledge on biodiversity; etc. There is also potential of reconnection to nature – so the  question that she addressed “Did citizen science changed biodiversity representation and knowledge? environmental values? pratcial knowledge? skills?” (all these are based on Stern framework). She looked at the butterfly collection programme and interview 30 regular volunteers who participate every year – They found that they were confident in science, and they discovered new aspects of biodiversity through participation and change their gardening practices. This can change representation but they were environmentally concern to start with. There was no issue of group identity  with this group of volunteers. The second study looked at a programme at school (vigienature école) with 400 pupils from 29 classes in 11-13 age group. They use a questionnaire to understand environmental value and other activities outside schools. In addition, they asked the children to draw an urban garden. Each drawing was analysed for natural elements, built elements and humans. Participation in nature monitoring showed higher presence of nature in drawing but no difference in environmental values. They think that it probably changed representation, but not values, there was no assessment of skills and there was some aspect of group social identity. In summary citizen science initative may change knwoeldge and attitdue of volunteers but this require attention and more evaluation.

Rachel Pateman (SEI) presented the an MSc project carried out by Sian Lomax  under the supervision of Sarah West (SEI) on ‘A critical assessment of a citizen science project‘. It’s an assessment of the science and impact of participants from the OPAL Soil and Earthworm Survey. Aims of citizen science are to answer scientific questions, but also to provide benefit to participants – learning, fun, change behaviours, or information for lobbying on behalf of nature. The challenges are how to find inclusive methods and have good quality data. The participants aim are not simple – there is not simple link between participation and pro-environmental behaviour. The way to deal with that is to evaluate and reflect critically during the development of a citizen science project, and inform the design process (this remind me a lot of Amy Fowler’s thesis, also about OPAL). The OPAL programme is aimed to be educational, change of lifestyle and inspire new generation of environmentalists and greater understanding of the environment. Sian evaluate the soil and earthworm survey which are usually run with an instructor (community scientist) but also can be done by ordering a self obtained pack. The methods – dig a pit, identify worms, and identify properties of the soil and then submit the inforamtion. The aim is that participants wil learn about soil properties and get interested in environmental issues. Sian recruited 87 participants from ages 5 to 60 and also evaluated the observations of participants in the lab, as well as running a questionnaire with participants. She found fairly poor results  (around 40% accurate) in comparison to her own analysis. The results are that 39% identified correctly, 44% functional group, 46% identified as immature – the reliability of the data that adult observers done is better. Results – ID to species level is challenging, especially without help (she didn’t trained the participants) and therefore there is a need of an OPAL community scientist to be an instructor. There was not enough testing of the material at the beginning of the survey and it haven’t been improved since 2009. There is a need to verify records – but should be emphasised further and included in apps. However, despite these limitation, the OPAL survey did yield useful information and they managed to use the data to find abundance of information. Only in 29% of the cases she agreed with participants about the classification of soil granularity. When evaluating the pH of the soil – 63% was within the correct category of acid/alkaline but not correct on the value – the issue might be with the instrument that was provided to participants and yields wrong reading.

From @Simon_Wilcock

In terms of knowledge and experience – the questionnaire was done before, immediately after the survey and then 3 months later. Knowledge increased immediately after but drop-off after – so conclusion is that need to reinforce it after the event. In terms of interest in nature they didn’t find difference – but that because there was high level of interest to start with.

Jodey Peyton (CEH/BRC)  ‘Open Farm Sunday Pollinator Survey: Citizen science as a tool for pollinator monitoring?‘. The decline in pollinators in the UK is a cause of concern. Their estimated value is £510 m a year. The Big Bumbelebee discovery is an example for a project that focus on pollinators. However, we’re lacking abundance data about them. The Open Farm Sunday is a project to open farms to the public (run by LEAF) and about 4 years ago they contacted CEH to do some work with visitors collect information on pollinators

They ask participants to observe a 2×2 m of crop and non-crop area. They have an ecologists on site so they do the same as the participants – carry 2 min observations in both habitats. The event included teaching people the process and giving them information. The forms use to be 4 pages but turned out to be too complex so simplified a form with just 2 pages. They also reduce time from 5 min to 2 min. They run  surveys in 2012 to 2014 with different number of farms – and looked at different factors during the day. They found that public was over-recording (compare to ecologists), not by much – they also got data from other parts of the plant so not only on the flowers because they wanted to report something. Conclusions – on the broad level public data was similar to ecologists. Lots of interest and enthusiasm and understand what they’re seeing. It is great opportunity to highlight the issue of pollinator. Want to run it every second year because of the effort of the ecologists on the day. They also want to deal with challenge of ‘recording zero. Want to see more collaboration with universities and schools.

Charlotte Hall (EarhtWatch Institute) provided an overview of FreshWater Watch: lessons from a global mass Citizen Science programme. The programme focused on fresh water quality. A global programme that look at water quality in urban areas – each location they partner with local research institute, and Earthwatch bring the citizen scientists with the local researchers. The data that is collected is managed by EarthWatch on a specially designed website to allow sharing knowledge and communictation. The evolving motivation of participants, they looked at Rotman et al 2012 model. Initial involvment stemming from interest or existing knowledge, although in the case of EarthWatch they are getting employees of Shell or HSBC who sponsor them, they also work with teachers in Teach Earth and also expanding to work with local groups such as Thames 21 or Wandle Trust. They have over 20 research partners. With such a mix of researchers, participants and organisations, there are different motivations from different directions. They start with training in person and online Research and learning- EarthWatch is interested in behaviour change, so they see learning as a very important issue and include quizzes to check the knowledge of participants. They pay special attention to communication between EarthWatch and the scientists and between EarthWatch and the citizen scientists. There is a community feature on the website for citizen scientists and also for the scientists. There is also an app with automated feedback that tell them about the outcomes of the research they are doing. They have an element of gamification -points on communication, science and skills that participants gained and they can get to different levels. They try to encourage people to move to the next step so to continue their involvement through learning in webinars, refresher session, research updates, points and prizes and even facility for the participants to analyse the data themselves. Involvement in FreshWater watch is exhibiting participation inequality. 2014-12-12 14.43.10They would like to make it shallower but it is very strongly skewed. In Latin America there is better participation, and also differences in participation according to the researcher who lead the activity. This is new citizen science approach for EarthWatch, with different audience, so it’s important to re-evaluate and understand participants. EarthWatch is still learning from that and understanding motivation.

Emma Rothero (Open University) Flight of the Fritillary: a long-running citizen science project linking Snakeshead fritillaries flowers and bumblebees. The work started in 1999, this is a rare plant that is growing only in few places in the UK. The Bees are critical to the flower, and they set a 15% secondary count to evaluate the success of volunteers. They also started winter workshops for discussions. To engage volunteers, they’ve done wide advertising and also used naturalist networks. She described a comparison between three sites where monitoring was carried out this year . In Lugg Meadow the monitoring is done during guided walks and family outreach events. In North Meadow, many people come to see – so they have a gate presence and offered free lunch for volunteers. In Clattinger Farm they haven’t done any specific activity. In 2008 – 20011 only 20 volunteers, now they’ve got 90 volunteers, and about 30-40 who come to winter workshops. Level of volunteering – once 120 , 40 participated twice and 20 three times – there is some enthusiastic people who do it regularly. The volunteers survey show that 88% heard about the monitoring project by word of mouth (despite the advertising and media access), and 87.5% are already recorders – but 88% thought that they had improved their skills. and 65% said that they improve their skills. 54% would like to get involved in other aspects of the project, and 100% enjoyed the activity. In terms of comparison with recounts – they do 4000 1sq m quads using very accurate (1 cm) GPS. They see that there wasn’t difference between recounts in some sites but significantly difference in another site (because of difficulties in frame orientation so implementation of the protocol) – recognising problem in their method. There is also scientific discovery, where they found a case that plants didn’t appear one year but bounced back the next year.

There was no time for much discussion, but a question that was raised and discussed shortly is that most of the projects are ‘top-down’ and led by the scientists, so what is the scope for co-created projects in the area of ecological observations and monitoring?

 

Notes from the second day of the BES/sfé annual meeting (see first day notes here)

Several talks in sessions that attracted my attention:

Daniel Richards (National University of Singapore) looked at cultural ecosystem services from social media sources. He mentioned previous study by  Casalegno at al 2013 study on social media and ecosystem services . In Singapore they carry out a study for the few green spaces that are used for leisure and nature reserves – the rest of the place is famously highly urbanised. There are patches of coastal habitat that are important locally. The analysis looked at Flickr photos to reveal interest. There are 4 study sites, with 760 photos that were returned and of them 683 related to coastal habitat. They use classification of content, with 8 people analysing the photos. Analysis of Flickr showed different aspects – landscape in one site, and wildlife in another site. In one site there are research photos due to the way it is used locally. Looking closely to one coastal site, focal points in the route where people stopped  to take a picture stood out, and landscape photos. All the photos follow the boardwalk in the area of Changi which is the only route. Simulation showed that after 70 photos they can get a good indication of the nature of the place, no need to look through all the images.

Barbara Smith explored the role of indigenous and local knowledge as part of a multiple evidence base for pollinator conservation. The context is India in agricultural area – looking at places where there is more extensive agriculture and less. The project aim is to record pollinators and then explore the impact of landscape and crop productivity . In this study, the starting point was the belief that traditional knowledge has a lot of value, and it is a knowledge that can be integrated with scientific information.  She mentioned Tengo et al 2013 discussion paper in IPBES on the value of local knowledge, and also Sutherland et al 2014 paper in Oryx about the need to integrate indigenous knowledge in ecological assessment. The aim to collate knowledge of trends, they created a local peer-review process to validate local knowledge. Understanding  factual data collection and separate it from inferences which are sometime wrong. They carry out small group discussions, in which they involved 5-7 farmers, in each of the 3 study area they had 3 groups. They asked questions that are evidence gathering (which crop you grow?) and also verification (how do you know?) they also ask opinion scoping (perceptions ) and then ‘why did you observed the change?’. In the discussions with the farmers they structured in around questions that can be explored together. After the first session, the created declarations – so ‘yields have fallen by 25%’ or crop yield declined because of the poor soil’ the statements were accepted or rejected through discussion with the farmers – local peer-review. Not all farmers can identify pollinators, and as the size goes down, there is less identification and also confusion about pests and pollinators. The farmers identified critical pollinators in their area and also suggestions on why the decline happen.

In the workshop on ‘Ecosystem assessments – concepts, tools and governance‘ there was various discussion on tools that are used for such purposes, but it became clear to me that GIS is playing a major role, and that many of the fundamental discussions in GIScience around the different types of modelling – from overlaying to process oriented modelling – can play a critical role in making sense of the way maps and GIS outputs travel through the decision making. It can be an interesting area to critically analysed – To what degree the theoretical and philosophical aspects of the modelling are taken into account in policy processes? The discussion in the workshop moved to issues of scientific uncertainty and communication with policy makers. The role of researchers in the process and the way they discuss uncertainty.

In the computational ecology session, Yoseph Araya presented a talk that was about the use of citizen science data, but instead he shared his experience and provide an interesting introduction to a researcher perspective on citizen science. He looked at the data that is coming from citizen science and the problem of getting good data. Citizen Science gaining attention – e.g. Ash die-back and other environmental issues are leading to attention. Citizens are bridging science, governance and participation. Citizen Science is needed for data at temporal, spatial and social scales and we should not forget that it is also about social capital, and of course fun and enjoyment. There is an increase in citizen science awareness in the literature. He is building on experience from many projects that he participated in include Evolution Megalab, world water monitoring day, floodplain meadows partnership, iSpot and OPAL, and CREW – Custodians of Rare and Endangered Windflowers (that’s a seriously impressive set of projects!). There are plenty of challenges – recruitment, motivation; costs and who pays; consideration of who run it; data validation and analysis and others. Data issues include data accuracy, completeness, reliability, precision and currency. He identified sources of errors – personnel, technical and statistical. The personal – skills, fitness and mistakes and others. Potential solutions – training with fully employed personnel,  then also monitor individual and also run an online quiz. Technically, there is the option of designing protocols and statistically, it is possible to use recounts (15%), protocols that allow ‘no data’ and other methods.

The poster session included a poster from Valentine Seymour, about her work linking wellbeing and green volunteering

As part of the citizens observatories conference, I represented Mapping for Change, providing an overview of community-led air quality studies that we have run over the past 4 years. Interestingly, as we started the work in collaboration with London Sustainability Exchange, and with help from the Open Air Laboratories programme the work can be contextualised within the wider context of NGOs work on citizen science, which was a topic that was covered in the conference.

The talk covered the different techniques that were used: eco-badges for Ozone testing, Wipe sampling, Diffusion tubes and particulate matter monitoring devices. In the first study, we also were assisted by Barbara Maher team who explore tree leaves for biomonitoring. The diffusion tubes are of particular importance, as the change in deployment and visualisation created a new way for communities to understand air quality issues in their area.

The use of a dense network of diffusion tubes became common in other communities over the past 4 years. I also cover the engagement of local authorities, with a year-long study in the Barbican with support from the City of London. There is a lesson about the diffusion of methodologies and approaches among community groups – with the example of the No to Silvertown Tunnel group carrying out a diffusion tubes study without linkage to Mapping for Change or London Sustainability Exchange. Overall, this diffusion mean that over 20 localised studies are emerging across London.

If you have been reading the literature on citizen science, you must have noticed that many papers that describe citizen science start with an historical narrative, something along the lines of:

As Silvertown (2009) noted, until the late 19th century, science was mainly developed by people who had additional sources of employment that allowed them to spend time on data collection and analysis. Famously, Charles Darwin joined the Beagle voyage, not as a professional naturalist but as a companion to Captain FitzRoy[*]. Thus, in that era, almost all science was citizen science albeit mostly by affluent gentlemen and gentlewomen scientists[**]. While the first professional scientist is likely to be Robert Hooke, who was paid to work on scientific studies in the 17th century, the major growth in the professionalisation of scientists was mostly in the latter part of the 19th and throughout the 20th century.
Even with the rise of the professional scientist, the role of volunteers has not disappeared, especially in areas such as archaeology, where it is common for enthusiasts to join excavations, or in natural science and ecology, where they collect and send samples and observations to national repositories. These activities include the Christmas Bird Watch that has been ongoing since 1900 and the British Trust for Ornithology Survey, which has collected over 31 million records since its establishment in 1932 (Silvertown 2009). Astronomy is another area in which amateurs and volunteers have been on a par with professionals when observation of the night sky and the identification of galaxies, comets and asteroids are considered (BBC 2006). Finally, meteorological observations have also relied on volunteers since the early start of systematic measurements of temperature, precipitation or extreme weather events (WMO 2001). (Haklay 2013 emphasis added)

The general messages of this historical narrative are: first, citizen science is a legitimate part of scientific practice as it was always there, we just ignored it for 50+ years; second, that some citizen science is exactly as it was – continuous participation in ecological monitoring or astronomical observations, only that now we use smartphones or the Met Office WOW website and not pen, paper and postcards.

The second aspect of this argument is one that I was wondering about as I was writing a version of the historical narrative for a new report. This was done within a discussion on how the educational and technological transitions over the past century reshaped citizen science. I have argued that the demographic and educational transition in many parts of the world, and especially the rapid growth in the percentage and absolute numbers of people with higher education degrees who are potential participants is highly significant in explaining the popularity of citizen science. To demonstrate that this is a large scale and consistent change, I used the evidence of Flynn effect, which is the rapid increase in IQ test scores across the world during the 20th century.

However, while looking at the issue recently, I came across Jim Flynn TED talk ‘Why our IQ levels are higher than our grandparents (below). At 3:55, he raise a very interesting point, which also appears in his 2007 What is Intelligence? on pages 24-26. Inherently, Flynn argues that the use of cognitive skills have changed dramatically over the last century, from thinking that put connections to concrete relationship with everyday life as the main way of understanding the world, to one that emphasise scientific categories and abstractions. He use an example of a study from the early 20th Century, in which participants where asked about commonalities between fish and birds. He highlights that it was not the case that in the ‘pre-scientific’ worldview people didn’t know that both are animals, but more the case that this categorisation was not helpful to deal with concrete problems and therefore not common sense. Today, with scientific world view, categorisation such as ‘these are animals’ come first.

This point of view have implications to the way we interpret and understand the historical narrative. If correct, than the people who participate in William Whewell tide measurement work (see Caren Cooper blogpost about it), cannot be expected to think about contribution to science, but could systematically observed concrete events in their area. While Whewell view of participants as ‘subordinate labourers’ is still elitist and class based, it is somewhat understandable.  Moreover, when talking about projects that can show continuity over the 20th Century – such as Christmas Bird Count or phenology projects – we have to consider the option that an the worldview of the person that done that in 1910 was ‘how many birds there are in my area?’ while in 2010 the framing is ‘in order to understand the impact of climate change, we need to watch out for bird migration patterns’. Maybe we can explore in historical material to check for this change in framing? I hope that projects such as Constructing Scientific Communities which looks at citizen science in the 19th and 21th century will shed light on such differences.


[*] Later I found that this is not such a simple fact – see van Wyhe 2013 “My appointment received the sanction of the Admiralty”: Why Charles Darwin really was the naturalist on HMS Beagle

[**] And we shouldn’t forget that this was to the exclusion of people such as Mary Anning

 

The Association of American Geographers is coordinating an effort to create an International Encyclopedia of Geography. Plans started in 2010, with an aim to see the 15 volumes project published in 2015 or 2016. Interestingly, this shows that publishers and scholars are still seeing the value in creating subject-specific encyclopedias. On the other hand, the weird decision by Wikipedians that Geographic Information Science doesn’t exist outside GIS, show that geographers need a place to define their practice by themselves. You can find more information about the AAG International Encyclopedia project in an interview with Doug Richardson from 2012.

As part of this effort, I was asked to write an entry on ‘Volunteered Geographic Information, Quality Assurance‘ as a short piece of about 3000 words. To do this, I have looked around for mechanisms that are used in VGI and in Citizen Science. This are covered in OpenStreetMap studies and similar work in GIScience, and in the area of citizen science, there are reviews such as the one by Andrea Wiggins and colleagues of mechanisms to ensure data quality in citizen science projects, which clearly demonstrated that projects are using multiple methods to ensure data quality.

Below you’ll find an abridged version of the entry (but still long). The citation for this entry will be:

Haklay, M., Forthcoming. Volunteered geographic information, quality assurance. in D. Richardson, N. Castree, M. Goodchild, W. Liu, A. Kobayashi, & R. Marston (Eds.) The International Encyclopedia of Geography: People, the Earth, Environment, and Technology. Hoboken, NJ: Wiley/AAG

In the entry, I have identified 6 types of mechanisms that are used to ensure quality assurance when the data has a geographical component, either VGI or citizen science. If I have missed a type of quality assurance mechanism, please let me know!

Here is the entry:

Volunteered geographic information, quality assurance

Volunteered Geographic Information (VGI) originate outside the realm of professional data collection by scientists, surveyors and geographers. Quality assurance of such information is important for people who want to use it, as they need to identify if it is fit-for-purpose. Goodchild and Li (2012) identified three approaches for VGI quality assurance , ‘crowdsourcing‘ and that rely on the number of people that edited the information, ‘social’ approach that is based on gatekeepers and moderators, and ‘geographic’ approach which uses broader geographic knowledge to verify that the information fit into existing understanding of the natural world. In addition to the approaches that Goodchild and li identified, there are also ‘domain’ approach that relate to the understanding of the knowledge domain of the information, ‘instrumental observation’ that rely on technology, and ‘process oriented’ approach that brings VGI closer to industrialised procedures. First we need to understand the nature of VGI and the source of concern with quality assurance.

While the term volunteered geographic information (VGI) is relatively new (Goodchild 2007), the activities that this term described are not. Another relatively recent term, citizen science (Bonney 1996), which describes the participation of volunteers in collecting, analysing and sharing scientific information, provide the historical context. While the term is relatively new, the collection of accurate information by non-professional participants turn out to be an integral part of scientific activity since the 17th century and likely before (Bonney et al 2013). Therefore, when approaching the question of quality assurance of VGI, it is critical to see it within the wider context of scientific data collection and not to fall to the trap of novelty, and to consider that it is without precedent.

Yet, this integration need to take into account the insights that emerged within geographic information science (GIScience) research over the past decades. Within GIScience, it is the body of research on spatial data quality that provide the framing for VGI quality assurance. Van Oort’s (2006) comprehensive synthesis of various quality standards identifies the following elements of spatial data quality discussions:

  • Lineage – description of the history of the dataset,
  • Positional accuracy – how well the coordinate value of an object in the database relates to the reality on the ground.
  • Attribute accuracy – as objects in a geographical database are represented not only by their geometrical shape but also by additional attributes.
  • Logical consistency – the internal consistency of the dataset,
  • Completeness – how many objects are expected to be found in the database but are missing as well as an assessment of excess data that should not be included.
  • Usage, purpose and constraints – this is a fitness-for-purpose declaration that should help potential users in deciding how the data should be used.
  • Temporal quality – this is a measure of the validity of changes in the database in relation to real-world changes and also the rate of updates.

While some of these quality elements might seem independent of a specific application, in reality they can be only be evaluated within a specific context of use. For example, when carrying out analysis of street-lighting in a specific part of town, the question of completeness become specific about the recording of all street-light objects within the bounds of the area of interest and if the data set includes does not include these features or if it is complete for another part of the settlement is irrelevant for the task at hand. The scrutiny of information quality within a specific application to ensure that it is good enough for the needs is termed ‘fitness for purpose’. As we shall see, fit-for-purpose is a central issue with respect to VGI.

To understand the reason that geographers are concerned with quality assurance of VGI, we need to recall the historical development of geographic information, and especially the historical context of geographic information systems (GIS) and GIScience development since the 1960s. For most of the 20th century, geographic information production became professionalised and institutionalised. The creation, organisation and distribution of geographic information was done by official bodies such as national mapping agencies or national geological bodies who were funded by the state. As a results, the production of geographic information became and industrial scientific process in which the aim is to produce a standardised product – commonly a map. Due to financial, skills and process limitations, products were engineered carefully so they can be used for multiple purposes. Thus, a topographic map can be used for navigation but also for urban planning and for many other purposes. Because the products were standardised, detailed specifications could be drawn, against which the quality elements can be tested and quality assurance procedures could be developed. This was the backdrop to the development of GIS, and to the conceptualisation of spatial data quality.

The practices of centralised, scientific and industrialised geographic information production lend themselves to quality assurance procedures that are deployed through organisational or professional structures, and explains the perceived challenges with VGI. Centralised practices also supported employing people with focus on quality assurance, such as going to the field with a map and testing that it complies with the specification that were used to create it. In contrast, most of the collection of VGI is done outside organisational frameworks. The people who contribute the data are not employees and seemingly cannot be put into training programmes, asked to follow quality assurance procedures, or expected to use standardised equipment that can be calibrated. The lack of coordination and top-down forms of production raise questions about ensuring the quality of the information that emerges from VGI.

To consider quality assurance within VGI require to understand some underlying principles that are common to VGI practices and differentiate it from organised and industrialised geographic information creation. For example, some VGI is collected under conditions of scarcity or abundance in terms of data sources, number of observations or the amount of data that is being used. As noted, the conceptualisation of geographic data collection before the emergence of VGI was one of scarcity where data is expensive and complex to collect. In contrast, many applications of VGI the situation is one of abundance. For example, in applications that are based on micro-volunteering, where the participant invest very little time in a fairly simple task, it is possible to give the same mapping task to several participants and statistically compare their independent outcomes as a way to ensure the quality of the data. Another form of considering abundance as a framework is in the development of software for data collection. While in previous eras, there will be inherently one application that was used for data capture and editing, in VGI there is a need to consider of multiple applications as different designs and workflows can appeal and be suitable for different groups of participants.

Another underlying principle of VGI is that since the people who collect the information are not remunerated or in contractual relationships with the organisation that coordinates data collection, a more complex relationships between the two sides are required, with consideration of incentives, motivations to contribute and the tools that will be used for data collection. Overall, VGI systems need to be understood as socio-technical systems in which the social aspect is as important as the technical part.

In addition, VGI is inherently heterogeneous. In large scale data collection activities such as the census of population, there is a clear attempt to capture all the information about the population over relatively short time and in every part of the country. In contrast, because of its distributed nature, VGI will vary across space and time, with some areas and times receiving more attention than others. An interesting example has been shown in temporal scales, where some citizen science activities exhibit ‘weekend bias’ as these are the days when volunteers are free to collect more information.

Because of the difference in the organisational settings of VGI, a different approaches to quality assurance is required, although as noted, in general such approaches have been used in many citizen science projects. Over the years, several approaches emerged and these include ‘crowdsourcing ‘, ‘social’, ‘geographic’, ‘domain’, ‘instrumental observation’ and ‘process oriented’. We now turn to describe each of these approaches.

Thecrowdsourcing approach is building on the principle of abundance. Since there are is a large number of contributors, quality assurance can emerge from repeated verification by multiple participants. Even in projects where the participants actively collect data in uncoordinated way, such as the OpenStreetMap project, it has been shown that with enough participants actively collecting data in a given area, the quality of the data can be as good as authoritative sources. The limitation of this approach is when local knowledge or verification on the ground (‘ground truth’) is required. In such situations, the ‘crowdsourcing’ approach will work well in central, highly populated or popular sites where there are many visitors and therefore the probability that several of them will be involved in data collection rise. Even so, it is possible to encourage participants to record less popular places through a range of suitable incentives.

Thesocial approach is also building on the principle of abundance in terms of the number of participants, but with a more detailed understanding of their knowledge, skills and experience. In this approach, some participants are asked to monitor and verify the information that was collected by less experienced participants. The social method is well established in citizen science programmes such as bird watching, where some participants who are more experienced in identifying bird species help to verify observations by other participants. To deploy the social approach, there is a need for a structured organisations in which some members are recognised as more experienced, and are given the appropriate tools to check and approve information.

Thegeographic approach uses known geographical knowledge to evaluate the validity of the information that is received by volunteers. For example, by using existing knowledge about the distribution of streams from a river, it is possible to assess if mapping that was contributed by volunteers of a new river is comprehensive or not. A variation of this approach is the use of recorded information, even if it is out-of-date, to verify the information by comparing how much of the information that is already known also appear in a VGI source. Geographic knowledge can be potentially encoded in software algorithms.

Thedomain approach is an extension of the geographic one, and in addition to geographical knowledge uses a specific knowledge that is relevant to the domain in which information is collected. For example, in many citizen science projects that involved collecting biological observations, there will be some body of information about species distribution both spatially and temporally. Therefore, a new observation can be tested against this knowledge, again algorithmically, and help in ensuring that new observations are accurate.

Theinstrumental observation approach remove some of the subjective aspects of data collection by a human that might made an error, and rely instead on the availability of equipment that the person is using. Because of the increased in availability of accurate-enough equipment, such as the various sensors that are integrated in smartphones, many people keep in their pockets mobile computers with ability to collect location, direction, imagery and sound. For example, images files that are captured in smartphones include in the file the GPS coordinates and time-stamp, which for a vast majority of people are beyond their ability to manipulate. Thus, the automatic instrumental recording of information provide evidence for the quality and accuracy of the information.

Finally, the ‘process oriented approach bring VGI closer to traditional industrial processes. Under this approach, the participants go through some training before collecting information, and the process of data collection or analysis is highly structured to ensure that the resulting information is of suitable quality. This can include provision of standardised equipment, online training or instruction sheets and a structured data recording process. For example, volunteers who participate in the US Community Collaborative Rain, Hail & Snow network (CoCoRaHS) receive standardised rain gauge, instructions on how to install it and an online resources to learn about data collection and reporting.

Importantly, these approach are not used in isolation and in any given project it is likely to see a combination of them in operation. Thus, an element of training and guidance to users can appear in a downloadable application that is distributed widely, and therefore the method that will be used in such a project will be a combination of the process oriented with the crowdsourcing approach. Another example is the OpenStreetMap project, which in the general do not follow limited guidance to volunteers in terms of information that they collect or the location in which they collect it. Yet, a subset of the information that is collected in OpenStreetMap database about wheelchair access is done through the highly structured process of the WheelMap application in which the participant is require to select one of four possible settings that indicate accessibility. Another subset of the information that is recorded for humanitarian efforts is following the social model in which the tasks are divided between volunteers using the Humanitarian OpenStreetMap Team (H.O.T) task manager, and the data that is collected is verified by more experienced participants.

The final, and critical point for quality assurance of VGI that was noted above is fitness-for-purpose. In some VGI activities the information has a direct and clear application, in which case it is possible to define specifications for the quality assurance element that were listed above. However, one of the core aspects that was noted above is the heterogeneity of the information that is collected by volunteers. Therefore, before using VGI for a specific application there is a need to check for its fitness for this specific use. While this is true for all geographic information, and even so called ‘authoritative’ data sources can suffer from hidden biases (e.g. luck of update of information in rural areas), the situation with VGI is that variability can change dramatically over short distances – so while the centre of a city will be mapped by many people, a deprived suburb near the centre will not be mapped and updated. There are also limitations that are caused by the instruments in use – for example, the GPS positional accuracy of the smartphones in use. Such aspects should also be taken into account, ensuring that the quality assurance is also fit-for-purpose.

References and Further Readings

Bonney, Rick. 1996. Citizen Science – a lab tradition, Living Bird, Autumn 1996.
Bonney, Rick, Shirk, Jennifer, Phillips, Tina B. 2013. Citizen Science, Encyclopaedia of science education. Berlin: Springer-Verlag.
Goodchild, Michael F. 2007. Citizens as sensors: the world of volunteered geography. GeoJournal, 69(4), 211–221.
Goodchild, Michael F., and Li, Linna. 2012, Assuring the quality of volunteered geographic information. Spatial Statistics, 1 110-120
Haklay, Mordechai. 2010. How Good is volunteered geographical information? a comparative study of OpenStreetMap and ordnance survey datasets. Environment and Planning B: Planning and Design, 37(4), 682–703.
Sui, Daniel, Elwood, Sarah and Goodchild, Michael F. (eds), 2013. Crowdsourcing Geographic Knowledge, Berlin:Springer-Verlag.
Van Oort, Pepjin .A.J. 2006. Spatial data quality: from description to application, PhD Thesis, Wageningen: Wageningen Universiteit, p. 125.

At the end of June, I noticed a tweet about new words in the Oxford English Dictionary (OED):

I like dictionary definitions, as they help to clarify things, and the OED is famous for careful editing and finding how a term is used before adding it. Being in the OED is significant for Citizen Science, as it is a recognised “proper” term. At the same time, the way that OED define citizen science, and its careful work in finding out when it was first used can help to emphasise some interesting aspects. This is how.

Here is the definition, in all its glory:

citizen science n. scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional scientists and scientific institutions.

1989   Technol. Rev. Jan. 12/4   Audubon involves 225 society members from all 50 states in a ‘citizen science’ program… Volunteers collect rain samples, test their acidity levels, and report the results to Audubon headquarters.
2002   M. B. Mulder & P. Coppolillo Conservation xi. 295/1   Citizen science has the potential to strengthen conservation practice in the developing world.
2012   M. Nielsen Reinventing Discov. vii. 151   Citizen science can be a powerful way both to collect and also to analyze enormous data sets.

citizen scientist n.  (a) a scientist whose work is characterized by a sense of responsibility to serve the best interests of the wider community (now rare);  (b) a member of the general public who engages in scientific work, often in collaboration with or under the direction of professional scientists and scientific institutions; an amateur scientist.

1912   Manch. Guardian 11 Sept. 4/2   Trafford, thus serenely established, should…have returned to his researches with a new confidence and content and become a noble citizen-scientist.
1936   Headmaster Speaks 65   Could not Science…turn out a race of citizen scientists who do not make an absolute religion of the acquisition of new scientific knowledge however useless or harmful it may be?
1949   Collier’s 16 July 74/3   By 1930 most citizen-scientists had perfected a technique which brought gin to its peak of flavor and high-octane potency five minutes after the ingredients had been well mixed.
1979   New Scientist 11 Oct. 105/2   The ‘citizen-scientist’, the amateur investigator who in the past contributed substantially to the development of science through part-time dabbling.
2013   G. R. Hubbell Sci. Astrophotogr. xiii. 233   A citizen scientist in the astronomical field has a unique opportunity because astronomy is a wholly observational science.

Dictionaries are more interesting than they might seem. Here are 3 observations on this new definition:

First, the core definition of ‘citizen science’ is interestingly inclusive, so community based air quality monitoring to volunteer bird surveys and running climate models on your computer at home are all included. This makes the definition useful across projects and types of activities.

Second, the term ‘citizen scientist’ captures two meanings. The first is noteworthy, as it is the one that falls well within Alan Irwin’s way of describing citizen science, or in Jack Stilgoe’s pamphlet that describes citizen scientists. Notice that this meaning is not now the most common to describe a citizen scientist, but scientists that are active in citizen science usually become such citizen scientists (sorry for the headache!).

Third, it’s always fun to track down the citations that OED uses, as it tries to find the first use of the phrase. So let’s look at the late 20th century citations for ‘citizen science’ and ‘citizen scientist’ (the ones from the early 20th century are less representative of current science in my view).

The first use of ‘citizen science’ in the meaning that we now use can be traced to an article in MIT Technology Review from January 1989. The article ‘Lab for the Environment’ tells the story of community-based laboratories to explore environmental hazards, laboratory work by Greenpeace, and Audubon’s recruitment of volunteers in a ‘citizen science’ programme. The part that describes citizen science is provided below (click here to get to the magazine itself). Therefore, groups such as the Public Laboratory for Open Technology and Science are linked directly to this use of citizen science. 

MIT Technology Review 1989

Just as interesting is the use of ‘citizen scientist’. It was used 10 years earlier, in an article in New Scientist that discussed enthusiasts researching Unidentified Flying Objects (UFO) and identified ‘ufology’ as a field of study for these people. While the article is clearly mocking the ufologists as unscientific, it does mention, more or less in passing, the place of citizen-scientists, which is “all but eliminated” by the late 1970s (click here to see the original magazine). This resonates with many of the narratives about how citizen science disappeared in the 20th century and is reappearing now. 

NewScientist1979-Details

 

If you would like to use these original references to citizen science and citizen scientists, here are the full references (I’ll surely look out for an opportunity to do so!):

Kerson, R., 1989, Lab for the Environment, MIT Technology Review, 92(1), 11-12

Oberg, J., 1979, The Failure of the ‘Science’ of Ufology, New Scientist, 84(1176), 102-105

 


 

Thanks to Rick Bonney who asked some questions about the definition that led to this post!

Follow

Get every new post delivered to your Inbox.

Join 2,697 other followers