Geothink & Learn citizen science session

The following recording is from the Geothink & Learn lunchtime webinar.

The call for the event stated:

“Should it be only people with graduate degree who make extraordinary scientific discoveries? Maybe not. Citizen scientists around the world have contributed to new discoveries in fields such as astronomy, biology, meteorology, geography, public health, and more. It can also address social and environmental inequalities, and allow individuals and communities to address issues that concern them through the application of scientific methods and tools. Efforts to harness the work of many hands or crowdsource important data collection or transcription have gained popularity because of their ability to help scientists in tasks that they wouldn’t be able to accomplish, increase public engagement with science, and potentially raise awareness and understanding of scientific issues. They also open up new lines of data in important areas of research, to the benefits of scientists and society. Citizen science requires the participation of ordinary citizens outside of scientific research in universities, governmental bodies, or other research institutions. Participation in citizen science provides individuals with new skills in technology, science, and community organization, as well as informal education on scientific issues. Crowdsourcing can take place as part of citizen science as it relates to large-scale participation that can include tens of thousands of people joining projects online.”

The webinar included me, Victoria Slonosky, principal organizer for ACRE–Canada and the Data Rescue: Archives and Weather Project (DRAW); and,  Caren Cooper, a research associate professor at North Carolina State University.

Advertisements

Lessons learned from Volunteers Interactions with Geographic Citizen Science – Afternoon session

The context of the workshop and the notes from the first part of the workshop is available here. The theme of the second part of the day was Interacting with geographical citizen science: lessons learned from urban environments

Volunteer interactions with flood crowdsourcing platforms – Avi Baruch talk is based on a completed PhD on the aspects of volunteers in flood monitoring and response. There are different types – incident reporting floodline, media outreach, online volunteering, and collaborative mapping. He looked at Tomnod as a system that is currently used to engage volunteers in tagging satellite images. Looked at forums and interviews with the most active participants. Most volunteers where over 50 and there is a good balance by gender. 23% stated that they had a long-term health problem – finding it addictive and spending 8-10 hours a day. Engaging volunteers is an issue: there was not enough feedback on how the information was used and how they are performing, which Tomnod team haven’t done. at least 10% of comments were concerned with the quality of their contributions. Without feedback, it is hard to judge. Tomnod allow people to explore the map and they can share location, but then people concentrate in one area. Restricting people to an area didn’t work well. Core motivations were based on altruistic reasons, and retirement, disability and health were reasons for engagement. The second part of the PhD project includes the development of a citizen science platform to report (floodcrowd.co.uk) and doing the development through an iterative process. The form allows people to report flooding incidents. All the information that is provided is location, and type of flooding, and then people can report further details. In communities, that experience flooding preferred a hard copy. All sort of information was submitted, mostly about surface mapping – many people who are potential participants didn’t want to engage with the app. Need further co-production with the people who contribute the data.

Volunteers Interaction in Technology Driven Citizen Science contexts: Lessons learned from senseBox and openSenseMap – Mario Pesch & Thomas Bartoschek SenseBox have been developed over the past 4 years. It came out from teaching computer science in school – focus on environmental sensing which the students wanted to see on the web. Sensing temp, humidity, pressure, light, UV-light. People wanted to participate in the project from outside school. in 2014, they had 50 sesneBoxes – most connected only for few weeks (8m records). After a while, people find it complicated and they wanted to do something on their own. They created a DIY sensing box for home and for school. The component allows people to create things without soldering. They set out reference stations next to official monitoring station – people asked about it. You always need to consider the limitation of the system. SensorBox home 2.0 was looking more at air quality and more options to send the data – measuring in places without WiFi so they added GSM and now they have 1500 sensing stations and people also want to work with the data and you can do basic interpolation. The platform is device independent and people use it for other systems. Also supporting mobile stations, They keep the project open – it can be adjusted to people own needs.

Lessons learned from volunteers’ use and feedback of the Cyclist GEO-C App – Diego Pajarito, Suzanne Maas, Maria Attard and Michael Gould the experience of the cycling app is part of the PhD network GEO-C – open city toolkit. A lot of application target sports or data collected but not linked to the experience on the road. The location Cyclist GEO-C app is for Android and can be competition or cooperation based, and collect GPS tracks and up to 3 tags. Tried in Castello, Munster and Valletta and Malta. There are different levels of cycling used. 20 participants – that commute regularly and using an Android phone. Different participation methods – as a group to get common views. They captured 793 trips, the response was generally positive. People seeing a potential for personal use but also to lobby and promote cycling. Can be a motivational tool for beginners. They also identify the issue of remembering to use the app when the need to use it, improve control over recording and improving the tags. Ideas about mapping interface and using wearable devices, social interaction and gamification were suggested.

Invisible Citizen Science: the case of Járókelő in Hungary – Bálint Balázs & Le Marietta thinking of the citizen science in Eastern Europe, which thinking about modes of public participation in scientific discourse and policy-making, there are multiple silences: there are many projects that offer it, and in the level of initiative – the term haven’t exist and used. The interview from an NGO suggested lack of familiarity. In eastern countries in Europe, citizen science is only recently emerging, not many initiatives, and little-published articles and only a few members of ECSA, and how it is connected. Methods are limited. Need to reconceptualise. There is invisible citizen science – the specific knowledge that is produced in the projects that they are looking at it are uncommon to scientists. An example for this is jarokelo – for addressing local issues – looking at the example for “fix my street” (or “letter to the mayor” in the Czech Republic). Civic technology to report street fixing and there are 20 volunteers who can transfer it to the authority – there are 50-100 reports per day and the reporting back from the authority can take 30 days. Most authorities report back, they also received reports on homeless people and had to agree on what to do with this types of report. The issue of participants is about trust in the state and also think of cooperative research ideas – analysing users’ statistics, thinking of involvement pathways and better communication.

Citizens as Shoppers: Lessons learned from the EnvBodySens application – Eiman Kanjo  looking into mobile sensing – the challenge for retail in the centre of cities and there is also all sort of noise and air pollution that people are concerned about. Done work around a popular shopping area in Nottingham city centre – what kind of sensors – environment, physiology, motion, timestamps, location, continuous self-reporting and the zoning (understanding which shops they are in, or the area that they are visiting). Issues of collecting data involve selecting types of sensors (e.g. the characteristics of the sensors). There was issue of demography, shopping behaviour (men/women), challenges with how many volunteers you get and how to prepare volunteers – but for shopping, we need them to be relaxed and enjoy the shopping and how you start the experiment. There is also the aspects of the journey (real-life shopping experience and temporal aspect of it) which also raise ethical concerns. They needed to consider if the phone is on all the time or should it use voice and audio information. Self-reporting and self-assessment is something that needs consideration. They ended with 50 participants, wristband devices and mobile phone and a 45 shopping journey – they looked at the impact of noise and they also consider how they can visualise all this information.

Lessons learned from the recent landlside mitigation efforts: citizen science as a new approach – Sultan Kocaman & Candan Gokceoglu volunteer contribution can provide important information – increase world population and climate change (extreme weather) is a major natural hazard. Wanted to explore how citizen science is relevant to address uncertainties because there is a lack of reliable temporal data. Risk assessment s base on knowledge of past events – then assessing susceptibility, hazard assessment and then you can understand the risk assessment and manage it. Landslide susceptibility requires a lot of information and data. The risk assessment needs all this information as otherwise there will be too much uncertainty. The majority of landslides are in mountainous areas and we can’t have sensors, but information is coming from observers evidence, and volunteers can provide the time and location in a better way. Shallow landslides disappear after a short period. Need volunteers at the right time and the right place – distributed participation. The scale of movement can also be measured with volunteers. Currently working on the project and consider what can be done – what the frequency and quality of spatial and temporal data and in any case rely heavily on local knowledge but need to be improved.

Citizens as volunteer cartoghraphers: A pedestrian map case study – Manousos Kamilakis exploring the field of cartography for pedestrian – based on ideas from VGI so people can share information. Most of the online maps are focusing on motorised transport, and less about the aesthetic pleasantness of the journey, the condition of the pavement etc. The two journeys are suggested as equivalent and only one of them is offering a better journey. Created an app for pedestrian reporting and recording the journey, then evaluate and review the journey and also editing a path. They carried out an experiment with people who never edited a map and had various motivations – the leaderboard wasn’t of interest, although half were motivated by gamification and were willing to cheat to score points. Creating motivation is difficult – need to design gamification carefully and external incentives encourage unacceptable data uploading – consider peer review. People do not volunteer to all tasks in an equal way.

Interacting with Community Maps – Mapping for Change Louise Francis and Rosa Arias cover the development of international odour observatory.  Building on Principle 10 of Rio and the right of access environmental information – different authorities produced maps, such a noise map.When talking with communities, people are pointing that they have a different experience and reflect their own understanding of their local conditions using citizen science. Citizen collect information and Mapping for Change visualise it on behalf of the community. Community evolved over the years. it is a flexible system that allows people to decide on the grouping of information – the themes are being groups in different ways. There is also a need to make conversation – interact with contributions that other people added. The data is to drive change – for example leading to a change in buses through campaign and publicity to change things around them. Lessons learned: communities, where adding data – demonstrating that community members wanted to share a lot of data and they wanted information on their balcony and putting a point on top of a point, wasn’t possible in the past and require changed. The map is allowing clustering that shows 115 points in a small area. Some communities wanted to have their own classification – so they took the data and created their own visualisation. We learned that and want to be part of the D-Noses: odour pollution. The top-down approaches to address issues of odour and there is fairly little addressing of issues. OdourCollect focuses on bottom-up approach – using the nose to notice odour problems. The OdoucrCollect allow data capture.

A Case Study on the Impact of Design Choices on Data Quality in Geographic Citizen Science – Jeffrey Parsons, NL Nature design choices – ecologist and looking at data management and data quality. Looking at a specific design choice. Looking at two archetypes of systems – on one end well defined and stable use of data (close) precise focus on data collection and data collection standards – citizen scientists with requisite domain knowledge and motivated to do the work well. The other end ill-defined, open use, which provides opportunities for data collection in an opportunistic way, ambiguous data collection standard and unclear domain knowledge. eBird is an example of a project that is towards the closed version. The research setting in traditional science lead to design principles for closed citizen science and these don’t work in open and that can lead to a problem in the application. Information quality is a major challenge in User Generated Content (UGC) – there is all sort of comments about it. Fitness for use is a major one – in close: training, data collection protocol, clean data – but this is a problem in an open environment and it can inhibit contributors from communicating unique knowledge. They suggest crowd IQ – from the contributors’ perspective (Lukyanenjo et al 2014). The question is how do we design in such a way that matches the contributors’ mental models of the information and align with contributors’ capabilities. Design principles focus on conceptual modelling – describing in a way that you use a class-based approach of setting the categories and the model drive the design. Design choice of conceptual model of the producer and not necessarily of the contributors. The alternative is to do instance-based modelling which is based on an ontological view of a world made of things and cognitive approach. The information quality impacts – if you think about data completeness as a way to describe the engagement of volunteers to add information. They checked a website that was focused on species only and another one that focuses on the attributes. The hypotheses are that they’ll get more observation and novel species. NLNature.com is about observations of wildlife. They allow people to type species name or the Latin name, the other option is typing whatever you want. They collected data over 6 months, they have 4 times more observations in the instance based condition, and also observe that class-based condition frustrated the contributors and left compared to the class-based case. They got many more species in the instance based when it is open to people to define insects, fish. They even discover a new type of wasp. The bottom line, modelling choices affect dataset completeness – class based lead to fewer observations and especially of species that are not in the schema.

 

From paper prototyping to citizen participation: Co-designing geolocated cultural heritage applications that trigger personal reflection – Kate Jones – looking at cultural heritage. The aim is to create a serendipitous outdoor exhibition to reflect on historical topics and encourage thoughtful play on historical issues. The topic that they focused on was that of migration – 45% of the population is made of migrants in Luxembourg and that influence way to thinking of a location for historical and contemporary memories and experiences. Two places – Luxembourg city and Valletta and they are both touched by migration and are UNESCO sites. They have Mobile app, moderator app, and point of interest management system and they check the information and want to use CrowdFlower to moderate. The application is to allow people to tag history places and be able to record journeys and stories about spaces and memory. Complexity is being hidden behind the levels. The app informs the user that they are being tracked. It was designed in an iterative process – user scenario, requirements, wood game to try how people use the application action – then develop and evaluate. A board game prototyping allowed the development of scenarios. Postcards symbolic of the user interface. The content needs to be valid, and interesting – want to reflect when people are out and about it the city. They included game designer and the developer and they can see the perspective of the player. People used stickers on the board card to indicate what they liked and disliked – people wanted a stronger connection between migration and experience. They used a digital humanities methods and figured out that it can be too complex so the levels can help in unlocking it. Questions had to be changed to address the emotional response of participants, and the multi-city connection was complex and need to develop carefully. A board game for the design was fun and collaborative but also helped in the development of the game. Going t the field, the launched the application in September 2017. out of 500 students, 40 app download, and only two trajectories. They created a new iteration. People don’t like reading the lengthy text – so they put it text to voice and that brought different issues with the interface. They see different types of people in the user population. Exploration have led to a change in perspective in the final application and grounded in participant experience. How do we give people the motivation to give it a go?

Geographical expertise and citizen science: planning and -design implications – Colin Robertson & Robert Feick considering different levels of geographical expertise – what does it mean to be a geographical expert – what are the expert/non-expert into a spectrum. We can look at some ideas of expertise: Collins 2013 pointed to the 3 dimensions of expertise – contributory, interactional and esotericity – exposure to tacit knowledge in a domain, recognised accomplishments or is it expertise that is common or uncommon we can look at it in a continuum – locale familiarity: place-based expertise related – might be fuzzy. Other geographical knowledge is about place-types – say urban environments or glacial environment. We can think about expertise in the cube – for a soil scientist it is in position A, long-term residents of the area might be a huge locale expertise B and so on. We can think of different projects – from Stresscapes – tweets as a place-based emotional expression but realise that this need validation with participants to check if the tweet related to the surroundings. The engagement was trying to be generic and ignored place and context. Everything was done through surveys on Twitter. RinkWatch looked at outdoor rink skateability – over 2000 rink people are passionate about it. The – a level of skateability level. the level of expertise is high in local knowledge and in thematic specificity. The Wildlife Health Tracker – where dead animals are – knowledge from hunters to capture information about what they have seen. Information that was reported is the type of animal – moderate thematic and local knowledge and low domain knowledge. The participants weren’t involved and much interested. The GrassLander is looking at private land – birds and habitats. Looking at farming community reporting. The cases here are where they’ve seen two types of birds (bobolink or eastern meadowlark) and – high thematic specificity, and moderate to high local knowledge and moderate domain knowledge (two species identification). Farmers were involved and there was a need to restrict access between participants. No project required high domain knowledge, the successful cases include place type or locale familiarity knowledge – though it’s a small sample. Many questions: metrics, credibility and trust models are all interesting.gfg

Following the day, group discussions explored the issues with people, technology, and future directions. Here are the future directions that were supposed in the group that I chaired with the help of Dan Artus (a future report from the workshop will be available)

 

 

Lessons learned from Volunteers Interactions with Geographic Citizen Science – Morning session

On the 27th April, UCL hosted a workshop on the “Lessons learned from Volunteers Interactions with Geographic Citizen Science“. The workshop description was as follows:

“A decade ago, in 2007, Michael Goodchild defined volunteered geographic information (VGI) as ‘the widespread engagement of large numbers of private citizens, often with little in the way of formal qualifications, in the creation of geo­graphic information, a function that for centuries has been reserved to official agencies.’ (p.2). The collection and use of this type of crowdsourced geographic data have grown rapidly with amateurs mapping the earth’s surface for all kind of purposes (e.g. collecting and disseminating information about accessibility in urban centres, for crisis and emergency response purposes, mapping illegal logging in remote areas and so on). A subset of these activities has been described as ‘geographic citizen science’ and includes scientific activities in which amateur scientists (volunteers) participate in geographic data collection, analysis and dissemination within the context of a scientific project (Haklay, 2013) or simply by using scientific methods and equipment. Although, there is an extensive discussion in the VGI and geographic citizen science literature about opportunities as well as implications (e.g. data coverage, data quality and trust issues, motivation and retainment of volunteers and so on), examples from the actual interaction are not so widely discussed, neither has evidence been collected from a broad spectrum of case studies to demonstrate how volunteers interact with those technologies and applications, what they are looking for and what it is that they need/try to accomplish (at a scientific, project and personal level) and what are the common design mistakes that influence interaction.” The following is a summary of the talk and presentations:

Welcome & Instructions – Artemis Skarlatidou the workshop is linked to our ERC funded project Intelligent Maps (ECSAnVis) and  EU funded Doing It Together science (DITOs) and the COST action – our work deal with geographical applications of citizen science and data collection. There is the COST Action CA15212 which got 243 members in 39 countries – all exploring aspects of citizen science – Work Group 1 (WG1) for scientific quality, WG2 education, WG3 society-science policy, WG4 the role of volunteers in citizen science, WG5 data and interoperability, and the synergies in WG6. In WG4, which Artemis lead. we’re looking at stakeholder mapping, motivation, needs and interaction issues, and mapping citizen science across Europe. Another relevant group is the ICA Commission on use user and usability issues, the International Society for Photogrammetry & remote sensing that have a WG V/3 that look at citizen science and crowdsourced information. Sultan Kocaman explained the ISPRS link – WG V/3 focus on the promotion of regional collaboration in citizen science and geospatial technologies within the focus of ISPRS area of education and outreach.

Louis Liebenberg presents Smartphone Icon User Interface design for Oralate Trackers – Louis Liebenberg who for 3 decades have been developing software to allow hunter gatherer to protect their knowledge of tracking. One of the challenges that Louis address is the understanding how our scientific thinking evolved. Louis suggests that tracking is an example for hypothesis testing and rational thinking that evolved in in tracking by hunter gatherers. He worked with !Nate from the San people since 1985 – the context of technology use by San for a long time. Already 100 years ago, hunters discovered that arrow points can be made from fence wire and started using them. This is an example of how hunter-gatherers adopt to technologies around them. Hunter-gatherers are not isolated: they always interacted and traded. Developing a software for a smartphone (you can get an Android phone for $10 in South Africa today), is similar to adopting the fence wire for the arrows 100 years ago. He learned from master trackers – the level of sophistication of trackers is astonished him since the mid 1980s. In the Kalahari, dogs were introduced in the 60s, and therefore the knowledge of tracking and the practices of hunting change. He used tracking and certification in it in order to secure employment. Master trackers are expected in an egalitarian society to show humility, so it is possible to miss them if you go and ask “who’s the best tracker here?” – the certification is a way to provide recognition and work. The tracking provided employment in the 1990s in surveying the movement of animals in the Kalahari. The persistent hunt – when you do it without any equipment, running animals down until they die from exhaustion which is an adaptation that humans have to be able to do that. Karoha was one of the persistence hunters but also able to use CyberTracker and use the system. Parallel to the software, Louis develop the tracker certification, to know if the data is reliable. As Master Trackers die, the knowledge is lost, so the certification provides an opportunity to encourage the younger generation to develop the knowledge and benefit from it. The level of details in animal tracks is very high. There is a high level of ambiguity in tracking and requirement to learn about claw marks and knowing what are the possibilities then it is possible with high certainty to understand which animal it was. Trackers also develop hypotheses on why the shape of hoofs is the way it is, and interpret activities of animals from the track – for example, identifying new ways of interpreting the behaviour of an animal that was not observer before. For example, the ability to guess that caracals are jumping upright in an attempt to catch a bird. CyberTracker started with the early Apple Newton with a GPS module, and then evolved into the Palm Pilot and continue to evolve. The interface was very limited in drawing icons – icons are either phonetic symbols (e.g. using a wheelbarrow to describe an item that sounds similar to the word in Africans). The details can be very extensive – species, age, number, male/female and so home. The data can provide information on abundance and potential of work are the communities. In a project in the Congo, they follow the trackers of different animals and they could show they Ebola impact Chimpanzees, Gorilla, but also other animals and then this was important to understand that you can identify Ebola in wildlife before it spreads into the human population. There is also a wide use of CyberTracker in citizen science on monitoring endangered species, and different projects by indigenous communities  Australia. They can also show that there are different results from what ecologists identify. A paper from 1999 about Rhino was co-authored by a tracker, demonstrating different models of publishing with citizen scientists. The first high impact that was co-authored by trackers was published recently in biological conservation. Questions: how to communicate from hypothesis by hunter-gatherers to the scientific sphere? The need is collaboration: data collected and organised by the trackers, and then the scientists write the report, but providing a report is challenging. The reality is co-authoring as there is always need for mentoring, reciprocal approach between scientists. Louis also circulates papers with experienced scientists to improve the paper. We all need peer review support. In terms of consent and engagement: there is a need to develop the relationship of trust and understanding – the first people who were involved in CyberTracker worked with Louis for 5 years, and Louis engaged as a tracker before they were willing to work with him. Some of the early papers in the Kalahari used trackers without mentioning their name even though the trackers carried out the research. Scientific institutions are one of the last authoritarians institutions – citizen science. Scientific elitism is intransigent and this makes citizen science exciting.

Lessons from supporting non-literate forest communities in the Congo-Basin to record their Traditional Ecological Knowledge – Michalis Vitos & Julia Altenbuchner the context of the Congo-basin is the second largest rainforest. This is a forest with 29 million people, with at least 500,000 nomadic communities that rely on resources. The forest is divided into concessions and then they are sued for resource extraction – how to make local groups heard? Local communities are excluded from protected areas. In the last few years, some legislation is changing – e.g. the FLEGT of the EU to control timber import and request for social payback and responsibility. ExCiteS collaborated with communities to support such process with technology. The challenges are dealing with non-literate groups who are also non-technologically literate. We use pictures as a way to communicate: the application working in a simple fashion – showing categories of things that people want to map, each category is leading to more specific options – the information can be captured and deciding if we want to save information and we can collect video and audio that are geotagged. In 3 simple steps, information can be captured. The process starts with a dialogue of what important for the communities, and then with this agreement on what will be collected. We do explore the usability of the application. About 70% can use the application, but 30% have a problem with categories – you follow a path of mapping banana, avocado and cacao – this requires categories, e.g. one of the set. Some participants found that confusing. Adding more icon to the category is becoming more complex. One approach was to test audio feedback in a local language – explaining the icons and what they mean. The experiments with the audio feedback help a bit, but not a lot. The next step was to go directly to the final icons and go directly to the final card – adding an NFC chip and adding the control to it. Participant finds the specific icon and then touch the card with the phone. With Tap&Map the success rate gets close to 100%.

Julia – the next issue is making sure that communities can manage their data- the vision is of intelligent maps  – having data collection, then local data repository and management, and then visualisation. But there is a challenge of the mapping and this was done by using UAVs and creating within a short time a high-resolution imagery. However, people don’t need maps as they know their area, but the maps are for communication. The maps are being used to check how the map is used – people felt under a lot of pressure when using the map. and the next experiment was not to put under pressure, and instead of doing a treasure hunt: going and looking for data by trying to find German Christmas decorations. The tracks of the people who participated in the study we can see how they looked for information. What we know is that people can use maps and understand them – the reference map. Now we want the thematic information – so when people take ownership and correct issues: this was done using the icons that were used as a resource and then to correct information. People were doing well in correcting information using a Tap&Map approach. We get feature corrections over 90%. This an ad-hoc approach: even without much exposure – we need to allow people to be sensors and the brains behind it.

Forest hunter-gatherers and Extreme Citizen Science: Reporting wildlife crime in collaboration with local and indigenous communities in Cameroon through community-led co-design – Simon Hoyte work in Cameroon for the last year and a half with Baka hunter-gatherers. Working in Cameroon in the south-east corner.Working with Dja reserve, working ZSL and 5 communities. In Cameroon, there are many issues with conservation – gorillas, chimpanzees, parrots, pangolins and elephants. Indigenous communities are lots of time are forgotten – those groups are familiar with the forest, with knowledge of 50,000 years and colonial approaches exclude. The technologies that are being used are Sapelli data collection tool, then there is the data management tool GeoKey and the CommunityMaps from Mapping for Change. The process starts with the community free prior informed consent – first starting with the concerns of the community and also building trust by staying overnight in the village and connect on a personal level. That is an important recommendation. Icons are being drawn from the sand, to a paper and then into the app. Functional actions changed from tick to thumbs app, or recording changed. XML layout of the project allow changes in the field. The second recommendation is the co-design that increases motivation. Audio and video are allowing information to be shared, including tracks – it allows a verification. Audio provides more information. Describing what people found. Indicators on the device are important – when recording is active a red icon allows you to see that something is working. The phone is checking for connection every 4 minutes. Using ID screen to recognise reported – can be used elsewhere. The community protocol also addresses who manage who will manage the phone and look after it. The report is upload and shared with the authorities – we need the diverse outcome. So in summary: trust building, co-design, media, feedback, simple tools, anonymous ID, community-led, and diverse outcomes. The map providing further more information.

Community based monitoring of tropical forests using information and communication technology (ICT) – Søren Brofeldt an example for a study that rely on Sapelli and expand the software to create the Prey Lang App: working in Cambodia, in the Prey Lang – 200,00 people who rely on the forest, and huge pressure of deforestation and a lot of the logging is illegal and it is supposed to be protected. The Prey Lang Community Network (PLCN) created around 2005-2007 and it is now a group of 600 people who are doing work over that last 10 years, and patrolling the area, confiscating chainsaw and catch wood and logs. Trying to address logging in the area. 2013 they try to communicate the problem to international society – to do what they wanted to set a forest monitoring programme and create a system to document illegal logging and provide evidence-based advocacy. The issue is to compile information and document breaches. The data is captured by Sapelli, and the information is validated by PLCN and scientists, which then helped in compiling report locally and globally, which then led to the positive platform. The platform was tweaked a bit and include information through a decision tree, they have different aspects. The things that they developed: unique functions – choosing icons or doing activities – they had basic activities in the first version: they have seen it as too simple. They started with 9 basic functions with 614 end-points of activities. By the third version, they had 9 functions, and 1663 options: types of trees, types of information, species and so on. They now have 10 functions (e.g. dropdown, word complete). Complexity does not lead to incorrect use (if training is adequate and added functionality is done in co-designed way). When people are experienced – people who use the app for 2 years can get into more complex functionality over time. Some of the issues with data – poor documents, double counting. over time, human errors are decreasing, and also technical issues. Poor connectivity and technical issue are a major issue – more than local ability to use. High quality is possible with active data management is needed.

Designing Human-Computer Interaction for Citizen Science Initiatives in Rural Developing Regions – Veljko Pejovic & Artemis Skarlatidou we need to understand how we move initiative from developed to developing regions in citizen science application. ICT4D point to environmental constraints: roads, electricity, There are also that this area lack skills in the workforce and cultural constraints. Clashes with assumptions. in the Extreme Citizen Science context: we need to identify solution adaptation in participatory design, there is a need for holistic implementation, and we need to make sure that we think about the whole process – from data collection to policy and this challenging. Finally, we also to consider the champions and engaging then (the book “Geek Heresy” by Toyama talks about it). The aim is to identify guidelines – this was done through participatory studies that are similar in the rural developing world and carried out 9 interviews with researchers with extensive experience in the field. An hour-long interviews x 2. The questions explored different aspects including interactions. The finding – need to mobilise the community by taking into account societal organisation (e.g. egalitarian aspects). Need to find local champions. We need to identify the ecosystem of the technology: chargers, cables. Also need to consider how the technology that was built to a different context work: rough fingertips, reflection in the screens and so on. There is also the issue of using hierarchical icon organisation which is pretty intuitive for educated people but it is challenging for participants (users) and also navigation buttons. This matches evidence from Medhi et al. Chi 2013. Juxtaposing this with illiterate users in urban Brazil, they managed to deal with hierarchical organisation and navigation – might be that the exposure to smartphones helped in developing these hierarchies. Icon design is different, but we can see that realistic icons with context are more suitable to use, not just an object. There are issues of actions and how to represent them. Getting honest feedback on the spot is a challenge – users don’t criticise before (Dell et al CHI 2012 – “yours is better”). Long trust relationship help in getting honest feedback. The participants lack the vocabulary to discuss HCI issues. To maintain motivation, there is a need to make data collection visible and ensure the real-world impact of data collection. Recommendation: develop context-specific apps – not genetic, and consider application interface that matches user’s skills and geographical information is a key.

Introducing user issues of the Global Forest Watch application – Jamie Gibson – developing with Vizzuality better maps and visualisation. Trying to think of citizen-focused GIS, interacting with the citizen in the design. Global Forests Watch (GFW) was developed in the last 3 years, and it is allowing to see the world’s forest and how they change. They wanted to tell a simple story: where forest is gained and lost. With few clicks, you can see the impact of conservation. GFW allow seeing how deforestation is implemented and how it is stopped. There is a need for global engagement – opening it to a whole crowd of people. Forest don’t have a connection to the web, and try to take data online to the field, walk to the area, investigate recent forest loss and report new areas – 4000-5000 users. They aim to integrate citizens into the design process. Forest Watcher is being used in important areas of the world and not where the most connected people area. They analyse where people use the app – when there are forest fires in Spain, people are updating GFW and explore. Use the analytics to find the places where we want more people to look and explore. This is integrated with interviews and usability testing. Working with experts who been working for a long time – including Jane Goodall Institute, Amazon Conservation Team, CAGDF, and BirdLife. As people use the application they build ownership and they provide a better feedback and richer information. In terms of what they learn, including the use of persona to think about monitors: need to have lots of other things that try to sync after the 14 days offline – the internet is slow and changed the app and the back end to make it faster. Use it to understand frustrations and find ways to wow moment. Face, name and story improve the quality of the thinking and understand their frustrations.

Lessons learned from Missing Maps – Jorieke Vyncke Her personal background is in interest in work that links to humanitarian purposes, and since 2017 is the missing maps coordinator. She is looking at the humanitarian organisation focus -more than 34,000 staffers in MSF and about 470 locations around the world. In many parts of the world there are empty maps and not geographical data. They discover OpenStreetMap and working with the American and British Red Cross, HOT and over 40 partners. They have principles from the Ostrom on working with groups. They compare rural and urban parts. In Idjiwi in DRC, the east of Congo – working with a multitude of problems: violence, refugees and more. Due to a measles outbreak, they needed population and mapping data. Included 250 remote volunteers who mapped 28,000 building in about a week. This helped in creating population estimation – critical for the logistical planning. They managed to identify 94% of the population. An example from Bangladesh in Hazaribagh informal settlement. The area was mapped with both local and remote mapping – including factories and tanneries – locating the workers that they wanted to reach – combining students from the university with workers that were reached through the union. The experience of mapping is done by the technical local students to make things happen. Using smartphones and field papers process. Paper is still effective, and then also the edit data in pairs on how to do the mapping – the end result provided an occupational health survey. The process motivated the community and they continue to use it. In different areas, they use remote mapping but the most important thing is to create a local mapping community and that makes a decision between empowerment and remote mapping with the importance of saving life.

Keynote: Approximated Reality: the use of digital tools by traditional communities in the Amazon – Vasco van Roosmalen working in Ecam – Equipe Conservacao Amazonia in Brazil since 1999. The big challenge is how to reconcile different visions of what the world is. In the Xingu area in Brazil, there was a need to create an ethno-map of the region. The community discusses what they want to map and how they want to represent them, but it also needed to be cartographically accurate as this is how you communicate with external bodies. The whole map is created for the community: to use resources, to remember the dead and to defend their land (using patterns of body paint). We can see that protected areas in the Xingu. Another area that he was involved in mapping is near Surinam – in an area the size of Holland with 2000 people, the community recorded information about their region. This helped in justifying the resources and the protection of the area. An area that is very rough to access, and the local survey by the community managed to map the area done that in 6 maps. The community collected much more data than what the map can show – over the coming years, they mapped with different groups millions of hectares and they developed a process of creating the maps. The collaboration with Google Earth Outreach led to the interaction with Chief Amir of the Surui. The link with commitment with Rebecca Moore helped in filling up areas that are missing and attaching video and audio to the map. They then wanted to record illegal logging using mapping tools and this was done with OpenDataKit – the data collection challenges are accuracy, ease of use, speed, etc. In 2008 started to understand REDD and developed the Surui Carbon Project – need a tremendous amount of data from the air and from the ground. The use of information such as the circumference of trees was done with ODK. They use Garmin devices: they weren’t scratch resistance. Now they use a Samsung smartphones that are cheap and can be replaced easily. For the GPS in the rainforest, it is challenging and they use barcode on the trees. They used the ODK build but discovered that it is not an easy interface: using a programmer in the staff and that is a limitation in terms of allowing to build forms easily. The project managed to demonstrate that indigenous people can collect data but the REDD credits were more challenging and they got them in 2013. Cultural maps where created in other indigenous lands in Brazil. The importance not just to demarcate the land but to collect data and help them to manage the area. Today there are many challenges – 13% of the Brazilian territory. In the Brazilian Amazon, there are many communities – 25 mil people of which only 350,00 indigenous for example, Quilombola groups and many other groups. There was no information on other groups and some of them are disadvantaged – e.g. Quilombola required mapping 7000 communities, they are descendent of West African slaves – they were persecuted, faced a lot of violence, and when slavery was abolished they were forgotten, but from the 1980s they are recognised in the constitution, but not enough recognised officially. His team was involved in creating a new map of the 7000 communities for which only on a team of 40 is looking after in the government level in Brasilia. They used approaches that are similar to the Indigenous mapping in order to record information and manage the land. They had people who became experts in mapping and then demonstrating how to map the land using google earth and demonstrating data collection. The communities also collect socio-economic data – using ODK and understanding their community and developing a life plan for the area (plan for the next 10-30 years). The question is who is listening to the information but by whom. A social network analysis of Facebook (which is 83% of users in Brazil use) Looking at interactions show that local association are not linked to environment, human right and there is missing links to health, to a specific campaign on the Belo Monte Power Plant but it is not linked to the community. They care about health, education, income, and only fifth is the environment – need to talk about what matters to communities. How to make conversations about them in the centre of the discussion and move beyond putting them in the corner of the environment. We need to engage with people with their communities in a way that makes sense to them.

 

 

 

 

 

 

 

Citizen Science & Scientific Crowdsourcing – week 5 – Data quality

This week, in the “Introduction to Citizen Science & Scientific Crowdsourcing“, our focus was on data management, to complete the first part of the course (the second part starts in a week’s time since we have a mid-term “Reading Week” at UCL).

The part that I’ve enjoyed most in developing was the segment that addresses the data quality concerns that are frequently raised about citizen science and geographic crowdsourcing. Here are the slides from this segment, and below them a rationale for the content and detailed notes

I’ve written a lot on this blog about data quality and in many talks that I gave about citizen science and crowdsourced geographic information, the question about data quality is the first one to come up. It is a valid question, and it had led to useful research – for example on OpenStreetMap and I recall the early conversations, 10 years ago, during a journey to the Association for Geographic Information (AGI) conference about the quality and the longevity potential of OSM.

However, when you are being asked the same question again, and again, and again, at some point, you start considering “why am I being asked this question?”. Especially when you know that it’s been over 10 years since it was demonstrated that the quality is beyond “good enough”, and that there are over 50 papers on citizen science quality. So why is the problem so persistent?

Therefore, the purpose of the segment was to explain the concerns about citizen science data quality and their origin, then to explain a core misunderstanding (that the same quality assessment methods that are used in “scarcity” conditions work in “abundance” conditions), and then cover the main approaches to ensure quality (based on my article for the international encyclopedia of geography). The aim is to equip the students with a suitable explanation on why you need to approach citizen science projects differently, and then to inform them of the available methods. Quite a lot for 10 minutes!

So here are the notes from the slides:

[Slide 1] When it comes to citizen science, it is very common to hear suggestions that the data is not good enough and that volunteers cannot collect data at a good quality, because unlike trained researchers, they don’t understand who they are – a perception that we know little about the people that are involved and therefore we don’t know about their ability. There are also perceptions that like Wikipedia, it is all a very loosely coordinate and therefore there are no strict data quality procedures. However, we know that even in the Wikipedia case that when the scientific journal Nature shown over a decade ago (2005) that Wikipedia is resulting with similar quality to Encyclopaedia Britannica, and we will see that OpenStreetMap is producing data of a similar quality to professional services.
In citizen science where sensing and data collection from instruments is included, there are also concerns over the quality of the instruments and their calibration – the ability to compare the results with high-end instruments.
The opening of the Hunter et al. paper (which offers some solutions), summarises the concerned that are raised over data

[Slide 2] Based on conversations with scientists and concerned that are appearing in the literature, there is also a cultural aspect at play which is expressed in many ways – with data quality being used as an outlet to express them. This can be similar to the concerns that were raised in the cult of the amateur (which we’ve seen in week 2 regarding the critique of crowdsourcing) to protect the position of professional scientists and to avoid the need to change practices. There are also special concerns when citizen science is connected to activism, as this seems to “politicise” science or make the data suspicious – we will see next lecture that the story is more complex. Finally, and more kindly, we can also notice that because scientists are used to top-down mechanisms, they find alternative ways of doing data collection and ensuring quality unfamiliar and untested.

[Slide 3] Against this background, it is not surprising to see that checking data quality in citizen science is a popular research topic. Caren Cooper have identified over 50 papers that compare citizen science data with those that were collected by professional – as she points: “To satisfy those who want some nitty gritty about how citizen science projects actually address data quality, here is my medium-length answer, a brief review of the technical aspects of designing and implementing citizen science to ensure the data are fit for intended uses. When it comes to crowd-driven citizen science, it makes sense to assess how those data are handled and used appropriately. Rather than question whether citizen science data quality is low or high, ask whether it is fit or unfit for a given purpose. For example, in studies of species distributions, data on presence-only will fit fewer purposes (like invasive species monitoring) than data on presence and absence, which are more powerful. Designing protocols so that citizen scientists report what they do not see can be challenging which is why some projects place special emphasize on the importance of “zero data.”
It is a misnomer that the quality of each individual data point can be assessed without context. Yet one of the most common way to examine citizen science data quality has been to compare volunteer data to those collected by trained technicians and scientists. Even a few years ago I’d noticed over 50 papers making these types of comparisons and the overwhelming evidence suggested that volunteer data are fine. And in those few instances when volunteer observations did not match those of professionals, that was evidence of poor project design. While these studies can be reassuring, they are not always necessary nor would they ever be sufficient.” (http://blogs.plos.org/citizensci/2016/12/21/quality-and-quantity-with-citizen-science/)

[Slide 4] One way to examine the issue with data quality is to think of the clash between two concepts and systems of thinking on how to address quality issue – we can consider the condition of standard scientific research conditions as ones of scarcity: limited funding, limited number of people with the necessary skills, a limited laboratory space, expensive instruments that need to be used in a very specific way – sometimes unique instruments.
The conditions of citizen science, on the other hand, are of abundance – we have a large number of participants, with multiple skills, but the cost per participant is low, they bring their own instruments, use their own time, and are also distributed in places that we usually don’t get to (backyards, across the country – we talked about it in week 2). Conditions of abundance are different and require different thinking for quality assurance.

[Slide 5] Here some of the differences. Under conditions of scarcity, it is worth investing in long training to ensure that the data collection is as good as possible the first time it is attempted since time is scarce. Also, we would try to maximise the output from each activity that our researcher carried out, and we will put procedures and standards to ensure “once & good” or even “once & best” optimisation. We can also force all the people in the study to use the same equipment and software, as this streamlines the process.
On the other hand, in abundance conditions we need to assume that people are coming with a whole range of skills and that training can be variable – some people will get trained on the activity over a long time, while to start the process we would want people to have light training and join it. We also thinking of activities differently – e.g. conceiving the data collection as micro-tasks. We might also have multiple procedures and even different ways to record information to cater for a different audience. We will also need to expect a whole range of instrumentation, with sometimes limited information about the characteristics of the instruments.
Once we understand the new condition, we can come up with appropriate data collection procedures that ensure data quality that is suitable for this context.

[Slide 6] There are multiple ways of ensuring data quality in citizen science data. Let’s briefly look at each one of these. The first 3 methods were suggested by Mike Goodchild and Lina Li in a paper from 2012.

[Slide 7] The first method for quality assurance is crowdsourcing – the use of multiple people who are carrying out the same work, in fact, doing peer review or replication of the analysis which is desirable across the sciences. As Watson and Floridi argued, using the examine of Zooniverse, the approaches that are being used in crowdsourcing give these methods a stronger claim on accuracy and scientific correct identification because they are comparing multiple observers who work independently.

[Slide 8] The social form of quality assurance is using more and less experienced participants as a way to check the information and ensure that the data is correct. This is fairly common in many areas of biodiversity observations and integrated into iSpot, but also exist in other areas, such as mapping, where some information get moderated (we’ve seen that in Google Local Guides, when a place is deleted).

[Slide 9] The geographical rules are especially relevant to information about mapping and locations. Because we know things about the nature of geography – the most obvious is land and sea in this example – we can use this knowledge to check that the information that is provided makes sense, such as this sample of two bumble bees that are recorded in OPAL in the middle of the sea. While it might be the case that someone seen them while sailing or on some other vessel, we can integrate a rule into our data management system and ask for more details when we get observations in such a location. There are many other such rules – about streams, lakes, slopes and more.

[Slide 10] The ‘domain’ approach is an extension of the geographic one, and in addition to geographical knowledge uses a specific knowledge that is relevant to the domain in which information is collected. For example, in many citizen science projects that involved collecting biological observations, there will be some body of information about species distribution both spatially and temporally. Therefore, a new observation can be tested against this knowledge, again algorithmically, and help in ensuring that new observations are accurate. If we see a monarch butterfly within the marked area, we can assume that it will not harm the dataset even if it was a mistaken identity, while an outlier (temporally, geographically, or in other characteristics) should stand out.

[Slide 11] The ‘instrumental observation’ approach removes some of the subjective aspects of data collection by a human that might make an error, and rely instead on the availability of equipment that the person is using. Because of the increase in availability of accurate-enough equipment, such as the various sensors that are integrated in smartphones, many people keep in their pockets mobile computers with the ability to collect location, direction, imagery and sound. For example, images files that are captured in smartphones include in the file the GPS coordinates and time-stamp, which for a vast majority of people are beyond their ability to manipulate. Thus, the automatic instrumental recording of information provides evidence for the quality and accuracy of the information. This is where the metadata of the information becomes very valuable as it provides the necessary evidence.

[Slide 12] Finally, the ‘process oriented’ approach bring citizen science closer to traditional industrial processes. Under this approach, the participants go through some training before collecting information, and the process of data collection or analysis is highly structured to ensure that the resulting information is of suitable quality. This can include the provision of standardised equipment, online training or instruction sheets and a structured data recording process. For example, volunteers who participate in the US Community Collaborative Rain, Hail & Snow network (CoCoRaHS) receive standardised rain gauge, instructions on how to install it and online resources to learn about data collection and reporting.

[Slide 13]  What is important to be aware of is that methods are not being used alone but in combination. The analysis by Wiggins et al. in 2011 includes a framework that includes 17 different mechanisms for ensuring data quality. It is therefore not surprising that with appropriate design, citizen science projects can provide high-quality data.

 

 

Citizen Science for Observing and Understanding the Earth

Since the end of 2015, I’ve been using the following mapping of citizen science activities in a range of talks:

Range of citizen science activities
Explaining citizen science

The purpose of this way of presentation is to provide a way to guide my audience through the landscape of citizen science (see examples on SlideShare). The reason that I came up with it, is that since 2011 I give talks about citizen science. It started with the understanding that I can’t explain extreme citizen science when my audience doesn’t understand what citizen science is, and that turned into general talks on citizen science.

Similarly to Caren Cooper, I have an inclusive approach to citizen science activities, so in talks, I covered everything – from bird watching to DIY science. I felt that it’s too much information, so this “hierarchy” provides a map to go through the overview (you can look at our online course to see why it’s not a great typology). It is a very useful way to go through the different aspects of citizen science, while also being flexible enough to adapt it – I can switch the “long-running citizen science” fields according to the audience (e.g. marine projects for marine students).

An invitation for Pierre-Philippe Mathieu (European Space Agency) in 2015 was an opportunity to turn this mapping and presentation into a book chapter. The book is dedicated to “Earth Observation Open Science and Innovation and was edited by Pierre-Philippe and Christoph Aubrecht.

When I got to writing the chapter, I contacted two researchers with further knowledge of citizen science and Earth Observation – Suvodeep Mazumdar and Jessica Wardlaw. I was pleased that they were happy to join me in the effort.

Personally, I’m very pleased that we could include in the chapter the story of the International Geophysical Year, (thank Alice Bell for this gem), with Moonwatch and Sputnik monitoring.

The book is finally out, it is open access, and you can read our chapter, “Citizen Science for Observing and Understanding the Earth” for free (as well as all the other chapters). The abstract of the paper is provided below:

Citizen Science, or the participation of non-professional scientists in a scientific project, has a long history—in many ways, the modern scientific revolution is thanks to the effort of citizen scientists. Like science itself, citizen science is influenced by technological and societal advances, such as the rapid increase in levels of education during the latter part of the twentieth century, or the very recent growth of the bidirectional social web (Web 2.0), cloud services and smartphones. These transitions have ushered in, over the past decade, a rapid growth in the involvement of many millions of people in data collection and analysis of information as part of scientific projects. This chapter provides an overview of the field of citizen science and its contribution to the observation of the Earth, often not through remote sensing but a much closer relationship with the local environment. The chapter suggests that, together with remote Earth Observations, citizen science can play a critical role in understanding and addressing local and global challenges.

 

Chapter in Routledge Handbook of Mapping and Cartography – VGI and Beyond: From Data to Mapping

Hot on the heels of the Routledge Handbook of Environmental Justice is thThe Routledge Handbook of Mapping and CartographyRoutledge Handbook of Mapping and Cartography. The handbook was edited by Alex Kent (Canterbury Christ Church University) who is currently the President of the British Cartographic Society and Editor of The Cartographic Journal; and Peter Vujakovic (also from Canterbury Christ Church University) who edited The Cartographic Journal.

Like the other handbooks, this is an extensive collection of 43 chapters and almost 600 page about maps and mapping. The chapters provide a vivid demonstration that cartography and map making is art and science, and that it links to many sciences and practices – from cognitive psychology to geodesy. The list of authors is impressive and includes many of the people that are shaping current cartographic research.

However, with a price tag of £195 for the Book, this collection is expensive and suitable for university libraries and to professional or commercial mapping organisation. The eBook is £35, which makes it much more affordable, though having used the online system, the interface could be better. Luckily the policy of Routledge permits sharing the chapters on personal websites.

My contribution to the book is in a joint paper that was led by Vyron Antoniou titled VGI and Beyond: From Data to Mapping. The chapter is building on a collaboration between Vyron, myself and Cristina Capineri during the COST Action on Volunteered Geographic Information (ENERGIC). In the chapter, we look at the concept of Volunteered Geographic Information (VGI) within practices of mapping and cartography and we attempted to provide an accessible overview of the area. We define what VGI is, provide an overview of the area, look at the advantages and disadvantages of VGI in mapping and cartography, and then look at the impacts of VGI on national mapping agencies, the public, and public bodies. The chapter is available here and we would be very happy to hear comments on it.

 

 

Chapter in Routledge Handbook of Environmental Justice – Participatory GIS and community-based citizen science for environmental justice action

The Routledge Handbook of Environmental Justice has been published in mid-September. This extensive book, of 670 pages is providing an extensive overview of scholarly research on environmental justice

The book was edited by three experts in the area – Ryan Holifield from the University of Wisconsin-Milwaukee, Jayajit Chakraborty from the University of Texas at El Paso, and Gordon Walker from the Lancaster Environment Centre, Lancaster University, UK. All three have affiliations that relate to Geography, and geographic and environmental information play a major part in the analysis and action regarding environmental justice.

The book holds 51 chapters that are covering the theory and practice of environmental justice – from how it is analysed and understood in different academic disciplines, to the methods that are used to demonstrate that environmental justice issues happen in a place,  and an overview of the regional and global aspects of current environmental justice struggles. The range of chapters and the knowledge of the people who write them are making this collection a useful resource for those who are studying and acting in this area (though few top authors in this field are missing, but their work is well referenced)

However, with a price tag of £165 for the Book, the costs put an obstacle for those who need the information but suitable for universities and libraries. The eBook is £35, which makes it much more affordable, though having used the online system, the interface could be better. Luckily the policy of Routledge permits sharing the chapters on personal websites.

My contribution, together with Louise Francis, is in Chapter 24 –Participatory GIS and community-based citizen science for environmental justice action. In this chapter, we provide an overview of the use of participatory GIS in environmental justice action, but in particular, a detailed explanation of the methodology that we have developed a decade ago, with contributions from Colleen Whitaker, Chris Church and other people that worked with us a the time. The methodology is now used in the activities of Mapping for Change.  The methodology supports both participatory mapping and citizen science.

As we note in the chapter “Our methodology emerged in 2007, through the London 21 Sustainability Network project ‘A Fairer, Greener London’, which aimed to give six marginalised communities the opportunity to develop their own understanding of local environmental justice issues and supporting action plans to address them. The project was integrated closely with the project ‘Mapping Change for Sustainable Communities’ which was funded as part of the UrbanBuzz scheme. Both projects were based on accessible GIS technologies and available environmental information sources.

The methodology evolved into a six-stage process that is inherently flexible and iterative – so, while the stages are presented here as a serial process, the application of the methodology for a specific case is carried out through a discussion with the local community.” The chapter provides an example for the implementation of the methodology from the work that we carried out in the Pepys Estate.

If you want to read the whole chapter (and use the methodology) you can find it here. For any other chapter in the handbook, email the authors and they will probably share a copy with you.