Lessons learned from Volunteers Interactions with Geographic Citizen Science – Afternoon session

The context of the workshop and the notes from the first part of the workshop is available here. The theme of the second part of the day was Interacting with geographical citizen science: lessons learned from urban environments

Volunteer interactions with flood crowdsourcing platforms – Avi Baruch talk is based on a completed PhD on the aspects of volunteers in flood monitoring and response. There are different types – incident reporting floodline, media outreach, online volunteering, and collaborative mapping. He looked at Tomnod as a system that is currently used to engage volunteers in tagging satellite images. Looked at forums and interviews with the most active participants. Most volunteers where over 50 and there is a good balance by gender. 23% stated that they had a long-term health problem – finding it addictive and spending 8-10 hours a day. Engaging volunteers is an issue: there was not enough feedback on how the information was used and how they are performing, which Tomnod team haven’t done. at least 10% of comments were concerned with the quality of their contributions. Without feedback, it is hard to judge. Tomnod allow people to explore the map and they can share location, but then people concentrate in one area. Restricting people to an area didn’t work well. Core motivations were based on altruistic reasons, and retirement, disability and health were reasons for engagement. The second part of the PhD project includes the development of a citizen science platform to report (floodcrowd.co.uk) and doing the development through an iterative process. The form allows people to report flooding incidents. All the information that is provided is location, and type of flooding, and then people can report further details. In communities, that experience flooding preferred a hard copy. All sort of information was submitted, mostly about surface mapping – many people who are potential participants didn’t want to engage with the app. Need further co-production with the people who contribute the data.

Volunteers Interaction in Technology Driven Citizen Science contexts: Lessons learned from senseBox and openSenseMap – Mario Pesch & Thomas Bartoschek SenseBox have been developed over the past 4 years. It came out from teaching computer science in school – focus on environmental sensing which the students wanted to see on the web. Sensing temp, humidity, pressure, light, UV-light. People wanted to participate in the project from outside school. in 2014, they had 50 sesneBoxes – most connected only for few weeks (8m records). After a while, people find it complicated and they wanted to do something on their own. They created a DIY sensing box for home and for school. The component allows people to create things without soldering. They set out reference stations next to official monitoring station – people asked about it. You always need to consider the limitation of the system. SensorBox home 2.0 was looking more at air quality and more options to send the data – measuring in places without WiFi so they added GSM and now they have 1500 sensing stations and people also want to work with the data and you can do basic interpolation. The platform is device independent and people use it for other systems. Also supporting mobile stations, They keep the project open – it can be adjusted to people own needs.

Lessons learned from volunteers’ use and feedback of the Cyclist GEO-C App – Diego Pajarito, Suzanne Maas, Maria Attard and Michael Gould the experience of the cycling app is part of the PhD network GEO-C – open city toolkit. A lot of application target sports or data collected but not linked to the experience on the road. The location Cyclist GEO-C app is for Android and can be competition or cooperation based, and collect GPS tracks and up to 3 tags. Tried in Castello, Munster and Valletta and Malta. There are different levels of cycling used. 20 participants – that commute regularly and using an Android phone. Different participation methods – as a group to get common views. They captured 793 trips, the response was generally positive. People seeing a potential for personal use but also to lobby and promote cycling. Can be a motivational tool for beginners. They also identify the issue of remembering to use the app when the need to use it, improve control over recording and improving the tags. Ideas about mapping interface and using wearable devices, social interaction and gamification were suggested.

Invisible Citizen Science: the case of Járókelő in Hungary – Bálint Balázs & Le Marietta thinking of the citizen science in Eastern Europe, which thinking about modes of public participation in scientific discourse and policy-making, there are multiple silences: there are many projects that offer it, and in the level of initiative – the term haven’t exist and used. The interview from an NGO suggested lack of familiarity. In eastern countries in Europe, citizen science is only recently emerging, not many initiatives, and little-published articles and only a few members of ECSA, and how it is connected. Methods are limited. Need to reconceptualise. There is invisible citizen science – the specific knowledge that is produced in the projects that they are looking at it are uncommon to scientists. An example for this is jarokelo – for addressing local issues – looking at the example for “fix my street” (or “letter to the mayor” in the Czech Republic). Civic technology to report street fixing and there are 20 volunteers who can transfer it to the authority – there are 50-100 reports per day and the reporting back from the authority can take 30 days. Most authorities report back, they also received reports on homeless people and had to agree on what to do with this types of report. The issue of participants is about trust in the state and also think of cooperative research ideas – analysing users’ statistics, thinking of involvement pathways and better communication.

Citizens as Shoppers: Lessons learned from the EnvBodySens application – Eiman Kanjo  looking into mobile sensing – the challenge for retail in the centre of cities and there is also all sort of noise and air pollution that people are concerned about. Done work around a popular shopping area in Nottingham city centre – what kind of sensors – environment, physiology, motion, timestamps, location, continuous self-reporting and the zoning (understanding which shops they are in, or the area that they are visiting). Issues of collecting data involve selecting types of sensors (e.g. the characteristics of the sensors). There was issue of demography, shopping behaviour (men/women), challenges with how many volunteers you get and how to prepare volunteers – but for shopping, we need them to be relaxed and enjoy the shopping and how you start the experiment. There is also the aspects of the journey (real-life shopping experience and temporal aspect of it) which also raise ethical concerns. They needed to consider if the phone is on all the time or should it use voice and audio information. Self-reporting and self-assessment is something that needs consideration. They ended with 50 participants, wristband devices and mobile phone and a 45 shopping journey – they looked at the impact of noise and they also consider how they can visualise all this information.

Lessons learned from the recent landlside mitigation efforts: citizen science as a new approach – Sultan Kocaman & Candan Gokceoglu volunteer contribution can provide important information – increase world population and climate change (extreme weather) is a major natural hazard. Wanted to explore how citizen science is relevant to address uncertainties because there is a lack of reliable temporal data. Risk assessment s base on knowledge of past events – then assessing susceptibility, hazard assessment and then you can understand the risk assessment and manage it. Landslide susceptibility requires a lot of information and data. The risk assessment needs all this information as otherwise there will be too much uncertainty. The majority of landslides are in mountainous areas and we can’t have sensors, but information is coming from observers evidence, and volunteers can provide the time and location in a better way. Shallow landslides disappear after a short period. Need volunteers at the right time and the right place – distributed participation. The scale of movement can also be measured with volunteers. Currently working on the project and consider what can be done – what the frequency and quality of spatial and temporal data and in any case rely heavily on local knowledge but need to be improved.

Citizens as volunteer cartoghraphers: A pedestrian map case study – Manousos Kamilakis exploring the field of cartography for pedestrian – based on ideas from VGI so people can share information. Most of the online maps are focusing on motorised transport, and less about the aesthetic pleasantness of the journey, the condition of the pavement etc. The two journeys are suggested as equivalent and only one of them is offering a better journey. Created an app for pedestrian reporting and recording the journey, then evaluate and review the journey and also editing a path. They carried out an experiment with people who never edited a map and had various motivations – the leaderboard wasn’t of interest, although half were motivated by gamification and were willing to cheat to score points. Creating motivation is difficult – need to design gamification carefully and external incentives encourage unacceptable data uploading – consider peer review. People do not volunteer to all tasks in an equal way.

Interacting with Community Maps – Mapping for Change Louise Francis and Rosa Arias cover the development of international odour observatory.  Building on Principle 10 of Rio and the right of access environmental information – different authorities produced maps, such a noise map.When talking with communities, people are pointing that they have a different experience and reflect their own understanding of their local conditions using citizen science. Citizen collect information and Mapping for Change visualise it on behalf of the community. Community evolved over the years. it is a flexible system that allows people to decide on the grouping of information – the themes are being groups in different ways. There is also a need to make conversation – interact with contributions that other people added. The data is to drive change – for example leading to a change in buses through campaign and publicity to change things around them. Lessons learned: communities, where adding data – demonstrating that community members wanted to share a lot of data and they wanted information on their balcony and putting a point on top of a point, wasn’t possible in the past and require changed. The map is allowing clustering that shows 115 points in a small area. Some communities wanted to have their own classification – so they took the data and created their own visualisation. We learned that and want to be part of the D-Noses: odour pollution. The top-down approaches to address issues of odour and there is fairly little addressing of issues. OdourCollect focuses on bottom-up approach – using the nose to notice odour problems. The OdoucrCollect allow data capture.

A Case Study on the Impact of Design Choices on Data Quality in Geographic Citizen Science – Jeffrey Parsons, NL Nature design choices – ecologist and looking at data management and data quality. Looking at a specific design choice. Looking at two archetypes of systems – on one end well defined and stable use of data (close) precise focus on data collection and data collection standards – citizen scientists with requisite domain knowledge and motivated to do the work well. The other end ill-defined, open use, which provides opportunities for data collection in an opportunistic way, ambiguous data collection standard and unclear domain knowledge. eBird is an example of a project that is towards the closed version. The research setting in traditional science lead to design principles for closed citizen science and these don’t work in open and that can lead to a problem in the application. Information quality is a major challenge in User Generated Content (UGC) – there is all sort of comments about it. Fitness for use is a major one – in close: training, data collection protocol, clean data – but this is a problem in an open environment and it can inhibit contributors from communicating unique knowledge. They suggest crowd IQ – from the contributors’ perspective (Lukyanenjo et al 2014). The question is how do we design in such a way that matches the contributors’ mental models of the information and align with contributors’ capabilities. Design principles focus on conceptual modelling – describing in a way that you use a class-based approach of setting the categories and the model drive the design. Design choice of conceptual model of the producer and not necessarily of the contributors. The alternative is to do instance-based modelling which is based on an ontological view of a world made of things and cognitive approach. The information quality impacts – if you think about data completeness as a way to describe the engagement of volunteers to add information. They checked a website that was focused on species only and another one that focuses on the attributes. The hypotheses are that they’ll get more observation and novel species. NLNature.com is about observations of wildlife. They allow people to type species name or the Latin name, the other option is typing whatever you want. They collected data over 6 months, they have 4 times more observations in the instance based condition, and also observe that class-based condition frustrated the contributors and left compared to the class-based case. They got many more species in the instance based when it is open to people to define insects, fish. They even discover a new type of wasp. The bottom line, modelling choices affect dataset completeness – class based lead to fewer observations and especially of species that are not in the schema.


From paper prototyping to citizen participation: Co-designing geolocated cultural heritage applications that trigger personal reflection – Kate Jones – looking at cultural heritage. The aim is to create a serendipitous outdoor exhibition to reflect on historical topics and encourage thoughtful play on historical issues. The topic that they focused on was that of migration – 45% of the population is made of migrants in Luxembourg and that influence way to thinking of a location for historical and contemporary memories and experiences. Two places – Luxembourg city and Valletta and they are both touched by migration and are UNESCO sites. They have Mobile app, moderator app, and point of interest management system and they check the information and want to use CrowdFlower to moderate. The application is to allow people to tag history places and be able to record journeys and stories about spaces and memory. Complexity is being hidden behind the levels. The app informs the user that they are being tracked. It was designed in an iterative process – user scenario, requirements, wood game to try how people use the application action – then develop and evaluate. A board game prototyping allowed the development of scenarios. Postcards symbolic of the user interface. The content needs to be valid, and interesting – want to reflect when people are out and about it the city. They included game designer and the developer and they can see the perspective of the player. People used stickers on the board card to indicate what they liked and disliked – people wanted a stronger connection between migration and experience. They used a digital humanities methods and figured out that it can be too complex so the levels can help in unlocking it. Questions had to be changed to address the emotional response of participants, and the multi-city connection was complex and need to develop carefully. A board game for the design was fun and collaborative but also helped in the development of the game. Going t the field, the launched the application in September 2017. out of 500 students, 40 app download, and only two trajectories. They created a new iteration. People don’t like reading the lengthy text – so they put it text to voice and that brought different issues with the interface. They see different types of people in the user population. Exploration have led to a change in perspective in the final application and grounded in participant experience. How do we give people the motivation to give it a go?

Geographical expertise and citizen science: planning and -design implications – Colin Robertson & Robert Feick considering different levels of geographical expertise – what does it mean to be a geographical expert – what are the expert/non-expert into a spectrum. We can look at some ideas of expertise: Collins 2013 pointed to the 3 dimensions of expertise – contributory, interactional and esotericity – exposure to tacit knowledge in a domain, recognised accomplishments or is it expertise that is common or uncommon we can look at it in a continuum – locale familiarity: place-based expertise related – might be fuzzy. Other geographical knowledge is about place-types – say urban environments or glacial environment. We can think about expertise in the cube – for a soil scientist it is in position A, long-term residents of the area might be a huge locale expertise B and so on. We can think of different projects – from Stresscapes – tweets as a place-based emotional expression but realise that this need validation with participants to check if the tweet related to the surroundings. The engagement was trying to be generic and ignored place and context. Everything was done through surveys on Twitter. RinkWatch looked at outdoor rink skateability – over 2000 rink people are passionate about it. The – a level of skateability level. the level of expertise is high in local knowledge and in thematic specificity. The Wildlife Health Tracker – where dead animals are – knowledge from hunters to capture information about what they have seen. Information that was reported is the type of animal – moderate thematic and local knowledge and low domain knowledge. The participants weren’t involved and much interested. The GrassLander is looking at private land – birds and habitats. Looking at farming community reporting. The cases here are where they’ve seen two types of birds (bobolink or eastern meadowlark) and – high thematic specificity, and moderate to high local knowledge and moderate domain knowledge (two species identification). Farmers were involved and there was a need to restrict access between participants. No project required high domain knowledge, the successful cases include place type or locale familiarity knowledge – though it’s a small sample. Many questions: metrics, credibility and trust models are all interesting.gfg

Following the day, group discussions explored the issues with people, technology, and future directions. Here are the future directions that were supposed in the group that I chaired with the help of Dan Artus (a future report from the workshop will be available)




Usability, SatNavs and Which?

The Consumers’ Association Which? magazine  is probably not the first place to turn to when you look for usability studies. Especially not if you’re interested in computer technology – for that, there are sources such as PC Magazine on the consumer side, and professional magazines such as Interactions from Association for Computing Machinery (ACM) Special Interest Group on Computer-Human Interaction (SIGCHI).

And yet…

Over the past few years, Which? is reviewing, testing and recommending Satnavs (also known Personal Navigation Devices – PNDs). Which? is an interesting case because it reaches over 600,000 households and because of the level of trust that it enjoys. If you look at their methodology for testing satnavs , you’ll find that it does resemble usability testing – click on the image to see the video from Which? about their methodology. The methodology is more about everyday use and the opinion of the assessors seems to play an important role.

Link to Which Satnav video

Professionals in geographical information science or human-computer interaction might dismiss the study as unrepresentative, or not fitting their ways of evaluating technologies, but we need to remember that Which? is providing an insight into the experience of the people who are outside our usual professional and social context – people who go to a high street shop or download an app and start using it straightaway. Therefore, it’s worth understanding how they review the different systems and what the experience is like when you try to think like a consumer, with limited technical knowledge and understanding of maps.

There are also aspects that puncture the ‘filter bubble‘ of geoweb people – Google Maps are now probably the most used maps on the web, but the satnav application using Google Maps was described as ‘bad, useful for getting around on foot, but traffic information and audio instructions are limited and there’s no speed limit or speed camera data‘. Waze, the crowdsourced application received especially low marks and the magazine noted that it ‘lets users share traffic and road info, but we found its routes and maps are inaccurate and audio is poor‘ (both citations from Which? Nov 2012, p. 38). It is also worth reading their description of OpenStreetMap when discussing map updates, and also the opinions on the willingness to pay for map updates.

There are many ways to receive information about the usability and the nature of interaction with geographical technologies, and some of them, while not traditional, can provide useful insights.