The Potential of Volunteered Geographic Information (VGI) in Future Transport Systems

dsc01541An aspect of collaborative projects is that they start slowly, and as they become effective and productive, they reached their end! The COST Energic (European Network for Research into Geographic Information Crowdsourcing) led to many useful activities, with some of them leading to academic papers. From COST Energic, we’ve got the European Handbook on Crowdsourced Geographic Information, a paper on VGI quality assessment methods, and more.

One outcome came out from the close collaboration around the summer schools that were organised by the network. Prof Cristina Capineri was the chair of the COST network, and also the organiser of summer schools in Fiesole, near Florence. Prof Maria Attard organised the other summer school of the action, at the University of Malta. Based on our close working relationships (though Maria and I know each other since our PhD studies in CASA) we started working on a joint paper. Maria specialises in transport geography, so the support from COST Energic was a reason to consider how VGI will play out in future transport systems. The paper was published in the journal Urban Planning and the abstract reads:

“As transport systems are pushed to the limits in many cities, governments have tried to resolve problems of traffic and congestion by increasing capacity. Miller (2013) contends the need to identify new capabilities (instead of capacity) of the transport infrastructure in order to increase efficiency without extending the physical infrastructure. Kenyon and Lyons (2003) identified integrated traveller information as a facilitator for better transport decisions. Today, with further developments in the use of geographic information systems (GIS) and a greater disposition by the public to provide volunteered geographic information (VGI), the potential of information is not only integrated across modes but also user-generated, real-time and available on smartphones anywhere. This geographic information plays today an important role in sectors such as politics, businesses and entertainment, and presumably this would extend to transport in revealing people’s preferences for mobility and therefore be useful for decision-making. The widespread availability of networks and smartphones offer new opportunities supported by apps and crowdsourcing through social media such as the successful traffic and navigation app Waze, car sharing programmes such as Zipcar, and ride sharing systems such as Uber. This study aims to develop insights into the potential of governments to use voluntary (crowdsourced) geographic information effectively to achieve sustainable mobility. A review of the literature and existing technology informs this article. Further research into this area is identified and presented at the end of the paper.”

The paper is open, and can be found here

Has GIScience Lost its Interdisciplinary Mojo?

The GIScience conference is being held every two years since 2000, and it is one of the main conferences in the field of Geographic Information Science (GIScience). It is a special honour to be invited to give a keynote talk, and so I was (naturally) very pleased to get an invitation to deliver such a talk in the conference this year. The title of my talk is ‘Has GIScience Lost its Interdisciplinary Mojo?’ and I’m providing here the synopsis of the talk, with the slides.

My own career is associated with GIScience very strongly. In 1992, as I was studying for my undergraduate studies with a determination to specialise in Geographic Information Systems (GIS) by combining computer science and geography degrees, I was delighted to discover that such studies fall within a new field called GIScience. The paper by Mike Goodchild that announced the birth of the field was a clear signal that this was an area that was not only really interesting, but also one with potential for growth and prospects for an academic career, which was very encouraging. This led to me to a Masters degree which combined environmental policy, computer science, and GIS. During my PhD, I started discovering another emerging area – citizen science, with two main pieces of work – by Alan Irwin and Rick Bonney marking the beginning of the field in 1995 (I came across Irwin’s book while looking into public understanding of science, and learn about Bonney’s work much later). According to OED research, the use of citizen science can be traced to 1989. In short, GIScience and citizen science as a recognised terms for research areas have been around for about the same time – 25 years.

Over this period, I have experienced an inside track view of these two interdisciplinary research fields. I would not claim that I’ve been at the centres of influence of either fields, or that I’ve analysed the history of these areas in details, but I followed them close enough to draw parallels, and also to think – what does it mean to be involved in an interdisciplinary field and what make such a field successful? 

The use of terms in publications is a good indication to the interest in various academic fields. Here are two charts that tell you how GIScience grown until it stalled around 2010, and how citizen science have been quiet for a while but enjoying a very rapid growth now.

First, from Egenhofer et al. 2016 Contributions of GIScience over the Past Twenty Years, showing the total number of publications with the keywords GIS or GIScience, based on a Scopus query for the years 1991 through 2015, executed in July 2015. Notice the peak around 2009-2010.

gisciencepublications

And here is Google Trends graph for comparing GIScience and Citizen Science, showing that in the past 8 years citizen science has taken off and increased significantly more than GIScience:

gisciencecitizenscience

I think that it’s fair to say that these two fields as inherently interdisciplinary.

In GIScience, as Traynor a Williams identify already in 1995: “Off-the-shelf geographic information system software is hard to use unless you have sufficient knowledge of geography, cartography, and database management systems; are computer-literate” and to these observations we need to add statistics, algorithms development, and domain knowledge (ecology, hydrology, transport).

Citizen Science also includes merging knowledge from public engagement, education, science outreach, computer science, Human-Computer Interaction, statistics, algorithms and domain knowledge (e.g. ecology, astrophysics, life science, digital humanities, archaeology).

Both fields are more than a methodology – they are contributing to scientific research on different problems in the world, and only a very reductionist view about what they are will see them as ‘a tool’. They are more complex than that – which is why we have specific scholarship about them, periods of training, dedicated courses and books, conferences and all the rest.

A very shallow comparison will note that GIScience was born as an interdisciplinary field of study, and experience consolidation and focus early on with research agendas, core curriculum which was supposed to lead to stability and growth. This did not happen (see Patrick Rickles comments, from an interdisciplinary research perspective, on this). Take any measure that you like: size of conferences, papers. Something didn’t work. Consider the Esri UC, with its 15,000 participants who are working with GIS, yet only a handful of them seem to be happy with the identity of a GIScientists.

In contrast, Citizen Science is already attracting to its conferences audience in the many hundreds – the Citizen Science Association have 4000 (free) members, The European Citizen Science Association 180 (paid) – and that is in the first 2 years since they’ve been established. It doesn’t have an explicit research agenda, and have an emerging journal, but the field also benefits from multiple special issues – there is almost a competition among them.

As a GIScientist this is a complex, and somewhat unhappy picture. What can I offer to explain it? What are the differences between the two fields that led to the changes and what we can learn from them? It is worth exploring these questions if we want the field to flourish

Engaging with Interdisciplinary research

The wider engagement with these fields is also linked to my personal and direct engagement in GIScience research that goes beyond disciplinary boundaries. Over the years, I was also involved in about 20 multidisciplinary, cross-disciplinary, interdisciplinary, and transdisciplinary projects. I also found myself evaluating and funding x-disciplinary projects (where cross, inter, multi or trans  stand for x). The main observations from all these is that many times, projects that started under the interdisciplinary flag (integrating knowledge from multiple areas), ended with mostly multidisciplinary results (each discipline addressing the issue from its own point of view). However, here are nine lessons that I’ve learned, which can also help evaluating the wider fields of GIScience and citizen science.

First, Get them young & hungry – when established professors are joining an interdisciplinary project, usually they have a clear personal research agenda, and the likelihood that they will be open to radically new ideas about their area is low. You can get excellent multidisciplinary projects with experienced researchers, but it is much harder and rarer to have interdisciplinary or transdisciplinary project – there is too much to lose. That mean that early career researchers are the most suitable collaborators who can develop new directions. At the same time, in terms of job potential and publications, it is very risky for PhD students to get into interdisciplinary research as this can reduce their chances of securing an academic job. With appropriate funding (as we done in Bridging the Gaps) and specific support to people at the more secured stage of early career (after securing a lectureship/assistant professor position), we’ve seen interdisciplinary collaboration evolve.

Second, in x-disciplinary projects, you’ll find yourself being undermined, unintentionally which will hurt. Disciplines have different notion of ‘truth’ and how to get to it (in philosophy: epistemology and ontology). What is considered as an appropriate methodology (e.g. fixation with randomised control trials), how many people need to participate, how they are selected and more. When people from another discipline use these concepts to question your practice it can feel as undermining the expertise, and the disciplinary knowledge that you are offering to the project…

logo-ercThird, there are also cases of being undermined, intentionally. Interdisciplinary proposal are evaluated by experts from different fields, and no matter how much they are told to focus their comments on their discipline, they will comment on other aspects. Moreover, proposal evaluators can assess the novelty in their area, not the overall innovation, reducing the likelihood of ‘outstanding’ mark that make it more likely to get funded. For example, in an early version of what was now funded by both EPSRC and ERC, a Research Challenges Board rejected the proposal because it “seemed so high risk to us is that there are many links in the chain… is it clear that even if everything works there would be real value from these sorts of devices? You use the example that the forest people might be able to tell if there were poachers in the area. Yet can that really be shown? Do forest people understanding probabilistic reasoning? If there any evidence that illiterate people can use maps, digital or otherwise?“. It’s important to note that both ERC and the EPSRC programmes were aimed at risky, interdisciplinary projects, but in more standard programmes, it is difficult to get funded.

Fourth, look out for the disciplinary scrounger. They might not be aware that they are disciplinary scrounger, but this is how it happens: Interdisciplinary research open up new tools and methodologies and people who know how to use them for the research team as a whole. While there is a supposed shared goals that will provide benefits to all sides, a savvy researcher will identify that there is an opportunity for using resources to advance their own research in their discipline, and find ways to do that, even if there are no apparent benefits to the side that give the resources. This act is not necessarily malicious – from the researcher perspective, it is exactly a demonstration of interdisciplinary contribution.

Fifth, in an interdisciplinary research it is critical to develop a common narrative, early. As the project progresses, it will shift and change. Because of the disciplinary differences, it is very easy to diverge and work on different issues, with some relationship to the original proposal. Especially in case where the funder evaluate the project against the proposal (e.g. in Horizon 2020), it’s critical to have a common story. The project can be harmonious and show good progression, but without a common narrative that is shared across the team, there can be troubles when it come to evaluation by external people as the outputs do not all fit neatly to their idea of what the project is about. In another project, Adaptable Suburbs, we deliberately shared reading lists between teams to help understanding each other, which bring us to…

Sixth, highstreetconsider the in-built misunderstanding. Terminology is an obvious one. For Anthropology, scale, from small to large is individual, household, community – and for cartography city is small scale, while house is large scale. However, these are easy – it can take time, and long discussions to discover that you’re looking at the same thing but seeing something completely different. As Kate Jones suggested when she worked on the Successful Suburban Town Centres project. In the image above urban designers see the streets, but not the people, while human geographers who look at census data will tend to see the people, but not the urban structure that they inhibit. There are many other examples of subtle, complex and frustrating misunderstanding that happen in such projects.

Seven, there will be challenges with publications – those that are written. Publications are critical academic outputs, and important for the individuals, teams, and the project as a whole. Yet, they are never easy – different disciplines have very different practices. In some, the first position in the author list is the most important, in another, the last. Some value single author monograph (Anthropology), other conference paper with multiple authors (Computer Science). This creates tensions and a need for delicate discussions and agreement. Moreover, and linked to Six – writing joint publications is an opportunity to expose interdisciplinary misunderstanding, but that make the writing process longer.

Eight, it is important to realise that many times interdisciplinary publications will never be written  – because academic careers, promotion criteria, visibility, and recognition depends on disciplinary practices, within projects disciplinary papers and outputs are written first. The interdisciplinary outputs left to a later stage – and then the project end and they never get written. They are actually dependent on voluntary investment of multiple contributors, which make it very difficult to get them done!

Finally, nine, is the importance of coffee and lunch breaks (and going out together). Team members in interdisciplinary projects are usually coming from different departments, and it is challenging to organise a shared space. However, by putting people together – computer scientists sitting next to a geographer, designer, anthropologists – it is possible to achieve the level of trust, relationship and the development of new ideas that are needed in such projects. In ExCiteS, we have a designated ‘social officer’ for the group.

On the basis of these experiences, I’d argue that Interdisciplinarity is always hard, risky, require compromises, accommodations, listening, and making mistakes. The excitement from the outputs and outcomes does not always justify the price. Frequently, there is no follow-on project – it’s been too exhausting. The analysis that Patrick Rickles done across the literature can provide you with further information on challenges and solutions.

From projects to research fields

Considering the project level challenges, viewing interdisciplinary areas of studies emerging is especially interesting. You can notice how concepts are being argued and agreed on. You can see what is inside and what is outside, and where the boundary is drawn. You can see how methodologies, jargon, acceptable behaviour, and modes of operations get accepted or rejected – and from the inside, you can nudge the field and sometimes see the impact of your actions. Here are some observations about GIScience and citizen science evolution.

First, citizen science seem to be growing organically, without a deliberate attempt to set a research agenda, define core curriculum, or start with nationally focused research centres, in contrast to GIScience, who had all of these. There is an emergent research agenda: data quality, motivations & incentives, interaction design, management of volunteers, and more. These are created according to views of different people who join the research area, opening opportunities for new collaborations. It is noted that GIScience, in practice, allowed for many other areas to emerge – for example crowdsourcing, which was not in the last version of the research priorities that are listed on UCGIS website, and also seemed to stop doing these exercises.

Second, there is an interesting difference in inclusiveness. Although there are different variants of citizen science, across events, conferences and projects, there is an attempt to be inclusive to the different variants (e.g. volunteer computing or ecological observations) though tensions remain and need maintenance. In GIScience, there have been inclusive activities, of workshops that brought together people from Human-Computer Interaction in the late 1980s, or the excellent series of meetings about GIS and Environmental Modelling. There is clear separation, for example in spatial analysis, where different methods are now appearing in ecology, but they are not shared back with the general GIScience. It is worth considering how to make such events and consider active inclusiveness, where researchers from different areas will find their place and reasons to participate.

It might be that citizen science is also more inclusive because of the interaction with people outside academia (participants) and the need to focus on things that matter to them, whereas GIScience has largely been for/by scientists. However, citizen science gets backlash for “not doing REAL science”, but it’s still grown. Maybe, in the process of GIScience trying to validate itself, it’s cut itself off from other research areas (even though GIS use continues to grow)?

Third, there is a sharp difference in the relationship with practitioners – GIScience decided to focus on fundamental questions and laws, while citizen science is a deliberate integration between researchers (the science of citizen science) and practitioners who are running volunteering programmes. The interaction between practice and science is bringing research questions to light and provide a motivation for addressing them with interdisciplinary teams. It might be that separation between science and systems in GIScience need to be blurred a bit to open up new opportunities.
bookcoverFinally, GIScience benefited from having a disciplinary name, and attention by a growing group of researchers who are committed to the field – job titles, positions, journals and conference do matter in terms of visibility and recognition. Citizen science, on the other hand, is only now starting to have a proper home and networks. There are ongoing discussions about what it is, and not everyone in the field is using the term ‘citizen science’ or happy with it. The actual conference that led to the creation of the Citizen Science Association was titled ‘Public Participation in Scientific Research'(!). The coherence and focus on understanding how important key phrases are, more than dislike of their potential meaning is valuable for the coherence of a field and stating that you have knowledge that can be shared with others.

New areas for Interdisciplinary research

To complete this discussion, I point to the opportunities that citizen science open for interdisciplinary collaborations with GIScience – It provides examples for longevity of VGI data sources, that can be used to address different research questions. There are new questions about scales of operations and use of data from the hyper local to the global. Citizen science offer challenging datasets (complexity, ontology, heterogeneity), and also a way to address critical issues (climate change, biodiversity loss). There are also usability challenges and societal aspects.

In final account, GIScience got plenty of interdisciplinary activity in it. There are actually plenty of examples for it. In terms of ‘mojo’ as being attractive for researchers from other area to join in, there are plenty of opportunities – especially if the practice of using GIS within different research and practice problems is included in the framework of GIScience.


This post benefited from discussions and comments from Patrick Rickles, who is our local expert in GIS use in an interdisciplinary settings. You should check his work.

Published: Why is Participation Inequality Important?

bookcoverI’ve mentioned the European Handbook for Crowdsourced Geographic Information in the last post, and explained how it came about. My contribution to the book is a chapter titled ‘Why is Participation Inequality Important?. The issue of participation inequality, also known as the 90:9:1 rule, or skewed contribution, has captured my interest for a while now. I have also explored it in my talk at the ECSA conference on ‘participatory [citizen] science‘ and elsewhere.

In this fairly short chapter what I am trying to communicate is that while we know that participation inequality is happening and part of crowdsourced information, we need to consider how it influences issues such as data quality, and think how it come about. I am trying to make suggest how we ended with skewed contributions – after all, at the beginnings of most projects, everyone are at the same level – zero contribution, and then participation inequality emerge.

I have used the iconic graph of contribution to OpenStreetMap that Harry Wood created, but the chapter is discussing other projects and activities where you can come across this phenomena.

Here is a direct link to the chapter, and I’ll be very happy to hear comments about it!

 

New book: European Handbook of Crowdsourced Geographic Information

COST EnergicCOST ENERGIC is a network of researchers across Europe (and beyond) that are interested in research crowdsourced geographic information, also known as Volunteered Geographic Information (VGI). The acronym stands for ‘Co-Operation in Science & Technology’ (COST) through ‘European Network Researching Geographic Information Crowdsourcing’ (ENREGIC). I have written about this programme before, through events such as twitter chats, meetings, summer schools and publications. We started our activities in December 2012, and now, 4 years later, the funding is coming to an end.

bookcoverOne of the major outcomes of the COST ENERGIC network is an edited book that is dedicated to the research on VGI, and we have decided that following the openness of the field, in which many researchers use open sources to analyse locations, places, and movement, we should have the publication as open access – free to download and reuse. To achieve that, we’ve approached Ubiquity Press, who specialise in open access academic publishing, and set a process of organising the writing of short and accessible chapters from across the spectrum of research interests and topics that are covered by members of the network. Dr Haosheng Huang (TU Wien) volunteered to assist with the editing and management of the process. The chapters then went through internal peer review, and another cycle of peer review following Ubiquity Press own process, so it is thoroughly checked!

The book includes 31 chapters with relevant information about application of VGI and citizen science, management of data, examples of projects, and high level concepts in this area.

The book is now available for download hereHere is the description of the book:

This book focuses on the study of the remarkable new source of geographic information that has become available in the form of user-generated content accessible over the Internet through mobile and Web applications. The exploitation, integration and application of these sources, termed volunteered geographic information (VGI) or crowdsourced geographic information (CGI), offer scientists an unprecedented opportunity to conduct research on a variety of topics at multiple scales and for diversified objectives.
The Handbook is organized in five parts, addressing the fundamental questions:

  • What motivates citizens to provide such information in the public domain, and what factors govern/predict its validity?
  • What methods might be used to validate such information?
  • Can VGI be framed within the larger domain of sensor networks, in which inert and static sensors are replaced or combined by intelligent and mobile humans equipped with sensing devices?
  • What limitations are imposed on VGI by differential access to broadband Internet, mobile phones, and other communication technologies, and by concerns over privacy?
  • How do VGI and crowdsourcing enable innovation applications to benefit human society?

Chapters examine how crowdsourcing techniques and methods, and the VGI phenomenon, have motivated a multidisciplinary research community to identify both fields of applications and quality criteria depending on the use of VGI. Besides harvesting tools and storage of these data, research has paid remarkable attention to these information resources, in an age when information and participation is one of the most important drivers of development.
The collection opens questions and points to new research directions in addition to the findings that each of the authors demonstrates. Despite rapid progress in VGI research, this Handbook also shows that there are technical, social, political and methodological challenges that require further studies and research

 

New paper: Digital engagement methods for earthquake and fire preparedness

At the beginning of the Challenging Risk project, the project team considered that before we go out and develop participatory tools to engage communities in earthquake and fire preparedness, we should check what is available.

To achieve that, we have commissioned Enrica Verrucci to help us with the review, and later on other members of the team updated the information – including Patrick Rickles, David Rush, and Gretchen Fagg. We then thought about the development of a paper from the review, and an interesting interdisciplinary discussion ensue, with different potential emphasis and structures were suggested. It took us several iterations until we’ve agreed that the best way to communicate the purpose of the review is by linking the use of digital technologies to behaviour change, with the guidance of Gabriela Perez-Fuentes and Helene Joffe who are the psychological experts on the team.

The resulting paper have just been published in Natural Hazards. It is the first paper from the project that covers all the groups that are involved in the project. Here is the abstract:

“Natural or human-made hazards may occur at any time. Although one might assume that individuals plan in advance for such potentially damaging events, the existing literature indicates that most communities remain inadequately prepared. In the past, research in this area has focused on identifying the most effective ways to communicate risk and elicit preparedness by means of public hazard education campaigns and risk communication programmes. Today, web- and mobile-based technologies are offering new and far-reaching means to inform communities on how to prepare for or cope with extreme events, thus significantly contributing to community preparedness. Nonetheless, their practical efficacy in encouraging proactive hazard preparedness behaviours is not yet proven. Building on behaviour change interventions in the health field and looking in particular at earthquakes and fire hazards, the challenging RISK team has reviewed the currently active websites, Web, and mobile applications that provide information about earthquake and home fire preparedness. The review investigates the type of information provided, the modality of delivery, and the presence of behaviour change techniques in their design. The study proves that most of the digital resources focus on a single hazard and fail to provide context-sensitive information that targets specific groups of users. Furthermore, behaviour change techniques are rarely implemented in the design of these applications and their efficacy is rarely systematically evaluated. Recommendations for improving the design of Web- and mobile-based technologies are made so as to increase their effectiveness and uptake for a multi-hazard approach to earthquake and home fire preparedness.”

You can find the paper (which is Open Access) on the journal’s website: http://link.springer.com/article/10.1007/s11069-016-2378-x?view=classic

You can find the list of websites and apps here.

Participatory [Citizen] Science

Citizen Science as Participatory Science‘ is one of the most popular posts that I have published here. The post is the core section of a chapter that was published in 2013 (the post itself was written in 2011). For the first European Citizen Science Association conference I was asked to give a keynote on the second day of the conference, which I have titled ‘Participatory Citizen Science‘, to match the overall theme of the conference, which is  ‘Citizen Science – Innovation in Open Science, Society and Policy’. The abstract of the talk:

In the inaugural ECSA conference, we are exploring the intersection of innovation, open science, policy and society and the ways in which we can established new collaborations for a common good. The terms participation and inclusion are especially important if we want to fulfil the high expectations from citizen science, as a harbinger of open science. In the talk, the conditions for participatory citizen science will be explored – the potential audience of different areas and activities of citizen science, and the theoretical frameworks, methodologies and techniques that can be used to make citizen science more participatory. The challenges of participation include designing projects and activities that fit with participants’ daily life and practices, their interests, skills, as well as the resources that they have, self-believes and more. Using lessons from EU FP7 projects such as EveryAware, Citizen Cyberlab, and UK EPSRC projects Extreme Citizen Science, and Street Mobility, the boundaries of participatory citizen science will be charted.

As always, there is a gap between the abstract and the talk itself – as I started exploring the issues of participatory citizen science, some questions about the nature of participation came up, and I was trying to discuss them. Here are the slides:

After opening with acknowledgement to the people who work with us (and funded us), the talk turn the core issue – the term participation.

https://www.google.co.uk/search?q=sherry+and+george+arnstein
Sherry Arnstein with Harry S Truman (image by George Arnstein)

Type ‘participation’ into Google Scholar, and the top paper, with over 11,000 citations, is Sherry Rubin Arnstein’s ‘A ladder of citizen participation’. In her ladder, Sherry offered 8 levels of participation – from manipulation to citizen control. Her focus was on political power and the ability of the people who are impacted by the decisions to participate and influence them. Knowingly simplified, the ladder focus on political power relationships, and it might be this simple presentation and structure that explains its lasting influence.

Since its emergence, other researchers developed versions of participation ladders – for example Wiedmann and Femers (1993), here from a talk I gave in 2011:

These ladders come with baggage: a strong value judgement that the top is good, and the bottom is minimal (in the version above) or worse (in Arnstein’s version). The WeGovNow! Project is part of the range of ongoing activities of using digital tools to increase participation and move between rungs in these concept of participation, with an inherent assumption about the importance of high engagement.

Levels of Citizen Science 2011
Levels of Citizen Science 2011

At the beginning of 2011, I found myself creating a ladder of my own. Influenced by the ladders that I learned from, the ‘levels of citizen science’ make an implicit value judgement in which ‘extreme’ at the top is better than crowdsourcing. However, the more I’ve learned about citizen science, and had time to reflect on what participation mean and who should participate and how, I feel that this strong value judgement is wrong and a simple ladder can’t capture the nature of participation in Citizen Science.

There are two characteristics that demonstrate the complexity of participation particularly well: the levels of education of participants in citizen science activities, and the way participation inequality (AKA 90-9-1 rule) shape the time and effort investment of participants in citizen science activities.

We can look at them in turns, by examining citizen science projects against the general population. We start with levels of education – Across the EU28 countries, we are now approaching 27% of the population with tertiary education (university). There is wide variability, with the UK at 37.6%, France at 30.4%, Germany 23.8%, Italy 15.5%, and Romania 15%. This is part of a global trend – with about 200 million students studying in tertiary education across the world, of which about 2.5 million (about 1.25%) studying to a doctoral level.

However, if we look at citizen science project, we see a different picture: in OpenStreetMap, 78% of participants hold tertiary education, with 8% holding doctoral level degrees. In Galaxy Zoo, 65% of participants with tertiary education and 10% with doctoral level degrees. In Transcribe Bentham (TB), 97% of participants have tertiary education and 24% hold doctoral level degrees. What we see here is much more participation with people with higher degrees – well above their expected rate in the general population.

The second aspect, Participation inequality, have been observed in OpenStreetMap volunteer mapping activities, iSpot – in both the community of those who capture information and those that help classify the species, and even in an offline conservation volunteering activities of the Trust for Conservation Volunteers. In short, it is very persistent aspect of citizen science activities.

For the sake of the analysis, lets think of look at citizen science projects that require high skills from participants and significant engagement (like TB), those that require high skills but not necessarily a demanding participation (as many Zooniverse project do), and then the low skills/high engagement project (e.g. our work with non-literate groups), and finally low skills/low engagement projects. There are clear benefits for participation in each and every block of this classification:

high skills/high engagement: These provide provide a way to include highly valuable effort with the participants acting as virtual research assistants. There is a significant time investment by them, and opportunities for deeper engagement (writing papers, analysis)

high skills/low engagement: The high skills might contribute to data quality, and allow the use of disciplinary jargon, with opportunities for lighter or deeper engagement to match time/effort constraints

low skills/high engagement: Such activities are providing an opportunity for education, awareness raising, increased science capital, and other skills. They require support and facilitation but can show high potential for inclusiveness.

low skills/low engagement: Here we have an opportunity for active engagement with science with limited effort, there is also a potential for family/Cross-generational activities, and outreach to marginalised groups (as OPen Air Laboratories done)

In short – in each type of project, there are important societal benefits for participation, and it’s not only the ‘full inclusion at the deep level’ that we should focus on.

Interestingly, across these projects and levels, people are motivated by science as a joint human activity of creating knowledge that is shared.

So what can we say about participation in citizen science – well, it’s complex. There are cases where the effort is exploited, and we should guard against that, but outside these cases, the rest is much more complex picture.

The talk move on to suggest a model of allowing people to adjust their participation in citizen science through an ‘escalator’ that we are aiming to conceptually develop in DITOs.

Finally, with this understanding of participation, we can understand better the link to open science, open access and the need of participants to potentially analyse the information.

Securing funding and balancing efforts: a tale of 21 research applications

EU H2020 Participants Portal
EU H2020 Participants Portal

The last 3 months were a gradual sigh of relief for the Extreme Citizen Science group (ExCites), Mapping for Change (MfC), and for me. As the UCL engineering website announced, the ExCiteS group, which I co-direct, secured funding through 3 research grants from the European Union’s Horizon 2020 programme (H2020), with enough funding to continue our work for the next 3 years, which is excellent. As usual in publicity, UCL celebrates successes, not the work that led to it. However, there are implications for the effort of securing funding and it is worth reflecting on them – despite the fact that we are in the success camp. While the criticism of the application process to European projects on the ROARS website is a bit exaggerated, it does give a good context for this post. In what follows I cover the context for the need to apply for funding, look at the efforts, successes and failures from mid 2014 to early 2016 (mostly failures), and then look at the implications. 

This is not a piece to boast about success or moan about failure, but I find writing as a useful way to reflect, and I wanted to take stock of the research application process. I hope that it will help in communicating what is the process of securing funding for an interdisciplinary, research intensive group.

Background & context 

The background is that the ExCiteS group started at the end of 2011, with a large group of PhD students – as common to early stage research groups. With the support of the UK Engineering and Physical Science Research Council (EPSRC) award, which is about to end soon, it was possible to start a group. With additional funding from the European Union (EU) and EPSRC projects, including EveryAware (2011-2014), Citizen Cyberlab (2012-2015), Challenging Risk (2013-2018), and Cap4Access (2014-2016), it was possible to develop the activities of ExCiteS & MfC. This is evident in the software packages that are emerging from our work: Sapelli, GeoKey, and a new version of Community Maps, methodologies for using these tools within participatory processes. the academic and non-academic outputs, and the fact that people know about our work.

However, what was clear since 2011 was that 2015 will be a crunch point, when we will need funding to allow members of the group to move from PhD students to post-doctoral researchers (postdocs). The financial implication of funding a postdoc is about three-times the funding for a PhD student. In addition, while at earlier years members of the group (regardless of their career stage) participated in writing research proposals – and helped winning them (e.g. Citizen Cyberlab), when people write-up their PhD theses it is inappropriate to expect them to invest significant amount of time in research applications. Finally, all our funding come through research projects – we don’t have other sources of income.

Research Applications – effort, successes, failures 

UK Research Councils system (Je-S)
UK Research Councils system (Je-S)

So it was very clear that 2015 is going to be full of research applications. To give an idea of how many and the work that was involved, I’m listing them here – more or less in order of effort. I’m providing more details on successful applications but only partial on the failed ones – mostly because I didn’t check with the coordinators or the partners to see if they allow me to do so.

We started in mid 2014, when we started working on the first version of what is now DITOs. Coordinating an EU H2020 project proposal with 11 partners mean that between May and September 2014 we’ve invested an estimated 6 or 7 person months within the group in preparing it. We’ve submitted it early October, only to be disappointed in early March 2015 when we heard that although we scored high (13/15), we won’t be funded – only 1 project out of 19 that applied was funded. We then resurrected the proposal in July 2015, dedicated further 5 person months, resubmitted it and won funding after competition with 56 other proposals – of which only 2 were funded.

The next major investment was into a first stage proposal to the Citizen Observatories call of H2020. ExCiteS coordinated one proposal, and MfC participated in another. The process required an outline submission and then a full proposal. We worked on the proposal from December 2014 to April 2015, and it wasn’t a huge surprise to discover that 47 proposals were submitted to the first stage, of which 11 progressed to the second. The one coordinated by ExCiteS, with an investment of about 5 person months, scored 7/10, so didn’t progressed to the second stage. MfC also invested 2.5 person months in another proposal, as a partner. This proposal passed the first stage, but failed in the second.

Participating as a major partner in a proposal is also a significant effort, especially in H2020 projects in which there are multiple partners. The collaborative effort of MfC and ExCiteS in the proposal that emerged at WeGovNow! required about 4 person months. The proposal was submitted twice – first in July 2015 to a call for “Collective Awareness Platforms for Sustainability and Social Innovation” which received 193 proposals of which 22 were funded, and then again to a call for “Meeting new societal needs by using emerging technologies in the public sector” where only 2 proposals were submitted in December 2015 (you can be lucky sometimes!).

The proposal for the European Research Council (ERC) was developed between May and June 2015, with about 3 person months – and luckily was successful. It competed with 1953 applications in total (423 in the social sciences), of which 277 (59) were successful – about 14% success rate.

Another fellowship proposal in response to an EPSRC call passed the first round, and failed at the interview stage (where 2 out of 5 candidates were selected). This one was developed from May 2015 and failed in February 2016, after an effort of about 2.5 person months.

We also developed the Economic and Social Science Research Council (ESRC) responsive mode proposal, which mean that we’ve applied to the general funds, and not to a specific call. We collaborated with colleagues at the Institute of Education from January 2015 to July 2015 , with an effort of about 2.5 person months, but we learned that it was unsuccessful in March 2016.

Another 2 person months were dedicated to an ESRC call for methodological research, for which 65 applications were submitted out of which 6 were funded, with our proposal ranking 22 out of about 65. In parallel, I had a small part in another proposal for the same call, which was ranked 56.

We’ve invested a month in an unsuccessful application to Wellcome Trust Science Learning + call in July 2014.

Less time was spent on proposals where we had a smaller role – a failed H2020 ICT proposal in April 2015, or another H2020 about Integrating Society in Science and Innovation September 2015. This also include a successful proposal to the Climate and Development Knowledge Network (CDKN).  Because of all the other proposals, information such as the description of our activities, CVs and other bits were ready and adjusted quite easily.

ExCiteS and MfC also participated in an EU LIFE proposal – this was for funding for applied activities, with very low level of funding of only 50%, so there was a need to think carefully about which add-on activities can be used for it. However, as the proposal failed, it wasn’t an issue.

Along the way, there were also small parts in an application to the Wellcome trust in early 2015 (failed), in an EPSRC programme grant (a large grant with many partners) that was organised at UCL and on which we dedicated time from June 2014 to February 2015 (ditto), an outline for Leverhulme trust (ditto), an ERC research proposal (ditto), and finally  a COST Action application for a research network on Citizen Science (which was successful!)

So let’s summarise all these proposals, success, failure, effort in one table. Lines where the funder marked in bold mean that we’ve coordinated the proposal:

Funder Effort (Months) Success/Failure
1 H2020 7 Failure
2 H2020 5 Success
3 H2020 5 Failure
4 H2020 2.5 Failure
5 H2020 4 Failure
6 H2020 1 Success
7 ERC 3 Success
8 EPSRC 2.5 Failure
9 ESRC 2.5 Failure
10 ESRC 2 Failure
11 ESRC 0.5 Failure
12 Wellcome 1 Failure
13 H2020 0.25 Failure
14 H2020 0.25 Failure
15 CDKN 0.25 Success
16 EU LIFE 0.5 Failure
17 Wellcome 0.5 Failure
18 EPSRC 0.5 Failure
19 Leverhulme 0.5 Failure
20 ERC 0.25 Failure
21 COST 0.5 Success

So what?

We’ve applied to lots of funders and mechanisms – fellowships, calls for proposals, and open calls for research ideas. We applied to UK funders and EU. As we are working in an interdisciplinary area, we have applied to social science as well as engineering, Information and Communication Technologies (ICT) and in between these areas. In third of the case we led the proposal, but in the rest we joined in to proposals that were set by others. So the first point to notice is that we didn’t fixate on one source, mechanism or role.

As the table shows, we’re not doing badly. Out of the 7 proposals that we’ve led, 2 succeeded (30%). Also among the 14 that we’ve participated in 3 succeeded (20%). The overall success is about quarter. Over about 18 months a group of about 10 people invested circa 40 person months in securing future funding (about 20% of the time) for the next 3 years, which doesn’t sound excessive.

However, the load was not spread equally, so some people spent a very significant amount of their time on proposals. I was involved in almost all of these 21 proposals during this period, much more in those that we led, and in some of those that we participated as partner, I was the only person in the group that worked on the proposal. It was increasingly challenging to keep submitting and working on proposals with so much uncertainty and the very long gap between submission and results (look above, and you’ll see that it can be up to 9 months). Because of the uncertainty about success, and an assumption that only 20% will be successful at best (that’s 4 wasted proposals for every successful one), I felt that I need to keep on going, but there were moment when I thought that it’s a doomed effort.

There is also the issue of morale – as should be obvious from the fact that we’ve announced the successes recently, as the failures mounted up during the second part of 2015, it was harder to be cheerful. Because of the long gap between proposal submission and result that I mentioned, the future of the group is unknown for a significant period, and that influences decisions by people about staying or leaving, or how to use the funds that we do have.

Implications

Leaving aside that by early 2016 it became hard to find the energy to be involved in more proposal writing, there is an issue about how interdisciplinary research groups are funded. While we can apply to more funding opportunities, the responses from the failures indicated that it’s tough to convince disciplinary evaluators that the work that is being done is important. This mean that we knew all along that we need to apply more. Maybe it was a coincident, but the EU funding evaluations seem more open to these ideas than UK funders.

Second, such a high number of applications take time from other research activities (e.g. check my publications in 2014-2015). Applications, with all the efforts that is associated with them, are not seen as an academic output, so all the effort of writing the text, proofing it and revising it are frequently wasted when a proposal fail.

Third, all these proposals burn social capital, ‘business capital’, and cash reserves – e.g. having a consultant to help with H2020 project or covering the costs of meetings, asking for letter of support from business partners, raising hopes and making links with partners only to write at the end that we won’t be working together beyond the proposal. There are also negotiations with the Head of Department on the level of support from the university, requests for help from research facilitators, financial administrators and other people at the university.

Fourth, considering how much effort, experience, support – and luck – is needed to secure research funding, I’m not surprise that some people are so despondent about their chances to do so, but all the above is the result of a large team and I would argue that the clue to the ability to keeping the stamina is team spirit and having a clear goal on why the hell you want the funding in the first place (in our case, we want to materialise Extreme Citizen Science).

Finally, looking at the number of the submissions, the ranking and the general success rate of applications in the areas that we’ve applied to (about 15% or less), I have concerns that under such conditions there is a ‘crowding out’ situation, in which groups that got better resources around them (e.g. the institutional infrastructure at UCL, our internal experience) make it harder for new entrants or smaller groups. At a higher funding rate, we could have secured the funding in less proposals, at which point we wouldn’t continue to apply, and therefore allow others to secure funding.

Epilogue

I have no plans for another period like the one that led to the current results. I am incredibly grateful to have such a level of success, which is about the institution that I’m in, the hard work and the evolving experience in preparing proposals and, always, luck. It is very possible that this post would have counted 19 failures, so we’re very grateful to all the people who evaluated out proposals positively and gave us the funding.

Back to the funding, with all the successes, in people terms, we’ve secured funding for the 10 people that I’ve mentioned for 3 years, with further 6 PhD students joining us over that period. There are still other people in the group that will need funding soon, so probably we will put the accumulated knowledge and experience to use soon.