Citizen Science & Scientific Crowdsourcing – week 5 – Data quality

This week, in the “Introduction to Citizen Science & Scientific Crowdsourcing“, our focus was on data management, to complete the first part of the course (the second part starts in a week’s time since we have a mid-term “Reading Week” at UCL).

The part that I’ve enjoyed most in developing was the segment that addresses the data quality concerns that are frequently raised about citizen science and geographic crowdsourcing. Here are the slides from this segment, and below them a rationale for the content and detailed notes

I’ve written a lot on this blog about data quality and in many talks that I gave about citizen science and crowdsourced geographic information, the question about data quality is the first one to come up. It is a valid question, and it had led to useful research – for example on OpenStreetMap and I recall the early conversations, 10 years ago, during a journey to the Association for Geographic Information (AGI) conference about the quality and the longevity potential of OSM.

However, when you are being asked the same question again, and again, and again, at some point, you start considering “why am I being asked this question?”. Especially when you know that it’s been over 10 years since it was demonstrated that the quality is beyond “good enough”, and that there are over 50 papers on citizen science quality. So why is the problem so persistent?

Therefore, the purpose of the segment was to explain the concerns about citizen science data quality and their origin, then to explain a core misunderstanding (that the same quality assessment methods that are used in “scarcity” conditions work in “abundance” conditions), and then cover the main approaches to ensure quality (based on my article for the international encyclopedia of geography). The aim is to equip the students with a suitable explanation on why you need to approach citizen science projects differently, and then to inform them of the available methods. Quite a lot for 10 minutes!

So here are the notes from the slides:

[Slide 1] When it comes to citizen science, it is very common to hear suggestions that the data is not good enough and that volunteers cannot collect data at a good quality, because unlike trained researchers, they don’t understand who they are – a perception that we know little about the people that are involved and therefore we don’t know about their ability. There are also perceptions that like Wikipedia, it is all a very loosely coordinate and therefore there are no strict data quality procedures. However, we know that even in the Wikipedia case that when the scientific journal Nature shown over a decade ago (2005) that Wikipedia is resulting with similar quality to Encyclopaedia Britannica, and we will see that OpenStreetMap is producing data of a similar quality to professional services.
In citizen science where sensing and data collection from instruments is included, there are also concerns over the quality of the instruments and their calibration – the ability to compare the results with high-end instruments.
The opening of the Hunter et al. paper (which offers some solutions), summarises the concerned that are raised over data

[Slide 2] Based on conversations with scientists and concerned that are appearing in the literature, there is also a cultural aspect at play which is expressed in many ways – with data quality being used as an outlet to express them. This can be similar to the concerns that were raised in the cult of the amateur (which we’ve seen in week 2 regarding the critique of crowdsourcing) to protect the position of professional scientists and to avoid the need to change practices. There are also special concerns when citizen science is connected to activism, as this seems to “politicise” science or make the data suspicious – we will see next lecture that the story is more complex. Finally, and more kindly, we can also notice that because scientists are used to top-down mechanisms, they find alternative ways of doing data collection and ensuring quality unfamiliar and untested.

[Slide 3] Against this background, it is not surprising to see that checking data quality in citizen science is a popular research topic. Caren Cooper have identified over 50 papers that compare citizen science data with those that were collected by professional – as she points: “To satisfy those who want some nitty gritty about how citizen science projects actually address data quality, here is my medium-length answer, a brief review of the technical aspects of designing and implementing citizen science to ensure the data are fit for intended uses. When it comes to crowd-driven citizen science, it makes sense to assess how those data are handled and used appropriately. Rather than question whether citizen science data quality is low or high, ask whether it is fit or unfit for a given purpose. For example, in studies of species distributions, data on presence-only will fit fewer purposes (like invasive species monitoring) than data on presence and absence, which are more powerful. Designing protocols so that citizen scientists report what they do not see can be challenging which is why some projects place special emphasize on the importance of “zero data.”
It is a misnomer that the quality of each individual data point can be assessed without context. Yet one of the most common way to examine citizen science data quality has been to compare volunteer data to those collected by trained technicians and scientists. Even a few years ago I’d noticed over 50 papers making these types of comparisons and the overwhelming evidence suggested that volunteer data are fine. And in those few instances when volunteer observations did not match those of professionals, that was evidence of poor project design. While these studies can be reassuring, they are not always necessary nor would they ever be sufficient.” (http://blogs.plos.org/citizensci/2016/12/21/quality-and-quantity-with-citizen-science/)

[Slide 4] One way to examine the issue with data quality is to think of the clash between two concepts and systems of thinking on how to address quality issue – we can consider the condition of standard scientific research conditions as ones of scarcity: limited funding, limited number of people with the necessary skills, a limited laboratory space, expensive instruments that need to be used in a very specific way – sometimes unique instruments.
The conditions of citizen science, on the other hand, are of abundance – we have a large number of participants, with multiple skills, but the cost per participant is low, they bring their own instruments, use their own time, and are also distributed in places that we usually don’t get to (backyards, across the country – we talked about it in week 2). Conditions of abundance are different and require different thinking for quality assurance.

[Slide 5] Here some of the differences. Under conditions of scarcity, it is worth investing in long training to ensure that the data collection is as good as possible the first time it is attempted since time is scarce. Also, we would try to maximise the output from each activity that our researcher carried out, and we will put procedures and standards to ensure “once & good” or even “once & best” optimisation. We can also force all the people in the study to use the same equipment and software, as this streamlines the process.
On the other hand, in abundance conditions we need to assume that people are coming with a whole range of skills and that training can be variable – some people will get trained on the activity over a long time, while to start the process we would want people to have light training and join it. We also thinking of activities differently – e.g. conceiving the data collection as micro-tasks. We might also have multiple procedures and even different ways to record information to cater for a different audience. We will also need to expect a whole range of instrumentation, with sometimes limited information about the characteristics of the instruments.
Once we understand the new condition, we can come up with appropriate data collection procedures that ensure data quality that is suitable for this context.

[Slide 6] There are multiple ways of ensuring data quality in citizen science data. Let’s briefly look at each one of these. The first 3 methods were suggested by Mike Goodchild and Lina Li in a paper from 2012.

[Slide 7] The first method for quality assurance is crowdsourcing – the use of multiple people who are carrying out the same work, in fact, doing peer review or replication of the analysis which is desirable across the sciences. As Watson and Floridi argued, using the examine of Zooniverse, the approaches that are being used in crowdsourcing give these methods a stronger claim on accuracy and scientific correct identification because they are comparing multiple observers who work independently.

[Slide 8] The social form of quality assurance is using more and less experienced participants as a way to check the information and ensure that the data is correct. This is fairly common in many areas of biodiversity observations and integrated into iSpot, but also exist in other areas, such as mapping, where some information get moderated (we’ve seen that in Google Local Guides, when a place is deleted).

[Slide 9] The geographical rules are especially relevant to information about mapping and locations. Because we know things about the nature of geography – the most obvious is land and sea in this example – we can use this knowledge to check that the information that is provided makes sense, such as this sample of two bumble bees that are recorded in OPAL in the middle of the sea. While it might be the case that someone seen them while sailing or on some other vessel, we can integrate a rule into our data management system and ask for more details when we get observations in such a location. There are many other such rules – about streams, lakes, slopes and more.

[Slide 10] The ‘domain’ approach is an extension of the geographic one, and in addition to geographical knowledge uses a specific knowledge that is relevant to the domain in which information is collected. For example, in many citizen science projects that involved collecting biological observations, there will be some body of information about species distribution both spatially and temporally. Therefore, a new observation can be tested against this knowledge, again algorithmically, and help in ensuring that new observations are accurate. If we see a monarch butterfly within the marked area, we can assume that it will not harm the dataset even if it was a mistaken identity, while an outlier (temporally, geographically, or in other characteristics) should stand out.

[Slide 11] The ‘instrumental observation’ approach removes some of the subjective aspects of data collection by a human that might make an error, and rely instead on the availability of equipment that the person is using. Because of the increase in availability of accurate-enough equipment, such as the various sensors that are integrated in smartphones, many people keep in their pockets mobile computers with the ability to collect location, direction, imagery and sound. For example, images files that are captured in smartphones include in the file the GPS coordinates and time-stamp, which for a vast majority of people are beyond their ability to manipulate. Thus, the automatic instrumental recording of information provides evidence for the quality and accuracy of the information. This is where the metadata of the information becomes very valuable as it provides the necessary evidence.

[Slide 12] Finally, the ‘process oriented’ approach bring citizen science closer to traditional industrial processes. Under this approach, the participants go through some training before collecting information, and the process of data collection or analysis is highly structured to ensure that the resulting information is of suitable quality. This can include the provision of standardised equipment, online training or instruction sheets and a structured data recording process. For example, volunteers who participate in the US Community Collaborative Rain, Hail & Snow network (CoCoRaHS) receive standardised rain gauge, instructions on how to install it and online resources to learn about data collection and reporting.

[Slide 13]  What is important to be aware of is that methods are not being used alone but in combination. The analysis by Wiggins et al. in 2011 includes a framework that includes 17 different mechanisms for ensuring data quality. It is therefore not surprising that with appropriate design, citizen science projects can provide high-quality data.

 

 

Advertisements

Citizen Science & Scientific Crowdsourcing – week 2 – Google Local Guides

The first week of the “Introduction to Citizen Science and Scientific Crowdsourcing” course was dedicated to an introduction to the field of citizen science using the history, examples and typologies to demonstrate the breadth of the field. The second week was dedicated to the second half of the course name – crowdsourcing in general, and its utilisation in scientific contexts. In the lecture, after a brief introduction to the concepts, I wanted to use a concrete example that shows a maturity in the implementation of commercial crowdsourcing. I also wanted something that is relevant to citizen science and that many parallels can be drawn from, so to learn lessons. This gave me the opportunity to use Google Local Guides as a demonstration.

My interest in Google Local Guides (GLG) come from two core aspects of it. As I pointed in OpenStreetMap studies, I’m increasingly annoyed by claims that OpenStreetMap is the largest Volunteered Geographical Information (VGI) project in the world. It’s not. I guessed that GLG was, and by digging into it, I’m fairly confident that with 50,000,000 contributors (of which most are, as usual, one-timers), Google created the largest VGI project around. The contributions are within my “distributed intelligence” and are voluntary. The second aspect that makes the project is fascinating for me is linked to a talk from 2007 in one of the early OSM conferences about the usability barriers that OSM (or more general VGI) need to cross to reach a wide group of contributors – basically about user-centred design. The design of GLG is outstanding and shows how much was learned by the Google Maps and more generally by Google about crowdsourcing. I had very little information from Google about the project (Ed Parsons gave me several helpful comments on the final slide set), but by experiencing it as a participant who can notice the design decisions and implementation, it is hugely impressive to see how VGI is being implemented professionally.

As a demonstration project, it provides examples for recruitment, nudging participants to contribute, intrinsic and extrinsic motivation, participation inequality, micro-tasks and longer tasks, incentives, basic principles of crowdsourcing such as “open call” that support flexibility, location and context aware alerts, and much more. Below is the segment from the lecture that focuses on Google Local Guides, and I hope to provide a more detailed analysis in a future post.

The rest of the lecture is available on UCLeXtend.

Caren Cooper’s Citizen Science: How Ordinary People are Changing the Face of Discovery

Today, Caren Cooper new book Citizen Science: How Ordinary People are Changing the Face of Discovery is going on sale in the UK. The book has been out in the USA for about a year, and it is a good point to review it.

The library of citizen science books is growing – there are the more literary books such as a diary of a citizen scientist, or citizen scientist, and a growing set of books that are edited collections such as Dickinson and Bonney Citizen Science: Public Participation in Environmental Research or the accessible The Rightful Place of Science: citizen science

Caren Cooper book is adding to this collection something important – a popular science book that provides an overview of the field, phenomena, and the movement of citizen science. As I was reading the book, I recognised the major challenge that she faced. Introducing citizen science is a complex issue: it happens in many areas of science that don’t always relate to each other, it got different structures and relationships between the scientists and the participants, and it can be close and personal, or remote and involving many thousands of people in online activities. In addition to this, citizen science can lead to many outcomes: improving education, contributing to a scientific project, self-empowerment and learning, addressing a local environmental problem and community cohesion, to name but a few. Packing it all into an accessible and engaging book is quite a feat.

Cooper has the experience in communicating citizen science through various blog posts that she published over the past 5 years and some of them have set the ground for this excellent book. The way she balanced the different aspects of citizen science is by taking different scientific fields as the main classification for the chapters, with 10 chapters covering different areas where citizen science have been used – from meteorology to public health. Each chapter provides both the factual information about the type of citizen science that is being used in it, as well as engaging stories and descriptions of the participants in them so we have a real and concrete image of how citizen science is being practiced.

Through the chapters, the reader is becoming familiar with the different modes and mechanisms that are being used in citizen science. For example, she uses the Brony@home project as a way to introduce volunteer computing and showing how the interactions around it can be meaningful and engaging, thus not marginalising this form of citizen science. Another example is the discussions in a later chapter on the use of Patients Like Me as a platform for citizen science, and the way that some of its experiment are challenging common medical practices in the ALS study on the impact of lithium.

One fantastic aspect of the book is the way that it respects and values all the forms of citizen science and the participants, and provide the reader with an opportunity to understand that it can come in many shapes, and she describes the difficulties and triumphs that came out from different studies, different forms of engagement, and different disciplines. She is providing a clear thread to link all these cases through the progression that she makes throughout the book from scientist-led projects (opening with Whewell tide study) and moving towards community-led studies towards the end, with examples from environmental justice campaigns. All these cases are described with warmth and humour that makes the material accessible and enjoyable to read.

Throughout the book, Cooper is making it clear that she sees citizen science as a critical part of the current landscape of science and society relationship, and she addresses some of the issues that are being argued about citizen science – for example, data quality – heads on. The book is making a strong advocacy for scientists and people who are involved in science communication to use citizen science as a way to improve the linkage between society and science.

The book is focusing, mostly, on American projects, case studies and practices – including social and cultural ones, but not to a degree that it makes it difficult to a reader from outside the US to understand. Only in a handful of cases I had to check on Wikipedia what a term or a phrase mean.

Overall, the book is engaging, enjoyable and informative. If you want an up-to-date introduction to citizen science, this book will open up the field to you. If you are working in a citizen science project or involved in developing one, you will learn new things – I did! 

 

 

 

Chapter in ‘Understanding Spatial Media’ on VGI & Citizen Science

77906_9781473949683[1]The book ‘Understanding Spatial Media‘ came out earlier this year. The project is the result of joint effort of the editors Rob Kitchin (NUI Maynooth, Ireland), Tracey P. Lauriault (Carleton University, Canada), and Matthew W. Wilson (University of Kentucky, USA).

The book is filling the need to review and explain what happened in the part 20 years, with the increase use of digital geographic information that then became widespread and can be considered as a media – something that Daniel Sui and Mike Goodchild noted in 2001. The book chapters are covering the underlying technologies, the sources of the data and media that are part of this area, and the implications – from smart cities to surveillance and privacy.

My contribution to this book is in a chapter that belong to the middle section – spatial data and spatial media – and that provides an introduction to Volunteered Geographic Information and Citizen Science. If you’re interested, you can read the chapter here.

PhD studentship in collaboration with the Ordnance Survey – identifying systematic biases in crowdsourced geographic information

Deadline 10th November 2017

UCL Department of Geography and the Ordnance Survey are inviting applications for a PhD studentship to explore the internal systematic biases in crowd-sourced geographic information datasets (also known as Volunteered Geographic Information – VGI).

The studentship provides an exciting opportunity for a student to work with Ordnance Survey on understanding the use of crowd-sourced geographic information MH DSCN0571and potentially contributing to the use of such data sources by national mapping agencies. Ordnance Survey is an active partner in its sponsored research and offers students opportunities to work on-site and to contribute to workshops and innovation within the business. In addition, the student will be part of the Extreme Citizen Science group at UCL, which is one of the leading research groups in the area of crowdsourced geographic information and the study thereof.

For more information about the project, the studentship and details how to apply, please see below:

 

Start Date: January 2018

Funding status: Applications are invited from UK and EU citizenship holders.

Funding Body: EPSRC and Ordnance Survey

Funding Details: The scholarship covers UCL student fees at the Home/EU rate and provides a stipend of £16,553 per annum tax free. Travel expenses and research equipment will also be provided to the successful candidate.

Project Description:

UCL Department of Geography and the Ordnance Survey are inviting applications for a PhD studentship to explore the internal systematic biases in crowd-sourced geographic information datasets (also known as Volunteered Geographic Information – VGI).

There has been a rapid increase in information gathered by people from all walks of life who are using connected devices with an ability to collect and share geographic information, such as GPS tracks, photographs with location information, or observations of the natural environment in citizen science projects. There is now a vast array of projects and activities that use this type of information, and each project has its own characteristics. Yet, it can be hypothesised that some of the characteristics of crowd-sourced geographic information will be systematically biased, and these biases differ between projects and data sources.

Crowd-sourced datasets will have some systematic biases that repeat across crowd-sourcing platforms. For example the impact of population density, business activity, and tourism on the places where data is available, or a weekend or seasonal bias of the temporal period of data collection. Others biases are project-specific – for example, some projects manage to attract more young men, and therefore places that are of interest to this demographic will be over-represented. One of the major obstacles that limit the use of such data sources is understanding and separating systematic and project-level biases and then developing statistical methods to evaluate their impact. In order to use such datasets to identify hidden features and patterns, there is a need to identify what are the relationships between a dataset and the world.

The aim of this research project, therefore, is to create a large collection of crowd-sourced GPS tracks and pedestrian trajectories, and use conflation techniques and advanced analytics to develop methodologies to identify and estimate the biases. Once this is done, the aim will be to identify hidden characteristics to be more confident about the patterns that are being observed.

Studentship Description

The studentship provides an exciting opportunity for a student to work with Ordnance Survey on understanding the use of crowd-sourced geographic information, and potentially contributing to the use of such data sources by national mapping agencies. Ordnance Survey is an active partner in its sponsored research and offers students opportunities to work on-site and to contribute to workshops and innovation within the business. In addition, the student will be part of the Extreme Citizen Science group at UCL, which is one of the leading research groups in the area of crowdsourced geographic information and the study thereof.

The project will run for four years and will be supervised by Prof Muki Haklay from UCL and Jeremy Morley from Ordnance Survey. Professor Muki Haklay, who is a professor in the UCL Department of Geography and who has a track record of research and publication relating to crowdsourced data management and quality. Jeremy Morley is the Chief Geospatial Scientist at Ordnance Survey, leading the long-term business research programme, and has research experience in crowd-sourced geographic information.

 Person Specification

Applicants should possess a strong bachelor’s degree (1st Class or 2:1 minimum) or Masters degree in Computer Science, Spatial statistics, Ecology, Geomatics, Geographic Information Science or a related discipline. The skills required to build the required database of case studies and the programming and analytical skills to assess biases and develop algorithms for their identification, are highly desirable. Candidates will ideally have some relevant previous research experience and should also have excellent communication and presentation skills.

The funding is provided for 4 years, and will involve spending time at the Ordnance Survey in Southampton.

Eligibility

Applications are invited from UK and EU citizens residing in UK. In particular, applicants must meet EPSRC eligibility and residency requirements found here:

http://www.epsrc.ac.uk/skills/studentships/help/eligibility/

Application Procedure

Applicants should send the following by e-mail to Judy Barrett (judy.barrett@ucl.ac.uk) and Prof Haklay (m.haklay@ucl.ac.uk):

  1. Cover letter, including a personal statement explaining your interest in the project.
  2. Examples of academic writing and outputs from past work (e.g. a dissertation or assignment)
  3. Academic transcripts
  4. A CV

Shortlisted applicants will be invited to interview during November 2017. Any incomplete applications will not be considered.

 

Cambridge Conference 2017 – The Willing Volunteer

wp-1499185668092The Cambridge Conference is an event that is held every 4 years, organised  by the Ordnance Survey, and it is a meeting of many heads of National Mapping Agencies who come together to discuss shared interests and learn from each other.

The history of the conference is available here. This year, I was asked to provide a talk about volunteered geographic information and the role of crowdsourced information in the service of national mapping bodies. As common in these conferences, I was given a title for the talk and request on the topic – this was “The Willing Volunteer –
Incorporating voluntary data into national databases” – and the description was: At present few mapping databases contain crowd sourced or voluntary data. Consider how, in the future, this will be a valuable source of data for national geospatial, cadastral and mapping agencies.

The talk itself covered 4 parts – since the conference as a whole looked at the future needs of mapping in the next 15 years, I’ve mentioned the trends that will influence crowdsourcing over this period. I’ve included both the technical and the social trends that will influence this area. I then covered few examples, and paid attention to the need to think differently about crowdsourced information (using the metaphor of scarcity/abundance as a way to explain that), then provided two insights from the “crowdsourcing geographic information in government” study that I’m currently leading. I’ve finished with few slides that demonstrate that engagement can reach out to everyone, regardless of their literacy.

Here are the slides:

Has GIScience Lost its Interdisciplinary Mojo?

The GIScience conference is being held every two years since 2000, and it is one of the main conferences in the field of Geographic Information Science (GIScience). It is a special honour to be invited to give a keynote talk, and so I was (naturally) very pleased to get an invitation to deliver such a talk in the conference this year. The title of my talk is ‘Has GIScience Lost its Interdisciplinary Mojo?’ and I’m providing here the synopsis of the talk, with the slides.

My own career is associated with GIScience very strongly. In 1992, as I was studying for my undergraduate studies with a determination to specialise in Geographic Information Systems (GIS) by combining computer science and geography degrees, I was delighted to discover that such studies fall within a new field called GIScience. The paper by Mike Goodchild that announced the birth of the field was a clear signal that this was an area that was not only really interesting, but also one with potential for growth and prospects for an academic career, which was very encouraging. This led to me to a Masters degree which combined environmental policy, computer science, and GIS. During my PhD, I started discovering another emerging area – citizen science, with two main pieces of work – by Alan Irwin and Rick Bonney marking the beginning of the field in 1995 (I came across Irwin’s book while looking into public understanding of science, and learn about Bonney’s work much later). According to OED research, the use of citizen science can be traced to 1989. In short, GIScience and citizen science as a recognised terms for research areas have been around for about the same time – 25 years.

Over this period, I have experienced an inside track view of these two interdisciplinary research fields. I would not claim that I’ve been at the centres of influence of either fields, or that I’ve analysed the history of these areas in details, but I followed them close enough to draw parallels, and also to think – what does it mean to be involved in an interdisciplinary field and what make such a field successful? 

The use of terms in publications is a good indication to the interest in various academic fields. Here are two charts that tell you how GIScience grown until it stalled around 2010, and how citizen science have been quiet for a while but enjoying a very rapid growth now.

First, from Egenhofer et al. 2016 Contributions of GIScience over the Past Twenty Years, showing the total number of publications with the keywords GIS or GIScience, based on a Scopus query for the years 1991 through 2015, executed in July 2015. Notice the peak around 2009-2010.

gisciencepublications

And here is Google Trends graph for comparing GIScience and Citizen Science, showing that in the past 8 years citizen science has taken off and increased significantly more than GIScience:

gisciencecitizenscience

I think that it’s fair to say that these two fields as inherently interdisciplinary.

In GIScience, as Traynor a Williams identify already in 1995: “Off-the-shelf geographic information system software is hard to use unless you have sufficient knowledge of geography, cartography, and database management systems; are computer-literate” and to these observations we need to add statistics, algorithms development, and domain knowledge (ecology, hydrology, transport).

Citizen Science also includes merging knowledge from public engagement, education, science outreach, computer science, Human-Computer Interaction, statistics, algorithms and domain knowledge (e.g. ecology, astrophysics, life science, digital humanities, archaeology).

Both fields are more than a methodology – they are contributing to scientific research on different problems in the world, and only a very reductionist view about what they are will see them as ‘a tool’. They are more complex than that – which is why we have specific scholarship about them, periods of training, dedicated courses and books, conferences and all the rest.

A very shallow comparison will note that GIScience was born as an interdisciplinary field of study, and experience consolidation and focus early on with research agendas, core curriculum which was supposed to lead to stability and growth. This did not happen (see Patrick Rickles comments, from an interdisciplinary research perspective, on this). Take any measure that you like: size of conferences, papers. Something didn’t work. Consider the Esri UC, with its 15,000 participants who are working with GIS, yet only a handful of them seem to be happy with the identity of a GIScientists.

In contrast, Citizen Science is already attracting to its conferences audience in the many hundreds – the Citizen Science Association have 4000 (free) members, The European Citizen Science Association 180 (paid) – and that is in the first 2 years since they’ve been established. It doesn’t have an explicit research agenda, and have an emerging journal, but the field also benefits from multiple special issues – there is almost a competition among them.

As a GIScientist this is a complex, and somewhat unhappy picture. What can I offer to explain it? What are the differences between the two fields that led to the changes and what we can learn from them? It is worth exploring these questions if we want the field to flourish

Engaging with Interdisciplinary research

The wider engagement with these fields is also linked to my personal and direct engagement in GIScience research that goes beyond disciplinary boundaries. Over the years, I was also involved in about 20 multidisciplinary, cross-disciplinary, interdisciplinary, and transdisciplinary projects. I also found myself evaluating and funding x-disciplinary projects (where cross, inter, multi or trans  stand for x). The main observations from all these is that many times, projects that started under the interdisciplinary flag (integrating knowledge from multiple areas), ended with mostly multidisciplinary results (each discipline addressing the issue from its own point of view). However, here are nine lessons that I’ve learned, which can also help evaluating the wider fields of GIScience and citizen science.

First, Get them young & hungry – when established professors are joining an interdisciplinary project, usually they have a clear personal research agenda, and the likelihood that they will be open to radically new ideas about their area is low. You can get excellent multidisciplinary projects with experienced researchers, but it is much harder and rarer to have interdisciplinary or transdisciplinary project – there is too much to lose. That mean that early career researchers are the most suitable collaborators who can develop new directions. At the same time, in terms of job potential and publications, it is very risky for PhD students to get into interdisciplinary research as this can reduce their chances of securing an academic job. With appropriate funding (as we done in Bridging the Gaps) and specific support to people at the more secured stage of early career (after securing a lectureship/assistant professor position), we’ve seen interdisciplinary collaboration evolve.

Second, in x-disciplinary projects, you’ll find yourself being undermined, unintentionally which will hurt. Disciplines have different notion of ‘truth’ and how to get to it (in philosophy: epistemology and ontology). What is considered as an appropriate methodology (e.g. fixation with randomised control trials), how many people need to participate, how they are selected and more. When people from another discipline use these concepts to question your practice it can feel as undermining the expertise, and the disciplinary knowledge that you are offering to the project…

logo-ercThird, there are also cases of being undermined, intentionally. Interdisciplinary proposal are evaluated by experts from different fields, and no matter how much they are told to focus their comments on their discipline, they will comment on other aspects. Moreover, proposal evaluators can assess the novelty in their area, not the overall innovation, reducing the likelihood of ‘outstanding’ mark that make it more likely to get funded. For example, in an early version of what was now funded by both EPSRC and ERC, a Research Challenges Board rejected the proposal because it “seemed so high risk to us is that there are many links in the chain… is it clear that even if everything works there would be real value from these sorts of devices? You use the example that the forest people might be able to tell if there were poachers in the area. Yet can that really be shown? Do forest people understanding probabilistic reasoning? If there any evidence that illiterate people can use maps, digital or otherwise?“. It’s important to note that both ERC and the EPSRC programmes were aimed at risky, interdisciplinary projects, but in more standard programmes, it is difficult to get funded.

Fourth, look out for the disciplinary scrounger. They might not be aware that they are disciplinary scrounger, but this is how it happens: Interdisciplinary research open up new tools and methodologies and people who know how to use them for the research team as a whole. While there is a supposed shared goals that will provide benefits to all sides, a savvy researcher will identify that there is an opportunity for using resources to advance their own research in their discipline, and find ways to do that, even if there are no apparent benefits to the side that give the resources. This act is not necessarily malicious – from the researcher perspective, it is exactly a demonstration of interdisciplinary contribution.

Fifth, in an interdisciplinary research it is critical to develop a common narrative, early. As the project progresses, it will shift and change. Because of the disciplinary differences, it is very easy to diverge and work on different issues, with some relationship to the original proposal. Especially in case where the funder evaluate the project against the proposal (e.g. in Horizon 2020), it’s critical to have a common story. The project can be harmonious and show good progression, but without a common narrative that is shared across the team, there can be troubles when it come to evaluation by external people as the outputs do not all fit neatly to their idea of what the project is about. In another project, Adaptable Suburbs, we deliberately shared reading lists between teams to help understanding each other, which bring us to…

Sixth, highstreetconsider the in-built misunderstanding. Terminology is an obvious one. For Anthropology, scale, from small to large is individual, household, community – and for cartography city is small scale, while house is large scale. However, these are easy – it can take time, and long discussions to discover that you’re looking at the same thing but seeing something completely different. As Kate Jones suggested when she worked on the Successful Suburban Town Centres project. In the image above urban designers see the streets, but not the people, while human geographers who look at census data will tend to see the people, but not the urban structure that they inhibit. There are many other examples of subtle, complex and frustrating misunderstanding that happen in such projects.

Seven, there will be challenges with publications – those that are written. Publications are critical academic outputs, and important for the individuals, teams, and the project as a whole. Yet, they are never easy – different disciplines have very different practices. In some, the first position in the author list is the most important, in another, the last. Some value single author monograph (Anthropology), other conference paper with multiple authors (Computer Science). This creates tensions and a need for delicate discussions and agreement. Moreover, and linked to Six – writing joint publications is an opportunity to expose interdisciplinary misunderstanding, but that make the writing process longer.

Eight, it is important to realise that many times interdisciplinary publications will never be written  – because academic careers, promotion criteria, visibility, and recognition depends on disciplinary practices, within projects disciplinary papers and outputs are written first. The interdisciplinary outputs left to a later stage – and then the project end and they never get written. They are actually dependent on voluntary investment of multiple contributors, which make it very difficult to get them done!

Finally, nine, is the importance of coffee and lunch breaks (and going out together). Team members in interdisciplinary projects are usually coming from different departments, and it is challenging to organise a shared space. However, by putting people together – computer scientists sitting next to a geographer, designer, anthropologists – it is possible to achieve the level of trust, relationship and the development of new ideas that are needed in such projects. In ExCiteS, we have a designated ‘social officer’ for the group.

On the basis of these experiences, I’d argue that Interdisciplinarity is always hard, risky, require compromises, accommodations, listening, and making mistakes. The excitement from the outputs and outcomes does not always justify the price. Frequently, there is no follow-on project – it’s been too exhausting. The analysis that Patrick Rickles done across the literature can provide you with further information on challenges and solutions.

From projects to research fields

Considering the project level challenges, viewing interdisciplinary areas of studies emerging is especially interesting. You can notice how concepts are being argued and agreed on. You can see what is inside and what is outside, and where the boundary is drawn. You can see how methodologies, jargon, acceptable behaviour, and modes of operations get accepted or rejected – and from the inside, you can nudge the field and sometimes see the impact of your actions. Here are some observations about GIScience and citizen science evolution.

First, citizen science seem to be growing organically, without a deliberate attempt to set a research agenda, define core curriculum, or start with nationally focused research centres, in contrast to GIScience, who had all of these. There is an emergent research agenda: data quality, motivations & incentives, interaction design, management of volunteers, and more. These are created according to views of different people who join the research area, opening opportunities for new collaborations. It is noted that GIScience, in practice, allowed for many other areas to emerge – for example crowdsourcing, which was not in the last version of the research priorities that are listed on UCGIS website, and also seemed to stop doing these exercises.

Second, there is an interesting difference in inclusiveness. Although there are different variants of citizen science, across events, conferences and projects, there is an attempt to be inclusive to the different variants (e.g. volunteer computing or ecological observations) though tensions remain and need maintenance. In GIScience, there have been inclusive activities, of workshops that brought together people from Human-Computer Interaction in the late 1980s, or the excellent series of meetings about GIS and Environmental Modelling. There is clear separation, for example in spatial analysis, where different methods are now appearing in ecology, but they are not shared back with the general GIScience. It is worth considering how to make such events and consider active inclusiveness, where researchers from different areas will find their place and reasons to participate.

It might be that citizen science is also more inclusive because of the interaction with people outside academia (participants) and the need to focus on things that matter to them, whereas GIScience has largely been for/by scientists. However, citizen science gets backlash for “not doing REAL science”, but it’s still grown. Maybe, in the process of GIScience trying to validate itself, it’s cut itself off from other research areas (even though GIS use continues to grow)?

Third, there is a sharp difference in the relationship with practitioners – GIScience decided to focus on fundamental questions and laws, while citizen science is a deliberate integration between researchers (the science of citizen science) and practitioners who are running volunteering programmes. The interaction between practice and science is bringing research questions to light and provide a motivation for addressing them with interdisciplinary teams. It might be that separation between science and systems in GIScience need to be blurred a bit to open up new opportunities.
bookcoverFinally, GIScience benefited from having a disciplinary name, and attention by a growing group of researchers who are committed to the field – job titles, positions, journals and conference do matter in terms of visibility and recognition. Citizen science, on the other hand, is only now starting to have a proper home and networks. There are ongoing discussions about what it is, and not everyone in the field is using the term ‘citizen science’ or happy with it. The actual conference that led to the creation of the Citizen Science Association was titled ‘Public Participation in Scientific Research'(!). The coherence and focus on understanding how important key phrases are, more than dislike of their potential meaning is valuable for the coherence of a field and stating that you have knowledge that can be shared with others.

New areas for Interdisciplinary research

To complete this discussion, I point to the opportunities that citizen science open for interdisciplinary collaborations with GIScience – It provides examples for longevity of VGI data sources, that can be used to address different research questions. There are new questions about scales of operations and use of data from the hyper local to the global. Citizen science offer challenging datasets (complexity, ontology, heterogeneity), and also a way to address critical issues (climate change, biodiversity loss). There are also usability challenges and societal aspects.

In final account, GIScience got plenty of interdisciplinary activity in it. There are actually plenty of examples for it. In terms of ‘mojo’ as being attractive for researchers from other area to join in, there are plenty of opportunities – especially if the practice of using GIS within different research and practice problems is included in the framework of GIScience.


This post benefited from discussions and comments from Patrick Rickles, who is our local expert in GIS use in an interdisciplinary settings. You should check his work.