New paper: Usability and interaction dimensions of participatory noise and ecological monitoring

The EveryAware book provided an opportunity to communicate the results of a research that Dr Charlene Jennett led, together with two Masters students: Joanne (Jo) Summerfield and Eleonora (Nora) Cognetti, with me as an additional advisor. The research was linked to the EveryAware, since Nora explored the user experience of WideNoise, the citizen science noise monitoring app that was used in the project. There is also a link to the Citizen Cyberlab project, since Jo was looking at the field experience in ecological observation, and in particular during a BioBlitz. The chapter provides a Human-Computer Interaction (HCI) perspective to the way technology is used in citizen science projects. You can download the paper here and the proper citation for the chapter is:

Jennett, C., Cognetti, E., Summerfield, J. and Haklay, M. 2017. Usability and interaction dimensions of participatory noise and ecological monitoring. In Loreto, V., Haklay, M., Hotho, A., Servedio, V.C.P, Stumme, G., Theunis, J., Tria, F. (eds.) Participatory Sensing, Opinions and Collective Awareness. Springer. pp.201-212.

The official version of the paper is on Springer site here.

New paper: Using crowdsourced imagery to detect cultural ecosystem services: a case study in South Wales, UK

Map showing the numbers of contributors for all three photo-sharing platforms across all grid units of the study area. Numbers in parentheses in the legend indicate the number of grid units in the specified range.

Gianfranco Gliozzo, who is completing his Engineering Doctorate at the Extreme Citizen Science group, written up his first case study and published it in ‘Ecology and Society’.  Cited as
Gliozzo, G., N. Pettorelli, and M. Haklay. 2016. Using crowdsourced imagery to detect cultural ecosystem services: a case study in South Wales, UK. Ecology and Society 21(3):6. http://dx.doi.org/10.5751/ES-08436-210306

The paper went through many iterations and took its time, but it is finally out. The abstract is provided below, and the paper, in open access, can be found here

The paper is exploring the role of crowdsourced imagery, and it building on some work that Vyron Antoniou done in 2010 about understanding the geographical aspects of multiple photo-sharing websites. Gianfranco is demonstrating how such information can be used to address the policy issue of assessing the cultural benefit of open and protected spaces, which is known as cultural ecosystem services.

Within ecological research and environmental management, there is currently a focus on demonstrating the links between human well-being and wildlife conservation. Within this framework, there is a clear interest in better understanding how and why people value certain places over others. We introduce a new method that measures cultural preferences by exploring the potential of multiple online georeferenced digital photograph collections. Using ecological and social considerations, our study contributes to the detection of places that provide cultural ecosystem services. The degree of appreciation of a specific place is derived from the number of people taking and sharing pictures of it. The sequence of decisions and actions taken to share a digital picture of a given place includes the effort to travel to the place, the willingness to take a picture, the decision to geolocate the picture, and the action of sharing it through the Internet. Hence, the social activity of sharing pictures leaves digital proxies of spatial preferences, with people sharing specific photos considering the depicted place not only “worth visiting” but also “worth sharing visually.” Using South Wales as a case study, we demonstrate how the proposed methodology can help identify key geographic features of high cultural value. These results highlight how the inclusion of geographical user-generated content, also known as volunteered geographic information, can be very effective in addressing some of the current priorities in conservation. Indeed, the detection of the most appreciated nonurban areas could be used for better prioritization, planning, and management.

Esri User Conference – Science Symposium

 

Esri Science Symposium

As part of the Esri User Conference, Dawn Wright, Esri Chief Scientist, organised a Science Symposium that gave an opportunity for those with interest in scientific use of Esri GIS to come together, discuss and meet.

Dawn Wright opened and mentioned that the science symposium is aimed to bring people people from different areas: hydrology, ecology or social sciences – together. The Esri science programme is evolving – and there is official science communication approach. There are different ways to support science including a sabbatical programme. Esri will launch a specific challenge for applications of data sets for students with focus on land, ocean and population. Esri will provide access to all the data that is available and the students are expected to carry out compelling analysis and communicate it. It is an activity in parallel to the global year of understanding. There are also sessions in the AGU meeting that are support by Esri staff.DSCN1690

Margaret Leinen (president, American Geophysical Union) who is working on marine and oceanography gave the main talk ‘what will be necessary to understand and protect the planet…and us?‘. Her talk was aimed at the audience in the conference – people who’s life focus is on data. What is necessary to understand the planet is data and information – it’s the first step of understanding. There are many issues of protecting and understanding the planet – we need to understand planetary impacts on us. The first example is the way we changed our understanding of climate change on the ocean. When we look at the change in sea surface temperature in the 1990 we can see changes up to 2 degrees F. The data was mostly collected in traditional means – measurements along the paths of ships. Through studies from ship records over the years, we have created a view of ocean heating – with different results between groups and researchers with lots of hand crafted compilation of records. In the last decade things have changed: ARGO floats are going up and down through ocean, and make all the data is available – there are 3839 operational floats, reporting every week. This is a completely new way or seeing the data, with huge scale international collaboration. Now we can see the annual cycle and determined the slope in the change in heat content. We have a 10 years time series for the depth of 0-2000m. We have a much more detailed information of the changes. There is an approach to make these devices that will understand the full planetary budget on heat through the whole depth of the ocean. The EarthScope Facilities also provide a demonstration of detailed sensing data – understanding the Earth and it’s movements. Many seismometers that are used for over a decade – the US array provided a massive increase in the resolution of seismic measurements. In 2011, the network identified the Japanese Honshu earthquake. The measurement provided a new class of earthquake modelling that can be used in engineering and science. GPS also provides new abilities to understand deformation o earth. Permanent GPS receivers – many of them – can provide the resolution and accuracy to notice subtle movement, by using very sophisticated statistical filtering. HPWREN – High Performance Wireless Research and Education Network – provide a way to transfer information from sensors who are very remote, then then linked through line of sight communication, and the network provide a reliable and resilient public safety network. The network support many sensing options. There are fire cameras that are linked to it, that alert to provide real time information to the fire department. WiFire is a programme that aim to deliberately work on this issues. GIS data is used to assess surface fuel. In summary: Earth science is going through huge transformation through collaboration of large groups of researchers who are using dense sensing networks. We can now monitor different processes – from short to long term. We gain new insights, and it is rapidly transform into local, regional, national and global responses.

After her talk, a set of responses was organised from a panel, including: Mike Goodchild, John Wilson, Marco Paniho , Ming Tsou, and Cyrus Shahabi.

John: discussion about GIScience – the examples that we’ve seen point to future challenges. We can train people in the spatial sciences, and insist that they’ll learn another area, or change the earth sciences, so people learn about spatial issues, or somewhere in between, with people becoming aware of each other language. Spatial scientists have little capacity to learn a new areas – and same is true for earth scientists. The only viable path is to work together – it’s about working in interdisciplinary teams and enabling people to work with them. Data acquisition is moving fast and it is a challenge to train graduates in this area. Only recently we start thinking about solutions. Academics are experts in dealing with problems in the world, and instead we need to suggest solutions and test them.

Marco: the principle and ideas are problems that are familiar in GIScience although the specific domain of the problem was not familiar. Issues of resolution and scale are familiar in GIScience. We have a long way to go in terms of details of describing a phenomena. We need to see how systematic are we now in acquiring data? We need details of the maps of the heating of the ocean, and understanding what is going on. What is the role of remote sensing in helping us in monitoring global phenomena? We need to think about down-scaling – get from aggregate data to more detailed understanding something locally. What is the role of citizens in providing highly local information on phenomena?

Ming: we need to remembers about ‘how to lie with maps?’ – we need to be very careful about visualisations and cartographic visualisation. Each map is using projections, cartographic representation, and we need to think if it is the appropriate way to ask if that is the appropriate way to present the information? How can we deliver meaningful animation. Cartography is changing fast, but today we need to look at 2000-5000 scale, but we are using now levels and not scale. The networks and models of wildfire are raising questions about which model is appropriate, how many variables we need and which sources of information, as well as the speed of the modelling. Need to think which model is appropriately used.

Cyrius: there are more and more sensors in different context, and with machine learning we have an increased ability to monitor cities. In case of existing models – we have cases of using more data analysis in computer science.

Margaret: we have new ability to move from data, model, analysis and keep the cycle going. In the past, there was gulf between modelling or observations, we don’t see a divide any more and see people going between the modelling and the data.

Discussion points: We need to consider what is the messages that we want to communicate in our maps – we need to embrace other disciplines in improving communication. We need to implement solutions – how much uncertainty you are willing to accept. Every single map or set of data is open and other people can look and change it – this is a profound change.

The earth system is an interrelated system – but we tend to look at specific variables, but data is coming in different resolutions, and details that make it difficult to integrate. Spatial statistics is the way to carry out such integration, the question is how do we achieve that.

It’s not enough to have data as open but the issue is how to allow people to use it – issues of metadata, making it able to talk with other data sets. Esri provide a mechanism to share and address the data.

There is uncomfortable relationships between science and policy – the better the models, there is more complex the issue of discussing them with the public. How to translate decimal points to adjectives for policy making. This creates an issue to communicate with the public and policy makers. There is a need to educate scientists to be able to communicate with the wider public.

Another issue of interdisciplinarity – encouraged as graduate, but not when it come to the profession. There are different paths. Once you land a job, it is up to how you behave and perform.

Considering the pathways of integration, the challenge between the modellers and the observationalists. We can think about identifying a path.

Machine learning might need to re-evaluate how we learn and know something. There is also need to think about which statistics we want to use.

Margaret: what is different now – a growing sense of lack of being able to characterised the things that are going on. The understanding about our ignorance: in the past we had simple linear expectations of understanding. We finding that we don’t understand, and the biosphere and what is does to the world. There are so many viruses in the sea air, and we don’t know what it does to the world. The big revolution is the insights into the complexity of the earth system. How not to simplify beyond the point that we will loose important insight!

Esri User Conference 2016 – plenary day

The main Esri User conference starts with a plenary day, where all the participants (16,000 of them) join together for a set of presentation from 8:30 to 3:30 (with some breaks, of course). Below you’ll find some notes that I took during the day:

wp-1467087487123.jpgThe theme of the keynote was GIS – Enabling a Smarter World. After an inspirational video (emphasising environmental applications of GIS, including dealing with sustainability and biodiversity), Jack Dangermond, opened the conference by covering a range of applications that fall under smart GIS. Examples include environmental monitoring, energy management for renewable energy and grids. Using the management of land information and urban design (green infrastructure plans, corridors for wildlife etc.), transport –  smart routing reduce environmental impacts, and increase efficiency. Engineering and public work, utilities and telecommunication, business analytics (an area that finally is taking off), public safety and also humanitarian support. We have an increasing understanding of citizen engagement through open data, and the UN is using GIS to share open data in data management for the Sustainable Development Goals. Story telling, and story maps are becoming central to the way information is shared.
We’re living in a world that is undergoing a massive digital transformation – how do we go forward in this wired planet? GIS is a language for understanding the world. We need to address the crisis of sustainability – we need to address the problems together. GIS allow integration, visualisation – a framework to design for the future through geodesign. Turn information to action – from measuring to affecting the world. GIS itself is getting smarter – through technologies and tools, sensors, types of data. Smart GIS is a variety of things: ability to connect to real time information – IoT, remote sensing, connecting everyone – assisting communities to understand what they are doing and acting. It mean integrating spatial data and records with system of engagement. This is possible through Web GIS pattern. Earthquake alerting from USGS tell people to get ready, and also flood analytics. There is an emerging ‘Community GIS’

A leading example of this change is the City of Los Angeles GeoHub– Lilian Coral – chief data officer described how she try to ensure that the city is using data for helping the management of the city. To assist with that, they have developed geohub.lacity.org to enable community organisations to do things with city data. It is using open data and open applications to allow new applications to solve problems. From running a clean Street Index to compare the information between different areas. GeoHub helps to unlock data in the city and can provide  support a range of application. People are used for community data collection on Exide Battery Contamination that happened in LA. LA is aiming to reduce death from accidents on the road, and trying to improve performance over time. They even try to explore walking in LA and reduce car dependency. They learn that the GeoHub is foundation for smart cities and develop a range of hubs for generating and using geographic information for residents.

Awp-1467087506737.jpgfter the GeoHub presentation, Jack Dangermond noted that we have an ability to share geographical knowledge like never before.  The concept of ArcGIS evolved to see it as a hub between a system of records, system or analysis, and system of engagement. Growing important of web services and apps. ArcGIS tools are evolving – collector and Survey123 apps are linking to field workers and data collection. In terms of GIS technology, there is more effort on exploratory spatial data analysis tools (Insights for ArcGIS) and making it possible to analyse Big Data – for example billion transactions – using distributed computations using computer clusters. Application such as Drone2Map can speed up the process of turning drone imagery. There are more development tools for apps, with over 500,000 appearing. The open source apps allow people to developing further. Esri has run 4 MOOCs and may learning resources that are free for use by users of Esri. Esri support 11,000 university and higher education institutions around the world.  The people who are working in GIS, engaged and committed, are the people who are creating a smarter and more sustainable world.

wp-1467087511310.jpgLater in the day, some of the technologies that were discussed include the living atlas which is a whole catalogue of updated base maps, and the use of vector data allow restyling of information in many ways. A growing range of apps for the field, office and for the community support a range of activities. Information for communities include story maps, open data, photo survey, crowdsourced reporter, manager, and polling.

An example for the utilisation of the apps was provided by the talk “Civic Responsibility – Changing Our Approach” from the City of New Orleans (Lamar Gardere, Greg Hymel & James Raasch). In New Orleans they used collector to work with volunteers to coordinate and record a progression of a campaign to raise awareness to mosquito that can be vectors of disease. They also created a very fast survey methods based on images of building, using a crowdsourced image analysis that includes 6 attributes. The photos where collected throughout the city using geolocated wide angle camera. They then prepare the images and created a way of capturing information. They ask people to help in crowdsourcing. An example for geographical crowdsourcing in government, with micro tasks: https://propertysurvey.nola.gov/photosurvey/ . They have also created an application to link people relating to basins and reports from 311 calls. When someone agreed to adopt a ‘catch basin’ (a drain in the street) then they are sharing responsibility to check that it is not blocked before storms arrive and volunteer to clean the drain. They also have a story map, to let people share their pictures and images that are integrated into a story map.

wp-1467087515436.jpgAfternoon session opened with the main keynote “The Invention of Nature: Alexander von Humboldt’s New World” by Andrea Wulf. She told the story of Alexander von Humbolt, who spend his fortune on a journey of 5 years in south America, the most famous person in his time after Napoleon. He inspired Darwin to go on the Beagle journey. Many people relate to him and his insights. Died in 1859, and after his death people celebrated him – but he is almost forgotten today. Humboldt invented the concept of Nature, noticing the connection between different aspects of the living world, and geography. He also defined global climate and vegetation zones. Pioneering mapping and visualisation – using scientific data as a basis for fantastic maps. He can also be associated with concepts of environmentalism. Her book explores him and his insights. The journey from Quito to Chimborazo was similar to a journey from the tropic to the arctic, and realised that it’s like movement between different regions of the world. He was capable of linking many things together. Humboldt also created new forms of cartography, and have an appreciation to indigenous knowledge. Humboldt ‘Cosmos’ made a physical description of the universe, linking many aspects of nature together and this was his most popular contribution. The network of GIS and the creation of a living atlases in GIS is knowledge that bring power to people and communities – we can see a link to practices in GIS to von Humboldt.

Another major announcement was the effort of “Designing and Creating a Green Infrastructure” with Arancha Muñoz Criado (City and Strategic Planner) and
Kaitlin Yarnall (National Geographic Society). A common initiative of conservation organisations to create a common set of information about green spaces and wild spaces. Esri and National Geographic are joining forces to create information system for this. The notion is to protect green infrastructure across America – a GIS for the whole country, to define the area that need protection. They will provide extensive information and will provide geodesign tools to allow many people to use the information.

wp-1467087519514.jpgAnother important presentation was about “The AmzonGISnet” with Richard Resl and Domingo Ankuash in Ecuador, who use GIS in new ways. 20 years ago, Domingo started to use GIS to help the indigenous tribes that he leads to protect their lands. Many local indigenous members of the community who have GIS skills and who create a self made life plan – their own atlas representing their land and views. He noted that his community “We do not live in the forest, we are part of it”. The are not thinking themselves are poor, but need the support of other people to protect their land – having maps that are strategic and mindful. Using GIS not to navigate the forest but to protect it.

The final talk in the event was about Connecting GIS with Education, noting that  there is more work on GIS in schools across the US and the world. San Andreas High School started only 18 months ago with GIS, with only one teacher getting into GIS, but alrady achieving results through collaboration with GIS Mentors. An area with 98% students who receive free lunches. The GIS is a force for good. They created a story map about teens and drinking & Alcohol abuse, showing analysis and considerations within the process. Students also created data collection for surveying the state of sidewalks using Survey123.

Esri Education Conference 2016 – day 1

I’ve been working with Geographic Information Systems (GIS) since 1988. During the first 2 years, I wasn’t even aware that what we were doing was GIS – it was a mapping/inventory system that run on second generation PC (80286 processors) and was used to map facilities. Once I’ve discovered that this was a GIS, the next thing was ‘and ESRI Arc/Info is the software that you should check’. I’ve heard a lot over the years about the Esri User Conference, but haven’t had the chance to attend it, so this year I’m filling in the gap in my experience of the world of GIS. I’m giving a keynote at the Esri Education User Conference (EdUC), and I’ll attend some other parts of the general User Conference, and report on the experience.

The Education UC opened up with interest in creativity. Angie Lee, who opened the conference, noted her inspiration to the theme from learning about the makers movement and the growing interest in teaching students to code. She noted that many aspects of GIS encourage creativity: developing a story map or building an app. The opportunity that are emerging with new technologies  This is also true in science for the development of hypotheses and methodologies.

The two keynotes on the first day are by Dave Zaboskiwp-1466874989913.jpg
(Professional Artist / Creative Consultant) a former Senior Animator with Disney. Dave suggest that people have an innate ability of creativity. Creativity that is common at a young age but disappear later. Creativity is the willingness to do try things, it’s the courage to take risks. It’s the act of turning a thought into a thing, but we tend to get lost in the process. Resolute imagination is leading to magical results. The approach that he suggests is highlighting a spiral iteration instead of trying to move directly at a goal. There are five key attributes: first – believe in what they are doing. Have a clear narrative, and trying to reduce noise to increase the signal. You can think of things you have no control about them (A list),  issues that you can influence (B list), and a D+ list of things that make you lighter & happy, that make the ‘signal’ in life. The A list should not considered – someone else is going to take care of that and it will be their D+ list. The B list, can be solved at one thing a day, and then focus on the main issues. Then you can focus on the D+ list. Need to collaborate, and risk in a powerful way – when iterating you need to be confident that you can try again. Need to know that it might require throwing work that you have already done. Need to learn to trust you own abilities. Need to be able to allow instead of cognitive dissonant to have allow for information to have creative confusion until the information organise itself in a way that make sense. Completion is also important: acknowledge that people that were involved, analyse and celebrate. When you can declare that you have completed, you can move to the next challenge. Creators believe, iterate , collaborate, risk, and complete.
wp-1466981055360.jpgThe second keynote was by Dominique Evans-Bye, Science Teacher, Clark Magnet High School. She gave a detailed presentation on teaching GIS in high school through project based learning – including diving and operating ROVs. She’s working with low income students, many are from immigrants families. The first project in 2006 that the students done was to analyse sediments and heavy metals in the Los Angeles Harbour. The students collected samples and then analysed the level of contamination in them, and visualising the information on a GIS. In the next stage, they examined contamination in lobsters and they analysed the tissues, with mass spectrometers. They found high arsenic levels. The students gain confidence, and learn through iteration and use the online tutorials of ArcGIS Online and offline to develop skills and use them to analyse their information. The exploration of the problem lead to new questions and ways to represent the information. The students are doing also applications with the Esri collector app, to understand how the litter can end up in the ocean. Another project involved them analysing albatross  migration. Through classes on environmental GIS which was problem oriented and based on all the skills that they’ve gain in operating the research process itself. The students are collaborating, and see the process from projects where they’ve been involved in data collection to analysis. Students experience collaboration with scuba divers from NOAA and other bodies. The students won awards and scholarships as a result of their effort. There are major benefits for creating a creative learning environment with high school students and allowing them to develop their learning through problem solving.

In an afternoon session, I presented a talk that Patrick Rickles prepared, titled GIS and Citizen Science: Combining Open Source and Esri Technologies. The presentation focused on the way that some of the technologies that are developed in the ExCiteS group, such as GeoKey and Community Maps, can work with Esri technologies. The presentation open by explaining the needs and requirements – the interdisciplinarity of the group, and the type of areas that we work with. He then demonstrate, using the work of Challenging Risk project in Seattle, which looks at participatory methods for community preparedness to earthquake and fire. The context of the project mean that there can be 2 way data sharing – from the community to local government so they can see the information in ArcGIS Online, and information from the open data store can be shared on Community Maps. Several other examples for Esri technologies that are in use are shown.

 

Esri Education User Conference talk: Citizen Science & Geographical Technologies: creativity, learning, and engagement

The slides below are from my keynote talk at the Esri Education User Conference 2016. The conference focused on creativity and its relevant to education and the utilisation of GIS (especially Esri software) at different levels of education.

My talk explored the area of citizen science and extreme citizen science and the way geographical technologies contribute to creativity and learning. As I continue to assume that many of the audience don’t know about citizen science, I start with a review of the field as a way to contextualise what we, as a group, try to do.

[The talk is similar, in parts, to other talks that are captured here on my blog (workshop on theory, practice and policy, standards and recommendation for citizen science, or the current developments in ExCiteS). I’m updating the slides with lessons on what seem to work or not in previous talks. Social media is helpful for that – I can see which points people found most useful/meaningful!]

The talk starts with an historical perspective of citizen science, continue with the societal and technical trends that are at the basis of the current growth in citizen science. Having done that, I’m using a typology that looks at domain (academic discipline), technology, and engagement as a way to introduce examples of citizen science activities. I’m using the trailer for the TV series ‘the Crowd & the Cloud’ to recap the discussions on citizen science activities. I also mention the growth of practitioners community through the Citizen Science Associations.

Next, on this basis, I’m covering the concepts and practices of Extreme Citizen Science – what we do and how. I’m using examples from the work on noise, community resource management and earthquake and fire preparedness to demonstrate the concept.

The last part of the talk focuses specifically on creativity and learning from the Citizen Cyberlab project, and I explain the next steps that we will carry out in the Doing It Together Science project. I complete the talk by giving examples for activities that the audience can do by themselves.

Throughout the talk, I’m showing how Esri technologies are being used in citizen science. It wasn’t difficult to find examples – Esri’s GIS is used in BioBlitzes, Globe at Night, links to OpenStreetMap, and support the work that the ExCiteS group is doing. Survey123 and similar tools can be used to create novel projects and experiment with them. ArcGIS Online will be linked to GeoKey, to allow analysis of community mapping efforts. In short, there is plenty of scope for GIS as an integral part of citizen science projects.

A review of volunteered geographic information quality assessment methods

One of the joys of academic life is the opportunity to participate in summer schools – you get a group of researchers, from PhD students to experienced professors, to a nice place in the Italian countryside, and for a week the group focuses on a topic – discussing, demonstrating and trying it out. The Vespucci Institute in 2014 that was dedicated to citizen science and Volunteered Geographic Information (VGI) is an example for that. Such activities are more than a summer retreat – there are tangible academic outputs that emerge from such workshops – demonstrating that valuable work is done!

During the summer school in 2014, Hansi Senaratne suggested to write a review of VGI data quality approaches, and together with Amin Mobasheri and Ahmed Loai Ali (all PhD students) started to developed it. I and Cristina Capineri, as summer school organisers and the vice-chair & chair of COST ENERGIC network (respectively), gave advice to the group and helped them in developing a paper, aimed at one of the leading journal of Geographic Information Science (GIScience) – the International Journal of GIScience (IJGIS).

Hensi presents at the Vespucci summer school
Hansi presenting at the Vespucci summer school

The paper went through the usual peer review process, and with a huge effort from Hansi, Amin & Ahmed, it gone all the way to publication. It is now out. The paper is titled ‘A review of volunteered geographic information quality assessment methods‘ and is accessible through the journal’s website. The abstract is provided below, and if you want the pre-print version – you can download it from here.

With the ubiquity of advanced web technologies and location-sensing hand held devices, citizens regardless of their knowledge or expertise, are able to produce spatial information. This phenomenon is known as volunteered geographic information (VGI). During the past decade VGI has been used as a data source supporting a wide range of services, such as environmental monitoring, events reporting, human movement analysis, disaster management, etc. However, these volunteer-contributed data also come with varying quality. Reasons for this are: data is produced by heterogeneous contributors, using various technologies and tools, having different level of details and precision, serving heterogeneous purposes, and a lack of gatekeepers. Crowd-sourcing, social, and geographic approaches have been proposed and later followed to develop appropriate methods to assess the quality measures and indicators of VGI. In this article, we review various quality measures and indicators for selected types of VGI and existing quality assessment methods. As an outcome, the article presents a classification of VGI with current methods utilized to assess the quality of selected types of VGI. Through these findings, we introduce data mining as an additional approach for quality handling in VGI