Public participation through citizen science – A critical reflection on the challenges for evaluation and monitoring
New Challenges for the Evaluation of Science and Research: Political Goals, Theory and Methodological Challenges Vienna, 11th of June 2015 Dr Larissa Talmon-Gros, Dr Thomas Teichler – Technopolis Group
Citizen Science can be defined as...
2
..„scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional scientists and scientific institutions“1
1 Oxford English Dictionary
Characteristics of CiSi projects
Various types of activities
Application in all stages of the research process possible
Diverse actor base Intensity of citizen
involvement
New technologies as enablers Different Objectives
Two empirical examples illustrate the range of of CiSi projects.
3 Quellen: www.igel-in-bayern-br.de http://sensebox.uni-muenster.de/
In practice, Citizen Science can take very several forms, with different qualities of interaction.
4
Type of project Intensity of interaction
Objective
Passive sensors
Computing capacity
Volunteer thinking
(Environmental) monitoring
Participative sensors
Community science
‚Extreme‘ CiSi
passive
active
trained
creative
Scientific
Science/environmental education
Engagement/dialogue
We suggest a theory-based approach to evaluation of CiSi.
5
Objectives Inputs Outputs
Needs Problems
Issues
Evaluation
Efficiency
Value Added
Impact and Sustainability
Other public interventions Outcomes
Impacts
Effectiveness Coherence &
Complementarity
Relevance Citizen Science projects/programmes
Society Research Other
Suggestions for evaluation questions (1/2)
6
Relevance • To what extent do project objectives correspond to societal needs? • To what extent does the project setup (scope, mechanisms etc) correspond to project
objectives? • …
Effective-ness
• To what extent do the projects contribute to major objectives such as science education/engagement of the public/the generation of scientifically relevant data?
• To what extent does the project contribute to other objectives, such as the involvement of underprivileged/other societal groups etc?
• To what extent is the implementation of the project effective? Which factors are hindering an effective implementation?
• …
Efficiency
• To what extent are monetary costs proportionate to benefits? • To what extent are non-monetary costs (eg free-time of volunteers) proportionate to
benefits? • Are costs and benefits in line with initial expectations? • …
Value Added
Suggestions for evaluation questions (2/2)
7
Coherence & comple-mentarity
• To what extent do other activities cover the objectives of citizen science projects (for instance science education, science communication, public participation)?
• Which additional benefits can CiSi projects bring? • …
Impact & Sustain-ability
• To what extent did the project contribute to a change in behaviour/change of the agenda/RRI/choice of study etc?
• How sustained is the project conducted? • How sustainable is citizen involvement in CiSi activities? • …
• Value added compared to existing approaches in science education/engagement • Contribution to reaching objectives • To what extent is action/funding by public bodies required? • Value added compared to current research practice • …
Indicators: Inputs
8
Inputs
• Expenditures for human resources (eg researchers) • Expenditure equivalents for unpaid human resources
(volunteers) • Other related expenditures
Indicators: Outputs
9
Outputs
• # of people reached / participants • # of projects • # of sites studied/ animals etc recorded • # of trainings for citizens • # of trained citizens • # of transdisciplinary projects • # and type of educational resources produced • # of academic publications generated • intensity of engagement • Type of project • Involvement of other societal/community actors (schools, clubs, etc) • Use of IT & type of IT use • Input for research/policy agenda • Research data generated • High quality data • ...
Indicators: Outcomes
10
Outcomes
• Textbook and process knowledge („how science is done“) • Confidence to contribute to further research • Engagement in and with science & technology • Political ownership • Formation of an opinion • Research results • Increased relevance of research for everyday life • ...
Indicators: Impacts
11
Impacts
• Change in behaviour (eg environmental engagement with S&T)
• Interest in STEM/environmental career • Responsible Research & Innovation through adjusted policy
agendas • Changed climate for science in the society • High quality research • ...
Summary & Discussion
12
• In your view, is the general approach suitable?
• Which other indicators and evaluation questions do you propose?
• Which importance do participative elements play? CiSi evaluation as a CiSi project?
• CiSi is receiving increasing attention, also by policy makers.
• CiSi projects are characterised by a large variety in terms of project type, intensity of interaction and objectives.
• When evaluating CiSi projects, we propose to handle this complexity by using a theory-based evaluation approach.
• We suggest as a first set of evaluation questions and several indicators for inputs, outputs and outcomes as well.
13
Thank you!
Dr Larissa Talmon-Gros technopolis |group| Deutschland GmbH Große Seestraße 26 60486 Frankfurt am Main Deutschland [email protected] www.technopolis-group.com technopolis |group| has offices in Amsterdam, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn und Vienna
Top Related