Citizen Science Study Design - US EPA...This presentation by Rachelle Duvall explains how to design...
Transcript of Citizen Science Study Design - US EPA...This presentation by Rachelle Duvall explains how to design...
Photo image area measures 2” H x 6.93” W and can be masked by a
collage strip of one, two or three images.
The photo image area is located 3.19” from left and 3.81” from top of page.
Each image used in collage should be reduced or cropped to a maximum of
2” high, stroked with a 1.5 pt white frame and positioned edge-to-edge with
accompanying images.
Rachelle Duvall, US EPA
Citizen Science Study Design
Office of Research and DevelopmentNational Exposure Research Laboratory July 9, 2015
Questions to Consider when Designing a
Citizen Science Study
• Why am I doing this?
• How am I going to do this?
• What type of useful and valid data do I need to collect
or use?
• Where should I collect my data?
• What resources (equipment, people, money) do I need
to do this?
• What will I do with this information?
1
Why am I doing this?
• What is my motivation?
• What question am I looking to answer?
• What is my goal?
• Am I the only one who wants to do this or are there
similar projects or organizations already doing this?
• Can I do it? Alone or in collaboration? Do I have a
team? Do I have money?
Let’s talk about your goal or goals…
2
My Goal or Goals
3
When designing a project with a goal or goals in mind,
consider something called the Gartner Hype Cycle
Source: http://en.wikipedia.org/wiki/Hype_cycle
My Goal or Goals
• It is a crucial element of your work
• The success of your work depends on it
• You have to agree to the goal or goals you choose
• Your goal should be something tangible and
quantifiable, clear and well articulated
– A good example: a source tracking study of pathogens to
identify contamination and keep your beaches open.
–A not so good example: to raise the awareness of health issues
in your community.
–A very bad example: monitoring a few spots in your
neighborhood -OR- counting birds in the County Park.
4
My Goal or GoalsThe Road To Success
5
So, WHAT do you do NOW?
You BRAINSTORM with your TEAM!
Potential Brainstorming Questions
• What are the questions you have?
• Are you doing things because they are needed or
because you can?
• Does your project or study make sense?
• How will it answer your basic question or questions?
• Does the progression of your study lead you from
awareness to recognition to action?
6
The Outcome?
7
We have a…
GOOOOOOAL!!!
Are We Ready for the Study Design?
Yes, we are!
We have…
One or more goals
A project team
Participants and/or collaborators
Funding and Resources
8
PROJECT/STUDY DESIGN
9
A Hypothetical Study and Its DesignMany possibilities
10
Which way will you choose to get to Hong Kong?
What will dictate your choice?
Do you have more than one choice?
Is your head spinning yet?
Basic Elements of Study Design
• Study/survey protocol development
– Location, frequency, equipment, methodology
– Quantity of data
– Quality of data
• Selecting and recruiting citizen scientists
• Technology requirements and use (web, computational resources
and capacity)
• Supporting materials and mechanisms (e.g. central site, labs)
• Plans for analyzing data, results and information
• Project/study evaluation
• Lessons learned and next steps
11
Study/Project Protocol DevelopmentLocation, frequency, equipment, methodology
• Location is key!
• A lot depends on WHERE you pick your sites
– Are they the right sites to help you achieve your goal?
– Is it safe to get to them and to sample?
– Additional security measures needed to protect equipment?
– How far are they from a lab or your location?
– Are there a sufficient number of sites in the plan?
• Can I start smaller and build?
• How often will samples need to be collected and under what conditions?
• Do I have or can I get the equipment I need?
• Do I know or can I learn and teach the necessary methods to my team?
12
Study/Project Protocol DevelopmentQuantity and Quality of Data
• Quantity of data
– Are we collecting discrete or continuous data, or both?
– Are we using only our new data or are we adding someone else’s data
into the mix?
– Based on the time line of my project, how much data will I have?
– Will it be enough or do we need more?
• Quality of data
– Understand the quality of data (Are your methods, equipments and/or
lab giving you what you need?)
– Do you have/are you ready to develop a criteria for accepting data?
– Are you training your volunteers to provide what you need?
– Do you know how to verify, evaluate and validate the data?
13
Quantity and Quality of DataA few more considerations
• Data complexity level vs. number of active participants and their
level of understanding/training
• Are you integrating data over time, space or in a geospatial
continuum?
• Are you collecting data for modeling or other purposes? Who will
be using your data?
• Big data = 4V
– Volume
– Velocity (frequency)
– Veracity (true reliability)
– Variety (environmental, weather, water, air, data systems, etc.)
14
Study/Project Protocol DevelopmentSelecting and Recruiting Citizen Scientists
You need citizens in order to conduct a citizen science study!
• Who will be involved or available to help collect the data? (Specific age
groups? Vulnerable populations? Anyone?)
– This may depend on the goal of your study
• How will you go about recruiting citizen scientists?
– Local citizen groups?
– Community centers?
– Social Media?
– School administration?
– Local/state environmental agencies?
Tip: If feasible, it’s good to include citizen scientists in the planning stages
15
Study/Project Protocol DevelopmentTechnology Requirements and Use
• Equipment
– The type of equipment you use will directly tie into the handling of your
data, your training, your costs, etc.
• Web services
– Do you have forums to serve as a focal point for your project? Do you
need to add on or create a brand new one?
• Computational resources
– Will you use apps, uploads via smartphone, social media, classic data
entry or a combination?
– Do you have the data entry and all aspects of data management, such
as statistical help covered?
• Capacity
– Do you have the computer know-how, power, and the data storage
you need for this project?
16
Study/Project Protocol DevelopmentSupporting Materials and Mechanisms
• Supporting Materials
– Do you plan to have a central location or space from which you will
manage all aspects of the project?
– What are your chain of command and accountability measures?
– How will you provide technical support if citizens have questions or
need help troubleshooting problems? Dedicated support staff?
– Do you have training materials, field procedures, and safety ready?
Mentoring of newbies?
Helpful Tip: Use existing materials if possible – don’t reinvent the wheel unless
you have to do it!
• Laboratory, statistics, dissemination
– Is your lab the right one for your project?
– Get the right statistical support!
– Mechanism for communicating intermediate and final results?
17
Study/Project Protocol DevelopmentPlans for Analyzing Data, Results, and Information
• Accepting data
• Data entry
• Data management
• Data evaluation and validation
• Data reconciliation with your goals
• Data use and sharing
• Results – presentation and use of best media
• Disseminating and promoting your project start to finish –
make that planning phase to finish!
18
Study/Project Protocol DevelopmentProject/study evaluation, lessons learned and next steps
Project/Study Evaluation
• Your Project should be evaluated throughout its useful life
• Baseline: prior to start, yardstick for measuring change
• Formative: during the project, strengthen and improve effectiveness
• Summative: at the end of the project or when you reach a decision
point
Lessons Learned and Next Steps
• Assemble your team as many times as you need and have a session on
meeting your goals, roadblocks, what to do the same or different and why
• Share your experience and invite outside feedback
19
PROJECT/STUDY DESIGN –
But HOW do I do it?
20
Outcome Mapping
Logic Model
The Logic Model and Study Design
In its simplest form a Logic Model is:
Key Terms:
Output: Activity or effort produced or provided
Outcome: Result, effect, consequence or impact
21
Inputs Outputs Outcomes-Impact
What is invested What is done What results
if then if then
All of the Elements
22
Adapted from Michelle Mandolia, EPA Office of Policy, Evaluation Support Division
Resources Activities Outputs Audience Short-Term
Outcomes
Intermediate
Outcomes
Long-Term
Outcomes
What we
invest
• Time
• Money
• Partners
• Equipment
• Facilities
(lab)
What we do
• Plan
• Train
• Monitor
sites
• Analyze,
interpret
data
What we
produce or
deliver
• Workshops
• Baseline data
on local
environment
• Reports,
publications,
events
Whom we
reach
• Volunteers
• Board of
Directors
• Sponsors
• Partners
• Public
• Decision-
makers
Change in:
• Knowledge
• Skills
• Attitude
• Awareness
• Motivation
Change in:
• Behaviors
• Practices
• Procedures
Change in:
These are the
impacts of the
project
• Environment
• Social
conditions
• Economic
conditions
• Policies
A few other considerations….
23
Test Your Study/Project Design
• Do a dry run from A to Z BEFORE you go live if you can
• Ground truth your sites if you are selecting new or unknown
locations
−Consider alternate sites if not safe or problematic in other ways
• Run through your training, field, lab and data management
procedures
• Determine how long it will take to collect a sample or data point
• Do a mock informational session with results and information if
you can
• DON’T BE AFRAID TO CHANGE ANYTHING THAT DOESN’T
WORK
24
Have a Backup Plan
Murphy’s Law…
Anything that can go wrong, will go wrong
• A number of things could go wrong with a study…
− Citizen scientists can drop out of a study
− Data quality could be compromised
− Sampling and analytical equipment can fail or get damaged
− Data loss
− Other unforeseen circumstances
• Always good to have an alternative plan for the “what-ifs”
25
Resources
• Ask for help and be brave – your local college/university or big
cooperation may have the math department, computers or lab
that you need. A new friendship may be born.
• Collaborate – it’s easier than going it alone
• Resources and materials on the web. For example…
EPA’s Air Sensor Tool Box for Citizen Scientists
http://www.epa.gov/heasd/airsensortoolbox/index.html
EPA Region 2 Citizen Science Page
http://www.epa.gov/citizenscience/
• Work with your State, County, local officials and schools – nurture
the next generation of Citizen Scientists
26
A Thought to Take Home With You
Providing information is not enough to lead to
better decisions. A two-way dialogue is
necessary to collectively determine what type
and information is needed to improve the
knowledge of decision makers.
27
Environment Canada – Community Based Monitoring Program
Thank you!
Questions?
Rachelle Duvall
(919) 541-4462
28
Acknowledgement: Paula Zevin (EPA R2)