CCLIMATE LIMATE LLITERACY AND ITERACY AND EENERGY NERGY AAWARENESS WARENESS NNETWORK PATHWAYETWORK PATHWAY
SUMMARY OF A RIGOROUS REVIEW PROCESS
CLN Webinar December 7, 2010CLN Webinar December 7, 2010
Outline
CLEAN project and CLEAN collection
Review Process
Results of first review round and gap analysis
Conclusions and Outlook
CLEAN Pathway Project CLEAN Pathway Project GoalsGoals
Stewarding a collection of excellent teaching resources on climate and energy science
Professional and Community Development
Online communities
Facilitate the Climate Literacy Network
CLEAN Pathway: CLEAN Pathway: CollectionCollection
500 excellent digital teaching resources addressing climate science or energy awareness for grades 6-16
Resources scientifically and pedagogically reviewed
Annotations reflect reviewer comments
Resources aligned with Climate Literacy: Essential Principles of Climate Science Energy Awareness Principles National Science Education Standards AAAS Project 2061Benchmarks for Science Literacy NAAEE Excellence in Environmental Edu. Guidelines for
Learning
Framework for collectionFramework for collection
Energy Awareness Principles (newly developed)
A. Earth System Energy B. Human Energy Sources C. Impacts of Energy Use D. Energy Distribution/Costs E. Energy Access and Equity F. Decarbonization Challenges
http://cleanet.org/clean/literacy/energy.html
Every activity included in collection has to address either Climate Literacy: Essential
Principles of Climate Science http://www.climatescience.gov/Library/Literacy/
CLEAN review process
Phase 1 of CLEAN focused on teaching activities
Following phases: broadening of scope to videos, visualizations, lab activities, background materials etc.
Developing Review Criteria
Review criteria specific to teaching activities
Informed by NSDL and SERC guidelines, Merlot criteria, DLESE, Climate Change collection scorecard
Review of e-learning materials requires additional considerations (multi-media elements, navigation etc.)
Climate and energy science evolving science – faster turn-around and need more media to convey
Developing Review Criteria
Initially long list of possible criteria: convergence in phrasing of questions during 7 test review rounds
Test reviews: good agreement for good resources, wide spread in answers for low quality resources
Weekly telecons for resource collectors
What is an excellent What is an excellent activity? activity?
Definition Teaching Activity:Relatively brief set of instructional materials that is presented as a whole, where all the parts and ideas are linked and part of the same activity
Educator should be excited to find this activity when searching for teaching materials.
Avenues into collection
Existing resource pools found through online search
Resources suggested by public through online form
Targeted search after gap analysis
Collaboration with projects that submit resources directlyto CLEAN team (iterations)
Informal vetting
Resource pools assigned to team of 9 collectors
Scanning of resource pools and informal triage
Informal triage: Does this resource meet the CLEAN criteria?
Review questionnaireReview questionnaire
Initial Vetting Review
Scientific accuracy Pedagogic effectiveness Technical quality / Ease of use→ 6-12 questions for each category, overall
rating in rubric format, comment box for annotations
→ Questions help to consider all relevant aspects for each category and lead to overall rating
→ No quantitative, only qualitative recommendation (low – medium – high priority)
Formal vetting step
Basic check for relevance to CLEAN project
Decision: “Will resource likely pass the review?”
Recommendation: Move on to initial review or put in “holding tank”
Review: Scientific accuracy
Considerations for initial science review
Solid, current science
Original data cited and data from a quality source
Attribution
Valid concepts
Supporting references
Review: Pedagogic effectiveness Considerations for pedagogic review
Learning objectives
Accommodates diverse learners (learning styles, language, cultural diversity)
Prerequisite skills and understandings
Assessment strategies
Engaging for students in subject and approach
Requires independent/inquiry-based thinking
Review: Technical quality/ Usability
Considerations for technical/usability review
Ready for use, stands on its own
Clear presentation of content
Software/tools/resources commonly found in classroom
Amount of necessary guidance for students by instructor
Offers comprehensive guide for instructor
Digitally available resource
Expert science review
External expert with PhD in relevant field reviews scientific quality and accuracy of resource
Activity already passed lower level science review (75 % of resources that passed the CLEAN review were rated scientifically excellent by experts)
Limitation/Challenge: Grade-level appropriate science Difficult to find scientists with enough time
Panel review
Based on NSF-panel review system and AccessData Workshops
Panel provides necessary range of expertise
Teams of 4 educators and scientists review each teaching material based on prior reviews, final decision about inclusion in collection
Comments of all reviewers are compiled into annotation which includes teaching tips
Annotations
All reviewer comments, suggestions and tips are combined in notes to users (annotations)
Annotation draft reviewed during review panels, final clean-up during cataloging process
Annotations add considerable value, insight from scientists or experienced educators
Numbers first review cycle
Informal vetting of ~5000 teaching activities
~200 resources passed first review with medium or high priority > forwarded to second review
142 resources passed second review with medium or high rating > forwarded to panel review
94 resources passed panel review (18 passed on to editorial board)
5-7 different people reviewed each resource
Gap Analysis
Holes in collection are apparent – inform targeted search and hopefully future solicitationsTotal # Total #
of of activitiactivitieses
# of # of concepts concepts with no with no matchmatch
# of concepts # of concepts with ≤ 3 with ≤ 3 matchesmatches
Guiding Principle
22 2 5 of 7
EP 1 17 2 3 of 5
EP 2 20 0 3 of 6
EP 3 15 2 3 of 5
EP 4 21 1 2 of 7
EP 5 27 1 3 of 5
EP 6 8 1 5 of 5
EP 7 16 1 4 of 6
Energy Awareness
34 0 1 of 6
Conclusions
Rigorous and transparent review process
Ensures reliable and high-quality resources
Framework of Climate and Energy Literacy Principles allow for gap analysis in collection
Outlook
Targeted search to fill gaps in collection
Broaden scope of collection to other educational resources
Promote the collection and build a community of educators (Professional Development/Discussions)
Refining Energy Literacy framework for collection
Detailed Review Criteria
Details about the CLEAN review process: http://cleanet.org/clean/about/review.html
Link to Initial Vetting Questionnaire:http://cleanet.org/files/clean/about/
clean_vetting_questionnaire.pdf
Link to Review Questionnaire:http://cleanet.org/files/clean/about/
clean_review_questionnaire.pdf
Link to Expert Science Review Questionnaire:
http://cleanet.org/files/clean/about/clean_science_review_questionn.pdf
Top Related