Activities Review Process

On the Cutting Edge Workshop on Teaching Hydrogeology, Soils, and Low-T Geochemistry in the 21st Century

HSG Review Management Team

Devin CastendykSUNY, Oneonta

Managing Editor

John McDarisScience Education Resource Center (SERC)

Barbara TewksburyHamilton College

Cutting Edge PI

Maddy SchreiberVirginia Tech

Associate Editor

The collections SERC: Science Education Resource Center

at Carleton CollegeSERC office and staff helps develop and

manage web resources for many projects through collaboration

Many different collections of activities, submitted for different projects

On the Cutting Edge: the first of the projects hosted by SERCMost of the activities in the Cutting Edge

collections were submitted in connection with workshops.

Cutting Edge Reviewed Collection

On the Cutting Edge is conducting a review of activities in the Cutting Edge collections

Each activity reviewed twice and ranked:Exemplary (Part of Reviewed Collection)Pass (Part of Reviewed Collection)Keep (Not part of Reviewed Collection)De-accession

Cutting Edge Reviewed Collection

Activities ranked as “Exemplary”Come up first in searchesAre designated on individual the

ActivitySheet as being part of Exemplary Teaching Activities collection

Cutting Edge Reviewed CollectionActivities ranked as “Pass”

Come up second in searchesAre designated on individual ActivitySheet

as being part of the Peer Reviewed Teaching Activities collection

Cutting Edge Reviewed CollectionActivities ranked as “Keep”

Come up last in searchesHave no designation on the ActivitySheetWorth keeping as catalyst idea

Might be too local or items might be missing or has other problematic aspects

Review process

Any activity tagged with hydro, soils, low-T geochem, biogeochem, or ES that has not been reviewed yet

Each activity receives 2 reviews Authors of “Exemplary” and “Pass” activities

receive letters Explains review process Indicates activity rank Indicates that reviewer comments are available

on request if the author wishes to revise If author does revise, the activity will be reviewed

again

Plan for webinar

Explain the web interfaceClarify review criteriaAnswer questions

As we go along, please post questions in chat

Your list of itemsWhen you click on Review Tool on

Review Team Instructions page, your login will take you to a page that lists only your items to review

http://serc.carleton.edu/NAGTWorkshops/hydrogeo/HSG2013/review_team.html

Your list of itemsIf you have not yet completed the

review, you will see:

Click the URL to go to the ActivitySheet and download the actual activity and any supporting materials.

The review tool

Click Review It to bring up the Review Tool

The review tool

You will evaluate the activity in five categoriesScientific accuracyAlignment of goals, activity, and assessmentPedagogic effectivenessRobustnessActivity description

The review tool For each category, questions plus rubric provide

guidance for what to consider

The review toolSummary score will tabulate automatically

Exemplary = 4; Very good = 3; Adequate = 2; Problematic = 1

Comments help the editors understand your ranking – please don’t leave these boxes blank!!

The review tool At the end of the form, you will add your view

about what it would take to raise the activity to Exemplary status if it fell short in your review

The editors will use your comments to respond to authors on request. Please phrase your comments in a collegial fashion.

Your list of items

Once you have submitted a review, your review list indicates completion for that item

Can you revise a review? Yes – click on the Review It link and then

the link to what you submitted previously

Your original rankings will come up, and you can change them and add to/change your comments.

Click submit when done.

Pause for questions

Anyone??

Your reviewReview the activity in the context for

which it was designedNot just whether it’s good for a particular

upper level course – many will be for other courses (e.g., intro geo)

Not everything has to be a full lab or major assignment (e.g., a back-of-the-envelope calculation could be Exemplary)

Not every activity needs to be usable by all instructors (e.g., a lab requiring specific software/math/expertise background)

Your review

Make a summary list of the activities that you reviewed

List both the total numerical score and your overall assessmentExemplary“Exemplary minus”PassKeepDe-accession

Bring with you to review team meeting

Your review

Exemplary Must have good science, good pedagogy,

and all materials so that someone else can adapt/adopt, nothing “broken”

Can be “local” if it is also a good templateDoes not need to have answer key or to

provide an instructor with backgroundWe have never required these so cannot ding

someone for not including them

ScoringExemplary or very good in all categoriesExemplary in at least three of the five.18 or higher.

Your reviewExemplary minus

Could be made Exemplary with only a small amount of work, such as:fixing a URLuploading the latest version of the

assignment or adding instructor tipsfleshing out the ActivitySheet

This is not a formal category, but it would help us a lot to have your list of “Exemplary Minus” activities

These will be ranked as Pass, but knowing that they are “Exemplary minus” will help the editors craft feedback if authors request it.

Your review

Pass – these become part of the Reviewed CollectionThose that aren’t Exemplary but still have

value to othersMust be more than just the germ of an idea Must have all of the components

These must have no scientific errors. If you think there are errors, confirm this with

someone else on the review team. Those with scientific errors should go into the

Keep or De-accession category, depending on

the severity of the problem.

Your review

Keep – no designation on ActivitySheet, will come up last in a searchNucleus of a good ideaInsufficient info for someone to adapt or

adopt or has scientific errorsAuthor does not receive a letter

Your review

De-accessionNot an activity or very fragmentary Has truly egregious problems

Summary Review each activity using rubric Score the activity in each of 5 categories Write a summary evaluation for each

Remember that these were submitted voluntarily to a community collection

Be kind but clear Make a summary list

ExemplaryExemplary minus (technically a Pass)PassKeepDe-accession

Your assignment

Each team member has 10-11 activities to review

Reviews must be completed and submitted using review tool before the review team meeting in ABQ

Meeting in ABQ on June 4 at 8:30 amDiscuss issuesResolve discrepant reviewsArrive at final rankings

Thank you!

Email Barb with any questionsbtewksbu@hamilton.edu