Diane Ebert-May Michigan State University

ebertmay@msu.eduhttp://first2.org

Scientific Teaching and Interactive Learning:

Strategies to Maximize Student Understanding

QuickTime™ and aGraphics decompressorare needed to see this picture.

Engage

Question 1

• Active learning strategies enable students to learn science better than passive lectures.

Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly

disagree

Large Class Meeting

Class Meeting

Question 2

• Students learn science best by “doing” science.

Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

Question 3

• How important is it to use multiple kinds of assessments to determine student learning?

Please respond on a scale of 0-100 in increments of 10:

Question 4

• The proportion of assessments I use in my course that demonstrate students’ critical thinking abilities is....

Please respond on a scale of 0-100 (%) in increments of 10:

• Connections among concepts

• Organization of concepts

• Visual representations

• Model-based reasoning

• Test models

What is critical thinking?

Question 5

• In my department, excellence in teaching is rewarded at a level comparable to excellence in research.

Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly

disagree

Articles derived from journal papers

•

How People LearnBransford et al 1999,

2004

Explore: Out of Thin Air

What is going on?• Brainstorm: talk to your

neighbor

• 1. Diagnose situation - what is the learning challenge

• 2. Where is the missing link?...misconception?

How and when do you identify student learning difficulties?

Don’t have to grade

Pre-test (e.g., diagnostic questions - identify misconceptions)

Engagement activity - brain teaser, discussion starter, ‘need to know’ questions

Surveys or polls (clickers?)

Others

May use pretest or diagnostic (clicker) question

• Photosynthesis as Energy

• Biomass from Soil

• Energy as Biomass

• All Green

• Plant Altruism

• Thin Air

• Respiration as ‘breathing’

Misconceptions about Photosynthesis, Respiration, and the Carbon Cycle

Radish Problemin Ebert-May D, Batzli J, Lim H. 2003. Bioscience 53:1221-1228.

•Experimental setup:•Weighed out 3 batches of radish seeds •each weighing 1.5 g.

•Experimental treatments:•1. Seeds placed on DRY paper towels in LIGHT•2. Seeds placed on WET paper towels in LIGHT•3. Seeds placed on WET paper towels in DARK

Problem (cont)

• After 1 week, all plant material was dried in an oven overnight (no water left) and plant biomass was measured in grams. Predict the biomass of the plant material in the various treatments.

No Water, light [yellow]Water, light [pink]Water dark [green]No idea [orange]

Results Mass of Radish Seeds/Seedlings

1.46 g 1.63 g 1.20 g

Write an explanation about the results.

Explain the results.Write individually on carbonless

paper.

Midterm Assessment

Hypothetical scenario: Grandma Johnson had very sentimental feelings toward Johnson Canyon, Utah, where she and her late husband had honeymooned long ago. Her feelings toward this spot were such that upon her death she requested to be buried under a creosote bush overlooking the canyon. Trace the path of a carbon atom from Grandma Johnson’s remains to where it could become part of a coyote. NOTE: the coyote will not dig up Grandma Johnson and consume any of her remains.

Code Organisms Code Processes and pathways 1 Decomposers IA Cellular Respiration IB Release CO2 2 IIA Pathway of Carbon

Primary producers IIA _1: through Air IIA _2 : through Root IIA _3 : no mention about pathway IIB Make Glucose IIC Photosynthesis 3 Herbivore III Respiration

(glycolysis, Kreb cycle) 4 Carnivore IV Respiration

(glycolysis, Kreb cycle)

Coding Scheme

Corr

ect

Stu

dent

Resp

onse

s (%

)

Cellular Respiration by Decomposers

Bio1/Bio2 Other/Bio2

Friedmans, p<0.01

Pathway of Carbon in Photosynthesis

Bio1/Bio2Corr

ect

Stu

dent

Resp

onse

s (%

)

Other/Bio2

Friedmans, p<0.05

Deep within a remote forest of Guatemala, the remains of a spider monkey were buried under an enormous mahogany tree. Although rare, jaguars (big cats - carnivores) were spotted in this forest by local farmers. Explain how a carbon atom in carbohydrates contained within the muscle cells of the spider monkey could become part of a cell within the stomach lining of a jaguar. Note: the jaguar does not dig up the monkey and eat the remains!

Final Exam Question

Final Exam QDeep within a remote forest of Guatemala, the remains of a spider monkey were buried under an enormous mahogany tree. Although rare, jaguars (big cats - carnivores) were spotted in this forest by local farmers. Explain how a carbon atom in carbohydrates contained within the muscle cells of the spider monkey could become part of a cell within the stomach lining of a jaguar. Note: the jaguar does not dig up the monkey and eat the remains!

Make a clearly labeled box model of the system. Use the template on the Answer Sheet. You will not need all of the boxes. In the model, clearly label the processes (next to arrows), organisms or places, and forms (carbon pools) the carbon atom must go through to cycle within the ecosystem. Use the organisms/places and pools from the lists below.

To do so...

Place or Organism [not listed in a specific order]

Carbon Pools [form]

AtmosphereJaguar (carnivore)Mahogany tree (producer)Bacteria (decomposer)Spider Monkey (herbivore)Tapir (similar to a pig) (herbivore)

CO2 gasCarbohydrate

ProcessPool

Place

Pool

Place

Pool

Place

Process

Reminder: format for a box model:

Circles identify key portions of box model. Orange circles identify difficult portions for students.

Explain

What is assessment? Data collection with the purpose of

answering questions about…students’ understanding

students’ attitudes

students’ skills

instructional design and implementation

curricular reform (at multiple grain sizes)• Informing BOTH instructors and

students about learning.

Assessment GradientHigh

Ease of

Assessment

Low

Multiple Choice, T/F

Diagrams, Concept maps, Quantitative

response

Short answer

Essay, Research papers/ reports

Oral Interview

Low

Potential for

Assessment of Learning

High

Theoretical Framework• Ausubel 1968; meaningful learning• Novak 1998; visual representations• King and Kitchner 1994; reflective judgment• National Research Council 1999; theoretical frameworks for assessment

What level of learning do we ask of our students?

Bloom (1956) Cognitive Domain of Educational Objectives

6 categories - KnowledgeComprehensionApplicationAnalysisSynthesisEvaluation

•

How am I going to grade all this stuff??

• Subsample= You don’t need to grade everything!!

• Classroom Assessment Techniques (Angelo & Cross 1993); Muddiest Point, Minute papers etc..

• Pyramid Exams- Individual 75% + Group 25%

• Diagnostic Questions & Clickers

• Rubrics

Assessment and Feedback Approaches

Learning ObjectiveIdentify desired results

Learning OutcomeDetermine acceptable evidence

AssessmentsData collected & Feedback given

Instructional Design & ActivitiesPlanned learning experiences and instruction

Like This?

Backward Design

Adapted from Wiggins and McTighe 1998, 2005

Backward Design

Learning Outcome

Students will demonstrate understanding of evolution and natural selection by designing

and testing a research question, explaining the results.

Learning Outcome

•Changes in a population occur through a gradual change in individual members of a population.

•New traits in species are developed in response to need.

•All members of a population are genetically equivalent, variation and fitness are not considered.

•Traits acquired during an individual’s lifetime will be inherited by offspring.

Alternative Conceptions: Natural Selection

Write a scenario that explains the phenotypic changes in the trees and animals. Use your

understanding of evolution by natural selection.

Level of Achievement General Approach ComprehensionExemplary(5 pts)

• Addresses thequestion.• States a relevant,justifiable answer.• Presents arguments ina logical order.• Uses acceptable styleand grammar (noerrors).

• Demonstrates an accurate andcomplete understanding of thequestion.• Backs conclusions with dataand warrants.• Uses 2 or more ideas,examples and/or arguments thatsupport the answer.

Adequate(3 pts)

• Does not address thequestion explicitly,although does sotangentially.• States a relevant andjustifiable answer.• Presents arguments ina logical order.• Uses acceptable styleand grammar (oneerror).

• Demonstrates accurate but onlyadequate understanding ofquestion because does not backconclusions with warrants anddata.• Uses only one idea to supportthe answer.• Less thorough than above.

Needs Improvement(1 pt)

• Does not address thequestion.• States no relevantanswers• indicatesmisconceptions.• Is not clearly orlogically organized.• Fails to use acceptablestyle and grammar (twoor more errors).

• Does not demonstrate accurateunderstanding of the question.• Does not provide evidence tosupport their answer to thequestion.

No Answer (0 pts)

Scoring Rubric for Quizzes and Homework

Ebert-May http://www.flaguide.org/cat/rubrics/rubrics1.php

Coding Student Responses

Misconceptions Correct

Change in the individual Change in the population

Need to Change/ Must Change/ Choice

Change due to genes

All members of a population are equally fit

Individuals within a population have varying

fitness levels

Traits acquired during a lifetime are passed on

Genetic traits help the individual to survive and

reproduce

If faculty change....

Do students learn better?

FIRST/SI Study of Faculty Change

75 Facultyfrom Research and Comprehensive Universities, Liberal Arts and Community Colleges

FIRST -- multiple workshops, over 2-4 yearsSummer Institute at U Wisconsin - one week

RTOP

Assess

Path Model of Faculty Professional Development

Course Size

Self Report Assessment

Department

Teaching:Research

Pedagogy

ProfessionalDevelop

Self Report Practice

Does self report of teaching practice correlate with observed practice?

RTOP

AssessCourse Size

Self Report Assessment

Department

Teaching:Research

Pedagogy

ProfessionalDevelop

Self Report Practice

Does self report of teaching practice correlate with observed practice?

Standard Regression used to Test Hypothesis

RTOPSelf Report Practice

Does self report of teaching practice correlate with observed practice?

Does self report of teaching practice correlate with observed practice?Does self report of teaching practice

correlate with observed practice?

Standard Regression used to Test Hypothesis

Self Report of Practice

RT

OP

low high

highStrong relationship: use self report data to predict classroom practice.

No relationship: Must observe in classroom.

Path Model of Faculty Professional Development

RTOP

AssessCourse Size

Self Report Assessment

Department

Teaching:Research

Pedagogy

ProfessionalDevelop

Self Report Practice

Does the number of forms (types) of assessment influence the Bloom's level of

items on exams?

RTOP

AssessCourse Size

Self Report Assessment

Department

Teaching:Research

Pedagogy

ProfessionalDevelop

Self Report Practice

Does the number of forms (types) of assessment influence the Bloom's level of

items on exams?

AssessCourse Size

Self Report Assessment

Does the number of forms (types) of assessment influence the Bloom's level of

items on exams?

Standard Regression used to Test Hypothesis

Course Size

Me

an

Blo

om

low

high Class Logistics: Lower Bloom’s level questions are quicker to grade.

No relationship: Type and frequency of assessment are not related to Blooms.

Self Report Assessment

low high

high

What is the impact of course size on pedagogy?

RTOP

AssessCourse Size

Self Report Assessment

Department

Teaching:Research

Pedagogy

ProfessionalDevelop

Self Report Practice

What is the impact of course size on pedagogy?

RTOPCourse Size

What is the impact of course size on pedagogy?

Standard Regression used to Test Hypothesis

Number of Enrolled Students

RT

OP

low high

highStrong negative relationship: Active learning methods are easier to use lower enrollment courses. No relationship: There is no difference based upon course enrollment.

Sub-models for analysis...

RTOP

AssessCourse Size

Self Report Assessment

Department

Teaching:Research

Pedagogy

ProfessionalDevelop

Self Report Practice

Assessment Database: Structure and Function

De-identification Software

Data Upload SpreadsheetPseudonym Spreadsheet

Sample Spreadsheet for a Course

Secure Database

Data Upload Software

Public Database

Data Verification

Verified de-identified data

Fac

ulty

Com

pute

r

• -Find assessment instruments

• -Prepare metadata and upload student

• assessment data

• -Analyze and download data

• -Archive records of analyses

How can I use the database?

• Based on Ecological Metadata Standards (Michener 1997)

• Describe what collected, who collected, where collected, when collected, how collected, why collected

“Educational Metadata Standard”EdMS

What is in the Educational Metadata Standard?

• Where– Institution, class size

• How – Experimental and

sampling design

– Administration of assessments

– Instructional design

• Who– Project personnel

• What

-Assessment instruments

-Rubrics and assessment concepts

• Why

-Classroom study

Metadata also include...

•Taxonomies– Bloom’s Taxonomy - cognitive

•Concept Categories of Biology – Evolution – Carbon Cycling

•Identification of Published Instruments– Concept Inventory of Natural Selection (CINS)

Metadata - questions can be linked to more than one concept

In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each individual guppy within a population gradually change.b. The proportions of guppies having different traits within a population change.c. Successful behaviors learned by certain guppies are passed on to offspring.d. Mutations occur to meet the needs of the guppies as the environment changes.

Question on Assessment

Concept category - EvolutionAnderson’s 10 concepts - change in population - variation heritable

Instrument CINS

Concept category - EvolutionDino/Plant rubric

- P - change in population- G - genetic traits help individual survive and reproduce

Using the Database: Faculty Questions

Question: An instructor wants to determine student learning gains about evolution. What evolution assessment tools are available?

What evolution assessment tools are available?

• Search database by concept category

✓ Evolution

• Narrow questions by more specific classification criteria

✓ Change occurs in populations

✓ Change due to change in genes

• Narrow questions by availability of student results

✓ Large class size (>100 students)

✓ Majors Biology class

✓ R1 Institution

Finding Assessments: Search Database by Concept Category

Concept category - Evolution1. In guppy populations, what are the primary changes that occur gradually over time?

2. What would happen if a breeding pair of finches was placed on an island under ideal conditions?

3. What type of variation is passed on to finches over time?

1. In guppy populations, what are the primary changes that occur gradually over time?

2. What would happen if a breeding pair of finches was placed on an island under ideal conditions?

3. What type of variation is passed on to finches over time?

1. In guppy populations, what are the primary changes that occur gradually over time?

2. What would happen if a breeding pair of finches was placed on an island under ideal conditions?

3. What type of variation is passed on to finches over time?

1. In guppy populations, what are the primary changes that occur gradually over time?

2. What would happen if a breeding pair of finches was placed on an island under ideal conditions?

3. What type of variation is passed on to finches over time?

2000 questions from 1500 assessments

Narrow 2000 questions by more specific classification criteria

Concept category - Evolutiona. Anderson’s 10 Concepts/Misconceptions of Evolutionb. Dino/Plant Rubric (unpublished rubric by Ebert-May and Linton)c. Sexual Selection

i. P – change in populationii. C – change due to change in genes iii. V – individuals within a population

have varying levels of fitnessiv. G – genetic traits help the individual to

survive and reproduce

1. In guppy populations, what are the primary changes that occur gradually over time?

2. What would happen if a breeding pair of finches was placed on an island under ideal conditions?

3. What type of variation is passed on to finches over time?

3. What type of variation is passed on to finches over time?

200 questions from 150 assessments

Narrow 200 questions further by course criteria

Concept category - EvolutionDino/Plant rubric

- P - change in population- G - genetic traits help individual survive and reproduce

Criteria

Class Size: >100 students

Subject: Biology

Target Students: Majors

Institution Type: Research 1

1. In guppy populations, what are the primary changes that occur gradually over time?

2. What would happen if a breeding pair of finches was placed on an island under ideal conditions?

3. What type of variation is passed on to finches over time?

50 questions from 15 assessments

1. In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each individual guppy within ab. The proportions of guppies having different.c. Successful behaviors learned by.d. Mutations occur to meet the needs .

2. In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each individual .b. The proportions of guppies.c. Successful behaviors.d. Mutations occur to meet.

3. In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each b. The proportions of guppies having .c. Successful behaviors learned .d. Mutations occur to meet the.

Link Questions with Metadata

1. In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each individual guppy within ab. The proportions of guppies having different.c. Successful behaviors learned by.d. Mutations occur to meet the needs .

2. In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each individual .b. The proportions of guppies.c. Successful behaviors.d. Mutations occur to meet.

3. In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each b. The proportions of guppies having .c. Successful behaviors learned .d. Mutations occur to meet the.

Assessment #1 Assessment #2

Concept Categories of BiologyEvolution ConceptsWeber, 6/1/2006

Metadata - Rating Quesitons

In guppy populations, what are the primary changes that occur gradually over time?a. The traits of each individual guppy within a population gradually change.b. The proportions of guppies having different traits within a population change.c. Successful behaviors learned by certain guppies are passed on to offspring.d. Mutations occur to meet the needs of the guppies as the environment changes.

Question on Assessment

TaxonomyBloom 19562 - ComprehensionEbert-May, 2/27/2007

Taxonomy Bloom 19563 - ApplicationUrban-Lurain, 1/20/2007

TaxonomyBloom 19563 - ApplicationWeber, 2/27/2007

• Finding assessment instruments

• Administer the instrument, teach, and prepare student data

• Prepare metadata and upload student assessment data

• Analysis and download

• Results of analyses

How do I use the database?

Team at MSU•Rett Weber - Plant Biology (postdoctoral researcher)•Deb Linton - Plant Biology [Central Michigan U]•Duncan Sibley - Geology•Doug Luckie - Physiology•Scott Harrison - Microbiology [postdoctoral fellow]•Tammy Long - Plant Biology•Heejun Lim - Chemistry Education •Rob Pennock - Philosophy•Charles Ofria - Engineering•Rich Lenski - Microbiolgy•Janet Batzli - Plant Biology [U of Wisconsin]

•

•“...we note that successful people are the ones who take advantage of those around them to ultimately benefit students.”

•Ebert-May D, Weber R, Hodder J, Batzli J (2006)

Finally...