Transforming how we teach is good, but transforming what ... › downloads › bestpractice2015 ›...

Transforming how we teach is good, but transforming what we expect

students to learn is better.

Melanie M Cooper

Michigan State University

Why don’t students learn what we teach them?

Old Conventional Wisdom…

Current Approaches

• Active learning (interactive pedagogies, engaged learning, inquiry, clickers)

• Affective: motivation, expectations, values, identity…

• Cognitive: working memory, spatial skills …

• Appropriate learning spaces (scale-up, studio)

There is emerging evidence that while these aspects are important they are

not “enough”

For example

Year %DFW ACS %ile # students # DFW2001 23 61 1200 2702002 30 72 1199 3622003 35 72 1314 4532004 44 75 1429 6252005 23 72 1265 2902006 19 72 1260 2402007 11 76 1306 1502008 18 79 1300 2302009 15 75 1570 236

Reforms

• For all sections (up to 15, around 1500 students)– Reduce class size (to about 100 from 180)– Remove content (~30%)– Weekly meetings to negotiate “big ideas” and learning

outcomes and assessments (backward design)– Common syllabus, learning objectives and exams– Increased conceptual focus– Add active learning (group work, clickers etc.)

• Each faculty member used their own notes/class management style, homework

• There was typically no difference in grade distributions, despite the level of active engagement

Success!

Year %DFW ACS %ile # students # DFW2001 23 61 1200 2702002 30 72 1199 3622003 35 72 1314 4532004 44 75 1429 6252005 23 72 1265 2902006 19 72 1260 2402007 11 76 1306 1502008 18 79 1300 2302009 15 75 1570 236

However…

While these students perform well above the average on nationally normed exams, and have done

everything that we asked of them there are some problems…

Evidence

I will use a core idea in chemistry –structure-property relationships

0

10

20

30

40

50

60

70

80

90

2 3 4 5 6 7 8 9

Ave

rage

Su

cce

ss R

ate

(%

)

Number of Atoms

Average Success Rate vs. Number of Atoms

We found that even Ochem students had difficulties drawing chemical structures…

The transition from one to two carbon atoms!

Cooper, M. M.; Grove, N. P.; Underwood, S. M.; Klymkowsky, M. W. J Chem Educ. 2010, 87, 869-874 , DOI: 10.1021/ed900004y

Students did not know what these structures were for


Underwood and Cooper, Science Education (in minor revision)

Underwood and Cooper, Chem. Educ. Res. Pract., 2012, 13 (3), 195 - 200

Information

General

Chemistry (%)

N=32

Organic

Chemistry (%)

N= 134

Graduate

Students (%)

N=10

Structural information 100 100 100

Chemical/Physical

Properties 56 31 50


It became apparent that many students do not understand what Lewis structures are for

What information can be obtained from Lewis structures?

Underwood and Cooper, Science Education (in minor revision)

Underwood and Cooper, Chem. Educ. Res. Pract., 2012, 13 (3), 195 - 200

We also interviewed students to see how they would use chemical

structures to determine properties

Cooper, Corley and Underwood. J. Res. Sci. Teach. 2013, 50 (6), 699–721.

Difficulties with the model of phase change:

Brittany (GC2)

(a) (b)Melting Boiling

“Hold on, I’ve never thought about all this stuff before”

“more means more”

• “This carbon (ethanol) is more substituted so I would say this carbon (methanol) has a lower boiling point…it has less bonds to break.”

• “Because it [ammonia] has more, more bonds…The nitrogen has attached to three hydrogens but the oxygen’s only attached to two…More bonds means more intermolecular forces so it should be, it should have a higher melting point”

diSessa, A. A. (1993). Cognition and Instruction, 10(2 & 3), 105–225.Hammer, D. (2000). American Journal of Physics, (68), S52–S59

Pseudon

ymCourse

Phase/

Phase

Change

Represe

n-tations

Termino

logy

Heuristics

PersonalInstruc-

tional

More

Means

More

Noah GC2 1 0 1 1 1 1

Brittany GC2 1 0 1 0 1 1

Tina GC2 1 1 1 1 1 0

Susan GC2 0 0 1 1 0 0

Erin GC2 0 1 0 0 1 0

Lucy GC2 0 1 1 1 1 0

Justin GC2 0 1 0 1 0 1

Robin OC1 0 1 0 1 1 1

Ted OC1 0 0 1 1 1 1

Lily OC1 1 1 1 1 1 1

Marshall OC1 1 0 1 1 0 1

Victoria OC1 0 0 1 0 0 1

Daisy OC2 1 0 1 1 0 1

Joy OC2 0 1 1 1 1 1

Jill OC2 1 1 1 1 0 1

Jane OC2 1 0 1 1 0 1

Joe OC2 0 1 1 0 1 1Cooper, Corley and Underwood. J. Res. Sci. Teach. 2013, 50 (6), 699–721.

Students did not understand the difference between intermolecular

forces and bonds

Hydrogen Bonding drawing examples:

Between molecules code:

Within the molecule code:

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Hydrogen

bonding

Dipole-dipole LDFs Hydrogen

bonding

Dipole-dipole LDFs

Within Between

Comparison of IMFs drawing code frequencies for

Cohort 1 and Cohort 2

Cohort 1 (N=94) Cohort 2 (N=160)

How can we design a better learning environment?

And how will we know when we have been successful?

Creating a Coherent STEM Gateway at Michigan State

University

“The first two years of college are the most critical to the retention and recruitment of STEM majors”

- President's Council of Advisors on Science and Technology (PCAST, 2012)

A project funded by the AAU STEM Education Initiative Project

We decided to use the best available evidence

Disciplinary Core Idea:

Disciplinary significance

Explanatory Power

Generative

Biology Evolution

Cell Theory of Life

Energy and matter flows and transformations

Chemistry Bonding and

Interactions

Structure and Properties

Energy

Physics Interactions are

mediated by fields

Interactions can cause changes in motion

Energy is conserved in closed systems

Examples

Experts knowledge is organized into an underlying framework that reflects deep

understanding of the discipline

NRC: “How People Learn” (2000)

Prior knowledge

New knowledge

Future knowledge

Knowledge must be linked and contextualized

NRC: “How People Learn” (2000)

Knowledge is not “enough”Students must know what to do with

that knowledge

We are adapting the scientific practices from the framework

Scientific and Engineering Practices

1. Asking questions and defining problems

2. Developing and using models

3. Planning and carrying out investigations and design solutions

4. Analyzing and interpreting data

5. Using mathematics and computational thinking

6. Developing explanations and designing solutions

7. Engaging in argument from evidence

8. Obtaining, evaluating, and communicating information

How we put knowledge to use

NRC Framework for Science Education 2012

Crosscutting Concepts

1. Patterns

2. Cause and effect

3. Scale, proportion and quantity

4. Systems and system models

5. Energy and matter conservation, cycles and flows

6. Structure and function

7. Stability and change

NRC Framework for Science Education 2012

Ideas that cut across and are important to all science disciplines

A shared vision for curriculum reform:

Engage faculty in discussions to build consensus on key issues.

– What are the core ideas in the discipline?

– What scientific practices are important?

– What cross-cutting concepts make connections among disciplines

The result – three dimensional learning.

Our premise:

Engaging faculty to determine the core ideas,

science practices and cross cutting concepts

promote change

leads to changes in classroom practice

and changes in assessment

practices

Cooper et. al. Science, “accepted”

Our approach

Scientific

Practices

Crosscutting

Concepts

Disciplinary

Core Ideas

NRC. (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core

Ideas. Washington DC: The National Academies Press.

Blending the three dimensions

Scientific

Practices

Crosscutting

Concepts

Disciplinary

Core Ideas

Three Dimensional

Learning



Will affect both assessments and instruction

Assessments Instruction

Scientific

Practices

Crosscutting

Concepts

Disciplinary

Core Ideas

Three Dimensional

Learning



We are measuring change

• By investigating classroom practice

– Using the Three Dimensional Learning Observation Protocol (3D-LOP)

• And the nature of course assessments.

– Using the Three Dimensional Learning Assessment Protocol (3D-LAP)

Characterizing Instruction


Scientific

Practices

Crosscutting

Concepts

Disciplinary

Core Ideas

Three Dimensional

Learning

Existing Observation Protocols

• Such as TDOP, RTOP, COPUS, etc.

• Focused on “How” the class is taught

– What students are doing

– What instructor is doing

– Interactions between students and instructor

– Student-centered

– Active learning techniques

• Don’t tell you about “What” is being taught

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48ListeningIndividualClicker2GroupsAnswering2QuestionStudent2QuestionLecturingReal;time2WritingFollow2UPPose2QuestionClicker2QuestionAnswering2QuestionMoving/Guiding12on21Demo/VisualsAdministrationWaiting

Studen

tsInstructor

Using the COPUS to code a class

First half of class

Using the (3D-LOP) to code a class

First half of class

Three-Dimensional Learning Observation Protocol (3D-LOP)

The 3D-LOP can be used…

• to characterize both the instructor behavior, and whether the three dimensions are being addressed.

• in biology, chemistry, and physics.

• to generate evidence of change over time as part of a transformation effort.

• (Note it does not provide evidence of what is learned)

Characterizing Assessments


Scientific

Practices

Crosscutting

Concepts

Disciplinary

Core Ideas

Three Dimensional

Learning

Three dimensional Learning Assessment Protocol (3D-LAP)

Practices Does the item contain a practice (yes/no)

If so, which practice is presented)

Crosscutting Concepts Is there a crosscutting concept (yes/no)

If so which CCC is present

Disciplinary Core Ideas Is there a disciplinary core idea (yes/no)

If so, which DCI is present

Assessment as an evidentiary argument

Knowing what students know, NRC

Evidence Centered DesignC

on

stru

ct What knowledge do you want students to have and how do you want them to know it?

Evid

ence What will you

accept as evidence? How will you analyze and interpret it?

What task(s) will students perform that will allow them to communicate their knowledge?

Mislevy, R. J., & Risconscente, M. M. (2005). Educational Measurement: Issues and PracticeNRC 2014, Developing Assessments for the Next Generation Science Standards

Our Construct is 3D Learning

What will we accept as evidence of 3D Learning?

What does an assessment task look like?

21. Start with the number of protons in the nucleus of a lithium atom

… multiply by the number of 3s electrons in a magnesium atom in its ground state

… add the number of unpaired electrons in an oxygen atom in its ground state

… subtract the number of π orbitals in a triple bond… add the number of neutrons in the nucleus of the 14C isotope

What is the result?

a. 6 c. 10 e. 12 g. 14 i. 16b. 8 d. 11 f. 13 h. 15 j. 18

Core idea? No !

Science Practice? No

Crosscutting Concept? No

Not this!

Core idea? Forces and interactions



What about concept inventories?

FCI, Hestenes et. al.

What makes DNA a good place to store information?A. The hydrogen bonds that hold it together are

very stable and difficult to break. B. The bases always bind to their correct

partner. C. The sequence of bases does not greatly

influence the structure of the molecule. D.The overall shape of the molecule reflects the

information stored in it

Core idea? Information storage and transfer



BCI, Klymkowskyet. al.

Which has the highest boiling point?

A. CH3CH2CH3B. CH3OCH3C. CH3CH2OHD. They all have the same boiling point

Core idea? Structure property relationships



Even questions intended to elicit reasoning do not

Cooper, Corley and Underwood. J. Res. Sci. Teach. 2013, 50 (6), 699–721.

How might we elicit better evidence of what students know and can do?

Use the practices of modeling and explanation to make student thinking visible

How might we elicit better evidence of what students know and can do?

1. Draw the Lewis structures of CH3OCH3, CH3CH2OH

2. Draw three molecules of each substance and show where the strongest intermolecular forces are located

Follow up by using the representations to construct an explanation

a. Which substance do you think has the highest boiling point (claim)

b. What factors affect a substances’ boiling point? (evidence)

c. How do these factors affect the boiling point? (reasoning)

Core idea? Structure property relationships

Science Practice? Models/explanation

Crosscutting Concept? Cause and effect

But we have 450 students!

Pragmatically, we need to use multiple choice questions at some point

One of these two compounds is a liquid at room temperature, choose the compound, the evidence you are using to make the claim, and the reasoning that allows you to make this claim.Compound:

Evidence:III. Compound I is heavier than compound II.IV. Compound I has more hydrogens and can form more hydrogen bonds than II.V. Compound II has both hydrogens and oxygens capable of hydrogen bonding.

Reasoning:VI. Heavier molecules are more likely to cluster together and form liquids because they are attracted to each other strongly by London dispersion forces.VII. Molecules capable of hydrogen bonding are strongly attracted to each other and tend to cluster together to form liquids.

A. I, III, VI B. I, IV, VIIC. II, V, VII D. Not enough information

3D-LAP Comparison of Exams

Traditional Exam

Q# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

SPCCCI

Transformed Exam

Q# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

SP

CC

CI

The 3D-LAP can be used…

• to analyze formative and summative assessments.

• to generate evidence of change over time as part of a transformation effort.

• to help instructors develop assessment items that measure knowledge-in-use.

• as a mechanism for influencing instructional design.

An example course transformation

“CLUE”

Our response to the need for a new approach to teaching and learning in general chemistry

CLUE is built on three core ideas

Cooper, M., Underwood, S. M., Hilley, C. Z. & Klymkowsky, M. W (2012). J. Chem. Educ., 89,

1351-1357.

Cooper, M., & Klymkowsky, M. W. (2013). J. Chem. Educ., 90, 1116–1122.

(held together by forces)

The course is developed as interconnected learning progressions

over two semesters

“Research-based descriptions of how students build knowledge and gain more expertise within and across a core idea over a broad

span of time”

1Shin, N., Stevens, S. Y., & Krajcik, J. (2010). In S. Rodrigues Routledge (Ed.), Using analytical frameworks for classroom research: Collecting data and analyzing narrative. London: Taylor & Francis. Corcoran, T., Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: An evidence based approach to reform.

Identify what students should know and be able to do

Design curriculum and assessments

Assess student learning

Use assessment results to revise curriculum, assessments

61

Hypothetical Learning

Progression

Empirically Tested

Learning Progression

Design research cycle

Brown, (1992) The journal of the learning sciences 2(2) 141-178.

A progression of ideas

Chapter 1: Atoms, interactions and models

Chapter 9: Networked biological reactions

SUMMARY: CLUE Materials

• Text (short: online or paperback)

• Interactive web-based learning and formative assessment materials (BeSocratic)

• Short “lectures” showing how to do common skills (YouTube)

• Assessments (Formative and Summative) on beSocratic

• Instructor Resource Materials

http://besocratic.colorado.edu/CLUE-Chemistry

To date we have taught CLUE in multiple institutions with multiple instructors

• Spring 2010 (pilot, 50 students)• Fall 2010 – spring 2011 (full year, one section 100 students) • Fall 2011 – Spring 2012 (full year, two sections: 200

students, multiple instructors)• Fall 2012 – Spring 2013 (full year, two sections: 200

students, single instructor)• Fall 2013 – Spring 2014 (full year, one section 430 students)• Fall 2014 - Spring 2015 (full year two sections enrollment

850 students, multiple instructors)• Fall 2015 –enrollment 3500 – so far so good!

We are now starting our 6th year of CLUE implementation and some

results are in:

Multiple Experimental Designs

Quasi Expermental, Control-Treatment:Matched cohorts of

studentsTargeted assessments

Design Research: Iterative rounds of

investigation leading to refinement of

curriculum

FallSemesterCLUE

Spring SemesterCLUE

After 2semester of O-Chem

Control (Traditional)

N = 120 N = 83 N = 32

Treatment (CLUE)

N = 93 N = 56 N = 24

Quasi-experimental design. Cohort 1: Students followed through 2 years of chemistry: 2010-2012

(replicated for 2011-2013 in Cohort 2)

Matched students by SAT, sex, major, MCAi, TOLT, motivation, etc.

Comparison of Lewis Structure drawing ability

0%

10%

20%

30%

40%

50%

60%

70%

Average for all 12 (post-Fall)

7 Harder structures(post-Fall)

Average for all (end ofSpring)

2 Most difficultstructures (end of Spring)

Control Mean Treatment Mean

Development and Assessment of a Molecular Structure and Properties Learning Progression,Cooper, Underwood, Hilley & Klymkowsky, J Chem Educ, 2012, DOI: 10.1021/ed300083a

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Within Between Ambiguous Not present Student DK

Hydrogen Bonding

Sp12 Traditional Univ. 1 (N=99)



Sp12 CLUE Univ. 1 (N=93)



Cooper, et. al. J Chem Educ, “in Press”

Surely organic chemistry students know about IMFs?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Hydrogen Bonding Dipole-dipole LDFs Hydrogen Bonding Dipole-dipole LDFs

Within Between

Longitudinal comparison of IMFs drawing code frequencies for CLUE

and Control at the end of gen chem (GC2) and Ochem (OC2)

Traditional GC2 (N=25) Traditional OC2 (N=25) CLUE GC2 (N=30) CLUE OC2 (N=30)

Identify what students should know and be able to do

Design curriculum and assessments

Assess student learning

Use assessment results to revise curriculum, assessments

80

Hypothetical Curriculum

Empirically Tested

Curriculum

We continue with the design research cycle

Brown, (1992) The journal of the learning sciences 2(2) 141-178.

We believe these results are a consequence of a carefully designed curriculum where ideas a explicitly connected and developed over the

two semesters.Where students are expected to

construct models, explanations and arguments – on a daily basis

Summary

• Curriculum reform should be based on theories of learning

• Learning should be carefully developed, scaffolded and linked

• Assessments should assess deeper learning and transfer to new situations (i.e. be 3-dimensional)

• Curriculum design should be informed by the results of assessments

• Its time to align what we teach with what we value as scientists

Acknowledgements

Marcos D. CaballeroDiane Ebert-MayCori L. Fata-HartleySarah E. JardelezaJoseph S. KrajcikJames T. LavertyRebecca L. MatzLynmarie A. PoseySonia M. Underwood

Leah C WilliamsChristopher MinterKeenan NoyesNicole BeckerKathryn KohnOscar Judd

Mike Klymkowsky

Transforming how we teach is good, but transforming what ... › downloads › bestpractice2015 ›...

Documents

Transcript of Transforming how we teach is good, but transforming what ... › downloads › bestpractice2015 ›...