Thinking about Methods: Perspectives on How We

Do What We DoJeffrey L. Bernstein

Department of Political Science

Eastern Michigan University

Why I Proposed This Workshop

• Carnegie experience – “methods anxiety”

• Recent scholarship (McKinney edited volume) on methods

• Chick

• Grauerholz and Main

• Are we talking past each other? Are we creating false dichotomies?

Tell Me Who You Are

• What methods do you use?

• What methods do you want to use?

• What thoughts/concerns do you have about methodological issues in the scholarship of teaching and learning?

A Potpourri of Methods Topics

• Do we need a control group?

• Do we need a pre-test/post-test?

• How “generalizable” must our work be?

• What constitutes evidence?

Indulge Me in an Example

• The drunk driving article:

• Powell, Jack L. and Aaron D. Drucker. 1997. “The Role of Peer Conformity in the Decision to Ride with an Intoxicated Driver.” Journal of Alcohol and Drug Education 43: 1-7.

Insights from the Drunk-Driving Study

• Internal validity – the degree of confidence we have that our treatment is causing the effect

• External validity – the degree of confidence we have that our results are generalizable to the big world out there

Reflections on Some of These Issues

• Control groups:

• Impractical

• Unnecessary

• Unethical

• Awesome

Reflections on Some of These Issues

• The Nature of our Evidence

• Must reflect our disciplinary conventions

• Could move beyond our disciplinary conventions

• Must be transparent and possibly replicable

• Rigor is not a bad word

Reflections on Some of These Issues

• Generalizability

• Apples and oranges

• Apples and tractors

• There are more oranges in the world than tractors

• The value of collaboration

Thinking about Your Work

• Consider a SOTL project on which you are working

• How can you make the project as internally valid as possible?

• How can you make the project as externally valid as possible?

• What do you need to acquire in a methodological trading zone?

Thanks for Joining Me!

Questions/comments/further discussion?

Jeffrey L. Bernstein

Department of Political Science

Eastern Michigan University

734-487-6970

jeffrey.Bernstein@emich.edu