Assessing student skills in laboratory and research

experiences.

Benjamin D. Caldwell

Department of Chemistry

Missouri Western State University, St. Joseph, MO

April 29, 2014 Experimental Biology 2014, San Diego

What are the expectations (minimum vs. hopes/dreams!) for students completing BMB courses/programs Perform basic tasks adequately vs. designing experiments

Set learning objectives

What are the expectations for students coming into an introductory BMB course? Pre-course survey

Determine assess strategy for achievement of objectives Lab reports, written research/project proposals,

presentations, quizzes, exams, practical exams, etc…

BMB Lab Skills

Focused on basic BMB lab skills and methods

What skills do we want to teach in our courses?

What are our objectives for program?

How in depth can we want go? Introductory, theory, usage, proficient, mastery

First efforts

04/22/23 4

Lab skills Review of matrix and discussion

BMB SKILLS Adapted from BMBEd

Vol. 32, No. 1, pp. 11–16, 2004

WHEN SKILL IS LEARNED AND LEVEL TYPE

Lower Level Courses Upper Level BMB Courses

Quanti-tative

Quali-

tativeLAB TECHNIQUES Theory

Usage

Proficiency

Theory

Usage

Competence

Theory

Usage

Competence

Acid/Base chemistry                      

Affinity techniques                      

Amino acid analysis                      

cDNA                      

Cell Culture-Aseptic/Sterile Techniques                      

Cell Culture: Adherents vs. suspension                      

Cell Culture-Bactierial/insects                      

Cell Culture-Mamallian                      

Centrifugation-high speed                      

Centrifugation-separation and ppt                      

Centrifugation-subcellular fractionation                      

Centrifugation-ultraspeed                      

Characterization of carbohydrates                      

Characterization of lipids                      

Characterization of proteins                      

Chromatography-Affinity                      

Chromatography-ion exchange                      

Chromatography-size exclusion                      

Chromatography-HPLC                      

Cloning and Selection                      

Dialysis and Desalting                      

DNA array                      

DNA digests                      

Electrophoresis-DNA.RNA, agarose                      

Electrophoresis-proteins, PAGE                      

Enzyme kinetics                      

Fluorescence/ fluorimetry                      

Gene expression/transcription                      

Immunocytochemistry-Blots-Northern                      

Immunocytochemistry-Blots-Southern                      

Immunocytochemistry-Blots-Western                      

Immunocytochemistry-Immunoprecipitation                      

Immunocytochemistry-Immnofluorescence                      

Immunocytochemistry- In Situ Hybridization                      

Ligand-binding, ELISA                      

Membranes                      

Microscopy - light                      

Microscopy - phase                      

Microscopy - fluorescence                      

BMB SKILLS Adapted from BMBEd

Vol. 32, No. 1, pp. 11–16, 2004

WHEN SKILL IS LEARNED AND LEVEL TYPE

Lower Level Courses Upper Level BMB Courses

Quanti-tative

Quali-

tativeLAB TECHNIQUES

Theory

Usage

Proficiency

Theory

Usage

Competence

Theory

Usage

Competence

Acid/Base chemistry

C111 C120 C120 C120

C321 C370

C321 C370

C321 C370

C321 C370     X  

Affinity techniques      C370 C426 370   C370 C370      X

Amino acid analysis       C370     B313 B313     X

cDNA B215 B215    B421 B421 B421B215 B421

B215 B421 B421 X X

Cell Culture-Aseptic/Sterile Techniques

B215 B215 B215 B390 B390

B390, B421

B390 B421

B390 B421

B390 B421 B431  X

Cell Culture: Adherents vs. suspension

B205B215

B205 B215

B205 B215

B390 B411 B421

B390 B411B421

B390 B411 B421

B205 B215 B390 B421

B205 B215B390 B421

B390 B421 X X

Cell Culture-Bactierial/insectsB215 B215 B390 B390

B390 B421

 B390 B421

B390 B421

 B390 B421  X X 

Cell Culture-Mamallian        B411 B411            X 

Centrifugation-high speed B215 B215 C370 C370   C370 C370     X

Centrifugation-separation and ppt       C370 C370           X 

Centrifugation-subcellular fractionation

B215 B215 C370 C370   C370 C370     X

Centrifugation-ultraspeed                      

Characterization of carbohydrates                      

Characterization of lipids                      

Characterization of proteins       C370 C370  C370 C470 C370   X X

Chromatography-Affinity       C370 C370           X 

Chromatography-ion exchange      C370 C426    

C370 C426       X 

Chromatography-size exclusion       C370 C370   C370 C370

       X

Chromatography-HPLC       C426 C426          X  X

Experiments, procedures are available in variety of venues

BMBEd, CBE Life Sci Ed., J. Chem Ed., …

A few lab manuals available FLABB, Boyer, Farrell/Taylor ….

Course design – project based/inquiry based research

Assessment of laboratory skills???

What’s out there on lab assessment?

Assessment of student progress through entire lab course

Pre-course survey – assess student comprehension/retention from earlier courses Includes attitudinal survey + skills assessment

Midterm-assessment Final Exam

First half of course focuses skill building Second half – project based

Protein purification, enzyme kinetics

Our efforts

Pre-course skills survey Use the following scale, circling the appropriate statement that best matches your feeling about the following laboratory related questions. Strongly Somewhat Neutral/ Somewhat Strongly Disagree Disagree No Opinion Agree Agree

1 2 3 4 5 1. I am able to solve basic mathematic problems that require scientific notation or exponents.

1 2 3 4 5

3. I can accurately determine the correct number of significant digits in a number.1 2 3 4 5

029.0

3.211.545 x

79. Complete the following mathematical equation, and give your answer with the proper number of significant digits.

a. 4.0 x 105

b. 4.00 x 105

c. 4.00367 x 105

d. 400,367

Complete the following mathematical equation, and give your answer with the proper number of significant digits

Pre-Course Survey Sample QuestionsAttitudinal Question: I am able to solve basic mathematical problems that require scientific notation or exponents

Strongly Disagree

Somewhat Agree

No Opinion

SomewhatAgree

Strongly Disagree

0% 0% 0% 35% 65 %

Attitudinal Questions: I can determine the accuracy and precision of a micropipette from data.

Strongly Disagree

Somewhat Agree

No Opinion

SomewhatAgree

Strongly Disagree

6% 0% 12% 23% 59 %Pre-test: 6% Could correctly determine accuracy/precisionMid-term: 47% could determine accuracy/precision

029.0

3.211.545 x

Pre-test: 24% Could correctly perform calculationMid-term: 88% Could correctly perform calculation

1. Density2. Calculating Averages3. Understand Accuracy4. Understand Precision5. Understand Equipment Tolerances6. Recognize Units: Molarity7. Recognize Solution & Dilution Properties8. Can Perform Dilution Calculations9. Understand Concepts of Acidity10. Can Perform pH Calculation from Molarity11. Can Calculate [H+] from pH Data12. Understand How Buffers Maintain pH

I understand & can Identify dependent variables 0 0 6 54 40 69

I understand the differences between accuracy & precision 0 0 0 27 73 48

I understand basics of chromatography 0 19 21 29 31 67

I can generate a standard curve & determine concentrations of unknowns.

0 6 15 46 33 42

I understand the need for more than 1 type of protein assay 8 13 21 44 15 56

I understand the need for more than 1 type of protein assay 8 27 23 35 6 67

I understand the basic concepts & importance of enzyme kinetics

13 35 27 19 6 73

I understand how electrophoresis is used to separate protein/DNA

8 2 10 23 56 54

Str

ongly

Dis

agre

e

Str

ongly

Agre

e

Som

ewhat

Dis

agre

eNeu

tral

/No

Opin

ion

Som

ewhat

Agre

e

Lab T

ask

Quest

ions

(% C

orre

ct)

Given: Formal report on purification of protein characterization Introduction to the enzyme Materials and Methods Data and results

No Conclusions or Discussion

Final Exam

Exam Question 5

What type of experiment was used to detect the presence of the enzyme specifically in fractions collected from the gel filtration column? What substance and physical property were measured directly in this experiment to detect the presence of ATPase acivity?

Learning Objectives:• Demonstrate understanding of enzyme assays and assay methodologies.• Demonstrate ability to compare data from different types of experimental procedures and data.

Exam Question 8

Were the pipettes used in the protein assays all accurate and precise? Indicate which pipettes were accurate, precise, both or neither. Also, indicate why any pipettes were determined to not be accurate or precise.

Learning Objectives:• Demonstrate ability to perform basic statistical calculations and comparisons.• Demonstrate understanding of the concepts of accuracy vs. precision.

Exam Question 9What are the Km and Vmax values for the enzyme in the presence of MgATP and the analog CrATP. Which compounds binds more tightly to the enzyme, MgATP or CrATP, and explain how the data support your conclusion.

Learning Objectives:• Demonstrate understanding of enzyme kinetics and inhibition.• Demonstrate ability to use data to perform biochemical calculations

Exam Question 10Write a conclusion paragraph for this lab report. Be sure to comment briefly on a) the steps and procedures used to purify the enzyme, b) methods used to specifically identify fractions containing the protein, and c) compare the relative kinetics of the enzyme using MgATP or CrATP binds more tightly to the enzyme and possible mechanism of inhibition as demonstrated by the presented data.

Learning Objectives:• Demonstrate ability to synthesize concluding statements based on a large body of work using different laboratory methods and multiple forms of data representations.• Demonstrate the ability to communicate effectively.

Class Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Bonus Q's

Average Exam

Score (%)Spring 2010 92.2 76.3 92.5 75.6 66.7 64.4 68.1 80.4 56.3 81.5 70.4 79.2

Spring 2011 95.6 68.0 83.5 75.7 73.2 73.5 68.5 81.4 69.7 84.6 65.2 82.9

Student Performance on final exam questions.

Scores represent percentage of students receiving full credit for each question.

Able to identify areas still needing improvement (indicated by yellow) include:

• Interpreting graphic data (Q’s 2, 6, 9) – graphical/SDS-PAGE results• Comprehension of enzyme assays and kinetics (Q’s 5, 9)• Performing biochemical calculations (Q’s 7, 9)

Final Exam Results

Conclusion- Course wide approach to assessing and tracking student achievement of introductory biochemistry laboratory lab skills

Future work:Incorporate attitudinal component into final

exam study, possibly include exit interviews

AcknowledgmentsDr. Melissa Daggett, MWSU Biology Department

Merrill Schaeffer, MWSU/Boehringer Ingelheim Vetmedica

CHE 370 Students, MWSU