1

2

Using ExamSoft to Facil itate Active Retrieval and Promote Student Success

David J. Caldwell, PharmD, AAHIVPDirector of Professional Affairs, ULM School of Pharmacy

Elizabeth M. Lafitte, PharmD, BCPSAssistant Professor – Clinical Sciences, ULM School of Pharmacy

3

Objectives• Describe active retrieval theory.

• Elucidate how active retrieval can be used as a formative assessment method.

• Describe the implementation of an active retrieval exercise to promote student success using ExamSoft.

• Demonstrate the value of ExamSoft in facilitating an active retrieval exercise.

4

Let’s do a Kahoot!

5

6

7

8

Let’s do a Kahoot!

9

Myth #1The key to learning is to study, study, study.

10

Active retrievalActive retrieval theory•Retrieval enhances learning•Each act of retrieval improves the ability to re-retrieve knowledge

What is active retrieval?•The active reconstruction of knowledge based on context and retrieval cues

11

Studying versus testing and effects on retention

Roediger HL, Karpicke JD. Psychological Science, 2006; 17:249–255.

12

Studying versus testing and effects on retention

Butler AC. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2010; 36:1118–1133.

13

Active retrieval as formative assessment

• The only way to assess knowledge is by engaging in the act of retrieval

• We can’t examine what a student has stored; we can only examine what they reconstruct

• Use self-quizzing for retrieval practice

Karpicke JD. Current Directions in Psychological Science. 2012;21(3):157-163.

14

Formative assessment with feedback

Roediger HL and Butler AC. Trends in Cognitive Sciences. 2011;31:20-27.

15

Let’s do a Kahoot!

16

Myth #2Learners are good at knowing when they know something.

17

Active retrieval as a metacognitive activity

Nunes LD, Karpicke JD. Retrieval-Based Learning: Research at the Interface between Cognitive Science and Education. In: Scott R and Kosslyn S, eds. Emerging Trends in the Social and Behavioral Sciences. Hoboken, NJ: John Wiley and Sons; 2015.

18

Getting practical:Implementation and utility of ExamSoft

19

The ULM experience: overview

Identify classes where academic difficulty is

common

• Pathophysiology I• 2 semester credit hours

• Principles Drug Action I• 5 semester credit hours

Identify students who have a higher likelihood of

academic difficulty

• GPA < 3.0• Multiple Ds or Fs in pre-

pharmacy curriculum

Implement early intervention program

• Active retrieval self-quizzing• Attend honor society tutoring

and review sessions• Review missed items and

strengths and opportunities reports after exams

• Attend office hours for professors

• Participate in faculty-led mentoring program

20

The ULM experience: preparing the quizzes

• Incentivized weekly quiz for each class•Posted for download the Monday after a week of material complete•Open until corresponding exam date

•Rho Chi (honor society) students wrote questions•Created template for ease of import•Students used rationale feature to reference notes/textbook pages

•Allowed 5 downloads for each weekly quiz

21

The ULM experience: question writing

• Provided written instructions• Provided RTF template

• ~ for rationale• * for correct answer

22

ExamSoft feature: import questions

23

ExamSoft feature: bulk question coding

24

ExamSoft feature: bulk question coding

25

ExamSoft feature: rationale• Rationale provides

feedback for self-quizzing

• Easily imports from RTF file with ~ after the question stem

26

Quiz setup• Categorized questions

by week and course• Enabled multiple

downloads• Enabled secure review

27

E x a m S o f t f e a t u r e : e d i t / s a v e p o s t

• Easy to update without re-creating the assessment• Settings• Secure Review• Additional Security

Options

28

ExamSoft feature: exam taker results

29

ExamSoft feature: longitudinal reporting

• Longitudinal report allowed for easy creation of “progress report”• Ran and downloaded longitudinal

report• Exported as Excel spreadsheet• Used conditional formatting to

highlight empty cells• Uploaded to view only

GoogleSheet• Students could see their attempts

in one place using a unique student ID

30

ExamSoft feature: elapsed time report

31

ExamSoft feature: elapsed time report

32

The ULM experience: student success

What changed?

• School-wide implementation of new early intervention and remediation policies

• Bonus points for • Boot camp exam (DA, Patho

and Pharmaceutics)• Practice quizzes (DA and

Patho)

P1s 2014

2015

PHRD 4002 - DA 1Passed post-remediationFailed post-remediationFailed outright

540

100

PHRD 4012 - Patho 1Passed post-remediationFailed post-remediationFailed outright

040

000

Total P1s modified 8* 0

33

The ULM experience: post-survey feedback

• 83/91 (91%) used provided self-quizzes

• 80/91 (88%) would like to see quizzes provided again

• 40/91 (44%) used active retrieval as primary study method

34

The ULM experience: student comments• These questions were very useful because it allowed me to think about

concepts in different ways from experienced students. • I often used the practice quizzes generated by older students to help me

to think of other ways that a question may be posed for the material, and what other students felt was an important thing to focus on.

• I thought the quizzes were great and helped me gauge how well I was remembering the material.

• The quizzes were very helpful in studying because I find testing myself to be a great way to learn.

• Re-taking the quizzes helped me learn the information quicker than my previous strategies.

35

Using ExamSoft to Facil itate Active Retrieval and Promote Student Success

David J. Caldwell, PharmD, AAHIVPDirector of Professional Affairs, ULM School of Pharmacy

Elizabeth M. Lafitte, PharmD, BCPSAssistant Professor – Clinical Sciences, ULM School of Pharmacy