Impact of a Structured Skills Lab Curriculum on

Operative Judgment Deb DaRosa, David Rogers,

Reed Williams, Linnea Hauge,

Ken Murayama, Alex Nagle,

Keith Millikan, and Gary Dunnington

PurposeDevelop and test an instructional

design template for skills lab curriculum that is:

1. Based on pertinent theories

2. Focuses on judgment & procedural skills

3. Is feasible and portable

4. Includes evaluation component

Research Question

Is there a difference….

in the judgment and technical skills of residents exposed to a skills lab curriculum based on a structured instructional design template?

TeamLeaders

Curriculum Team

David Rogers*

Linnea Hauge

Ken Murayama

Deb DaRosa

Evaluation TeamReed Williams*Keith MillikanAlex NagleGary DunningtonDeb DaRosa

Cognitive Task Analysis

• Arises from Human Factors

• Group of analytical techniques designed to capture expert thinking

Elstein AS. Decision analysis in surgical education. World J. Surg 1989;13:287-91.

CTA Example

During the course of a laparoscopic cholecystectomy, the surgery resident variously strays into the lumen of the gallbladder and the parenchyma of the liver.

Experimental Design

T group RO1 X O2

C group RO3 O4

Institution T C Total A 9 7 16

B 13 15 28C 10 9 19

Total 32 31 63

Instructional Design Template

Curriculum & Learning Theories• Whole-Part Method• Objectives• Feedback – written/oral• Distributed learning episodes• Active & learner-centered• Repetition and practice• Problem & experience-oriented

Curriculum: Session 1

Both T and C Groups - Baseline

• Orientation by Chair or PD

• Written pretest

• View MedAscend CD of Lap Chole

• Question/Answer period

• Box trainer exercises

Written Exam ItemsWhich trocar placement is optimal

for a laparoscopic cholecystectomy?

Example: Written TestWhich retraction of the gallbladder

provides optimal exposure of the cystic duct?

Example: Written Exam

List up to three methods for determining whether the veress needle has penetrated into the abdominal cavity.

Curriculum: Session 2Homework: Pblm centered exercise

Assigned readings

• Pre trial testimony

• Review designed CD-ROM

– 12 critical steps of LC

• Worksheet to record steps

• Error brainstorm session

• Summary

Curriculum: Session 3• Recall exercise w/

CD-ROM

• Review/Discussion

• Porcine Practice

– Think aloud rehearsal

– Peer feedback using Porcine checklist

Curriculum: Session 4

Both Groups

• Written post-test

• Porcine exercise (45 minutes) using think aloud rehearsal

• Faculty (blinded) direct observation with final feedback – Porcine checklist

Measures and AnalysisMeasurement Instruments

• Written pre and post-test

• LC Porcine Evaluation Form

• LC Evaluation Form for OR

Data Analysis

• Cronbach’s Alpha Coefficients

• Student’s T-test

Results: Reliability of Measures

Measure Knowledge Skill Errors

Pretest .75

Post test .64

LC Porcine

.83 .90 .53

LC OR

Eval

.74 .71 .50

Results: LC Porcine

Items T

M (sd)

C

M (sd)

t p

Knowledge 36(5.8) 36(6.9) -.10 ns

Technical 68(21.5) 56(26.7) -1.6 ns

Errors 47(52.9) 41(48.6) -.33 ns

Results – OR Evaluation

Items T

M (sd)

C

M (sd)

t p

knowledge 34(3.1) 32(4.5) -1.31 ns

Technical 25(2.7) 23(2.9) -0.10 ns

Errors 4.0(3.7) 4.2(3.3) 0.12 ns

Written Pre and Post-test

Measure T

M (sd)

C

M (sd)

t p

Pretest 48(11.1) 51(9.2) 1.16 ns

Post-test 61(6.2) 55(8.0) -2.47 .02*

Pre/Post

Difference

11 (0.6) 4(10.3) -2.38 .02*

Study Limitations• Small sample size

• Subject mortality – OR evaluation

• Insufficient practice time

• Changes in study personnel

• Outcome evaluation flaws– Talk aloud– Eubanks items– Direct observation by faculty

ConclusionsInstructional Design Template

Effective in:

• CTA methodology – blueprint for cohesive curriculum and reliable exams

• Enhancing intra-operative judgment as assessed on written examination

Revised Instructional Design Template: Part 1

Descriptive Information• Assumptions about learners• Description of skill

- Cognitive task analysis - Procedural task analysis

• Potential errors and complications - Prevention, anticipation, &management strategies

Revised Instructional Design Template: Part 2

Instructional Plan

• Objectives and Teaching plan

– Time, resources, sequence, etc

• Homework/assignments

• Guidelines for facilitator/learners

• Practice guidelines

– Frequency, intensity, duration

Practice, Practice, Practice

Ericsson KA, Krampe RT, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Acad Med, 2004.

Revised Instructional Design Template: Part 3

Feedback and Evaluation

• Practice performance standards – Time, conditions, etc.

• How feedback will be given

• Pre-test description

• Post-test description

• Outcome evaluation criteria

Thank You!

Foundation of the Association for Surgical Education (CESERT)

SIU: Janet Ketchum

Rush: Daniel Deziel, MD

Northwestern: John Coyle,MD, Tom Powers & Deb Rooney