Post on 26-May-2015
APPLICATION OF ASSESSMENT AND EVALUATION DATA TO IMPROVE A
DYNAMIC GRADUATE MEDICAL EDUCATION CURRICULUM
TIFFANY L. SMITH, PATRICK B. BARLOW, ROBERT E. HEIDEL, & WILLIAM METHENY
A LAYOUT OF THE PRESENTATIONSetting the Scene
– The Task– The Complexities
Explanation of the tools created and the process of creating them
1. Assessment Tool2. Precourse Tool3. Classroom Activities and
Homework4. An Evaluation Tool5. Fine-Tuning
The Result of Our Efforts– How this process informs
our task– A lesson– Suggestions
Questions
SETTING THE SCENE
Systems thinking and complexity concepts
as frameworks for conceptualizing
interventions (Patton, 2010)
THE TASK
The creation of a blanket syllabus and set of learning objectives for research design and statistics that can be
worked into any lecture series.
THE COMPLEXITIES
• Target Audience– Time constraint– Intrinsic motivation toward research
• Accreditation and Accountability
• Absence of Materials
• Stakeholder Buy-In
THE PROCESS
• What did we start with?
• What is it like to be without a formal set of objectives or a syllabus?
• What do we do to change?
Where do we go from here?...
FIRST: AN ASSESSMENT TOOL
• Implementation of an initial assessment for the purposes of understanding:
– Student baseline knowledge
– Student progress at the completion of the series
SECOND: A PRE-COURSE TOOL
• The initiation of a pre-course survey
– What do our students want to get out of the course?
– Who are they?
– What are their interests?
– Perceptions of the usefulness of the course
THIRD: CLASSROOM ACTIVITIES AND HOMEWORK
• New classroom activities related to applying what they’ve learned
• New homework assignments to assess their ability to apply course content outside of the classroom
• A final research proposal project – Incorporating the course content into an oral
presentation
FOURTH: A POST-COURSE TOOL
• Development of a post-course survey to assess:
– Change in perceptions
– Change in attitudes
– Evaluation of instructor contribution
– Utility of the modules
A Constant Process of Fine-Tuning
THE RESULT OF OUR EFFORTS TO UNDERSTAND AND OPERATE IN A
DYNAMIC SYSTEM
We implemented all of these sweeping curriculum and measurement changes almost
exactly one year after our first experience teaching the material, and it was universally the most successful and rigorous workshop to date.
HOW THIS PROCESS INFORMS OUR TASK
• As this academic year’s workshops get underway, there will be a constant turning of the feedback wheel
• Utilization of data• Constant adaptation is KEY• Constant, critical reflection on our own work
(Stevahn, King, Ghere, & Minnema, 2005)• Must remember the complex nature of the task• This was not only an implementation of a
course. Not just the implementation of a new assessment.
• It was also the implementation of evaluation into the process of course development and modification.
A LESSONË
Learning objectives, like many things, cannot
be looked at like a logic model, with
rigidity and fixedness, but instead like a
process that will change over time.
SUGGESTIONS
Promote feedback and utilization of results
Remember the nature of the organization
Collaborate with colleagues
Use reflection for improvement and development
Think outside the box
QUESTIONS?THANK YOU!