Dr. Ching Ling Prof(Addl) Dept. of Obs & Gyn TNMC/OSCE 2.pdf · Define OSCE/OSPE Difference between...
Transcript of Dr. Ching Ling Prof(Addl) Dept. of Obs & Gyn TNMC/OSCE 2.pdf · Define OSCE/OSPE Difference between...
ObjectivesDefine OSCE/OSPE
Difference between OSCE and Conventional exams
Realize the circumstances that necessitated introduction of OSCE
Advantages and Disadvantages of OSCE
Planning an OSCE/OSPE
Traditional Long CaseOne hour with patient
History taking and examination not observed
Interrogated for 20 minutes
No structured questioning or marking by examiners
Lack of ValidityTeachers of opinion it tests recall and not depth of knowledge. Clinical competencies not measured.
Students of opinion that scores depend on whether they are examined at beginning or end of session. Ridiculing students answer is not uncommon
Lack of ReliabilityInter examiner reliability is low
Evans et al showed that marks awarded correlated positively with no. of words used and speaking time of students and negatively with that of the examiner
The oral marks did not show any correlation between the same students marks on an objective test on the same content material
Lack of ObjectivityLack of clarity in questioning and questions of variable levels of difficulty are common.
Since questions posed to candidates are not uniform and there are no guidelines for marking, scores given to candidates are arbitrary
Sometimes the examiner expect a predetermined correct answer and are unwilling to consider the candidate’s answer as a possible alternative
Problem With Oral ExamVariability – Student, Examiner, Patient are the three
Variables.
Patient Variability:
The disease he is suffering from
His temperament
His attitude towards his disease
The process of examination
Problem With Oral ExamExaminer Variable:
Marks awarded by one examiner differs from another for the same performance
Marks awarded by same examiner, for similar competence, on same day varies
Hence the patient and examiner variable should be reduced to a minimum
Problem With Oral ExamCompetencies not examined:
One tests only the product not the process since marks are awarded based on overall impression candidate makes on the examiner
Little effort made to evaluate various components of clinical examination
Clinical Competencies Not
Examined 1.To obtain detailed and relevant history
2.Carry out systemic physical examination
3.Reach provisional diagnosis
4.Select appropriate investigations
5.Interpret the results of the investigations
6.Recommend and institute appropriate treatment
7.Impart education to the patient
8.Communicate effectively especially when patient suffering from diseases like cancer
Problem With Oral ExamDifficulty in Conducting Clinical Examination:
A large number of patients and examiners would be required for a given number of students to improve outcome of clinical examination
Each candidate would have to be given a large number of cases
A lot of time required
OSCE/OSPEHarden and Gleeson in 1979 originally described it.
Now used in the quantitative assessment of competence in the field of undergraduate and postgraduate medical education
Definition
A method of clinical/practical examination where predetermined decisions are made on the competencies to be tested and checklists incorporating important evaluable skills are prepared
MethodologyCandidates rotate sequentially around a series of
structured cases located in ‘stations’
Specific tasks have to be performed in each station(history taking, examination of a patient or practical skill) An observer observes and marks on a checklist or rating scale
Some stations candidate has to write answers to questions. X rays, specimen or instrument may be kept. These stations does not have an examiner or observer
MethodologyStations are of various types
� Procedural station
� Linked station
� Must pass station
� Rest station
The Exam Day1 Registration
2Orientation
3Escorting to exam position
4Station instruction time
5The encounter
6Post encounter period
7Exam ends/Escorting to dismissal area
Station ProfileInstruction to students:
� Patient profile
� Time duration to do the task
� An observer, using a checklist, will assess your performance while you interact with the patient
Station ProfileBriefing to the observer:
What the student is required to assess
Observe the students using the following checklist, without any probing or interference from you
Station ProfileBriefing to the Patient:
Do not give more information than asked for
Try to answer the questions as clearly as possible
Try to be consistent with all the students as they interview you
Real patientsConsistency- must give same history each time
Can fall sick
Develop new signs/ lose old ones
Can get tired (10 students a day)
Simulated patientsConsistency- reliability
Training
Briefing
Database of actors
Scenarios in advance
Practice with each other and with examiner
ASSESSMENTThe students answer sheets and the examiners checklists are marked according to previously agreed scheme by all the examiners
The fundamental principle being every candidate completes the same assignments in the same amount of time and is marked according to a structured marking schedule
Scoring with ChecklistAdvantages:
� Helps examiner know what station designers are looking for
� Helps examiners be objective
� Facilitates use of non expert examiners
Disadvantages:
� Rewards primarily process/thoroughness
� May not sufficiently reward exceptional candidates
� Ignores examiners expertise
Global ScoringExcellent: Perfectly performed clinical task with no omission; no unnecessary procedure done
Proficient: Performed clinical task satisfactorily with minor mistakes
Average: Performed the task at an acceptable standard with some mistakes and unnecessary procedure
Poor: Failed to carry out the task. Lot of mistakes and misconception
Global ScoringAdvantages:
� Utilizes expertise of examiners
� Examiners are in a position to make a judgement about the performance
Disadvantages:
� Examiners have to be expert examiners i.e. trained
� Examiners have to be familiar with expected standards for the level of the test
OSCEObjective: Examiners use a checklist for evaluating trainees
Structured: Trainee sees the same problem and performs the same tasks in the same time frame
Clinical: The tasks are representative of those faced in real clinical situations
OSPEBasic Medical Sciences
A multi station, multi task process of assessment
Reflect the objectives – construct validity
Understand and apply theory
Test practical skills
Developing clinical skills – related to Basic Medical Sciences
Advantages of OSCE/OSPEReliable
1. Consistent scoring by examiners according to predetermined criteria
2.Candidates performs tasks across clinical, practical and communication skill domains- more reliable picture of candidates overall competence
3. Each candidate examined by number of different examiners at different stations, so multiple independent observations are collated. Individual examiner bias is thus attenuated
Advantages of OSCE/OSPE� Valid, Reliable, Objective
� Consistent examination scenarios for students
� Uniform marking scheme for examiners
� Generates feedback for both learners and teachers
� More students assessed at a given time
� Students clinical and interactive competencies examined
Advantages of OSCE/OSPE� Examiners are required only at certain stations
� Junior examiners may be appointed for other stations as checklists are provided
� Reproducible
� Can be used with large number of students
� The variable of the examiner and the patient are to a large extent removed
� Activity within the Department which promotes teamwork
Disadvantages of OSCE/OSPE� Requires an extensive amount of organizing
� Expensive in terms of manpower, resource and time
� Students skill assessed in compartments
� Observer fatigue
� Needs space
� Training needed
Educational ImpactStudents focus on their assessments rather than the learning objectives of the course
There is a danger in the use of detailed checklists as this may encourage students to memorize the steps in a checklist rather than learn and practice the skill.
To Sum Up
OSCE can assess clinical, communication and practical skills but are still situated in the context of an examination setting. To assess doctors in the context of their professional practice requires the use of different formats in the workplace
OSCE
OSCE has proved to be so effective that it is now being adopted in disciplines other than medicine like dentistry, nursing, midwifery, pharmacy and even engineering and law
Objective Structured Long
Examination Record (OSLER)Developed by Gleeson in 1997 to improve objectivity, reliability and validity of long case examination.
History taking: 1. Pace and clarity of presentation
2. Communication process
3. Systematic approach
4. Establishment of case facts
OSLER Physical Examination: 5. Systematic approach
6. Examination technique
7.Correct physical findings
Management: 8. Appropriate investigations
9. Appropriate management
10. Clinical acumen
OSLEREach item graded as P+ (Very Good)
P (Pass)
P-(Below Pass)
Final grade for each item is decided by conferring with co-examiners and so is an overall grade. All candidates are assessed on the same 10 items by the examiners over 20-30 minutes.