Assessment instruments
description
Transcript of Assessment instruments
![Page 1: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/1.jpg)
![Page 2: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/2.jpg)
Assessment instruments
![Page 3: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/3.jpg)
Types classroom assessment
• Placement assessment• • Screening assessment• • Formative assessment• • Summative assessment• (Berk R A, 2002)Are the students qualified, to advance to the
next, course level?
![Page 4: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/4.jpg)
Millers Pyramid
• Types of classroom assessment• Placement assessment• • Screening assessment• • Formative assessment• • Summative assessment• (Berk R A, 2002)
![Page 5: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/5.jpg)
Exercise
Reflect for a moment and write downassessment instruments that youknow and how they influence yourlearning!Remember difference between method andinstrument:Method of travel = by land, sea, air…Instrument of travel = train, bus, ship, plane…
![Page 6: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/6.jpg)
Taxonomies
• Classification of objectives or skills• • Hierarchic• – To achieve higher level one must• master lower one• • Motivation for holistic form of• education
![Page 7: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/7.jpg)
Bloom’s taxonomy
![Page 8: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/8.jpg)
Millers pyramid
![Page 9: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/9.jpg)
Millers pyramid
![Page 10: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/10.jpg)
Millers pyramid
![Page 11: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/11.jpg)
Validity
Proper assessment instrument?– Thermometer -> Temperature– Thermometer x-> Weight– MCQ -> Know & understand
– MCQ x-> Show, demonstrate, do
![Page 12: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/12.jpg)
• • Content (=direct) validity– – Relevance?– – Representative?– – Subjective!
• • Construct (=indirect) validity– – Test result -> Competence?
![Page 13: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/13.jpg)
Reliability
• Test-retest reliability• • Equivalent forms reliability• • Split-half reliability• • Scorer/rater reliability
– – Inter- vs. intrarater reliability
![Page 14: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/14.jpg)
Factors in clinical skills assessment that lead to lower reliability
• Two few stations or too little testing time• Check list that do not discriminate (too
easy or too hard)• Unreliable patients or inconsistent role
players• Idiosyncratic examiners• Administration problems
![Page 15: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/15.jpg)
Engaging with data
• Write down some observationsabout the data. What do you see?
• 2. Suggest explanations (hypotheses)• for the data.• 3. How would you test the hypotheses?
![Page 16: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/16.jpg)
Feasibility
• Cost in terms of resources and time of Construction
• • Marking• • Organisation• • Quality feedback
![Page 17: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/17.jpg)
Utility
• • U = Utility• • R = Reliability• • V = Validity• • E = Educational impact• • A = Acceptability• • C = Cost
![Page 18: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/18.jpg)
Types of assessment
Multiple Choice Questions (MCQs) Modified Essay Questions (MEQs) Patient Management Problems
(PMPs) Traditional essays questions Structured questions .filling the
blanks True or false questions
![Page 19: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/19.jpg)
Able to do
![Page 20: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/20.jpg)
![Page 21: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/21.jpg)
12 curriculum outcomes. SMS
What the
Clinician is able
to do-doing the right
thing
How the clinician approaches
Their practice-doing the thing
right
The clinician as
professional
The right
person doing it
![Page 22: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/22.jpg)
What the doctor is able to do 1-7
• Competence in clinical skills• Competence to perform procedures• Competence to investigate a patient• Competence to manage a patient • Competence in health promotion and disease control• Competence in skills of communication• Competence to retrieve and handle information
![Page 23: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/23.jpg)
How the doctor approaches their practice 8-10
• With understanding of basic and clinical sciences
• With appropriate attitudes, ethical understanding and understanding of legal responsibilities.
• With appropriate decision making skills and clinical reasoning and judgment
![Page 24: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/24.jpg)
The doctor as a professional 11-12
• Appreciation of the role of the doctor within the health service
• Attitude for personal development
![Page 25: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/25.jpg)
Multiple Choice Questions
They are designed to test knowledge• Outcomes assessed- knowledge of basic skills
and clinical sciences– 8patient investigation– 3 Patient management – 4 health promotion– Clinical reasoning ands judgment
![Page 26: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/26.jpg)
Multiple Choice Questions
They are designed to test knowledgeOutcomes assessed- knowledge of basic skills
and clinical sciences– 8patient investigation– 3 Patient management – 4 health promotion– Clinical reasoning ands judgment
![Page 27: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/27.jpg)
What do multiple choices questions measure
• Knowledge of specific facts• Knowledge of terminology• Knowledge of principles• Knowledge of procedures• The understanding and application of facts
and principles• The understanding of procedures• Reasoning to some extent
![Page 28: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/28.jpg)
How to construct MCQs
• State the question positively• Construct options that are grammatically
consistent with the item.• Do not use MCQs when other instruments
are more appropriate.•
![Page 29: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/29.jpg)
Multiple choice questions- principles(rules)
• The question itself or the stem must be appropriate
• Have one correct response to the question• Distracters or the incorrect responses among
the choices should be plausible
![Page 30: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/30.jpg)
Extended Matched Items
Management of diabetes in the community?
1.
1. start on oral hypoglycemic2. do diabetic annual review3. three monthly blood tests and review4. monthly blood tests and review5. Refer to podiatrist6. refer to hospital diabetes specialist nurse7. Trial diet control for 3 months8. 2start on twice daily insulin injection 9. Do fasting blood sugars10. Do haemoglobin A/c blood test
![Page 31: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/31.jpg)
Extended matching
Select the most appropriate course of action for the following patients
1. Woman, mid 20s ,14 weeks pregnant to have glycosuria on routine resting in GP ante natal clinic
2. Woman,mid 50s ,BMI 30, just found to have fasting blood sugar 15mmol/l,HBA a18.5%
3. South Asian woman, mid 30s complaining of fatique, thirst and dry mouth
![Page 32: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/32.jpg)
Extended Matched Items (EMIs)
Purpose – test knowledge and reasoning skillsOutcomes assessed- all outcomes with an
emphasis on knowledge and clinical science– 8patient investigation– 3 Patient management – 4 health promotion– 5Clinical reasoning ands judgment
![Page 33: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/33.jpg)
MCQ
• Multiple Choice Questions are used to tests students’ factual knowledge and to some extent the critical thinking of students. An attempt is made to test higher levels of Bloom’s taxonomy and not mere recall of facts.
Integratory
![Page 34: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/34.jpg)
• 1. A cervical smear taken from a thirty five year old lady was found to have the following:
Mitotic cells Multinucleated cells Loss of chromatin granules Cells that have lost adhesiveness
![Page 35: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/35.jpg)
Mcq
• Which of the following will be the best classification of this smear?
• a) CIN I• b) CIN III• c) CIN IV• d) CIN V• e) CIN II
![Page 36: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/36.jpg)
Which of the following will be the best classification of this smear?
a) CIN Ib) CIN IIIc) CIN IVd) CIN Ve) CIN II
![Page 37: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/37.jpg)
2. Tissue Processing in Histology is the most critical stage in entire processes that go on in a Histology lab. Which of the following will determine the processing procedure?
Urgent diagnosis The element to be demonstrated The materials availableThe skills involvedThe size of tissue for examination
![Page 38: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/38.jpg)
MEQ
• Modified Essay Questions will be used to assess the students’ ability to integrate facts and problem solving skills, which are fundamental to good practice. Question requiring consolidation of facts and critical thinking will be asked.
![Page 39: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/39.jpg)
1. A tissue section was found to be having a brown colour that was intracellular upon staining with H/E staining technique. The pigment was found to be soluble in alcoholic picric acid.
![Page 40: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/40.jpg)
Possible scenarios
i. What is the possible name of this pigment?
ii. What features enabled you to identify the pigment?
iii. What class would this pigment fall into?Assuming the pigment was not soluble in alcoholic
acid picric acid, how would this affect your identification
![Page 41: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/41.jpg)
Formatively as learning tool to stimulate
discussion and top plan future development
• Can be used to stimulate and provide feedback – may be every three months what is going , intriguing / exciting ,worrying phenomenon.
![Page 42: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/42.jpg)
What can MEQ measure ? MEQ can be used as a formal assessment
technique• It can be used as an aid to learning.• It can be used as an adjunct to a lecture.• It presents good questions for group
discussion.• It can be used as a self-instruction method.• It can be used as a self-audit device.
![Page 43: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/43.jpg)
Construction of MEQ
• Drawing up a draft of the subject area.• Circulating the draft to a team of fellow
professionals to study it.• Each of the professional colleagues should be asked
to provide the answer to each of the questions.• These responses should be studied and forms the
basis for model answers.• The marks should be allocated according to
consensus decisions
![Page 44: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/44.jpg)
PATIENT MANAGEMENT PROBLEM (PMP)
• Patient management problem (PMP) consists of short questions that require also short answers that revolve around managing a patient. They are intended to equip the student with knowledge, skills and attitude that are necessary in professional practice.
• The main feature is that a series of questions are asked about a case and options are provided that leads to proper management of the patient. It tests students on a wide range of subjects.
![Page 45: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/45.jpg)
What PMP can assess
• Like the MEQ, PMP provides the opportunity for students to demonstrate their ability in patient management skills and problem solving skills in a variety of scenario.
• The students are presented with simulated patients, real patient or imaginary patient in a paper and pencil format. The unfolding scenario is presented logically and the student is evaluated on the basis of the proper steps taken leading to the proper management of the patient.
![Page 46: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/46.jpg)
Constructing a PMP
• Constructing a PMP begins with a description of the symptoms of the patient and the resources that are available. The student is presented with the problem followed by a series of questions relating to management, diagnosis and investigations. Each question has a number of options from which the student must select. Beside each option are hidden instructions about how the student should proceed.
![Page 47: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/47.jpg)
Steps in constructing PMP
• Clearly define what it is you wish to test in the problem. Set realistic objectives for the group you are testing.
• Select a problem that is meaningful and suitable.• Ensure that the sequence of management is logical.• Prepare the opening scene. This should compose
of: the actual setting for the case, the presenting features of the patient, and the role of the user, the resources available and clear instructions.
![Page 48: Assessment instruments](https://reader035.fdocuments.in/reader035/viewer/2022062800/568140a0550346895dac550c/html5/thumbnails/48.jpg)
Steps continue
• Prepare a list of clinical choices which are consistent with the problem.
• Pretest the PMP using a group of subject experts. Also try it out with one or two potential candidates.
• Score the PMP. Allocate a mark for each individual choice on the PMP.