Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr...
-
Upload
eleanor-gray -
Category
Documents
-
view
215 -
download
0
Transcript of Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr...
Do the new formative work based placed assessments improve
educational impact?Dr Hermione Race, Dr David James, Dr Caroline Fertleman
London School of Paediatrics Trainees Committee
Outline
• Implementation of two new work-based assessments nationally
• Why• How• Results of a survey assessing impact of new
WBA’s• Further developments
Introduction
• The usefulness of an assessment is a product of its reliability, validity, cost-effectiveness, acceptability and educational impact
• Traditional approaches to maximise the reliability or reproducibility of assessments can have a negative educational impact on the learner by reducing the opportunity for meaningful developmental feedback.
CbD
SAIL
Methods
New Safeguarding CbD
New DOC
Survey Design• Assessment subgroup of the trainees committee
designed survey on experience of new assessments• Had the new assessments improved educational
experience for both trainees and tutors?
• Sent to London School of Paediatric trainees and to college tutors
• June after ARCP• Responses from 161 trainees, evenly distributed
throughout training years• 35 tutor responses
Survey Results - CbD
• Majority of trainees completed one Safeguarding CbDs
• 15- 20 minutes to complete• 69% were completed with the trainer present
Survey Results - CbD
Compared with your previous experience of the old CbD, do you feel that the safeguarding CbD better allows the opportunity to:
– Assess your safeguarding knowledge, application and clinical skills?
– stimulate your learning more?– create clear and achievable learning objectives?
Survey Results – CbD
p = <0.05 p=<0.05
Trainees
Survey results - CbDTutors
p = <0.05 p = <0.05 p = <0.05
Survey Results - CbD
What are the barriers to completing a Safeguarding CbD?
Survey Results - CbD
Describe what you like and dislike about the safeguarding CbD.
Like Dislike
Like specific safeguarding CbD Assessments are still a tick box exercise
Better structure Only as good as the assessor
Opportunity to learn and get feedback
The need for reflection
Encourages trainers to make comment
Finding time with a consultant
Only suitable for certain types of placement
Survey Results - DOC
• Majority had performed 5 DOCs, taking 5-10 minutes each
• Only 49% performed in person with the trainer
Survey Results - DOC
Compared with your previous experience of SAIL, did you feel that performing the DOC better allowed you the opportunity to assess:
– Written correspondence?– Stimulate learning?– Create clear and achievable learning objectives?
Survey Results - DOCTrainees
Survey Results – DOCTutors
p = <0.05 p = <0.05 p = <0.05
Survey results - DOC
What are the barriers to completing a DOC?
Survey Results - DOC
Describe what you like and dislike about the DOC?
Like Dislike
Better Structure Difficulty in finding a consultant
Better opportunity to learn and get feedback
Less relevant for more senior trainees
Online better than paper for eportfolio purposes
Still not broad enough
Straightforward
Conclusions
• New Safeguarding CbD format favoured by trainers and trainees
• New DOC favoured by trainers but not trainees
• Trainees felt their learning was not stimulated more by the new assessments
• It is a step in the right direction
The future
• Need to change the culture of work based placed assessments in paediatrics– Possibly having more assessments
• An app to allow them to truly be supervised learning events done at the time of assessment
• Tailoring of assessments to GRID trainees and more senior trainees
Any questions?