Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr...
-
Upload
eleanor-gray -
Category
Documents
-
view
215 -
download
0
Transcript of Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr...
![Page 1: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/1.jpg)
Do the new formative work based placed assessments improve
educational impact?Dr Hermione Race, Dr David James, Dr Caroline Fertleman
London School of Paediatrics Trainees Committee
![Page 2: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/2.jpg)
Outline
• Implementation of two new work-based assessments nationally
• Why• How• Results of a survey assessing impact of new
WBA’s• Further developments
![Page 3: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/3.jpg)
Introduction
• The usefulness of an assessment is a product of its reliability, validity, cost-effectiveness, acceptability and educational impact
• Traditional approaches to maximise the reliability or reproducibility of assessments can have a negative educational impact on the learner by reducing the opportunity for meaningful developmental feedback.
![Page 4: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/4.jpg)
CbD
![Page 5: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/5.jpg)
SAIL
![Page 6: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/6.jpg)
Methods
![Page 7: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/7.jpg)
New Safeguarding CbD
![Page 8: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/8.jpg)
New DOC
![Page 9: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/9.jpg)
Survey Design• Assessment subgroup of the trainees committee
designed survey on experience of new assessments• Had the new assessments improved educational
experience for both trainees and tutors?
• Sent to London School of Paediatric trainees and to college tutors
• June after ARCP• Responses from 161 trainees, evenly distributed
throughout training years• 35 tutor responses
![Page 10: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/10.jpg)
Survey Results - CbD
• Majority of trainees completed one Safeguarding CbDs
• 15- 20 minutes to complete• 69% were completed with the trainer present
![Page 11: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/11.jpg)
Survey Results - CbD
Compared with your previous experience of the old CbD, do you feel that the safeguarding CbD better allows the opportunity to:
– Assess your safeguarding knowledge, application and clinical skills?
– stimulate your learning more?– create clear and achievable learning objectives?
![Page 12: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/12.jpg)
Survey Results – CbD
p = <0.05 p=<0.05
Trainees
![Page 13: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/13.jpg)
Survey results - CbDTutors
p = <0.05 p = <0.05 p = <0.05
![Page 14: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/14.jpg)
Survey Results - CbD
What are the barriers to completing a Safeguarding CbD?
![Page 15: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/15.jpg)
Survey Results - CbD
Describe what you like and dislike about the safeguarding CbD.
Like Dislike
Like specific safeguarding CbD Assessments are still a tick box exercise
Better structure Only as good as the assessor
Opportunity to learn and get feedback
The need for reflection
Encourages trainers to make comment
Finding time with a consultant
Only suitable for certain types of placement
![Page 16: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/16.jpg)
Survey Results - DOC
• Majority had performed 5 DOCs, taking 5-10 minutes each
• Only 49% performed in person with the trainer
![Page 17: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/17.jpg)
Survey Results - DOC
Compared with your previous experience of SAIL, did you feel that performing the DOC better allowed you the opportunity to assess:
– Written correspondence?– Stimulate learning?– Create clear and achievable learning objectives?
![Page 18: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/18.jpg)
Survey Results - DOCTrainees
![Page 19: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/19.jpg)
Survey Results – DOCTutors
p = <0.05 p = <0.05 p = <0.05
![Page 20: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/20.jpg)
Survey results - DOC
What are the barriers to completing a DOC?
![Page 21: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/21.jpg)
Survey Results - DOC
Describe what you like and dislike about the DOC?
Like Dislike
Better Structure Difficulty in finding a consultant
Better opportunity to learn and get feedback
Less relevant for more senior trainees
Online better than paper for eportfolio purposes
Still not broad enough
Straightforward
![Page 22: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/22.jpg)
Conclusions
• New Safeguarding CbD format favoured by trainers and trainees
• New DOC favoured by trainers but not trainees
• Trainees felt their learning was not stimulated more by the new assessments
• It is a step in the right direction
![Page 23: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/23.jpg)
The future
• Need to change the culture of work based placed assessments in paediatrics– Possibly having more assessments
• An app to allow them to truly be supervised learning events done at the time of assessment
• Tailoring of assessments to GRID trainees and more senior trainees
![Page 24: Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ef55503460f94c09181/html5/thumbnails/24.jpg)
Any questions?