Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T...
-
date post
15-Jan-2016 -
Category
Documents
-
view
213 -
download
0
Transcript of Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T...
![Page 1: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/1.jpg)
Keele University School of Medicine
Reliability of the Leicester Clinical procedure Assessment Tool (LCAT), a
tool to support holistic generic assessment of clinical procedure skills
RK McKinley, J Strand, L Schuwirth
T Gray, T Alun-Jones, H Miller
![Page 2: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/2.jpg)
Keele University School of Medicine
Background
• Assessment– Repertoire of competence
• Micro-certification
– Changing roles• Skills extension• Skills cascade
– Volume– Implications for assessment
![Page 3: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/3.jpg)
Keele University School of Medicine
Instrument specification• Generic• Holistic• Multi-professional • Multi-level• Multi-modal• Enable high quality assessment:
– reliable, valid, acceptable, feasible, educational impact
• Nationally accredited
![Page 4: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/4.jpg)
Keele University School of Medicine
Methods
• Multi stage:– Systematic review– Qualitative development– National validation– Psychometrics– Implementation
![Page 5: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/5.jpg)
Keele University School of Medicine
Stage 1: Systematic review• LCAT v1.7
![Page 6: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/6.jpg)
Keele University School of Medicine
Stage 2: Qualitative
• Focus groups
• Observed assessments, debrief interviews
• Trial ‘hi-fi’ OSCE
• LCAT v2.5
![Page 7: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/7.jpg)
Keele University School of Medicine
Stage 3: Delphi
• National sampling frame– Recruitment– Questionnaire
• Definition of consensus• Agreement
– Categories: All >95%– Components: All >89%
• LCAT v3.0
![Page 8: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/8.jpg)
Keele University School of Medicine
LCAT v3.0
Category– Communication and working with
the patient– Safety – Infection Prevention– Procedural Competence– Team working
Components9
7
6
12
4
![Page 9: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/9.jpg)
Keele University School of Medicine
Stage 4: Psychometrics
• Hi-fi OSCE– 8 stations (2X3 tracks)
• Prosthetics + simulator• Venepuncture, venous cannulation, IV drugs,
ABG, ♀&♂ urinary catheterisation, skin suture, ECG
• ‘Prep station’• 15 minutes + 5 feedback
![Page 10: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/10.jpg)
Keele University School of Medicine
Stage 4: Psychometrics
• Analysis– Used data from 1 assessor per station– Assessors not systematically assigned to
stations– Variable group of assessors– Unbalanced design
![Page 11: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/11.jpg)
Keele University School of Medicine
Stage 4: Psychometrics
• Analysis– P (candidate)– S (stations)– A:(PS) assessors within candidates x
stations– PS,e interaction candidates and stations +
error
![Page 12: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/12.jpg)
Keele University School of Medicine
Stage 4: Psychometrics
• 46 candidates
• 50 assessors– 19 medical – 27 nurses– 1 midwife – 3 HCA
![Page 13: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/13.jpg)
Keele University School of Medicine
Stage 4: Psychometrics
• Results: variance
Effect Df Variancecomponent
% of totalvariance
P (candidates) 41 2.28 24.1
S (stations) 7 0.98 10.3
PS,e (candidate x stations plus general error)
287 3.61 38.1
A:PS (assessors within candidates x stations)
149 2.59 27.3
![Page 14: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/14.jpg)
Keele University School of Medicine
Stage 4: Psychometrics
• Results: D analysis
N stations
N Assessors
1 2 3
6 0.66 0.70 0.71
7 0.69 0.73 0.75
8 0.72 0.76 0.77
9 0.74 0.78 0.79
10 0.76 0.79 0.81
12 0.79 0.82 0.83
15 0.83 0.85 0.86
0.76
0.790.79
![Page 15: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/15.jpg)
Keele University School of Medicine
Summary• Generic• Holistic• Multi-professional • Multi-level• Multi-modal• Enable high quality assessment:
– reliable, valid, acceptable, feasible, educational impact
• Nationally accredited
![Page 16: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/16.jpg)
Keele University School of Medicine
Summary• Generic• Holistic• Multi-professional • Multi-level• Multi-modal• Quality:
– reliable, valid, acceptable, feasible, educational impact
• Nationally accredited
![Page 17: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/17.jpg)
Keele University School of Medicine
Acknowledgements• Analysis:
– Ron Hoogenboom, Cees van der Vleuten and Arno Muijtjens
• Colleagues: – Linda Ward, Clinical Librarian UHL – University Hospitals of Leicester – Eastern Leicester PCT – Students and staff
• University of Leicester • De Montfort University
• Funding: – Nation Health Service University – Leicestershire, Northamptonshire and Rutland WDC
![Page 18: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/18.jpg)
Keele University School of Medicine
![Page 19: Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.](https://reader036.fdocuments.in/reader036/viewer/2022070412/56649d4a5503460f94a26bcf/html5/thumbnails/19.jpg)
Keele University School of Medicine