car.ca Lifelong Learning... · Identification of Interventional Radiology procedural and...
41
CAR ASM 2016 | May 14-17 | Montréal, QC Mirza, S. 1 , Athreya, S. 1,2 1 McMaster University. 2 St. Joseph’s Healthcare Hamilton (SJHH).
Transcript of car.ca Lifelong Learning... · Identification of Interventional Radiology procedural and...
C A R A S M 2 0 1 6 | M a y 1 4 - 1 7 | M o n t r é a l , Q C
Mirza, S.1, Athreya, S.1,2
1McMaster University. 2St. Joseph’s Healthcare Hamilton (SJHH).
No conflict of interest to declare on behalf of the presenter and research team
1. Introduction to simulation training
2. Identification of Interventional Radiology procedural and non-procedural skills taught using simulation technology
3. Description of education validity of commercially available simulation technologies
4. Review of assessment tools applied in simulation training
Ultrasound guided peripherally inserted central catheter cannulation Simulator at McMaster University, Centre for Simulation-Based Learning (Hamilton, ON, Canada)
EVE Real-time high-fidelity Endovascular Simulator for real-time interventional neuroradiology at Massachusetts General Hospital, The Sim Group (Cambrige, MA, United States)
PERC Mentor Simbionix for percutaneous nephrostomy access under real-time fluoroscopy at The Chaim Sheba Medical Center, MSR, the Israel Center for Medical Simulation (Tel Hashomer, Israel)
Increased pressure on the healthcare system to be safer and more efficient
Problem Solution Safety of patient is paramount Train in a risk-free environment
without real patients Time constraints on
training Training can be tailored to a learners
pace Training can be self-directed or part
of a curriculum
Lack of uniformity in training Wide range of training cases available
Simulation: “devices, life-like virtual environments and contrived social situations that mimic problems, events or conditions that arise in professional [medical] encounters”1
Procedural skills: practical physical skills performed by a physician to complete a medical procedure
Non-procedural skills: skills not associated with a medical procedure but are integral to medical practice such as interpersonal, cognitive and personal resource skills
Simulators
o Virtual Reality – immersive computer programs which recreate the real world
o Mannequins – more specifically Computer-Enhanced Mannequins, full body models with computer systems which can replicate physiological responses
o Phantoms or Part-Task Trainers – model of an anatomic region used for teaching a specific skill
o Computer-Based Learning – digital simulations on a computer
Ultrasound Guided Liver Biopsy
CT Guided Biopsy
Ultrasound Guided Needle Procedures
Ultrasound Guided Peripherally Inserted Central
Catheter (PICC) Cannulation
Ultrasound Scanning and Interpretation
Transvaginal Ultrasound
Focused assessment in sonography for trauma
(FAST)
Seldinger Technique for Vascular Catheterization
Percutaneous Transhepatic Cholangiography
Ablation of Liver Tumors
Fluoroscopy Guided Lumbar Puncture
C-Arm Training
Stenting
Angioplasty
Cannulation
Endovascular Ruptured Aortic Aneurysm Repair
(rEVAR)
Catheterization
Percutaneous Coronary Revascularization
Palpation and Needle Insertion
Contrast Reaction, Sedation and Analgesia
Management
Ultrasound needle guided procedures2
o Needle insertion phantom (Advance Medical Technologies LLC, Kirkland WA)
o US machine used to guide the needle to target
Ultrasound Scanning and Interpretation3
o UltraSim by MedSim ultrasound simulator used with mannequin to generate simulated US images
Part task trainer for needle guided procedures
UltraSim ultrasound simulator
Virtual Reality
o ImaGiNe Seldinger has 2 stations o 1st station: needle puncture
• Needle inserted into mannequin with a simulated pulse and haptics device for needle feedback
• User wears 3D stereoscopic glasses to see anatomical overlay on mannequin.
o 2nd station: catheterization • Real guidewire and catheter inserted into simulated skin
model
• 2 haptics devices give feedback for needle holder and tool insertion
• LED light blinks when vessel is punctured
• Computer displays simulated fluoroscopy images generated from real patient data
ImaGiNe Seldinger Station 1
ImaGiNe Seldinger Station 1I
Virtual Reality o Computer displays US images
created from CT data
o Simulated US probe and microwave antenna used with abdominal phantom (CIRS 071, Elkhart, IN, USA)
o Program localizes and displays probe and needle on US images
Computer Based Training and Virtual Reality
o VirtX teaches user how to use a C-arm to reduce dose to patient and medical team o Computer based mode allows user to move c-arm, patient and table using the program
interface to complete task by taking DRR image
o Virtual reality mode allows user uses computer program, rayless c-arm, mannequin and table to complete task
Procedures: Renal, Iliac, Carotid,
Virtual Reality
o Procedicus-VIST by Mentice8, 9 ,10
• Mannequin with haptics devices in femoral access point allows insertion of modified tools
• Computer screens display fluoroscopic images and virtual tools
• Joystick for user to move virtual fluoroscope, zoom, replay and capture images, and move the table
• Foot pedals and control panel for control of the fluoroscopy suite
• Use of extra modules to simulate the injection of contrast, perform angioplasty, deploy stents, and perform fluoroscopy with digital subtraction angiography
o AngioMENTOR by Simbionix9
• Desktop for displaying simulated fluoroscopic images
• Haptics module for real tool insertion
• PROcedure program by Simbionix can use real patient data to generate images
VIST screens displaying procedure
AngioMENTOR simulator modules
Procedures: iliac artery, renal artery, superficial femoral artery, coronary11,12,13,14
Virtual Reality
o Procedicus-VIST
o AngioMENTOR
o ICTS27
• modified instruments inserted into haptics module
• VR module displays patient ECG and other vital sign data
• fluoroscopy images made from CT data and Visible Body Project anatomy
o NeuroCath28 • uses real patient data to displays 3D and fluoroscopy images
• program allows user to choose tools and insertion point
• control panel to manipulate table and a haptics device for catheter insertion
ICTS simulator
NeuroCath simulator
Procedures: renal arteries, superior mesenteric, internal carotid
Virtual Reality
o In-house simulator called VPAS (videoscopic phantom based angiographic simulation for fluoscopic cannulation of arterial tree vessels)
o User cannulates a silicon phantom with mechanical blood flow system
o Camera records phantom and display as fluoroscopic images to guide user navigation since phantom is hidden from view
Procedures: cardiac, carotid, subclavian, ascending aorta, renal16,17,18
Virtual Reality
o Procedicus-VIST
o STRESS simulator
• User views screen which merges an Xray image with video of glass phantom
• Phantom model of abdominal aorta, renal and iliac arteries
• User inserts guidewire and catheter into phantom
• Program’s injection option simulates contrast injection
o In-House simulator19
• User inserts tools into homemade haptics device
• Real CT images used to create 3D images of vasculature
In-House simulator
STRESS simulator
Virtual Reality
o daVinci20 used for patient-specific treatment planning and IR device testing
o Uses CT and MRI data to reconstruct images
o Guidewires and catheters are inserted into haptics device which can be integrated with a mannequin interface
o Computer screen shows fluoroscopic images or 3D views so user can see vasculature
o User can make endoscopic views and save "vascular roadmaps" created during contrast injection
o VR program has contrast injection, balloon inflation, stent and coil replacement options
Virtual Reality o Palpsim o User views LCD screen which
displays their hands and real needle overlaid on a simulation of a patient's back
o Hands and tools are tracked using 2 cameras
o User inserts needle into haptics device and palpates a silicon gel skin phantom with embedded pulsation device
Computer Based Simulation
o SAC Computer simulator22
• Computer program where user must save patient undergoing contrast reaction by controlling the airway, breathing, circulation and administering medication
• Interface displays state of the airway (placement of nasal oxygen, face mask, anesthetic mask or intubation) and vital sign monitors
• Expert help option in program
o Sedation Simulator by Anesoft23
• Computer program where user is the sedation provider and must save patient undergoing sedation reaction
• User can manage the airway and breathing, administer fluids and drugs, interact with the radiologist performing the procedure and resuscitate the patient if necessary
• Interface displays patient’s history, vital signs and patient response
SAC program interface
Sedation Simulator program interface
Mannequin o SimMan by Lauderal24,25,26,27 o Mannequin can simulate tongue edema,
laryngospasm, and bronchospasm with audible wheezing
o Displays vital signs (HR, BP, PO2 and breathing rate) which change dynamically
o Speech controlled by a technician who can make the mannequin respond to user interaction
o User can administer injections, intravenous lines, nasal cannula and oxygen masks
Mannequin
o SimMan28,29,30
• In-situ simulation at a hospital involving nurses, doctors, etc.
• Manage a cardiac arrest, contrast reaction and biopsy complications
o Patient Simulator by Eagle Simulation31
• Simulation resembles a patient in a CT scanner radiology suite
• Mannequin has radial and carotid pulses, pupillary and lid reflexes, temperature changes,
• Breathing, heart sounds and voice controlled by technician
• Mannequin responds semi-automatically to user
• Users work in teams to manage reaction with IV placement and medication administration through brachial, antecubital, or forearm veins
Crisis management scenario using SimMan
CRM with Patient Simulator by Eagle
Computer based simulation32
o Night Shift computer game o User is a resident on duty at night and must
respond to different tasks in priority order
o Events are calibrated to reflect actual frequency and severity as recorded in NHS database
o During the game tasks deteriorate and go up in priority
o Actual floor plan of hospital incorporated into program allows users to learn hospital layout while playing
Definition: how well a simulation/simulator mirrors real life7
Coles, John et al. (2011)21 o Femoral palpation and needle insertion on
Palpsim VR simulator
o 7 interventional radiologists with 5+ years of experience
o 29 point questionnaire on realism of the simulation rated using Likert scale
o Results
• High sense of immersion • Realistic draping, pulse and location of
femoral artery
• Insertion device had good haptic feedback
• “Feeling of free space" was distorted when using the needle
Definition: how well a simulation measures the knowledge it intends to measure; does it train the skills it is supposed to34
Nicholson, Cates et al. (2006)35
o Left and right carotid angiography on Procedicus-VIST
o 100 interventionalists with a mean of 12.4 yrs of endovascular experience
o Survey to evaluate anatomy, tools, and overall procedural steps on 5-point Likert scale
o Results
• Good aortic and carotid anatomy but poor cranial anatomy
• Realistic haptics and feedback from tools
• Graphics appeared similar to real fluoroscopic images
• Simulation contains all necessary procedural steps
• Simulator allows user to make mistakes
Definition: how well a simulator measures the skill it is supposed to measure; differentiation based on skill level8
Van Herzeele, Aggarwal et al. (2007) 33
o Procedicus VIST used to perform angioplasty and stenting of right ICA with type 1 arch and stenosis in proximal region
o Study assessed simulator metrics: procedure time, contrast volume, number of angiograms performed, fluoroscopic time, # of catheter errors and procedure-specific errors
o 45 interventionalists from radiology, vascular surgery and cardiology:
• Group A (0 CAS procedures), n=12
• Group B (1to 20 CAS), n= 12,
• Group C (21 to 50 CAS), n =10 • Group D (>50 CAS, experienced group), n= 11
o Results
• Fluoroscopy time, procedure time and number of angiograms were valid metrics to differentiate between experienced and inexperienced groups; not able to differentiate between inexperienced skill levels
• Error metrics did not show statistical significance for skill level differentiation
Definition: how well simulation can train someone as compared to a validated method8
Mendiratta-Lala, Williams et al. (2015)36
o Evaluated knowledge using pre and post MCQ test after receiving training on percutaneous noncontinuous CT-guided fluoroscopic procedures using in-house phantom and web-based training module
o 40 radiology residents and 8 expert IRs
• Group A - IR experts
• Group B - 4th year residents without simulation training
• Group C - 2nd and 3rd year residents with simulation training and residency
• Group D - 1st year residents without residency and only simulation training
o Results
• Pre-test results were related to level of experience: B>C>D
• Post-test results
Group C score increase was statistically significant
Group D score approached Group B pre-test score
• Suggests simulation training enhances knowledge comparable to residency training
Definition: how well skills trained on a simulator translate to the operating room; can performance on a simulator predict performance in OR8
Chaer, Derubertis et al. (2006)16
o Mentice Procedicus-VIST used for simulation course
o 2 catheter based procedures for lower extremity occlusive disease in OR
o 20 residents on surgery rotation with no endovascular experience: 10 enrolled in simulation course and apprenticeship training, other 10 only received apprenticeship training
o Blinded expert evaluated the procedures using a procedural checklist with rating scale (18 steps) and an overall performance global rating scale (12 criteria)
o Results
• Simulation group got higher scores on procedural checklist and global rating scale
• 2nd procedure showed more statistic significance on checklist score for simulation versus control group
• Overall performance of 1st and 2nd operation did not change within groups
Assessment metrics generated by the simulator
Pros: objective, when used with other assessment methods can be robust
Cons: metrics are not always valid or designed with medical knowledge, need an expert dataset as benchmark
Johnson, Hunt et al. (2012)38
o US guided biopsy on VR in-house simulator designed using critical procedural steps from task analysis created with expert consultation
o 14 IRs and 26 trainees tried procedure on simulator
o Experience level led to significant differences in scores for targeting, probe usage time and mean needle length in beam
o Experienced participants received better overall scores from simulator
Assess procedural skills using a binary score or rating scale
Used for in-situ or post-hoc video assessment
Pros: objective scoring method, effective, reliable and validated method of assessment
Cons: subjective design, does not take into account timing and sequence of actions
When used in conjunction with global rating scales it does not add any additional value to assessment
Bagai, O'Brien et al. (2012)17
o 27 trainees on cardiac catheterization rotation
o Randomized into control and simulation group by residency year
o Simulation group got proctored training of cardiac catheterization on Procedicus-VIST
o Pre-test and post-test 1wk later on VIST
o Evaluated using procedural checklist with rating scale
o Change in pre test and post test score was significantly higher in simulation group versus in control group
Rating of general criterion of whole performance usually using Likert rating scale
Pros: rating proven to be tied to experience, good reliability
Cons: generic, inter-rater reliability needs to be considered, training needed for raters to use tool
Willaert, Aggarwal et al. (2011)37
o 33 endovascular physicians with varying level of experience with CAS procedures
o 3 groups: highly experienced, moderately and novice
o VR Simulator AngioMENTOR
o Procedure on simulator recorded and scored using Generic Endovascular global rating scale (GSR) tool and a procedure specific rating scale (PSRS)
o Experienced physicians had higher GSR scores, this was less significant using PSRS
o GSR and PSRS score was not significant between the highly and moderately experienced groups
Generally used for subjective assessment:
o Evaluating face validity
o Confidence in skills
o Utility of simulation experience
o Measured on Likert scale or
open-ended answer
Procedural and non-procedural skills
Wang, Schopp et al. (2011)27
o Contrast reaction management simulation with 44 radiology residents: 23 randomized into simulation group
o High fidelity simulator using SimMan
o Survey of comfort managing contrast reactions before and after simulation
o Simulation group reported higher comfort levels after training
Primarily used for knowledge assessment
Use same or different pre and post test
Generally MCQ format
Mendiratta-Lala, Williams et al. (2010)2
o 29 radiology residents participated in US needle guided procedural training
o Proctored and self-directed training on part task trainer for 6 months
o Improvement in pre test and post test score was statistically significant
Used to assess procedural skills
Breaks down a procedure into a series of steps and how long each one takes
Pros: objective analysis
Cons: lots of set up for videotaping, time needed for video analysis, time taken does not equal quality
Duncan, Kline et al. (2007)39
o Renal stenting procedure performed on AngioMENTOR and Procedicus-VIST
o 10 experienced angiographers and IRs
o Procedure videotaped and analyzed by blinded reviewers with no angiography experience
o Procedural steps outline used to determine placement of time-points in video
o Efficiency score calculated using: procedure time, contrast use, fluoroscopy time and supply use
o All participants met efficiency standards
Rating of performance based on number and types of errors made Good for differentiating skill level Need to define and weight types of errors Klass, Tam et al. (2008)10
o Left renal artery angioplasty performed on Procedicus-VIST o 12 new radiology residents o Access to simulator to practice during 1st year of residency o Experts noted the following errors: incorrect selection of equipment, incorrect use of
equipment and incorrect selection of the next appropriate stage of the task. o Group data shows errors did not decrease over the 3 trials o Individual data shows some participants did decrease the number of errors made over the 3
trials o Common mistakes were lack of familiarity with the equipment and skipping procedural steps
Simulation has the potential to revolutionize clinical skill training because it provides a risk-free, uniform, accessible medium to train a wide variety of procedural and non-procedural skills.
This potential can be hindered by sparse and non-uniform evidence of educational validity of commercially available simulation technology. o Face validity of simulators is easy to assess and many commercially available
simulations have been validated.
o The ultimate goal in simulation research is to prove predictive validity/transference; this will lead to widespread implementation.
A wide variety of assessment tools exist to measure knowledge and competence of trainees. o When designing a simulation training exercise, it is important to choose an appropriate
assessment tool to accurately measure the skill being trained.
1. Issenberg, S.B., et al., Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach, 2005. 27(1): p. 10-28.
2. Mendiratta-Lala, M., et al., The use of a simulation center to improve resident proficiency in performing ultrasound-guided procedures. Acad Radiol, 2010. 17(4): p. 535-40.
3. Monsky, W.L., et al., Using a sonographic simulator to assess residents before overnight call. American Journal of Roentgenology, 2002. 178(1): p. 35-39.
4. Johnson, S.J., et al., Development and validation of a virtual reality simulator: human factors input to interventional radiology training. Hum Factors, 2011. 53(6): p. 612-25.
5. Luboz, V., et al., ImaGiNe Seldinger: first simulator for Seldinger technique and angiography training. Comput Methods Programs Biomed, 2013. 111(2): p. 419-34.
6. Wu, W., et al., A simulator for percutaneous hepatic microwave thermal ablation under ultrasound guidance. Int J Hyperthermia, 2014. 30(7): p. 429-37.
7. Bott, O.J., et al., Use of a C-Arm Fluoroscopy Simulator to Support Training in Intraoperative Radiography. RADIOGRAPHICS, 2011. 31(3): p. E31-E41.
8. Neequaye, S.K., et al., Endovascular skills training and assessment. J Vasc Surg, 2007. 46(5): p. 1055-64.
9. Dayal, R., et al., Computer simulation as a component of catheter-based training. J Vasc Surg, 2004. 40(6): p. 1112-7.
10. Klass, D., et al., Training on a vascular interventional simulator: an observational study. Eur Radiol, 2008. 18(12): p. 2874-8.
11. Coates, P.J., I.A. Zealley, and S. Chakraverty, Endovascular simulator is of benefit in the acquisition of basic skills by novice operators. J Vasc Interv Radiol, 2010. 21(1): p. 130-4.
12. Aggarwal, R., et al., Virtual reality simulation training can improve inexperienced surgeons' endovascular skills. Eur J Vasc Endovasc Surg, 2006. 31(6): p. 588-93.
13. Hseino, H., et al., Skills transfer after proficiency-based simulation training in superficial femoral artery angioplasty. Simul Healthc, 2012. 7(5): p. 274-81.
14. Jensen, U.J., et al., The role of a simulator-based course in coronary angiography on performance in real life cath lab. BMC Med Educ, 2014. 14: p. 49.
15. Narra, P., et al., Videoscopic phantom-based angiographic simulation: effect of brief angiographic simulator practice on vessel cannulation times. J Vasc Interv Radiol, 2009. 20(9): p. 1215-23.
16. Chaer, R.A., et al., Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg, 2006. 244(3): p. 343-52.
17. Bagai, A., et al., Mentored simulation training improves procedural skills in cardiac catheterization: a randomized, controlled pilot study. Circ Cardiovasc Interv, 2012. 5(5): p. 672-9.
18. Willems, M.C., et al., Assessing endovascular skills using the Simulator for Testing and Rating Endovascular Skills (STRESS) machine. Eur J Vasc Endovasc Surg, 2009. 37(4): p. 431-6.
19. Li, S., et al., A catheterization-training simulator based on a fast multigrid solver. Computer Graphics and Applications, IEEE, 2012. 32(6): p. 56-70.
20. Andersen, P.O., et al., Development of a formative assessment tool for measurement of performance in multi-professional resuscitation teams. Resuscitation, 2010. 81(6): p. 703-11.
21. Coles, T.R., et al., Integrating haptics with augmented reality in a femoral palpation and needle insertion training simulation. Haptics, IEEE Transactions on, 2011. 4(3): p. 199-209.
22. Medina, L.S., J.M. Racadio, and H.A. Schwid, Computers in radiology. The sedation, analgesia, and contrast media computerized simulator: a new approach to train and evaluate radiologists' responses to critical incidents. Pediatr Radiol, 2000. 30(5): p. 299-305.
23. Chew, F.S., Section editor's notebook: Sedation simulation. AJR Am J Roentgenol, 2013. 201(5): p. 940.
24. Tubbs, R.J., et al., High-fidelity medical simulation as an assessment tool for radiology residents' acute contrast reaction management skills. J Am Coll Radiol, 2009. 6(8): p. 582-7.
25. Picard, M., et al., Comparison of High-Fidelity Simulation Versus Didactic Instruction as a Reinforcement Intervention in a Comprehensive Curriculum for Radiology Trainees in Learning Contrast Reaction Management: Does It Matter How We Refresh? Acad Radiol, 2015. 22(10): p. 1268-76.
26. Tofil, N.M., et al., Severe contrast reaction emergencies high-fidelity simulation training for radiology residents and technologists in a children's hospital. Acad Radiol, 2010. 17(7): p. 934-40.
27. Wang, C.L., et al., Prospective randomized comparison of standard didactic lecture versus high-fidelity simulation for radiology resident contrast reaction management training. AJR Am J Roentgenol, 2011. 196(6): p. 1288-95.
28. Sarwani, N., R. Tappouni, and D. Flemming, Use of a simulation laboratory to train radiology residents in the management of acute radiologic emergencies. AJR Am J Roentgenol, 2012. 199(2): p. 244-51.
29. Walker, S.T., et al., Unannounced in situ simulations: integrating training and clinical practice. BMJ Qual Saf, 2013. 22(6): p. 453-8.
30. Niell, B.L., et al., Prospective analysis of an interprofessional team training program using high-fidelity simulation of contrast reactions. AJR Am J Roentgenol, 2015. 204(6): p. W670-6.
31. Sica, G.T., et al., Computerized realistic simulation: a teaching module for crisis management in radiology. AJR. American journal of roentgenology, 1999. 172(2): p. 301-304.
32. Larkin, C., et al., 'Night Shift': A Task Simulation to Improve On-Call Prioritisation, Self-Management, Communication, and Route Planning Skills. 2014: p. 59-62.
33. Van Herzeele, I., et al., Virtual reality simulation objectively differentiates level of carotid stent experience in experienced interventionalists. J Vasc Surg, 2007. 46(5): p. 855-63.
34. Dawson, S., Procedural simulation: a primer. Radiology, 2006. 241(1): p. 17-25.
35. Nicholson, W.J., et al., Face and content validation of virtual reality simulation for carotid angiography: results from the first 100 physicians attending the Emory NeuroAnatomy Carotid Training (ENACT) program. Simulation in Healthcare, 2006. 1(3): p. 147-150.
36. Mendiratta-Lala, M., et al., Simulation center training as a means to improve resident performance in percutaneous noncontinuous CT-guided fluoroscopic procedures with dose reduction. AJR Am J Roentgenol, 2015. 204(4): p. W376-83.
37. Willaert, W.I., et al., Patient-specific endovascular simulation influences interventionalists performing carotid artery stenting procedures. Eur J Vasc Endovasc Surg, 2011. 41(4): p. 492-500.
38. Johnson, S.J., et al., Virtual reality, ultrasound-guided liver biopsy simulator: development and performance discrimination. Br J Radiol, 2012. 85(1013): p. 555-61.
39. Duncan, J.R., B. Kline, and C.B. Glaiberman, Analysis of simulated angiographic procedures. Part 2: extracting efficiency data from audio and video recordings. J Vasc Interv Radiol, 2007. 18(4): p. 535-44.
