Overcoming User-Centered Challenges with Complex Health...

26 Biomedical Instrumentation & Technology January/February 2018

Features

Overcoming User-Centered Challenges with Complex Health TechnologyPeter A. Doyle and Martha Vockley

About the Authors

Peter A. Doyle, PhD, is a human factors engineer at The Johns Hopkins Hospital in Baltimore, MD. Email: pdoyle6@

jhmi.edu

Martha Vockley, MAPW, is a principal of VockleyLang, LLC, in Reston, VA. Email: mmv@ cox.net

When an operator manual for a physiologi-cal monitor exceeds 800 pages and a manual for a hospital bed includes more than 100 distinct warnings and over 100 unique icons, it’s clear that medical equipment has become extraordinarily complex. The exponential growth in health technology brings wonderful innovations,1 many of which may be accompanied by challenges in device mastery for the people who are its prime users. Deploying an array of comput-ers, video systems, digitally driven control systems, and other complex devices for patient care places new demands on clinical users to understand, set up, operate, and troubleshoot both stand-alone medical devices and integrated systems.2 These challenges are further complicated by the different makes, models, and software revisions produced by suppliers.

Clinicians’ effective use of complex technology depends on a wide range of factors, including information technology (IT) integration, technical support require-ments from all stakeholders (e.g., IT, clinical engineering, vendors, users, patients), and ongoing system management (e.g., cyberse-curity, software upgrades, costs of licensing, service contracts).

This article focuses on a narrower set of user-centered issues that also relate to effective use of complex technology: design for usability, training, competency assess-ment, and procurement activities. At its

kickoff meeting in April 2017, the AAMI Foundation’s National Coalition to Promote the Safe Use of Complex Healthcare Technol-ogy identified these issues, among others, as priorities for patient safety. (For more on the coalition, see the sidebar on p. 29.)

The Rising Tide of Complex Technology Elevates RiskIn four years’ time, the types of nondispos-able medical devices at The Johns Hopkins Hospital in Baltimore, MD, have increased by 23%.3 Many of these devices are complex technologies designed to improve patient care, ease caregiver workload, or improve workflows. Although it is hard to dispute their value and contributions to the medical mission, the demands of mastering the rising tide of incoming equipment, especially equipment used in procedures with greater risk, cannot be understated.

We can consider complex technology as that which cannot be fully mastered for safe and reliable use in a typical 15-minute in-service training. Complex technology often has one or more of the characteristics described in the sidebar (p. 27). Unfortu-nately, the inherent complexity of many devices and their apparent never-ending influx into the healthcare workplace intro-duce opportunities for risk. Risk factors include use errors based on poor design for usability4 and lack of proficiency of use.

© Copyright AAMI 2018. Single user license only. Copying, networking, and distribution prohibited.

27Biomedical Instrumentation & Technology January/February 2018

Features

Usability Issues and Training Burden“I thought all the problems I had with devices were due to me,” said an experienced nurse after learning about usability during a Hopkins human factors workshop. Now she realizes that usability issues often account for her vexations. Equipment that is not easy or satisfying to learn, use, or remember—and by inference, is not safe to use—invites harm and legal liability. In the United States, nearly one-quarter of operating room (OR) errors involve medical devices,5 contributing to $1.7 billion in annual costs.6 The point here is not to hold suppliers accountable for these issues; rather, it’s to collaborate better with them so that users can develop a good mental model of how a device functions, how to troubleshoot it, and how to use it safely.

If tools are not designed for safe use, we may strive to compensate for clinicians’ misgivings with training. However, in-service conditions often are less than ideal due to limited time, settings with poor visual or physical access to the product, limited participant availability, and sometimes low participation levels. Time constraints often preclude addressing all hazards and critical learning objectives, and the opportunity for hands-on skill development may be lacking. By delegating training responsibility to a supplier, an institution delegates control of

training-related risks to the supplier while assuming associated liabilities.

By compensating for poor usability with training, we also must be cognizant of the burden medical device training imposes on organizations. The training list for all Hopkins OR nursing specialties includes 233 training activities for first-year comprehen-sive skills. That list is supplemented by 114 multimedia courses and a number of annual specialty courses (e.g., 38 for an obstetrics nurses). There are also 32 OR-specific policies to learn and 50 new orientees to train each year.

Adding to the training burden are vendor in-service training, internal device training, and ongoing refresher training. A case in point: Training for one Hopkins hospital bed involves in-services, online video courses, a quick reference guide, a tip sheet, and com-puter-based training embedded in the bed itself. Even so, these multimodal training and skill-building efforts do not include 20 significant warnings from the manual; therefore, Hopkins must provide supplemen-tal fact sheets and further training.

A typical ICU nurse must complete 45 courses annually and comply with 248 interdisciplinary clinical practice policies, in addition to learning multiple standard-of-care policies. All of this is augmented by device training. Not only do nurses feel overdosed on training, but each nurse in training requires another to be on the job—an expense the institution must bear. Hopkins surgical techs also have training requirements. They must master the setup, operation, and troubleshooting for more than 70 types of devices. The hospital has more than 50 types of lasers alone.

Many hospitals now make use of integrated systems with digital controls to manage the relationships and behaviors of multiple hospital devices or systems in a dynamic manner. For example, a nurse call system and physiological monitoring system may be connected to clinicians’ wireless phones. As a result, technology developed to improve workflow and increase efficiency requires further nurse training. Medical device training thus places considerable burdens on clinicians, beyond the equally substantial institutional training that covers

Characteristics of Complex Technology1. Difficult to learn2. Hard to remember how to operate3. Hard to develop a “mental map” of

how it works4. Has a large number of controls for

operation5. Has complicated menu-driven

controls6. Does not easily communicate its

operational status7. Promotes use errors due to poor

usability8. Difficult to troubleshoot or recover

from errors9. Is computer based 10. Has a high degree of operational

variability across makes and models

Not only do nurses feel overdosed on training, but each nurse in training requires another to be on the job—an expense the institution must bear.



Features

organizational policies, practices, and standards of care. Given this reality, both healthcare delivery organizations (HDOs) and clinicians value and appreciate devices that are easy to learn, use, and troubleshoot.

Developing, Assessing, and Defining CompetencyComplex devices require quality training to encourage proficient and consistent use of tools across users. We need to be wary of the quick in-service and on-the-job training that does not standardize content, develop troubleshooting skills, or identify error recovery actions. Training should provide thorough attention to significant hazards and the means to mitigate them. That means using established methods to develop training, such as the instructional systems design method7—a systematic approach to determine essential training content and to develop, deliver, evaluate, and validate the training.

Competency can be defined as the applica-tion of knowledge, skills, and behaviors, including interpersonal, technical, and critical-thinking behaviors. As such, compe-tency is best assessed via a hands-on return

demonstration—not as a test after an education session. A review of medical training and education literature focusing on education techniques, frequency, setting, and media found that passive instruction (reading or lecture) was not as effective as interactive approaches and had little or no beneficial effect.10 Interaction with the instructor and feedback on performance (both typical components of hands-on demonstrations) were shown to be important, supporting the view that achievement of competency depends on use, feedback, and practice.10

Simulations, mock reviews, and case studies are helpful to assess device compe-tency. However, bear in mind that credentialing addresses clinical procedures but not necessarily the skill levels needed for proper use of tools and technologies. The Joint Commission has highlighted the importance of training at a level that estab-lishes competencies. Ultimately, however, hospitals must define their own training needs and provide training as they see fit.11

Improved Design for Usability, Better Approaches to ProcurementA challenge lies before us. Considering the many complex devices “in play,” do our institutions have sufficient resources to define and assess all needed competencies given the demands of a high-production environment? Can we identify all critical learning objectives? Can we provide adequate time and opportunity to develop skills with hands-on demonstrations? Are resources sufficient to manage competency develop-ment and retention? The need to perform these functions is supported by evidence of the OR error rates and associated costs cited above. Hospitals bear other costs as well, including those associated with forensic analyses and disruptions to productivity, not to mention patient harm, death, and the suffering experienced by second victims.

These user-centered patient safety issues can be addressed in good part by improved design for usability and better approaches to procurement. Usable devices are not only safer, but they reduce the training burden because training is easier, quicker, and more effective as knowledge retention is improved.12–15 From a procurement stand-

Selecting and Purchasing Safe and Effective Health Technology1. Determine if the device has

undergone adequate usability testing according to Food and Drug Administration guidance.

2. Evaluate the device in-house for user-centered design qualities, in addition to reviewing third-party evaluations.

3. Test equipment in the actual physical use environment.

4. Consult ANSI/AAMI HE75:2009/(R)2013, Human factors engineer-ing—Design of medical devices,15 which provides detailed guidance for design and evaluation.

5. Ask specific questions to deter-mine whether the command input process and output sequences are well designed.

Does the device:a. Offer the actions I wish to

initiate?b. Understand my instructions?c. Deny inappropriate inputs?d. Offer the opportunity to

change inputs as needed?e. Provide easy access to appro-

priate status information given the actions performed?

f. Provide proper cues for subsequent command options?

g. Have adequate external or embedded instructions for use?

h. Do all this in a manner that is safe and easy to use?

This sidebar content was adapted from A Roundtable Discussion: Understanding Medical Devices and Users in Context.17

Competency is best assessed via a hands-on return demonstration—not as a test after an education session.


29Biomedical Instrumentation & Technology January/February 2018

Features

point, we at Hopkins and others16 have found value in performing human factors evalua-tions when purchasing infusion pumps and other medical equipment with a direct influence on patient safety.

HDOs can collaborate more closely with suppliers to develop designs that are easy to learn, use, and troubleshoot. Engaging the assistance of a human factors engineer in such activities can prove beneficial. Working with suppliers to identify needed hazard miti-gations and procedures deemed important for training also is advisable. Then, HDOs can determine:• What standards and controls are in place

to ensure that instructor skills, course content, and competency assessments support proper skill development.

• Whether suppliers will provide training to ensure that they are assessing staff competency on key skills.Finally, to minimize the training burden,

support clinician mastery, and improve patient safety, consider the user-centered recommendations in the sidebar (p. 28) for selecting and purchasing medical devices.

ConclusionThe exponential growth of complex medical devices introduces risk related to challenges in developing and maintaining mastery of device use. Overcoming these challenges in a healthcare work environment that is already stressed by high-production pressures and a rapid pace of change requires greater attention to design for usability, careful attention to the quality of training and competency assessment, and thoughtful practices in the procurement of medical devices. n

References1. Vockley M. Game-Changing Technologies:

10 Promising Innovations for Healthcare.

Biomed Instrumen Technol. 2017; 51(2):96–108.

2. Connor MA, Reinbolt JA, Handley PJ.

Perioperative Nurse Training in Cardiothoracic

Surgical Robotics. AORN J. 2001;74(6):851–7.

3. Doyle PA, Gurses, AP, Pronovost P. Mastering

Medical Devices for Safe Use: Policy, Purchasing

and Training. Am J Med Qual. April 2016.

4. Zhang J, Patel VL, Johnson TR, et al. Evaluating

and Predicting Patient Safety for Medical Devices

with Integral Information Technology. In:

Henriksen K, Battles JB, Marks ES, Lewin DI,

eds. Advances in Patient Safety: From Research to

Implementation. Vol. 2: Concepts and Methodology.

Rockville, MD: Agency for Healthcare Research

and Quality; 2005:323–36.

5. Weerakkody RA, Cheshire NJ, Riga C, et al.

Surgical Technology and Operating-Room Safety

Failures: A Systematic Review of Quantitative

Studies. BMJ Qual Saf. 2013;22(9):710–8.

Overcoming Challenges Surrounding the Purchase and Use of Complex TechnologyThe AAMI Foundation’s National Coalition to Promote Safe Use of Complex Healthcare Technology is taking on the challenges of complex healthcare technology for clinicians, hospitals, and suppliers. Launched in spring 2017, the coalition consists of a diverse group of stakeholders, including nurses, physicians, human factors engineers, and industry representatives. During its first two years, the coalition is seeking to develop the following: • Best practices for selecting

and purchasing complex technology. Create a toolkit.

• Best practices for educating and training for the use of complex technology. 1) Provide guidance to healthcare facilities to help them determine which tech-nologies to focus on first (i.e., those that cause the most adverse events in their facili-ties). 2) Identify models: manufacturer/hospital partner-ships, content management, examples of high-performing organizations.

• Best practices for establishing minimal levels of proficiency and for assessing that profi-ciency. Identify critical elements for assessment and how proficiency can be meas-ured.

• Business case. 1) Build the case for allocating financial resources to improve how clinicians are prepared for safe use of complex technologies, highlighting patient safety for financing the initiative. 2) Create a return-on-investment template and identify data sources to help frame discus-sion.

• Design and development considerations. 1) Develop ongoing discussions regarding what the design and develop-ment of products may look like and what the impact on end users may be. 2) Create informa-tion on proven tools to help address the concepts of design that should be included. 3) Provide tools for having discus-sions with manufacturer partners about existing fleet support, upgrades, and product development.

To learn more, visit www.aami.org/complextechnology.



Features

6. Van Den Bos J, Rustagi K, Gray T, et al.

The $17.1 Billion Problem: The Annual Cost of

Measurable Medical Errors. Health Aff (Millwood).

2011;30(4):596–603.

7. Clark D. Why Instructional Systems Design

and ADDIE? Available at: www.nwlink.com/

~donclark/hrd/sat1.html. Accessed June 15, 2017.

8. Department of Defense. Standard Practice

for System Safety. Available at: www.faa.gov/

regulations_policies/handbooks_manuals/

aviation/risk_management/ss_handbook/media/

app_h_1200.pdf. Accessed May 5, 2017.

9. ANSI/AAMI/ISO 14971:2007/(R)2010. Medical

Devices—Application of Risk Management to

Medical Devices. Arlington, VA: Association for the

Advancement of Medical Instrumentation.

10. Bluestone J, Johnson P, Fullerton J, et al.

Effective In-Service Training Design and Delivery:

Evidence from an Integrative Literature Review.

Hum Resour Health. 2013;11:51.

11. The Joint Commission. Comprehensive

Accreditation Manual for Hospitals (CAMH):

The Official Handbook (Human Resources Chapter,

HR.01.05.03 and HR.01.06.01). Oakbrook Terrace,

IL: Joint Commission Resources; 2016.

12. Ross R. It’s Not a Training Issue: Practical

Usability: Moving Toward a More Usable World.

Available at: www.uxmatters.com/mt/archives/

2010/12/its-not-a-training-issue.php. Accessed

Sept. 12, 2017.

13. Segall N, Saville JG, L’Engle P, et al. Usability

Evaluation of a Personal Health Record.

AMIA Annu Symp Proc. 2011;2011:1233–42.

14. Johnson CM, Johnson TR, Zhang J. A User-

Centered Framework for Redesigning Health

Care Interfaces. J Bio Inform. 2005;38(1) 75–87.

15. ANSI/AAMI HE75:2009/(R)2013. Human

Factors Engineering—Design of Medical Devices.

Arlington, VA: Association for the Advancement

of Medical Instrumentation.

16. Ginsburg G. Human Factors Engineering:

A Tool for Medical Device Evaluation in Hospital

Procurement Decision-Making. J Bio Inform.

2005;38(3):213–9.

17. A Roundtable Discussion: Understanding Medical

Devices and Users in Context. Biomed Instrum

Technol. 2013;47(fall suppl):8–13.

Get the Most Out of Your CMMS

Computerized Maintenance Management Systems for Healthcare Technology Management3rd Edition

Ted Cohen, MS, FACCE Matthew F. Baretich, PE, PhD

Computerized Maintenance Management Systems for Healthcare Technology Management, 3rd Edition


From handling patient needs to managing data, the CMMS helps keep your facility running smoothly. In this book, authors Ted Cohen and Matthew Baretich provide guidance and best practices for:

• Proper setup of your CMMS

• Generating meaningful data

• Performing better equipment maintenance

Computerized Maintenance Management Systems for Healthcare Technology Management3rd Edition


Advancing Safety in Health Technologywww.aami.org

To purchase a print or PDF copy, visit www.aami.org/Store. Product Codes: CMMS or CMMS-PDF


Overcoming User-Centered Challenges with Complex Health...

Documents

Transcript of Overcoming User-Centered Challenges with Complex Health...