Copyright © 2015 BSI. All rights reserved.
Usability Where we’ve been…
reactive… ”oops I forgot the manual” Where are we going… proactive…. ”Understanding with minimal use of manual”
Usability: BSI Roadshow 2015 Richard Stein
3 Copyright © 2015 BSI. All rights reserved.
What is Usability?
• The ability for a human to interact easily and relatively error-free with a system, product or procedure
• What is Human Factors? A synonym for usability engineering
• Assume the system or product is a medical device for this presentation
08/12/2015
4 Copyright © 2015 BSI. All rights reserved.
Users
08/12/2015
Who* Where*
Patients On the go Labs
Caregiver Home Clinics
Clinicians (doc, nurse, tech) Emergency transport In the field
EMTS ORs ERs
Hospital room Nursing home
*consider age, dexterity, education/training, context
This presentation doesn’t focus on the function
6 Copyright © 2015 BSI. All rights reserved.
What types of devices may apply: implantable devices
Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI.
08/12/2015
7 Copyright © 2015 BSI. All rights reserved.
What types of devices may apply: implantable devices
Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI.
08/12/2015
small
8 Copyright © 2015 BSI. All rights reserved.
What types of devices may apply: Robotic Surgery
Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI.
08/12/2015
Users?
9 Copyright © 2015 BSI. All rights reserved.
What types of devices may apply: CPAP
Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 08/12/2015
12 Copyright © 2015 BSI. All rights reserved.
What types of devices may apply: Ventilator
Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI.
08/12/2015
15 Copyright © 2015 BSI. All rights reserved.
What types of devices may apply: Medical Apps with Sensors
Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI.
08/12/2015
18 Copyright © 2015 BSI. All rights reserved.
What types of devices may apply • CPAP
• nebulizers
• compression therapy
• EKGs
• Blood Pressure monitors
• Infusion pumps
• Implantable devices
• Catheters
• LVADs
• Surgical equipment (manual and powered)
• Endoscopes
• Apps, some with sensors
• Robotic surgery
• Ultrasound
• Ventilator
• Defibrillator
• Infant incubator
• Combination devices
• Software
• Expect sensors to facilitate the “human OS”
• More…….
15/09/2015
19 Copyright © 2015 BSI. All rights reserved.
Why implement usability in Your medical device design? • Understandable • Useful device • Successful • Safer • Sales • Directives (EU) • Standards (IEC) • Guidance (FDA) • Serious Adverse Events, serious injury or death
08/12/2015
20 Copyright © 2015 BSI. All rights reserved.
Why the concern? In the past:
• Often, there has not been enough time to correct the usability shortcomings in the medical device that was found in validation, so the manual was re-written.
• If manual was not enough to address shortcomings, then training was provided. • If training did not address the remainder of the shortcomings, then complaints may
have occurred which were addressed via instructions on the phone, etc. Now:
• It is expected to provide a safe product that the user understands, errors minimized Still:
• SAE’s have occurred, see MAUDE, some due to Use Errors • Complaints to the manufacturers are still occurring
15/09/2015
21 Copyright © 2015 BSI. All rights reserved.
Some history
• Human Factors (Usability) first addressed by Peter Carstensen of FDA • Initiatives to make anesthesia equipment safer through application of human factors principles • Motivated the creation of an international committee to create standards
• ANSI/AAMI HE-48* Human Factors Engineering Guidelines and Preferred Practices for the Design of Medical Devices
o *Much of which adopted from MIL-STD-1472 Human Engineering
• ANSI/AAMI HE-74 Human factors design process for medical devices
• IEC 60601-1-6 Medical electrical equipment – Part 1-6: General requirements for basic safety and essential performance – Collateral standard: Usability
• IEC 62366:2007 Application of usability engineering to medical devices
• ANSI/AAMI HE-75 Human Factors Engineering – Design of Medical Devices (a Tutorial to HE-74)
• Today • IEC 62366-1:2015 Part 1: Application of usability engineering to medical devices
• Soon • IEC/TR 62366-2:2 Part 2: Guidance on the application of usability engineering to medical devices
08/12/2015
22 Copyright © 2015 BSI. All rights reserved.
Injuries associated with anesthesia* “The anaesthetized patient is at risk of complications resulting from the actions, or inaction, of the anesthetist, from the actions of the surgeon, and from failure or malfunction of anesthetic equipment.” One of many of the root causes attributable to usability:
• Failure to check equipment, lack of vigilance, distraction, and carelessness were the most frequently associated human factors
• More see FDA CDRH, MAUDE Adverse Events Report
• Why?
*British Journal of Anesthesia Volume 95, Issue 1, pages 95-109
08/12/2015
23 Copyright © 2015 BSI. All rights reserved.
Injuries associated with Infusion Pumps* • From 2005 through 2009, FDA received approximately 56,000 reports of Adverse Events associated
with the use of infusion pumps, including numerous injuries and deaths • These adverse event reports and device recalls have not been isolated to a specific manufacturer, type
of infusion pump, or use environment; rather, they have occurred across the board. Some examples: • Infusion pump screen confuses the user • Infusion pump screen.. Which units of measurement to enter.. Decimal point error
• pump labels…or...become damaged under routine use…labels worn • IFU or cues for mechanical set-up are not specific or clear enough • Infusion pump screen design is clunky or confusing to users, cause delays
• IFU are confusing, inadequate, outdated, or unavailable.
• In 2010 the FDA announced three steps it would take to improve infusion pump safety. These steps were to:
• (1) increase user awareness, • (2) proactively facilitate device improvements, and • (3) publish new guidance for industry
*See FDA CDRH Infusion Pumps
08/12/2015
24 Copyright © 2015 BSI. All rights reserved.
Examples of Different screens
Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI 08/12/2015
25 Copyright © 2015 BSI. All rights reserved.
Why IEC 62366-1:2015 we already have 62366:2007?
• Created to be more “usable”, easier to understand than original 62366 • Original 62366 required an expert to understand • Implements practices by developers • Attempts to harmonize methods to meet FDA and international regulators • Contains the “what” requirements, the “how” is in 62366-2 • Closer ties to risk management, EN ISO 14971 • Hazard related use scenarios • No reference to usability goals, however goals may still have a need • Errors and/or shortcomings in the design to be addressed through risk evaluation • Reduced emphasis on functional analysis • Closer to FDA guidance • Legacy still addressed in Annex C (UOUP)
*Annexes are very helpful in implementation
08/12/2015
26 Copyright © 2015 BSI. All rights reserved.
Major activities (shalls*) in IEC 62366-1:2015 • Usability Engineering activities shall be planned • To reduce risk do: safe design, protective measures, and/or information on safety
• Establish a usability engineering process • Documents Usability activities to a usability file • Use specification • ID user interface** characteristics • ID hazards & situations (foreseeable, the unforeseeable is gone) • ID hazard use scenarios for summative evaluation • User interface spec • Prepare user interface evaluation plan (summative & formative)
• Perform the design & summative evaluations
* Most of the requirements
**what the user looks at, uses
08/12/2015
27 Copyright © 2015 BSI. All rights reserved.
What’s in the new EN 62366-2:2016?
This is the “how” document or “tutorial” • How safety relates to usability • Reasons to invest in usability • How to implement a usability program • Overall usability process • Prepare the use specification • Identify characteristics related to safety and potential use errors • Identify hazard-related use scenarios • Select hazard-related use scenarios for summative evaluation • Establish user interface specification • Establish user interface evaluation plan • Design/implement user interface • Perform formative evaluations • Perform summative evaluations
*note: the above is paraphrased by the author and the final version is not yet released
08/12/2015
28 Copyright © 2015 BSI. All rights reserved.
Isn’t the usability process an addition to my development?
When you think about it, the activities are part of the development process
08/12/2015
Design process Usability process
Concept development Preliminary analysis
Design User interface design
Prototype development Formative evaluation
Verification & validation
Summative evaluation
29 Copyright © 2015 BSI. All rights reserved.
Challenges
• Process to include usability • Conduct adequate formative evaluation • Conduct adequate summative evaluation • Address and reduce use risks • Just an IFU to address usability may no longer be enough
08/12/2015
30 Copyright © 2015 BSI. All rights reserved.
Notified Body expectations of manufacturers
• Address usability in the Essential Requirements, in the Directives • Asking a couple of clinicians if they like the product is not enough……be careful of KOLs • Follow a process, ideally the harmonised standard or rationale, to meet ERs • Specifications should include the intended user, environment and procedure (as applicable) • Include usability in the requirements of the device/system and risk management • Verify the usability requirements of the device/system • Reduce usability risks as low as possible • Legacy products will need to be addressed, (See Annex 3)
Expectations are rising in Europe. For example: MHRA has task force on HF that is starting to look deeper into usability and how it can be improved to reduce use errors. A presumption is that the needs will require the NB’s to audit for usability.
FDA’s expectations have also increased.
8/12/2015
32 Copyright © 2015 BSI. All rights reserved.
When does BSI apply EN62366-1:2015?
• Now? Auditors may use EN62366-1:2015 as a benchmark, BSI will use the Directives. • Auditors need training • Auditors will look for evidence of compliance • Legacy is addressed • If I do not have high risk, must I apply it? Remember the shalls and risk based in
standard • If EN 62366-2 (TR) is guidance, then must I apply it? What is in it for me?
08/12/2015
33 Copyright © 2015 BSI. All rights reserved.
Case study: Abbott Symbiq™ IV Infusion System
Abbott’s Process: • Literature Review • User Analysis • Use Scenarios • “paper prototypes” use screens (GUI) • Formative evaluation
• Simulations & models • Design • Summative evaluation (some simulated)
08/12/2015
Case study supplied by usability consultant, not assessed by BSI
34 Copyright © 2015 BSI. All rights reserved.
Case study: Intelliject Epicard
Consultant’s Process • Development motivated by needs, mistakes, complaints • Conduct preliminary evaluations to understand mistakes
• Observed on how used • Learned from use errors
• Create the mock-up design • Conduct formative evaluation with mock-ups
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
35 Copyright © 2015 BSI. All rights reserved.
Case study: Intelliject epicard Multiple concepts were evaluated in formative, resulted in refine design
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
36 Copyright © 2015 BSI. All rights reserved.
Case study: Intelliject epicard
• Final design then evaluated in summative evaluations (simulated, many users)
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
37 Copyright © 2015 BSI. All rights reserved.
Case study: Drug injection for Rheumatoid Arthritis
• Previous systems
• Difficult to use with hands • Time • Multiple steps • Mistakes • Know the user • Understand the context
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
38 Copyright © 2015 BSI. All rights reserved.
Case study: Drug injection for Rheumatoid Arthritis
Consultant’s Process • Created the mock-up design • Conducted formative evaluation with mock-ups, simulated, many users • Created design based on learning from formative evaluation
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
39 Copyright © 2015 BSI. All rights reserved.
Case study: Drug injection for Rheumatoid Arthritis
• Summative design, many users
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
40 Copyright © 2015 BSI. All rights reserved.
Case Study: Mobile Apps
• Mobile apps may be largest increasing segment in medical devices
Consultant’s Process • Formative evaluation, simulated, many users
• What was learned • Simplify • Minimize • Don’t repeat unnecessarily • Use regular “English” • Visually appealing • Show progress • Provide actionable choices • Encourage action
08/12/2015
41 Copyright © 2015 BSI. All rights reserved.
Case Study: Mobile Apps
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
42 Copyright © 2015 BSI. All rights reserved.
Case Study: Mobile Apps
• Summative evaluation, not simulated, many users,
Case study supplied by usability consultant, not assessed by BSI
08/12/2015
43 Copyright © 2015 BSI. All rights reserved.
Where can I learn more?
• AAMI HF Committee: • AAMI Website: for webinars and training • ANSI/AAMI HE-74 • ANSI/AAMI HE-75 • IEC 62366-1:2015 • IEC/TR 62366-2 • Handbook of Human Factors Medical Device Design, Matthew Weinger
et.al. ISBN 978-0-8058-5627-9
• FDA Guidance, http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm259748.htm
• The psychology of everyday things, Donald A. Norman ISBN 0-465-06709-3
• Set Phasers on stun, Steven Casey, ISBN 0-936178-8-5
• Usability testing of medical devices, Michael Wicklund et. al. ISBN 978-1-4398-1183-2
• The growing role of human factors and usability engineering for medical device, http://medicaldevices.bsigroup.com/en-GB/resources/Whitepapers-and-articles/
08/12/2015
44 Copyright © 2015 BSI. All rights reserved.
Conclusion
• Usability has become a vital part of the medical device development process • Usability activities should be conducted throughout all phases of the development
process, include it not a separate process • Usability should be part of the overall risk management process
08/12/2015