NISTMedical Device Communication Testing

Semantic interoperability of Medical Devices

Test Tool Update

HL7 Healthcare Devices WG / IEEE x73 Upper Layers

John J. Garguilo

National Institute of Standards and Technology

8 May, 2014 – Phoenix, AZ

Contact: john.garguilo@nist.gov, 301-975-5248

2

NIST MDC Testing Staff

• John J. Garguilo

301-975-5248

john.garguilo@nist.gov

• Nicolas Crouzier(GR) – RTMMS, HL7 V2 Tools

• Art Griesser + Michael Faughn (Contractor -

Prometheus Computing, LLC ) – DIM Modeling +

“MyDevice” (next gen ICSGenerator plus)

• New GR (half time) starting June ‘14

Semantic Interoperability of Medical Devices

3

Work Areas

• HL7 V2 (2.6) IHE-PCD Validation Tools

– Enhancements

“Off-cycle” added to Pre-Con tool (i.e., between Cycles 8 and 9)

– Cycle 9 CP updates being code for 2014-15;

Fall/Winter Pre-Connectathon + Virtual Testing,

January/February 2015 Connectathon

– IHE-PCD Pre-Connectathon, “Isolated Environment”

– IHE-PCD Connectathon, “Instance Environment”

• Rosetta Terminology Mapping Management System (RTMMS)

Overview + Update

– RTMMS Deployment Update and service status

• -10101a + IEEE Agreement to make available the “Works” + latest numbers…

• IGAMT – Implementation Guide Authoring Management Tool

– Continue to develop PCD-01 (ORU^R01)… targeting Sept WG mtgs

4

Work Areas, continued

• ISO/IEEE 11073 ‘Domain Information Model Editor’ -> “MyDevice”

– Progress/Update/Status (Michael Faughn + Art Griesser)

– UML Defined x73 DIM (-10201) basis for tooling

– DIM Editor / ‘MyDevice’ pilot tool demo

– Auto producing ISO/IEEE documentation (to meet SDO’s templates)

HL7 V2.6 Validation ToolingIHE-PCD Pre-Connectathon Cycle 9

Semantic interoperability of Medical Devices

Test Tool Update

Joint HL7/IEEE 11073 Healthcare Devices Working Group

National Institute of Standards and Technology

8 May, 2014, Phoenix AZ

Contact: john.garguilo@nist.gov, 301-975-5248

6

2013-14 IHE-PCD Pre-Connectathon Tool

Status

• IHE-PCD: NIST “Isolated Environment” Test Tools required

– HL7 V2.6

– Supports 6 Integration Profiles (number of test cases)

• DEC – Device Enterprise Communication (10)

– SPD – Filter (4)

– POI – Pulse Ox (3)

• PIV – Patient Infusion Verification (11)

• IPEC – Infusion Pump Event Communication (6)

• IDCO – Implantable Device Cardiac Observation (4)

• ACM – Alarm Communication Management (2)

• RDQ – Retrospective Data Query (1)

• OMS, RTLS/CMS white papers/WGs

• 30+ Test Cases

• 60+ existing User Accounts

– Contributing Orgs, SDO (leads/Co-chairs), Reviewers

– In total since tool made publically available (2011)

7

2013-14 IHE-PCD

Pre-Connectathon + Connectathon Status, Continued

• Cycle 9 - Test Cases/demographics reviewed used as basis

for tooling at 2015 NA Connectathon

• IHE-PCD Spring F2F– week of April 1st in Cleveland at new HIMSS Interoperability Center

• CPs documented and submitted (see PCD wiki)– Note: One withdrawal (adding an “alarm consumer”), others balloted and in process of

comment resolution (official ballot closed last Friday), CP usage change to NK1

Segment in ballot

– 9 CPs received and processed: http://wiki.ihe.net/index.php?title=PCD_CP_grid

• Roadmap of activities updated… see PCD wiki

http://wiki.ihe.net/index.php?title=Patient_Care_Devices

8

IHE-PCD 2013/14 Pre-Connectathon + Connectathon

NIST Testing Support

The NIST V2 (2.6) Tools perform the following validation:

• Syntax and Semantic Content Validation

– Against HL7 conformance profile

– Against IHE-PCD Technical Frameworks/Supplements constraints

(e.g., PCD-01: ORU^R01, Communicate Device Data)

– Against HL7 and/or user [local] provided tables

• Example of user provided table is RTM for Ref_IDs, Units, body sites, etc.

– Against ‘validation context’, including specific values

• Defined in XML (e.g., specific test case values)

9

Testing Environments

• Instance Testing– Conformance (e.g., against HL7 2.x or CDA)

• Test object conforms to specification on which it is based

• IHE Model: ~Virtual and Pre-Connectathon

• NIST IHE-PCD v2 Message Validation Test Tool

• Isolated System Testing– Includes Instance Testing Activities

– Protocol Conformance

– Functional Behavior Conformance

• Features and operational behavior correspond to specifications

• IHE Model: ~Virtual and Pre-Connectathon

• NIST IHE-PCD v2 Test Tool

• Peer-to-Peer System Testing– Includes Isolated System Testing Activities

– Interoperability Testing

• Testing complete application environment

• May include interacting with Database, using Network Communications, or interacting with

other hardware, applications, or systems if appropriate

• IHE Model: ~Connectathon

10

NIST V2 HL7 IHE-PCD Test Tool: Operational Process

END-USER

(VENDOR)

SYSTEM UNDER TEST

(SUT)

NIST IHE-PCD

HL7 v2/v3 TEST TOOL

SPECIFICATIONS

(test material that defines test assertions)

INTERACTION/REPORTS

MESSAGES (TEST OBJECTS)

MANUAL OR AUTOMATED SUT

Web Application

Interface

(via the communication protocol

currently only MLLP)

V3 – Future Work

STIMULUS OR RESPONSE (MESSAGES)

11

HL7 V2 Tool Updates – New/In Progress

• Reference to standard/date for every MDC terms (e.g. 10101 or

10101a) – To add column once completed

• Working on code generation taking into account discriminators with

the help of Paul Schluter.

• Adding Alerts attributes in RTMMS (work in progress with

Monroe Patillo)

• Various test cases corrections (messages examples, validation

contexts…)

• IPEC containment tree validation is in testing phase; targeting

ready for this summer (2014)

• Pre-Connectathon tool, fixed test case scenarios and validation

files for DOC-DOR-DOF test cases.

• Currently implementing PCD CPs (97-102 from

http://wiki.ihe.net/index.php?title=PCD_CP_grid)

12

HL7 V2 Tool Updates, continued - update

• Documentation Tab (see coming slides)

– Conformance Profile Tab

– Patient Demographics

– IDCO Patient Demographics

– PIV Drugs

– Other Resources

– Cycle 9: Looking into capability to upload libraries +

demographics

• incorporated [automatically] into validation context files

used by tooling

13

New Tool Updates: Test Event Selection – Out of cycle

• Top Right Corner – Out of Cycle to enable separation

(historical) of test event results (for/by paticipants, last

being Cycle 8 (Fall – Winter 2013-14)

Default Test Event

14

New tool updates – Profile Viewer – updating to reflect CPs

15

HL7 V2 Tool Updates (Started with Cycle 8 test events)

• Current Version / Release Notes – new CPs not yet integrated

– ‘About’ Tab (yellow ribbon)

16

HL7 V2 Tool Updates (Starting with Cycle 8 test events)

• Documentation Tab

– Conformance Profiles

17

HL7 V2 Tool Updates (Starting with Cycle 8 test events)

• Documentation Tab

– Patient Demographics

18

HL7 V2 Tool Updates (Starting with Cycle 8 test events)

• Documentation Tab

– IDCO Patient Demographics

19

HL7 V2 Tool Updates (Starting with Cycle 8 test events)

• Documentation Tab

– PIV Drugs

20

Cycle 8 Organizations using NIST Pre-Connection Tooling

# of Organizations: Total 21, Unique 17

Organization Name Organization Name

USST Hospira

True Process Inc. Draeger Medical

*Carefusion Cardiopulmonary Corp.

B Braun Baxter

Smiths Medical *West Health

iSirona BIT Computer Co. Ltd.

Covidien Masimo

*GE Healthcare *Philips Healthcare

Fukuda Denshi*Multiple Users from same Org

21

IHE-PCD Cycle 8 2013-14 Pre-Connectathon

Test Result Statistics – Organizations and Tests Executed

• Total Organizations (by User Accounts): 21

(Cycle 7: 19)

• Total Organizations (by *unique Company Name): 17

(Cycle 7: 14)

Registered (stored) at NIST site (over Pre-Con 2013-14 Cycle 8)

• Total number of Tests Executed:116

(Cycle 7: 82)

• Total Number of Tests Passed: 108

(Cycle 7: 81)

• Total Number of Tests Failed: 8

(Cycle 7: 1)

22

IHE-PCD Cycle 8 2013-14 Pre-Connectathon

Test Result Statistics – by Actor

Cycle 7

Actor Total Pass Fail

DEC Reporter 18 17 1 17

DEC Consumer 2 2 1

DEC Filter 1 1 1

PIV Programmer 1 1 0

PIV Consumer 4 4 5

ACM Reporter 11 10 1 6

ACM Manager 2 2 1

IDCO Reporter 1 1 0

IDCO Consumer 1 1 0

IPEC Reporter 5 5 4

IPEC Consumer 1 1 0

RDQ Consumer 1 1 0

RDQ Responder 0 0 0

Cycle 8 213-2014 Cycle 8

Actors: 13

48 Total:

46 Pass

2 Fail

Cycle 7:

Actors: 11

35 Total

Rosetta Terminology Mapping Management System - RTMMS

Semantic interoperability of Medical Devices

Test Tool Update

Joint HL7/IEEE 11073 Healthcare Devices Working Group

National Institute of Standards and Technology

21 January, 2014 – San Antonio, TX

Contact: john.garguilo@nist.gov, 301-975-5248

24

RTMMS Overview

• A web application* that allows vendors and reviewers access,

retrieval, and reporting of Rosetta Tables over the internet in

conformance to IHE-PCD RTM Profile

• An electronic resource/tool providing the capability of saving

data in xml format (as defined by RTM Profile)

• Aids the harmonization process by:

– Identifying missing terms

– Automatic generation of the “Harmonized Rosetta Table”

– Providing latest up-to-date view of hRTM table

• Facilitates the proposal of New Terms to IEEE 11073

Nomenclature standard

• Facilitates Conformance Tooling

– Message verification and conformance (syntax and semantics)

– Leading to interoperability…

– *developed by and currently hosted at NIST

– Integrated with ICSGenerator Tool (hRTM also imported into DIM Editor)

25

Rosetta Terminology Mapping Data Base

• Rosetta Table

– Maps vendor supported observations, units and enumerations to

ISO/IEEE x73 nomenclature

• Units Table

– Defines allowed units-of-measure

– Defines groups of related units-of-measure

• Enumerations Table

– Defines groups of enumerated values

• hRTM Table

– Generated from the original Rosetta

X73 Nomenclature DB

• Linked to RTMMS

Security DB - Stores users information

RTMMS Key Tables

26

• Access to NIST IEEE 11073 Nomenclature database

– Appendix A terms (from ISO/IEEE 11073 10101: Nomenclature)

– Appendix B terms (from ISO/IEEE 11073 10101: Nomenclature)

– IDCO terms (ISO/IEEE 11073-10301 – implantable device cardiac)

– aECG (annotated Electrocardiography)

– PHD terms (personal health domain)

• Access to RTM database

• Ability to propose terms in Rosetta

• hRTM, units, and enumeration download-able in XML format

• User registration

– Email confirmation, approval process… controlled through ‘admin’

• Filtering based on regular expressions

• Rosetta validation against hRTM

• Management capabilities for SDO users

Integrated w/ ICSGenerator / exported to ‘DIM Editor’

RTMMS Key Features

27

• General user

– Views Rosetta Tables

• Reviewer

– Participates in discussions

• Contributing Organizations

– Vendor/Contributer ‘sandbox’

– Modifies Vendor Rosetta Table

– Suggests new terms

– Modifies Units and Enumerations Table

• SDO (Standard Development Organization)

– Modifies Units and Enumerations Table

– Register new terms

• Admin

– Manages User Accounts

RTMMS Users

28

RTMMS by the numbers (as of 8 May 2014)

(since Jan 2014)

• 1619 (+4) Rosetta terms

• 671 (+1) hRTM terms

• 224 (0) Groups

• 8 (0) Unit Groups

– 445 (0) Units (MDC)

– 429 (0) Units (UCUM)

• 28 (0) Enumeration Groups

– 273 (+12) Enumerations

• 46 (+4) Contributing

Organizations now

Registered!

29

RTMMS Terminology Access and Views

Agreement finalized between IEEE and NIST (Dec 2012)• Allows users free and public access of terminology attributes that were

formally ‘pay-for’ via the NIST tool• The “Works”

Definition of the “Works” [from IEEE/NIST Agreement]

The following data items within the approved standards , existing and future versions, currently

designated as:

1 – IEEE 11073-10xxx

2 – IEEE 11073-20xxx

Limited to the following data items within, currently designated as:

1 – Ref ID

2 – Code

3 – Description

4 – Systematic Name

5 – Common Term – to be added to agreement (working w/ Kathryn Bennett/IEEE to amend)

30

RTMMS Primary Updates – Signed Agreement 12/20/12

31

RTMMS Updates, cont.

• User can download (an XML, HTML, and CSV) version of:– Rosetta – “hRTM” – Units – Term Approval::“Proposed terms” [192]– Term Approval::”Ready Terms” [286]– Note: Basis from prior WG discussions - this version enables greater flexibility

to work with when working outside the tool

• “Standard Table”– A new column “Standard Tbl” is now present in all Rosetta, hRTM and review

tables – Provides association of the term with corresponding table in the base

standard – Only the terms given by Jan W. (275) REFIDs are currently linked– NIST can add more if community provides the/additional mapped terms.

• Term Approval Updates (see subsequent slides)

32

Term Approval

Revised “term approval” tables/tab“Proposed terms” tab (192 rows)

Re-groups all the newly proposed terms with “MDCX_” prefixThe ‘Assign REFID’ button allows to change the REFID from an “MDCX_” term

to a “MDC_ term” (or anything else)

33

• “Ready to review terms” tab: 286 rows:– Terms with MDC_ prefix and null or 0 term code

Term being worked on by SDOs/MDC experts These processed terms would become / be added to the harmonized

Rosetta terms (hRTM) upon approval by SDO user and balloting

Term Approval, cont.

34

• “Ready to review terms” tab: 286 rows:– The ‘Edit RefID’ button allows approved user to edit a term and it’s attributes

– Edit a REFID, Standard Table, Block, Partition, Code(base 10), Systematic Name, Common Term, Acronym, and Description to the x73 database

– Pull down menus available (where enumerations provided/exist)

Term Approval, cont.

35

Terms being vetted (by the numbers)

• “Proposed Terms”: 192 rows• “Ready Terms Table”: 286 rows• “Mapped Terms”: 27 rows• “Proposed Units”: 101 rows• “Proposed Enumerations”: 133 rows

Term Approval, Proposed cont.

36

NIST MDC Testing Project

Web Sites• Project Web site: www.nist.gov/medicaldevices

• NIST HL7 V2 Test Tooling Web sites:

IHE-PCD Pre-Connectathon:http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/

IHE-PCD Connectathon: http://hit-testing.nist.gov:13100/PCD-HL7WebCon/

• NIST Medical Device Terminology Service:

Rosetta Terminology Mapping Management System (RTMMS):

http://hit-testing.nist.gov:13110/rtmms/

• NIST Implementation Conformance Statement

Generator (ICSGenerator):

– http://hit-testing.nist.gov/medicaldevices/ICSGenerator/ics_download.html

• DIM Work

– http://mdcx73-working.wikispaces.com/_NIST-ArtG

Semantic Interoperability of Medical Devices36

Domain Information ModelUML / DIM Editor, Progress Update

Semantic interoperability of Medical Devices

Test Tool Update – Michael Faughn

Prometheus Computing

Joint HL7/IEEE 11073 Healthcare Devices Working Group

National Institute of Standards and Technology

8 May 2014– Phoenix, AZ