Joint Exchange / Interop Work Group Test Workgroup

John Donnelly/Judith Huntman/Joe LamyAug 15, 2012

Roll

2

Name Organization RepresentationNick VanDuyne NYeC StateIvan Handler ILJohn Donnelly NJ Task Group Co-Chair - IWGMarty Prahl SSA

Federal agencyDennis Peterson, Curtus Browning, Elaine Hunolt, Taimur Aslam, Willie Singletary, Monica VanDyke VHA

Kevin Brady NIST NISTDavid Tao, Dan Huber Siemens

EHR VendorDave Cassel, Rob Klootwyk Epic Bill Howard, Charles Parisot GEJeff Cunningham, Tim Dunnington ICA

HIE vendor

Mike LaRocca Intersystems

Vladi Reznikov, Jennifer Puyenbroek CONNECT PMO

Matt Bell, Tom Wilson, Kathy Lin AxolotlTeddy Gedamu MedfxMelissa Owens Marshfield Clinic Health SystemSteve Leighty MedVirginia

Regional HIE Brian Bonnington, Dave Trepanier, Dave Smith, Steve Michael IHIE

Anuj Desai Interop WG Interop WGEric Heflin THSA Task Group Co-Chair - ExchangeMariann Yeager Exchange Exchange Ed O'Connor, Judith Hutman Nitor ONC Testing Contractor

Dennis Wilson, Alisa Ray, Sue Rieber CCHIT

ATCBs and Testing SMEs

Jim St. Clair IHE USA

Amit Trivedi, Kevin Brown, Michella Knighton ICSA Labs

Traci Mapps SLI Global SolutionsKyle Meadors, Timothy Bennett Drummond Group

Milton Padilla, Steve Wilson, Mark Shin, Doublas Biggs, Les Biggs InfoGuard

• Roll Call – Cayla

• Announcements – Eric

• Review of Phases – Judith (10 mins)

• Vote on Harmonized Phase I Test Cases – John (10 mins)

• Execution of Test Cases – Judith/Joe (10 mins)

• Test Case Metrics – Judith/Joe (7 mins)

• Open Discussion – All

• Wrap Up - John

• Schedule Review - John

Agenda

3

• Harmonized PD/QD/RD/MP/AF (aka Patient Record Lookup) Tests Are Ready for Vote!• Future changes are possible, but will be incorporated into the next

phase of test cases design and implementation (post Oct 2012)

• RFP• Has 5 respondents

• Contracting is expected to begin this week

Announcements

4

Harmonization Phases

Phase 1 – SOAP Send/Receive

Phase 2 – Direct Send/Receive

Phase 3 – Secondary Methods, Service

Discovery, & Provider Directories

•Parent Use Cases (Done)•Test Case harmonization (Done)•Tests: Mess & Auth Framework (Done)•Tests: Patient Discovery (XCPD, PIX, PDQ), Query for Documents, Retrieve Documents (Done)

•Test Execution Guides (Done)

•Parent Use Cases•Test Case harmonization•Tests: Direct Applicability Statement (Done)

•Tests: Direct XDM/XDR-Bridge (Done)•Tests: Direct Certificate Discovery (Done)

•Test Execution Guide

•Parent Use Cases•Test case harmonization•Tests: Web Services Registry (Done)•Tests: HPD+•Tests: Document Submission (Done)•Tests: Administrative Distribution (Done)•Test Execution Guides

Unified Test Process – Has Been Proposed

Vote on Harmonized Artifacts

• PD, QD, RD, Messaging & Auth (Phase I)

• Vote to ratify this workgroups decision that the current written test cases for PD, QD, RD, MP, AF (aka Patient Record Lookup) are in “final” status

• ‘round-the-room vote:• Approve

• Abstain – with reason stated

• Do not approve

• Artifacts may be found at: http://exchange-iwg.wikispaces.com/Test+Artifacts+Harmonization

Phase I: SOAP send/receive Status

Parent Use Cases Done

Test Cases: Patient Discovery (XCPD, PIX, PDQ) Done

Test Cases: Query for Documents Done

Test Cases: Retrieve Documents Done

Test Cases: Messaging Platform & Authorization Framework Done

Test Case harmonization Done

Test Execution Guides Done

Harmonization Activity Status

8

Phase II: Direct send/receive Status

Parent Use Cases Done

Test Cases: Direct Done

Test Case harmonization In progress

Test Execution Guides (deprecate Mod Spec reference, document self-guided process through multiple modular testing artifacts)

In progress

Harmonization Activity Status

Harmonization Activity Status

Phase III: Secondary Specs Status

Parent Use Cases

Test Cases: Document Submission Done

Test Cases: Administrative Distribution Done

Test Cases: Web Services Registry Done

Test Cases: HPD+

Test Cases: Content Not started

Test Case harmonization Not started

Test Execution Guides In progress

Direct Harmonization: Deployment Models

• Deployment models:• EHR-HIE Test Spec identifies actors: assumes specific deployment models

• ONC Test package: agnostic

• Direct Project specifications do not specify a model

• Issues:• Specifying a deployment model may restrict testing or add requirements outside the

scope of the specs

• Solution:• Present EHR-HIE Test Spec deployment models and actors as examples of

implementation models

• Map EHR-HIE Test Spec deployment models to our use cases

• Show the outliers (what they are testing that is not in our scope)

Harmonization: Modular Approach (PD-QD-RD)

End User Auth

TC1

TC2

TC3

PD

PD1

PD2

PD3

QD

QD1

QD2

QD3

RD

RD1

RD2

RD3

Message

Render

MR1 MR2

MR3

EHR-HIE ApplicantProduction Participant

Messaging & Auth

TC1

TC2

TC3

Eve

nt-

Bas

ed

Inte

rop

Tes

tin

g TC1

TC2

TC3

PD1

PD2

QD1 RD

2

TC1

TC2

MR1

MR3

MR2

Harmonization: Modular Approach (Direct)

End User Auth

TC1

TC2

TC3

Direct Messagi

ng

DM1 DM2

DM3

Cert Discove

ry

CD1

CD2

CD3

Message

Render

MR1 MR2

MR3

EHR-HIE ApplicantProduction Participant

Deplmnt A Reqts

A1 A2

A3

Eve

nt-

Bas

ed

Inte

rop

Tes

tin

g TC1

TC2

TC3

DM1

DM2

CD1

CD2

A1

A2

MR1

MR3

MR2

Deplmnt B Reqts

B1 B2

B3

DM3

CD3

A3

Deplmnt X Reqts

X1 X2

X3

Automating Test Case Execution

Messaging & Auth

• Modular: New scripts for each web service

• Some test cases implicitly covered by test cases in core specs

• Testing invalid SAML assertions• Critical to exchange security

• Probably can’t be executed using CONNECT as test tool

Automating Test Case Execution

Direct

• NIST: Direct messaging tool in development

• ONC Test: Prototype Direct Cert Discovery tool

• September 2012: Incorporated into NIST/ONC-Cert MU2 Test Procedures

Automating Test Case Execution: soapUI scripts complete

Priority PD QD RD TOTALS

High 7 18 4 29

Med 3 16 1 20

Low 3 0 1 4

TOTALS 13 34 6 53

‘round-the-room

What risks or other concerns do you have at this point in the process related to the harmonized test cases or automation approach?

Open Discussion

•RFP released — July 2nd 

•Questions received by 6 organizations

•Answers distributed — July 13th

•Bids due — July 20th

•Expected award — August 15th

•Contract start — TBD

Schedule Review

18

We are here