Conformance Testing of Interoperability

Conformance Testing of Interoperability

in Health Information Systems in Finland

Medinfo 2007, Brisbane

Wednesday, 22 August, Session S103, 11 AM

Tanja Toroia, Anne Eerolaa, Juha Mykkänenb

University of Kuopio, Finlanda Department of Computer Scienceb Health Information System (HIS) R&D Unit

2Conformance Testing of Interoperability...

in this presentation

• Introduction and background• Survey of conformance testing of interface

specifications in healthcare applications in Finland

• Findings of the survey • Recommendations based on the results of the

survey


introduction:conformance testing of interfaces

• Conformance testing = testing to determine whether a system meets a specification or not

• Interface (standard) specifications are central for interoperability

• Conformance testing of interfaces includes– conformity of applications and interfaces to standards– testing of the interfaces between applications

• Does not cover testing of the non interoperability-specific features of the applications


background of the study• In most healthcare facilities, interoperability and safety require attention on

cross-application level• Integration, distribution, security and interoperability emphasized lately in

software testing• Testing related to interoperability

– focus on deployment and integration phases of systems lifecycle– standards compliance– certification

• Conformance to specifications a key area in interoperability• However, few / no empirical studies on the level of conformance testing in

software companies in healthcare and in health service provider organizations– Survey: how much and how often conformance testing is performed in relation to the

interface specifications in healthcare applications


main research questions

1. How can customers, i.e. hospital districts, influence the conformance of the software they acquire?

2. How much conformance testing do software vendors perform and how often are standards used?

3. What are the main goals of, problems with, and opinions about external interface testing services?


the survey

• Themes– General software testing practices– Conformance testing and standards– The organization of interface testing services

• Web-based questionnaires with open and structured questions

• The questionnaires are shown on

http://www.cs.uku.fi/~ttoroi/questionnaire.html

Topic in this talk


sample and responses

• Healthcare software vendors in Finland– 27 software companies which belong to a Health and

Welfare IT R&D cluster in Finland (response rate 52%)

– responses focused on companies producing EPR and departmental systems + consultancy services

• Customers of the software vendors– 20 Finnish hospital districts, all mainland public

healthcare districts (response rate 75%)

– respondents mostly from specialized care facilities


results - Conformance testing and standards

• Term 'conformance testing' was misunderstood by two respondents

• 1 - 4 new versions each year for most applications• Interoperability standards and open interfaces used widely• Conformance testing performed variedly for entire applications

and between specific systems• Demand for the use of UML models and XML in standards • Reported problems:

– In some situations, no choice but to accept evident errors in applications or new versions

– testing among the first areas to be "cut down" in some companies– often difficult to apply and interpret standards, also national specifics

and laborious implementation hinder the adoption


results - Organization of interface testing services

• Suspiciousness of external interface testing services, lack of knowledge about such services

• Interoperability challenges not focused in testing– "testing services cannot help in integration"

• Certification of application interfaces was not considered very necessary nor useful

• Respondents were not familiar with the international developments of interoperability testing or certification– e.g. IHE Connectathons, CCHIT, European certification

initiatives


discussion and recommendations 1

1. Perform interoperability conformance testing more rigorously

surprisingly missing link between quality requirements and testing, more time for testing and various conformance testing activities required for diverse requirements

2. Utilize open interfaces and use interface testing services open interfaces are steadily increasing, promoting cost

effectiveness in general and in testing activities

3. Provide proper skills and knowledge needed: education about the benefits of standards-based

interfaces, common definitions and policies for interoperability testing


discussion and recommendations 2

4. Enforce the recommendations by authorities respondents were in favor of official recommendations in

relation to interoperability, security and usability

5. Organize neutral testing services little market possibilites for commercial testing services possibilities: automation of conformance testing neutral services / national mandates need further exploration

6. Reuse testing experiences and information some encouraging sharing of experiences from system

introductions, could be extended to expert groups

7. Improve the testability of specifications and standards conformance requires accuracy and quality from specifications


concluding remarks• Results can not be directly generalized

• Standards: perceived necessary but burdensome

• Conformance testing: hindered by attitude problems and lack of knowledge

• Improvements needed in relation to demand, investments and standardization

• But:– Interoperability testing is already part of common policies

– National / large-scale initiatives draw attention to interoperability and shared requirements, solutions and services

– Further discussions and developments encouraging (national healthcare IT services and related certification, interest and participation to interoperability testing on international level)

• first author: [email protected]

• speaker: [email protected]


other results from the SerAPI project in Medinfo'07

• Model-Centric Approaches for the Development of Health Information Systems (Tuomainen et al.)

• A National Study of eHealth Standardization in Finland - Goals and Recommendations (Mykkänen et al.)


THANK YOU

Tanja Toroi, [email protected],

Anne Eerola,

Juha Mykkänen, [email protected],

This work is part of the SerAPI

and AVOINTA projects:

www.serapi.fi/

www.centek.fi/avointa/Projects are funded by the Finnish funding agency

for Technology and Innovation TEKES together with a group of hospital districts / cities / joint

municipal boards for healthcare and companies.

Conformance Testing of Interoperability

Documents

Transcript of Conformance Testing of Interoperability