Health Informatics Evaluation and Health Technology

Assessment: Lessons and Warnings

Juzwishin, D., Borycki, E., Kushniruk, Nohr, C

University of Victoria & Aalborg University

June 2010

Dublin, Ireland

2

Objectives

• Foundations

• Emergent issues

• Compare and contrast

• Strengths and

weaknesses

• Opportunities for

collaboration

3

Foundations of HTA & HIT Evaluation

• HTA

– Focus on broad spectrum

of health care interventions

but non-IT focus

– Safety

– Efficacy

– Clinical effectiveness

– Cost effectiveness

– Social contextualization

– Perspective

– Outcomes oriented – e.g.

clinical controlled trials

• HIT Evaluation

– Focus on IT and software

– System safety, effectiveness and

efficiency

– System Usability

– Integration and interoperability

– Cost effectiveness

– Accuracy and Correctness of

Information/Data and Processing

– Reusability and Extensibility

– System Reliability

– Outcome as well as increasing focus

on Process Oriented Evaluation (i.e.

both quantitative and qualitative)

4

Emergent issues

• HTA

– Integration of medical

devices and information

systems

– In Canada patient

management software

fits the definition of a

medical device

– Uncertainty surrounding

effectiveness

– Examples of harm

emerging from IS elements

of health interventions

• HIT

– Problems with Large Scale

Deployments (e.g. local,

regional and national efforts)

– Socio-technical fit

– Adoption issues

– Usability

– System interoperability

– Integration with medical

devices

– Proof of financial benefits

– Slow progress

5

2010 World Health IT Conference and the EU

Ministerial High Level Conference on eHealth

• "eHealth Conference Declaration" adopted.

• "... we consider that the following points should be

addressed:

• 2a ... To evaluate eHealth, as much as possible in a

standardised manner, with respect to health outcomes,

benefits and cost effectiveness, including patient safety,

accessibility to care and quality of care"

6

Strengths from collaboration

• Establish a working linkage

between International HIT and

International HTA community

• Undertake an exploratory

paper to Identify issues of

common interest,

responsibility, and opportunity

• Identify mutually beneficial

methodologies and

approaches

• Test the level of interest

7

Resources

• STARE-HI—Statement on reporting of evaluation studies in

Health Informatics

– http://www.imia.org/endorsed/Stare-HI_as_published.pdf

• Health IT Evaluation Database

– http://evaldb.umit.at

– 1400 evaluation papers 1980 – 2009

• Bad Health Informatics can Kill

– http://iig.umit.at/efmi

• INAHTA Checklist

– http://www.inahta.org/HTA/Checklist/