Quality Reporting and Improvement Using

TechnologyMike Hindmarsh

Hindsight Healthcare StrategiesCincinnati, OH

June 18, 2010

How Do We Know That a Change is an Improvement?

“When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it,

when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.”

Lord Kelvin, May 3, 1883

Measuring Performance for Improvement

We cannot improve what we cannot measure IT is a critical success factor in gather

improvement data Understanding how to get data out of your

EMR is essential Garbage in, Garbage out “Why bother, I’m doing okay in practice!”

Sometimes gathering data can

bring new and

surprising knowledge!

Different Purposes for MeasurementAspect Improvement Research

Aim Improvement of care New knowledge

Methods: Test Observability

Test observable Test blinded or controlled

Bias Accept consistent bias Design to eliminate bias

Sample Size “Just enough” data, small sequential samples

“Just in case” data

Flexibility of Hypothesis

Hypothesis flexible, changes as learning takes place

Fixed hypothesis

Testing Strategy Sequential tests One large test

Determining if a Change is an Improvement

Run charts or Shewhart control charts

Hypothesis, statistical tests (t-test, F-test, chi square),

p-values

Confidentiality of the Data

Data used only by those involved with improvement

Research subjects’ identities protected

Getting Started:Identify your population

Create a query that identifies the population (e.g., ICD-9 codes)

Be sure to code populations the same way. Don’t “over-worry” about the accuracy of case

finding: you can clean it up as you go. Get baseline data in the easiest, fastest fashion

possible…..if it’s hard to program, leave it for later! Don’t obsess about baseline data accuracy! Select measures that you can impact and have clinical

relevance

For Each Measure Use measures from evidence-based guidelines Create concrete operational definitions to

capture data from you IT in a reliable, consistent manner

Use the reporting template to display your measures as quickly as possible.

Set goals based on evidence or best pratice Gather data monthly (quarterly at a minimum!)

Example of Improvement Measures: Diabetes

% with A1c<7 % with BP<130/80 % on ACE1 or ARB % with LDL<2.0 nmol/l % screened for microalbumen % with eye screen in last 24 months % with foot exam in last 12 months % with A1c in last six months % with documented self-mgmt goals in medical

record

Annotated Time Series

Percent of Patients with Documented Self-mgmt. Goals

0

10

20

30

40

50

60

70

80

90

100

Mar

-99

Apr

-99

May

-99

Jun-

99

Jul-9

9

Aug

-99

Sep

-99

Oct

-99

Nov

-99

Dec

-99

Jan-

00

Feb-

00

Mar

-00

Apr

-00

May

-00

Jun-

00

Jul-0

0

Aug

-00

Sep

-00

Perc

ent o

f Pat

ient

s in

Reg

istry

Goal=90%

Cycle 1: Dr. Smith/ 3 patients

Cycle 2: Test of Group VisitCycle 3: 8 patients self measuring blood glucose

“This type of data reporting isn’t reliable!!”

Yes it is….for quality improvement purposes! Trending is what you are looking for…not statistical

reliability. Your “goal” is to improve population outcomes over

your usual care by changing care for every patient: the annotated time series (run chart) will tell you if you are succeeding!

70

35

01020304050607080

Before Change After Change

Perc

ent o

f Dia

betic

s in

Reg

istr

y

Improvement in Glycemic Control Percent of (Patients with HbA1c >9 in Clinic A)

Improvement in Glycemic Control (% of Population with HbA1C >9 in Clinic A)

0102030405060708090

100da

te

Jan

Feb

Mar Ap

r

May Jun

Jul

Aug

Sep

Oct

Nov

Dec

Change Implemented

% o

f Dia

betic

s in

Reg

istr

y

70

35

01020304050607080

Before Change After change

Perc

ent o

f Dia

betic

s in

Reg

istr

y

Improvement in Glycemic Control (% of Population with HbA1C >9 Clinics B and C)

Improvement in Glycemic Control (% of Population with HbA1C >9 in Clinic B)

0102030405060708090

100

date

Jan

Feb

Mar Ap

r

May Jun

Jul

Aug

Sep

Oct

Nov

Dec

Change Implemented

% o

f Dia

betic

s in

Reg

istr

y

Improvement in Glycemic Control (% of Population with HbA1C >9 in Clinic C)

0102030405060708090

100da

te

Jan

Feb

Mar Ap

r

May Jun

Jul

Aug

Sep

Oct

Nov

Dec

Change Implemented

% o

f Dia

betic

s in

Reg

istr

y

Month

Mar-99Apr-99May-99Jun-99Jul-99

Aug-99Sep-99Oct-99Nov-99Dec-99Jan-00Feb-00Mar-00Apr-00May-00Jun-00

Key Measur

e

0

10

20

units

Percent of Patients with 2 HbA1c's in Last Year

0

20

40

60

80

100

Mar-99 Apr-99 May-99 J un-99 J ul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 J an-00 Feb-00 Mar-00 Apr-00 May-00 J un-00

perc

ent

Goal

Percent of Patients with HbA1c > 9.0%

2030405060708090

100

Mar-99 Apr-99 May-99 J un-99 J ul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 J an-00 Feb-00

perc

ent

Goal

Average HbA1c Values for Diabetic Patients

77.5

88.5

99.510

10.511

Mar-99 Apr-99 May-99 J un-99 J ul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 J an-00 Feb-00

aver

age

Goal

Percent of Patients with Documented Self-Management Goals

0

20

40

60

80

100

Mar-99 Apr-99 May-99 J un-99 J ul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 J an-00 Feb-00

perc

ent Goal

Population and Currrent Size of Registry

0

50

100

150

200

Mar-99 Apr-99 May-99 J un-99 J ul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 J an-00 Feb-00

num

ber

in r

egis

try

Population size

Run Charts of Monthly Measures - Diabetes Population

Summary Define your population with carefully thought

out queries Obtain baseline data on relevant measures

through structured queries Produce regular reports for improvement

activities Set targets/goals for imporvement based on

best practice Be ambitious!