EFFECTIVE GENERAL PRACTICE: AUDIT & FEEDBACK FOR THE IMMUNISATION

PRIMARY SERIES.Gary Reynolds, Mareta Timo, Anjileena Dev, Susannah Holt, Tracey Poole, &

Nikki Turner

Immunisation Advisory CentreUniversity of Auckland, New Zealand

September 2015

QUOTE “Someone told me that each equation I included in the

book would halve the sales”

Stephen Hawking –Theoretical physicist

BACKGROUND – NZ Context High Immunisation rates are a Govt. Target

The Target - 95% by age 2 years

With Imm Rates just over 93%, the last 2% towards the Govt & WHO target of 95% - Just how challenging?

Targets – 95% immunised by 8 months since late 2013

Recent years - Emphasis on timeliness

There is enormous variation in immunisation delivery between General Practices

Establishing the denominator – ie. the number eligible for immunisation hase been a priority

BACKGROUND –Audit & Feedback

A Plethora of Stats generated for General Practice

Meaningful? Accurate? Relevant?

Need to develop audits that are useful and identify deficiencies in vaccine delivery

Use of A&F Tools allow quality improvements

Audit = collection of data (clinical performance)

Feedback = data to practice with or without recommendation

Need to develop best practice systems

AIM To examine the Primary Series (6w, 3m & 5m) delivery

in General Practice in NZ

To determine the accuracy of current auditing systems for the primary series

To establish a rigorous auditing system for the primary series

To identify the rules of vaccine delivery that improve outcomes

OBJECTIVES To confirm the accuracy of NIR data and different practice

based audits for individual general practices for the primary series

To develop an audit system which establishes an accurate numerator (no. immunised) and denominator (no. eligible) for immunisation

To specifically examine the last 5 -10% of delivery towards the target 95% immunisation rates

To examine European, Maori and Pasifika providers

To re audit the same practices an interval later

METHODS

Setting 1-Pasifika Provider Pasifika Horizon Healthcare – Inner West Auckland Pasifika practice - 1640 patients PMS - Medtech 32

Setting 2- Practice 1 A typical Practice – Inner SW suburbs of Auckland Practice 1 - 3000 enrolled patients PMS – Medtech 32 So-called “average” practice –(90% immunisation rate)

Setting 3-Maori Provider A Maori Provider –Urban South Auckland 5500 enrolled patients PMS – Medtech 32

METHODS The Dataset

“All infants enrolled at these practices requiring a primary series of 6w, 3m and 5m immunisationsaccording to the 2011 NZ immunisation schedule within a set 1 year period.”

1-2-2011 to 1-2-2012.

1-6-2011 to 1-6-2012.

1-6-2012 to 1-6-2013

METHODS Obtained the NIR report for the 1 year period for each

practice

Run a PMS Audit for each practice with the same NIR parameters (PMS Audit = NIR report)

The audit would identify the number immunised (Numerator) eligible (Denominator)

Run the Multi-Audit (A&F tool) on all practices

Correlate all immunisation events with the NIR Database

METHODS-The A&F Tool The Multi Audit = Multiple sub audits: 1) The 5m immunisation = the PMS based audit = NIR

report parameters. 2) The 6w immunisation not in the 5m audit. 3) The 3m immunisation not in the 6w and 5m audit. 4) No immunisations at 5m. 5) Total children born 6 months before the period to the

end of the period not in all the above audits.

Confirmation of Immunisation History Using NIR

ResultsRESULTS-Pasifika Horizon Healthcare

RESULTS – Pasifika audits All 3 initial audits were different NIR report 31/36 (86.1%) PMS 5m audit 39/43 (90.7%) {Manual audit 41/48 (85.4%)}

Requested a printout of NHI numbers from the NIR of the no. eligible (denominator)

RESULTS – Pasifika A&F Tool The Multi Audit identified (x=57) The 5m immunisation = the PMS based audit. (Group A =

43). The 6w immunisation not in the 5m audit (Group B = 7). The 3m immunisation not in the 6w and 5m audit (Group

C = 3). No immunisations at 5m (Group D = 2). Total children born 6 months before the period to the end

of the period not in all the above audits (Group E = 2).

RESULTS - exclusions A total of 10 of the total 58 candidates (17.2%) were

excluded for following reasons.

Input errors = 2 (3.4%)

Transferred out of the practice before the primary series was complete = 5 (8.6%)

Casual Patients seen but enrolled at other practices = 2 (3.4%)

Enrolled after the period = 1 (1.7%)

Multi-Audit

RESULTS-Multi Audit All 48 (100%) started their primary series

95.8% (46/48) fully immunised for their primary series. NIR report 31/36 (86.1%) -9.7% difference PMS 5m audit 39/43 (90.7%) {Manual audit 41/48 (85.4%)}

4.2% (2/48) were partially immunised missing only one of their injections.

None (0%) were unimmunised despite the initial audit revealing 2

RESULTS - timeliness Using strict timeliness criteria, 52.1% (25/48) were

immunised on time by 5 months

43.7% (21/48) were late for some or all of their immunisations

A total of 83.3% (40/48) would satisfy the new target of completion of the primary series immunisations by the 8-month milestone.

RESULTS - rules for PHHC Give injections sequentially 6w, 3m then 5m

Patient transfers out of practice Advise NIR -?other way round

Patient transferred in Run Status query with NIR database to update your PMS immunisaton history

Partially immunised need to stay in the inbox until the catch up injections have happened

Casual patients immunise and enter into NIR database. Invoice but don’t count. Complete a NIR Status

query

Avoiding Input Errors Don’t input all vaccines on the same day. Run NIR status query

Shared care patients Important for Maori -Run NIR status query

ResultsRESULTS-Practice 1-The Average

NIR report for Practice 1

RESULTS- Practice 1

Audit Practice 1

NIR 38/42 (90.5%)

PMS 37/40 (92.5%)

Multi 44/49 (89.8%)

Requested a printout of NHI numbers from the NIR of the no. eligible (denominator)

RESULTS- Comparison

Multi Audit Practice 1

Total 44/49 (89.8%)

5 month 37

6 week 2

3 month 0

NO 5 month imms 1

Total Babies 4

RESULTS- Audit Exclusions

Audit Practice 1

Multi 20/69 (30%)

Input errors 2 (2.8%)

Transfer Out 5 (7.2%)

Went Overseas 3 (4.3%)

Enrolled Outside Period 10 (14.5%)

RESULTS- Comparison

Multi Audit Practice 1

Multi 44/49 (89.8%)

Unimmunised 1/49 (2%)

Partial imms 0

Decline 4/49 (8.2%)

Timeliness 43/49 (87.8%)

Government Targets 95% by 8 months of Age

Results –Practice 1 to 95%

Excellent systems and NIR contact 44/49 (89.8%) Declines 4/49 (8.2%) Unimmunised referred to outreach 1/49 (2.0%) Met the new govt. target of 8m 43/49 (87.8%)

Practice 1NIR Audit 38/42 90.5%

PMS Audit 37/40 92.5%

Multi-Audit 44/49 89.8%

Feedback for Practice 1 Excellent nurse –NIR interaction Avoiding input errors Partially immunised stay in their inbox (outreach) No partially immunised –once vaccinated, they finished to

primary series Plan of attack for “declining” parents

ResultsRESULTS-The Maori Provider

Maori Provider NIR report

RESULTS- Maori

Audit Maori

NIR 115/144 (80%)

PMS 139/162 (85.8%)

Multi 159/194 (82%)

Requested a printout of NHI numbers from the NIR of the no. eligible (denominator)

RESULTS- A&F Tool

Multi Audit Maori

Total 159/194(82%)

5 month 139

6 week 15

3 month 1

NO 5 month imms 2

Total Babies 2

RESULTS- Audit Exclusions

Audit Maori

Multi 120 /314 (38.2%)

Input errors 21 (6.6%)

Transferred 54 (17.2%)

Went Overseas-Died 2 (0.3%)

Enrolled Outside Period 43 (13.7%)

RESULTS- Comparison

Multi Audit Maori

Multi 159/194(82%)

Unimmunised 7/194 (3.6%)

Partial imms 35/194 (18%)

Decline 3/194 (1.5%)

Timeliness ?/194 (%)

Government Targets 95% by 8 months of Age

Results –Maori to 95%

Systems and NIR contact 159/194 (82%) Declines very low 3/194 (1.5%) Partially immunised 35/194 (18%) Unimmunised ?outreach 7/ 194 (3.6%) Met the new govt. target of 8m ?/194 (?%) Transient population 54/314 (17.2%)

Maori

NIR Audit 115/144 (80%)

PMS Audit 139/162 (85.8%)

Multi-Audit 159/194 (82%)

RESULTS - rules for Maori Provider

Give injections sequentially 6w, 3m then 5m

Patient transfers out of practice –extreme care Advise NIR -?other way round

Patient transferred in –extreme care Run Status query with NIR database to update your PMS immunisaton history

Partially immunised need to stay in the inbox until the catch up injections have happened

Casual patients immunise and enter into NIR database. Invoice but don’t count. Complete a NIR Status query

Avoiding Input Errors Don’t input all vaccines on the same day. Run NIR status query Nurse education re NIR interaction esp a NIR Status query

Shared care patients Important for Maori -Run NIR status query eg Born to and 6w at practice 3m elsewhere 5m elsewhere again. Plunket Is particularly helpful to this practice Care with NIR Assignation

CONCLUSIONS Possible to improve immunisation rates just with accurate counting

The Pasifika practice under reported rates by 9.7% on NIR (86.1%) when actually running at 95.8%. Above the current NZ government and WHO target

The “average” practice in this study will struggle to hit 95% due to the numbers declining vaccination

The Maori population in this study is highly mobile often receiving the primary series at multiple providers

Each audit had different Numerators & Denominators ?Why. The A&F Tool identifies a more accurate denominator

The A&F Tool identifies deficiencies in vaccine delivery

The NIR ‘status query’ is an excellent tool to determine immunisation history

Direct reconciliation with the NIR database is a key step for improving accuracy.

FUTURE Confirm and refine the A&F Tool

More Discussion with NIR

Reconcile the NIR and Practice Data

Best Practice Vaccine Delivery Systems

Acknowledgements IMAC / CONECTUS Staff

Mareta Timo

Anjleena Dev

Leane Els

Tracey Poole

RNZCGP funding