Evaluating a new model of Primary Health Care service delivery in remote Queensland: Lessons Learned

Kristine Battye, Peter Stanley-Davies and

Elaine Ashworth

Outline

• Describe NWQAHS to provide context for evaluation

• Structural issues encountered

• Difficulties in establishing a system to “measure” process and impact of PHC

• Type of data available for planning and evaluating PHC services

North West Qld Allied Health Service

Operational Hub and Spoke Model

Mt Isa Hub

Gulf precinct5 communities

Highway precinct3 communities

Mt Isa precinct3 communities

Key Features

•Functional teams

•6 month calendar

•6 weekly rotations

•2-3 days in each community

•Primary health care

•Centralized booking system

•Therapy assistants in each community

•Videoconference follow-up

•Case conference with resident health professionals

NWQPHC Board

Chief Executive Officer

Executive OfficerOperations & Outcomes

Executive OfficerAdmin & Finance

Area Manager

Area Manager

Area Manager

Area Manager

NWQAHS Manager

NWQPHC Board

CEO

NWQAHSService Manager

9 AHPs1 Admin

Community PanelAdvisory

Initial Management Structure (1.5 Years)

Domains of the Evaluation

• Recruitment and retention strategy

• Management and operation of service

• Service delivery – access and PHC

• Impact – community and individual

• Integration with other service providers

• Comparative cost effectiveness with alternate models

Methodology Proposed

• Qualitative - Direct intermittent information gathering

• Quantitative - Indirect and continuous monitoring, data collection, surveillance and use of sentinel communities, and health issues

Difficulties encountered

• Shared perspective of purpose of evaluation• Time frame • Information Mx system relevant to PHC

activity• Recognition of complexity of PHC service

delivery• Management capacity and multiple demands• Are we trying to collect the right data

anyway?

Shared purpose of evaluation

Management Capacity – Multiple demands

Time frame

Structural Issues impacting on Evaluation

Shared purpose of evaluation

Complexity of servicedelivery model

Info Mx system that captures complexity

Is it the “right” data?

PHC paradigm but what data set?

Management Capacity – Multiple demands

Time frame

Purpose of Evaluation

Management Funder Evaluator

Summative – Did the service do what it said it would do, and do it well enough to be refunded?

Developing internal systems in parallel to the evaluator

Do what it said it would do?Meet objectives of RHS programImprove access – occasions of service

Value for $$

Formative and Summative– Development of systems to support roll-outAspects of model needing modification to achieve goalsImpact on client & communities

Shared purpose of evaluation

Management Capacity – Multiple demands

Time frame

Structural Issues impacting on Evaluation

Structural issues: Lessons Learned and Implications for Policy

1. Realistic timeframes for service establishment and realistic expectations of deliverables in first 3 years

2. Adequate resource allocation to management in the service establishment phase (service and auspice)

3. Greater emphasis on formative evaluation by funders and service providers

4. Broader performance indicators for primary health care services – reduced emphasis on occasions of service

Practical Issues around “Measuring”

Primary Health Care

Shared purpose of evaluation

Complexity of servicedelivery model

Info Mx system that captures complexity

Is it the “right” data?

PHC paradigm – How is it measured?

Management Capacity – Multiple demands

Time frame

Information Management System

Specifications:

Client demographics and indigenous identifier

Clinical treatment records

Time use data – activities in conjunction with treatment

- community focused activities

Client outcomes/ client centred goals

Total community attributable time 2002/2003

0.0

100.0

200.0

300.0

400.0

500.0

600.0

Bur

keto

wn

Cam

oow

eal

Clo

ncur

ry

Daj

arra

Doo

mad

gee

Hug

hend

en

Julia

Cre

ek

Kar

umba

McK

inla

yM

orni

ngto

nIs

land

Nor

man

ton

Ric

hmon

d Community

Ho

urs Client

Community

Travel

Mornington Island - community attributable time 2002/2003

0.010.020.030.040.050.060.070.080.090.0

Discipline

Ho

urs

Client

Community

Travel

Hughenden - community attributable time 2002/2003

0.020.040.060.080.0

100.0120.0140.0160.0180.0200.0

Discipline

Ho

urs

Client

Community

Travel

Quantitative data

• Referrals as a proxy for occasions of service

• Used to measure access to service by indigenous and non-indigenous people at a community level (2nd yr)

• Management capacity increased – development of chart system

Analysis of Referrals by community 2002/03

Community No. Referrals

% Pop referred

No. Indig. Referrals

Est. Indig. Pop

% Indig. Pop referred

Burketown 103 44% 35 94 37%

Camooweal 23 7% 10 145 7%

Cloncurry 172 5% 31 880 4%

Dajarra 20 9% 14 195 7%

Doomadgee 154 11% 103 1,231 8%

Mornington Is 195 16% 107 1,115 10%

Normanton 81 5% 47 866 5%

Richmond 221 24% 5 103 5%

Hughenden 322 12% 45 260 17%

Julia Creek 62 7% 8 30 27%

Total 1,419 11% 409 5,064 8%

REAL LIFE Data Collection Issues

• AHPs record info/data relevant to their job, or see the value of it – data quality is better

• Outreach service - client info maintained in a number of places

• Reason for referral not “centralized” but recorded in client notes

• Centralized data base – maybe need a data “enterer”

• Coding? ICPC developed by WONCA

Do we collect the right data to evaluate and plan for PHC?

• Measure what we can measure• Occasions of service – proxy for workload?• How do you evaluate the number of oldies you keep out

of institutions because they have had access to allied health interventions?

• How do you measure the impact of early intervention in services that operate across the age continuum?

• Risky!! Why do we try and plan PHC (wellness) services at a local level using secondary care (sickness) data? Because that’s all there is!?

PHC: More Lessons Learned

• Need a mix of qualitative and quantitative measures to evaluate PHC – perhaps with equal emphasis

• Development of information management and evaluation processes need to be staged, and recognized in contracts with funders

• We need to re-think the data set and data collection processes we use to plan and evaluate primary health care services