Model for Monitoring and Evaluation of Overall Health System Performance for Comparison Based on the Study Conducted for the Ministry of Health

General Objective

Comparison of overall health system performance against selected indicators between districts using one index district from each province.

The mandate was to look at routinely collected and readily generated data at district level to measure the selected indicators.

Mandate

Do other countries use health system performance

indicators?

Cross-Country Comparison of Concepts of Health System Performance

Dimensions and Subcategories of Health

System Performance

OECD Proposed

framework

WHO framework

Australia’s Proposed

framework

Canada’s Health System

Performance framework

UK’s NHS High-level Performance Framework

Health Improvements/Outcomes

X X X X X

Appropriateness X X X

Capacity/Competence X X

Safety X X X

Responsiveness X X

Patient Satisfaction Patient experience/ accessibility Acceptability

X

X

X

X

X

Accessibility (in terms of timeliness of services

X X X

Continuity X X

Cross-Country Comparison of Concepts of Health System Performance…contd…

Dimensions and Subcategories of Health

System Performance

OECD Proposed

framework

WHO framework

Australia’s Proposed

framework

Canada’s Health System Performance

framework

UK’s NHS High-level Performance Framework

Equity X

Equity of Health Outcomes X X

Equity of access X X X X X

Equity of Finance X X

Efficiency X X

Macroeconomic efficiency X

Overall micro efficiency X X

Unit costs X X X

PROCESS

Initially a steering committee was established consisting of ministry officials & the AHF secretariat.

Based on the literature review of national and global materials and the documents available with the MoH a draft conceptual framework for measurement of health system performance was presented to Steering Committee and modified taking into account the views of the committee.

Concurrently the study team also looked at the indicators used world wide for performance measurements.

After identifying the readily available indicators at provincial and national levels with the concurrence of the steering committee it was decided to place them before high level ministry officials from centre as well as the managers from the provinces.

The following indexed districts one per each province were selected for comparison with the concurrence of the ministry officials & PDHS’s.

The selected districts were - Gampaha – Western Province - Ratnapura – Sabaragamuwa -Province- Anuradhapura – North Central-Province- Galle – Southern Province - Matale – Central Province - Trincomalee – North East Province- Badulla – Uva Province - Kurunegala – North Western Province

Sri Lankan Health Performance Framework- A Model

HOW HEALTHY ARE THE CITIZENS? IS IT THE SAME FOR EVERYONE?

WHERE IS THE MOST OPPORTUNITY FOR IMPROVEMENT?

Health condition Life expectancy and wellbeing

Deaths

%LBW Infectious diseases

-incidence of malaria

-incidence of dysentery

-TB new cases Prevalence of anaemia among pregnant womenIncidence of HIV/AIDS(NSACP)

Life expectancy IMR MMR NMR Case fatality rate for dengue fever/DHF

Health Status and Outcome Tier 1

IS THE SYSTEM GEARED TO MEET EXPECTATIONS & THE FELT NEEDS OF THE CITIZENS?

Responsiveness Accessibility & Equity

Safety

These dimensions are not going to be measured during this project.

-Indicators not identified. Data has to be obtained from community surveys.

Measuring equity in

access to health services requires household survey data at district level.

-Incidence of adverse drug reaction-incidence of nosocomial infection

Responsiveness & Access Tier 2

Effectiveness Efficiency Sustainability Health Services

Measles coverage

Tetanus toxoid (TT2 +) given to pregnant mothers

% of pregnant mothers tested for VDRL

Pap smear screening rate

Number of deaths within 48hrs of admission

In pt /staff

Hospital bed occupancy rate

Average length of stay (MSU)

% expenditure for health vs. total budget

% of expenditure on drugs

CS Rate

DOTS treatment success rate

Nurses or doctors/hospital bed (AHB)MSU

Hospital beds /1,000 population (AHB)MSU

Doctors /100,000 population (AHB)MSU

Nurses/100,000 population

PHM/ 100,000 population (AHB)MSU

Patient Transfers

No. of new cases of diabetics in hospital clinics for

-diabetics-Hypertension-Cancer

% of medical audits done for

-maternal deaths-still births

No. of major surgical operations

No. of minor surgical operations

Health System Performance Framework

Tier 3

Routine data was not available for the following indicators:

Prevalence of anaemia among pregnant women Inpatient to Staff Ratio Percentage of expenditure for health vs. total budget Percentage of expenditure on drugs Patient Transfers Number of New cases of NCDs in hospital clinics

for:Diabetes HypertensionCancer

Percentage of Medical Audits done for still Births

RESULTS - PILOT STUDY

Health Status and Outcome Tier 1

Health Conditions

Incidence of HIV/AIDS

0

1

2

3

3

4

4

7

9

0 1 2 3 4 5 6 7 8 9 10

Ratnapura Galle

AnnuradhapuraGampaha

MataleAverage Badulla

KurunegalaTrincomalee

Dis

tric

t

TB New Cases per 100,000 Population

93

97

108

141

142

169

189

211

549

0 100 200 300 400 500 600

Galle

Ratnapura

Trincomalee

Matale

Gampaha

Kurunegala

Average

Annuradhapura

Badulla

Dis

tric

t

DeathsInfant Mortality Rate (2005)

Infant Mortality Rate per 1000 live births

DefinitionNumber of deaths to infants under one year of age per 1,000

live births in a given year

5.3

6.3

9.1

9.9

10.15

10.4

12

12.4

13.55

0 2 4 6 8 10 12 14 16

Trincomalee

Galle

Matale

Average

Gampaha

Badulla

Ratnapura

Kurunegala

Anuradhapura

%

Dis

tric

t

Health System Performance Tier 3

Effectiveness Percentage of pregnant mothers tested for VDRL (2005)

58

66

68

70

71

72

74

75

77

50 55 60 65 70 75 80

Trincomalee

Ratnapura

Gampaha

Average

Galle

Annuradhapura

Matale

Badulla

Kurunegala

District

%

Percentage of Pregnant mothers Tested for VDRL

DefinitionNumber of mothers tested for VDRL as a percentage of total number of

deliveries reported.

Average length of stay(2005)

1.4

2.1

2.2

2.5

2.6

2.7

2.8

3.0

3.0

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5

Kurunegala

Ratnapura

Gampaha

Average

Trincomalee

Galle

Matale

Annuradhapura

Badulla

District

Number of Days

Average Length of Stay in Hospital

DefinitionThe average length of stay a patient spends in a government hospital. It is measured by dividing the total number of days

stayed by all inpatients in government hospitals during a year by the number of admissions

Efficiency

Caesarean Section Rate (2005)

Caesarean Section Rate

14.6

19.5

20.0

21.9

22.5

22.7

23.2

23.9

28.6

0 5 10 15 20 25 30 35

Trincomalee

Ratnapura

Anuradhapura

Average

Badulla

Kurunegala

Galle

Matale

Gampaha

Rate

Dis

tric

t

DefinitionThe number of caesareans per 100 live births in government hospitals

Sustainability

Nurses per 100,000 population(2005)

74

80

84

86

104

113

137

157

184

0 50 100 150 200

Matale

Ratnapura

Gampaha

Anuradhapura

Kurunagala

Average

Trincamolee

Badulla

Galle

Dis

tric

ts

Total number of nurses

Health Services Percentage of Medical Audits done for: Percentage of Medical Audits done for

Maternal deaths(2005)

100

100

100

100

100

N/A

N/A

N/A

0 20 40 60 80 100 120

Trincomalee

Galle

Badulla

Matale

Gampaha

Kurunegala

Annuradhapura

Ratnapura

Dis

tric

ts

%

Percentage of medical audits done for maternal deaths

SUMMARY SHEETS FOR OVERALL HEALTH SECTOR PERFORMANCE BY DISTRICT

Figure 20.0 Summary sheet for overall health sector performance -Gampaha

0

1

2

3

4

5

6

7

8

9

Per

centa

ge o

f m

edic

al a

udits done

for m

ater

nal

dea

ths

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

R

ate

of e

pisodes

of D

ysen

tery

per

100,0

00

popula

tion

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

VD

RL

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

erPer

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

year

sPH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant m

oth

ers

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Figure 20.1 Summary sheet for overall health sector performance -Galle

0

1

2

3

4

5

6

7

8

9Per

centa

ge o

f m

edic

al a

udits done

for

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

Rat

e of e

pisodes

of D

ysen

tery

per

100,0

00

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

er

Per

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

PH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Figure 20.2 Summary sheet for overall health sector performance -Trincomalee

0

1

2

3

4

5

6

7

8

9

Per

centa

ge o

f m

edic

al a

udits done

for m

ater

nal

dea

ths

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

R

ate

of e

pisodes

of D

ysen

tery

per

100,0

00

popula

tion

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

VD

RL

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

erPer

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

year

sPH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant m

oth

ers

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Figure 20.3 Summary sheet for overall health sector performance -Kurunagala

0

1

2

3

4

5

6

7

8

9

Per

centa

ge o

f m

edic

al a

udits done

for m

ater

nal

dea

ths

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

R

ate

of e

pisodes

of D

ysen

tery

per

100,0

00

popula

tion

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

VD

RL

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

erPer

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

year

sPH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant m

oth

ers

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Figure 20.4 Summary sheet for overall health sector performance -Anuradhapura

0

1

2

3

4

5

6

7

8

9Per

centa

ge o

f m

edic

al a

udits done

for

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

Rat

e of e

pisodes

of D

ysen

tery

per

100,0

00

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

er

Per

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

PH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Figure 20.5 Summary sheet for overall health sector performance -Badulla

0

1

2

3

4

5

6

7

8

9

Per

centa

ge o

f m

edic

al a

udits done

for m

ater

nal

dea

ths

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

R

ate

of e

pisodes

of D

ysen

tery

per

100,0

00

popula

tion

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

VD

RL

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

erPer

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

year

sPH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant m

oth

ers

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Figure 20.6 Summary sheet for overall health sector performance -Ratnapura

0

1

2

3

4

5

6

7

8

9

Per

centa

ge o

f m

edic

al a

udits done

for m

ater

nal

dea

ths

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

R

ate

of e

pisodes

of D

ysen

tery

per

100,0

00

popula

tion

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

VD

RL

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

erPer

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

year

sPH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant m

oth

ers

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Figure 20.7 Summary sheet for overall health sector performance -Matale

0

1

2

3

4

5

6

7

8

9

Per

centa

ge o

f m

edic

al a

udits done

for m

ater

nal

dea

ths

Num

ber

of M

ajor Surg

ical

Oper

atio

ns

Per

centa

ge o

f Low

Birth

Wei

ghts

R

ate

of e

pisodes

of D

ysen

tery

per

100,0

00

popula

tion

Cae

sare

an S

ection R

ate

Num

ber

of M

inor Surg

ical

Oper

atio

ns

Rat

e of H

osp

ital

Bed

Occ

upan

cy

Annual

Par

asitic

Index

for M

alar

ia

Neo

-nat

al M

ort

ality

Rat

e

Hosp

ital

Bed

s per

1,0

00 P

opula

tion

Inci

den

ce o

f H

IV/A

IDS

Mat

ernal

Mort

ality

Rat

e

Infa

nt M

ort

ality

Rat

e per

1000 liv

e birth

s

Doct

ors

per

100,0

00 P

opula

tion

TB N

ew C

ases

per

100,0

00 P

opula

tion

Num

ber

of D

eath

s w

ithin

48 h

ours

Aver

age

Len

gth o

f Sta

y in

Hosp

ital

Per

centa

ge o

f Pre

gnan

t m

oth

ers Tes

ted for

VD

RL

Nurs

es p

er 1

00,0

00 P

opula

tion

Fat

ality

Rat

e fr

om

Den

gue

Fev

erPer

centa

ge o

f under

wei

ght ch

ildre

n b

elow

five

year

sPH

Ms per

100,0

00 P

opula

tion

Per

centa

ge o

f TT2+

giv

en to p

regn

ant m

oth

ers

DO

Ts Tre

atm

ent Succ

ess R

ate

Per

centa

ge o

f M

easles

Cover

age

Indicator

Po

sit

ion

Direction for Establishing & Using this Model for Systematic Monitoring for Comparison of Health System Performance between Districts in the Future

POLICY ISSUES

For future analysis the level of analysis of costing, whether provincial or district, needs to be identified. At present there is no provision to collect and compile the cost data at the district level as they are not accountable for them. Hence a policy decision needs to be taken whether performance comparison should be at district level or at provincial level.

It is better to look at all inputs from both line ministry and the provincial ministry for an accurate comparison, as the health outcomes etc. will be dependant on all resource inputs to a particular province, rather than through provincial health sources only. This could best be done at the central level.

It is also recommended to review the IMMR and hospital returns sent to the medical statistician and to modify them to include additional information which, at present, is already available at institution level but not collated and reported.

Wherever national figures are available for the selected indicators these should be compiled from them as they are more accurate and reliable. Since the numerator and the denominators used will be same and for comparison across the districts.

RECOMMENDATIONS

A system for regular monitoring of health system performance should be established.

Provincial health authorities need to adopt this model for their monitoring purposes & use this at review meetings with the district health authorities

For this purpose planning cells of PDHS/DPDHS offices has to be strengthened.

Timely & accurate electronic data flow to the DPDHS offices from the periphery for compiling the selected indicators should be established.

PDHS’s should provide expenditure data to DPDHs for monitoring purposes.

Expenditure on drugs by individual institutions should be monitored at DPDHS level.

These data bases from the DPDHS offices should be linked with the PDHS’s planning units & the MDPU of the MoH

For the present the responsibility of M & E of Health System Performance could be a joint effort of both organizations MDPU and the AHF secretariat. This responsibility should be transferred to the D/I of the MDPU of the MoH after AHF secretariat cease to function after the project period.

It is to be noted that in other countries too most of the health status indicators are calculated by a central organization for consistency.

It is recommended that to get data for the second tier - responsiveness and access, as well as for NCDs, including the risk factor prevalence a national health survey is carried out every three to four years, depending on the resource availability, or alternatively, to look at the feasibility of combining this with DHS survey to cut down costs to the health ministry.