KEY PERFORMANCE INDICATORS
IN HOSPITALS
From Measurement and Reporting to Actual Improvement and Action
DAY 2
Fadi El-Jardali, MPH, PhD.
March 2016
DEVELOPMENT AND
APPLICATIONS OF
BALANCED SCORECARD
SYSTEM IN HOSPITALS
The Balanced Scorecard Concept
The Balanced Scorecard
• Tracks indicators across different perspectives
• Provides a balanced view of performance
• Guides strategic decisions at both the provider- and health system- levels
• Tells a brief yet comprehensive story about the achievement and
performance of the organization toward goals.
Rationale
• No single aspect of the organization/ system causes poor or excellent
performance
• For this reason, performance-measurement activities must include measures
that provide insights into multiple dimensions of performance
El-Jardali F, Saleh S, Ataya N, Jamal D. Design, implementation and scaling up of the balanced scorecard for hospitals in Lebanon: policy coherence and application lessons for low and middle income countries. Health Policy. 2011 Dec;103(2-3):305-14. Zelman WN, Pink GH, Matthias CB. Use of the balanced scorecard in 686 health care. Journal of Health Care Finance 2003;29:1–16. Pink GH, McKillop I, Schraa EG, Preyra C, Montgomery C, Baker GR. 668 Creating a balanced scorecard for a hospital system. Journal of Health 669 Care Finance 2001;27:1–20.
What’s the difference between Dashboards
and the Balanced Scorecard?
http://www.bscdesigner.com/dashboard-vs-balanced-scorecard.htm
Dashboard Balanced Scorecard
Used for Performance measurement /
monitoring
Performance management and
progress
Focus Operational (short-term)
goals
Strategic (long-term) goals
It helps Visualize the performance to
understand the current state
Align KPI, objectives, and
actions to see the connection
between them
Example Automobile dashboard
(shows how your car is
operating)
GPS (shows when and how
you will arrive?)
International Applications of the BSC to Healthcare Sector
Ontario Acute Care
Hospitals
Hospital Flexibility Tracking
Project
National Health Service Trust
Hospitals
Participation Voluntary
Acute care hospitals in
Canada
Voluntary
Critical access hospitals in the
US
Compulsory
All NHS Trust hospitals
Primary
Audience
Providers & Consumers Providers & Policymakers Political and social agenda of
labour government
Purpose • Increase awareness of
performance variation
• Stimulate improvement
• Ensure accountability
• Improve quality of data
• Inform policymakers • Provide a balanced view of
NHS performance
Data
reported
• In Aggregate
• For Individual Hospitals
• In Aggregate • In Aggregate
• For Individual Hospitals
Domains • Financial Performance
Condition
• Patient Satisfaction
• Clinical Utilization and
Outcomes
• System Integration and
Change
• Financial
• Customer
• Process
• Infrastructure,
Governance, and
Community Relations
• Health improvement
• Fair access
• Effective delivery of
appropriate care
• Efficiency
• Patient experience
• Health outcomes of NHS care
Data 38 indicators for 89
organizations
47 indicators for 538 hospitals 36 indicators for All NHS Trust
hospitals
• First application of the BSC
in a developing country
• The BSC was used to
regularly monitor the
progress of the strategy of
the Ministry of Public Health
in Afghanistan to deliver a
basic package of health
services
• Facilities included: basic
health centres,
comprehensive health
centres, and the outpatient
and maternity wings at
district hospitals
• 29 core indicators and
benchmarks representing six
different domains of health
services.
Peters DH, Noor AA, Singh LP, Kakar FK, Hansen PM, Burnham G. A 696 balanced scorecard for health services in Afghanistan. Bulletin of the 697 World Health Organization 2007;85:146–51.
Ontario Hospital Association. Hospital Report 2007.
BSC used to
present
results for the
Ontario
Hospital
Association
Lessons learned
Lessons learned from international applications of the BSC
• The BSC allows comparability across organizations and the
relevant performance comparison is healthcare sector
benchmarks (which may be averages of the values for the
included organizations)
• Indicators have to be relevant for a large number of providers and
thus tend to be general, reflecting common services and
programs
• Reporting data in a standardized way from multiple organizations
presents data quality challenges
• The results from a BSC may serve as input for policy formulation
or change
Focus on Kuwait
• MOH reports indicators back to providers in a “dashboard”
specific to each Clinical Area.
Example: Appendectomy Indicators, Hospital X
Current Results
Range in 6 MOH Hospitals (Min- Max)
National Result
Compared to National Performance
Less than 12-Hours Interval between Admission & Appendectomy (H)
85% 4 - 90% 84%
Normal Pathology (L)
20% 4- 30% 9.8%
Perforated Appendicitis (L)
6% 4- 30% 13.9%
Missed Appendicitis (L)
3% 0- 14% 4.1%
Antibiotic Prophylaxis (H)
40% 14- 50% 47.1%
Post Appendectomy Complications (L)
3% 0- 11% 4.5%
Focus on Kuwait: Challenges
• Better reporting on a set of balanced indicators is needed to
provide “overall picture” on performance of healthcare
organization/ health system Need for a Balanced Scorecard
• Limited comparisons to international/ regional benchmarks or
targets
• Limited information on context to better understand
performance of healthcare providers
CONCEPT AND USE OF
BENCHMARKS IN
MONITORING
PERFORMANCE
Benchmarks: Definition and Sources
What is a Benchmark?
• An externally-agreed comparator to compare performance between
similar organisations or systems.
• International comparisons can help identify health system
performance issues and determine the extent to which other
countries also experience these.
• Benchmarking is an important tool to motivate improvement and
help healthcare providers understand where their performance falls in
comparison to others.
• Benchmarking can stimulate healthy competition.
Setting references for benchmarking
How high to set the bar?
• The identification of international, regional, or national reference
points is a critical exercise.
• It is helpful to consider which countries (or regions) might offer the
most relevant experience rather then focusing exclusively on the
countries with the greatest similarity to one’s own.
• Combining elements from the different reference groups is the
best way to make a sound comparison and, more importantly, to
identify opportunities for improving the health system.
Benchmarks: Definition and Sources
How to use benchmarks?
• Search for updated International Benchmarks in the references of the
Procedures Manuals and in accessible data that is clearly defined, regularly
collected and publicly reported on International databases (e.g., CIHI,
AHRQ, CDC, ACHS)
• Make sure measurement protocol (i.e., Formula/ inclusion & exclusion criteria,
reporting periods, risk adjustments, etc.) is consistent with protocol in Kuwait.
• Take note of these differences in measurement protocol in the report.
Differences may be expected since measurement protocol may have been
refined to the context of Kuwait.
• Compare averages to benchmarks. Benchmarking should act as a signal for
improvement.
• Compare results with targets, once targets are assigned.
SAMPLE BALANCED SCORECARD
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 9
! Indicators Results National
Average
Benchmark Benchmark
source
Q1 Q2 Q3
4 Health
Workforce Turnover rate- PHC 0.4620
0.3721
0.6822
0.50 15.60
US Kansas Hospital
Association (2013)
"
5 Satisfaction /
Experience Patients and Staff
Patient/ customer experience 98.2023 93.37
24 87.6025 93.06 i. 80.81 Canada (2005) "
Staff satisfaction—PHC Annual- will be reported
after Q4
NHS England
(2013)
!
!
! !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!20
Qatar Armed Forces and Qatar Red Crescent did not submit data. Qatar Petroleum Medical Services reported 8 terminations and no denominator, as such it was not included in Q1. 21
Qatar Petroleum Medical Services reported 6 terminations and Qatar Red Crescent Industrial reported 3 terminations and no denominator, as such it was not included in Q2. 22
Qatar Armed Forces and Qatar Red Crescent did not submit data. Qatar Petroleum Medical Services reported 1 termination and no denominator, as such it was not included in Q1. 23
ASPETAR and Al Emadi Hospital submitted data. There are concerns about data submitted from Al Emadi Hospital, Interpret results with caution. 24
ASPETAR, Doha Clinic Hospital, and Al Emadi Hospital submitted data. There are concerns about data submitted from Al Emadi Hospital, Interpret results with caution. 25
ASPETAR, Doha Clinic Hospital, and Al Emadi Hospital submitted data.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 10
Indicators Results National
Average
Benchmark Benchmark
source
Q1 Q2 Q3
6 Performance and
Efficiency Percent Medication
reconciliation at admission 96.60
26 66.41
27 88.80
28 83.93 60
CIHI Canada
National compliance rates/
Accreditation Canada (2011)
"!
Blood culture contamination rate (%)29
0.6630
0.54 0.58 0.59 2.89 US CAP Q-Tracks study (1999-2003) "!
! !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!26
ASPETAR and Al Emadi Hospital submitted data. Doha Clinic Hospital may have misinterpreted one of the criteria for this quarter, which could have affected overall results. 27
ASPETAR, Doha Clinic Hospital and Al Emadi Hospital submitted data. 28
ASPETAR, Doha Clinic Hospital and Al Emadi Hospital submitted data. 29
Not Applicable to American Hospital. 30
Doha Clinic Hospital did not submit data. Not Applicable to American Hospital.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 8
Indicators Results National Average
Benchmark Benchmark source
Q1 Q2 Q3
2 Access and Responsiveness
Waiting time in clinic/ outpatient
department11
84.63 86.21 77.60 82.81 Internal Internal !
Percent Referrals to hospitals 1.8312
1.4913
1.2514
1.52 Internal Internal !
3 Safety
Patients and Staff
Percent compliance with hand
hygiene
Annual- will be reported
after Q4
Australia National Data (2014)
Blood and body fluid staff
exposure rate (%) 0.10
15 0.21 0.25
16 0.19% Internal Internal !
Medication errors
7.7517
; 0
caused harm
1.518
; 0
caused harm
3019
; 0
caused harm
13.08 Internal Internal !
!! !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!11
% Patients seen within 30 minutes of their appointment time in clinic/ outpatient department in Hospitals. Interpret with caution as major corrections to data required. Not applicable to Doha Clinic Hospital. Only ASPETAR submitted data for Q1, Q2, and Q3 12
Ministry of Interior and QRC Al- Mamoura and Al Wafedeen Freej Abdel Aziz submitted data. Qatar Armed Forces reported 36 referrals and no denominator, as such it was not included in Q1. 13
Ministry of Interior, QRC Zikrit and Al Wafedeen Freej Abdel Aziz, Qatar Armed Forces, and Primary Healthcare Corporation submitted data. Qatar Petroleum Medical Services, reported 2,728 referrals and no denominator, as
such it was not included in Q2. 14
Ministry of Interior, QRC Al Mamoura, Indutrial and Al Wafedeen Freej Abdel Aziz, Qatar Armed Forces, and Primary Healthcare Corporation submitted data. Qatar Petroleum Medical Services, reported 3,636 referrals and no
denominator, as such it was not included in Q3. 15
ASPETAR, Al Emadi Hospital, and HMC submitted data. American Hospital reported 2 events and no denominator, as such it was not included in Q1. 16
American Hospital did not submit data. 17
HMC and Doha Clinic Hospital did not submit data. 18
HMC and Doha Clinic Hospital did not submit data. 19
HMC and American Hospital did not submit data.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 6
Results for Quarters 1, 2 & 3 (April 2014 and December 2014)
Legend
! Major deviation from benchmark " Meets or better than
benchmark
# Deviates slightly
from benchmark ! Benchmarking
requires internal target
! Indicators Results National
Average
Benchmark Benchmark
source
Q1 Q2 Q3
1 Processes,
Appropriateness, and Outcomes
In-hospital deaths rate (per 1000)
1.66 1.28 1.501 1.48 38
AHRQ US Nationwide (2010) "
Percent unplanned
readmissions within 28 days of discharge related to the
primary admission
1.562 0.77
3 1.40
4 1.25 1
ACHS National
Australian data (2013)
"
Percent unscheduled returns to the Emergency Department within 48 hours related to
primary visit
2.385 0
6 8.03
7 3.47 0.1
Government of
Western Australia: Lowest reported
rate (2014)
!
Postoperative pulmonary embolism or deep vein
thrombosis rate (per 1000)
0 0.02 0.05 0.02 8.27 AHRQ US
Nationwide (2012) "
Percent Women undergoing
general anesthetic for
Cesarean Section
41.018 46.11
9 53.49
10 46.87 6.40
ACHS Australia
nationwide (2013) !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1 American Hospital did not submit data for Q3
2 Al Ahli Hospital did not submit data for Q1
3 Al Ahli Hospital did not submit data for Q2
4 Al Ahli Hospital and American Hospital did not submit data for Q3
5 Not applicable to ASPETAR and Al Emadi Hospital. Doha Clinic Hospital and Al Ahli Hospital did not submit data for Q1.
6 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q2.
7 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q3.
8 Not applicable to ASPETAR and American Hospital. Doha Clinic Hospital did not submit data for Q1.
9 Not applicable to ASPETAR and American Hospital.
10 Not applicable to ASPETAR and American Hospital. HMC did not submit data for Q3.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 6
Results for Quarters 1, 2 & 3 (April 2014 and December 2014)
Legend
! Major deviation from benchmark " Meets or better than
benchmark
# Deviates slightly
from benchmark ! Benchmarking
requires internal target
! Indicators Results National
Average Benchmark Benchmark
source
Q1 Q2 Q3
1 Processes, Appropriateness,
and Outcomes
In-hospital deaths rate (per
1000) 1.66 1.28 1.50
1 1.48 38
AHRQ US Nationwide (2010) "
Percent unplanned
readmissions within 28 days of discharge related to the
primary admission
1.562 0.77
3 1.40
4 1.25 1
ACHS National
Australian data (2013)
"
Percent unscheduled returns to the Emergency Department
within 48 hours related to
primary visit
2.385 0
6 8.03
7 3.47 0.1
Government of
Western Australia: Lowest reported
rate (2014)
!
Postoperative pulmonary embolism or deep vein
thrombosis rate (per 1000)
0 0.02 0.05 0.02 8.27 AHRQ US
Nationwide (2012) "
Percent Women undergoing
general anesthetic for
Cesarean Section
41.018 46.11
9 53.49
10 46.87 6.40
ACHS Australia
nationwide (2013) !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1 American Hospital did not submit data for Q3
2 Al Ahli Hospital did not submit data for Q1
3 Al Ahli Hospital did not submit data for Q2
4 Al Ahli Hospital and American Hospital did not submit data for Q3
5 Not applicable to ASPETAR and Al Emadi Hospital. Doha Clinic Hospital and Al Ahli Hospital did not submit data for Q1.
6 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q2.
7 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q3.
8 Not applicable to ASPETAR and American Hospital. Doha Clinic Hospital did not submit data for Q1.
9 Not applicable to ASPETAR and American Hospital.
10 Not applicable to ASPETAR and American Hospital. HMC did not submit data for Q3.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 6
Results for Quarters 1, 2 & 3 (April 2014 and December 2014)
Legend
! Major deviation from benchmark " Meets or better than
benchmark
# Deviates slightly
from benchmark ! Benchmarking
requires internal target
! Indicators Results National
Average Benchmark Benchmark
source
Q1 Q2 Q3
1 Processes, Appropriateness,
and Outcomes
In-hospital deaths rate (per 1000)
1.66 1.28 1.501 1.48 38
AHRQ US Nationwide (2010) "
Percent unplanned
readmissions within 28 days of discharge related to the
primary admission
1.562 0.77
3 1.40
4 1.25 1
ACHS National
Australian data (2013)
"
Percent unscheduled returns to the Emergency Department within 48 hours related to
primary visit
2.385 0
6 8.03
7 3.47 0.1
Government of
Western Australia: Lowest reported
rate (2014)
!
Postoperative pulmonary embolism or deep vein
thrombosis rate (per 1000)
0 0.02 0.05 0.02 8.27 AHRQ US
Nationwide (2012) "
Percent Women undergoing
general anesthetic for
Cesarean Section
41.018 46.11
9 53.49
10 46.87 6.40
ACHS Australia
nationwide (2013) !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1 American Hospital did not submit data for Q3
2 Al Ahli Hospital did not submit data for Q1
3 Al Ahli Hospital did not submit data for Q2
4 Al Ahli Hospital and American Hospital did not submit data for Q3
5 Not applicable to ASPETAR and Al Emadi Hospital. Doha Clinic Hospital and Al Ahli Hospital did not submit data for Q1.
6 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q2.
7 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q3.
8 Not applicable to ASPETAR and American Hospital. Doha Clinic Hospital did not submit data for Q1.
9 Not applicable to ASPETAR and American Hospital.
10 Not applicable to ASPETAR and American Hospital. HMC did not submit data for Q3.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 6
Results for Quarters 1, 2 & 3 (April 2014 and December 2014)
Legend
! Major deviation from benchmark " Meets or better than
benchmark
# Deviates slightly
from benchmark ! Benchmarking
requires internal target
! Indicators Results National
Average
Benchmark Benchmark
source
Q1 Q2 Q3
1 Processes,
Appropriateness, and Outcomes
In-hospital deaths rate (per 1000)
1.66 1.28 1.501 1.48 38
AHRQ US Nationwide (2010) "
Percent unplanned
readmissions within 28 days of discharge related to the
primary admission
1.562 0.77
3 1.40
4 1.25 1
ACHS National
Australian data (2013)
"
Percent unscheduled returns to the Emergency Department within 48 hours related to
primary visit
2.385 0
6 8.03
7 3.47 0.1
Government of
Western Australia: Lowest reported
rate (2014)
!
Postoperative pulmonary embolism or deep vein
thrombosis rate (per 1000)
0 0.02 0.05 0.02 8.27 AHRQ US
Nationwide (2012) "
Percent Women undergoing
general anesthetic for
Cesarean Section
41.018 46.11
9 53.49
10 46.87 6.40
ACHS Australia
nationwide (2013) !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1 American Hospital did not submit data for Q3
2 Al Ahli Hospital did not submit data for Q1
3 Al Ahli Hospital did not submit data for Q2
4 Al Ahli Hospital and American Hospital did not submit data for Q3
5 Not applicable to ASPETAR and Al Emadi Hospital. Doha Clinic Hospital and Al Ahli Hospital did not submit data for Q1.
6 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q2.
7 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q3.
8 Not applicable to ASPETAR and American Hospital. Doha Clinic Hospital did not submit data for Q1.
9 Not applicable to ASPETAR and American Hospital.
10 Not applicable to ASPETAR and American Hospital. HMC did not submit data for Q3.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 6
Results for Quarters 1, 2 & 3 (April 2014 and December 2014)
Legend
! Major deviation from benchmark " Meets or better than
benchmark
# Deviates slightly
from benchmark ! Benchmarking
requires internal target
! Indicators Results National
Average
Benchmark Benchmark
source
Q1 Q2 Q3
1 Processes,
Appropriateness, and Outcomes
In-hospital deaths rate (per 1000)
1.66 1.28 1.501 1.48 38
AHRQ US Nationwide (2010) "
Percent unplanned
readmissions within 28 days of discharge related to the
primary admission
1.562 0.77
3 1.40
4 1.25 1
ACHS National
Australian data (2013)
"
Percent unscheduled returns to the Emergency Department within 48 hours related to
primary visit
2.385 0
6 8.03
7 3.47 0.1
Government of
Western Australia: Lowest reported
rate (2014)
!
Postoperative pulmonary embolism or deep vein
thrombosis rate (per 1000)
0 0.02 0.05 0.02 8.27 AHRQ US
Nationwide (2012) "
Percent Women undergoing
general anesthetic for
Cesarean Section
41.018 46.11
9 53.49
10 46.87 6.40
ACHS Australia
nationwide (2013) !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1 American Hospital did not submit data for Q3
2 Al Ahli Hospital did not submit data for Q1
3 Al Ahli Hospital did not submit data for Q2
4 Al Ahli Hospital and American Hospital did not submit data for Q3
5 Not applicable to ASPETAR and Al Emadi Hospital. Doha Clinic Hospital and Al Ahli Hospital did not submit data for Q1.
6 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q2.
7 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q3.
8 Not applicable to ASPETAR and American Hospital. Doha Clinic Hospital did not submit data for Q1.
9 Not applicable to ASPETAR and American Hospital.
10 Not applicable to ASPETAR and American Hospital. HMC did not submit data for Q3.
Briefing Note Health Service Performance Agreements (HSPAs) Qatar 6
Results for Quarters 1, 2 & 3 (April 2014 and December 2014)
Legend
! Major deviation from benchmark " Meets or better than
benchmark
# Deviates slightly
from benchmark ! Benchmarking
requires internal target
! Indicators Results National
Average
Benchmark Benchmark
source
Q1 Q2 Q3
1 Processes,
Appropriateness, and Outcomes
In-hospital deaths rate (per 1000)
1.66 1.28 1.501 1.48 38
AHRQ US Nationwide (2010) "
Percent unplanned
readmissions within 28 days of discharge related to the
primary admission
1.562 0.77
3 1.40
4 1.25 1
ACHS National
Australian data (2013)
"
Percent unscheduled returns to the Emergency Department within 48 hours related to
primary visit
2.385 0
6 8.03
7 3.47 0.1
Government of
Western Australia: Lowest reported
rate (2014)
!
Postoperative pulmonary embolism or deep vein
thrombosis rate (per 1000)
0 0.02 0.05 0.02 8.27 AHRQ US
Nationwide (2012) "
Percent Women undergoing
general anesthetic for
Cesarean Section
41.018 46.11
9 53.49
10 46.87 6.40
ACHS Australia
nationwide (2013) !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1 American Hospital did not submit data for Q3
2 Al Ahli Hospital did not submit data for Q1
3 Al Ahli Hospital did not submit data for Q2
4 Al Ahli Hospital and American Hospital did not submit data for Q3
5 Not applicable to ASPETAR and Al Emadi Hospital. Doha Clinic Hospital and Al Ahli Hospital did not submit data for Q1.
6 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q2.
7 Not applicable to ASPETAR and Al Emadi Hospital. Al Ahli Hospital and HMC did not submit data for Q3.
8 Not applicable to ASPETAR and American Hospital. Doha Clinic Hospital did not submit data for Q1.
9 Not applicable to ASPETAR and American Hospital.
10 Not applicable to ASPETAR and American Hospital. HMC did not submit data for Q3.
How to interpret the Balanced Scorecard with
benchmarks?
• Identify and prioritize improvement goals
• Track progress toward those goals
• Monitor maintenance of progress over time
• Report and Disseminate data using KT reporting tools: Briefing notes,
policy briefs, and dialogues
MONITORING &
EVALUATION (M&E) Key to Evidence-based management in
making decisions and improvement plans
Monitoring & Evaluation (M&E)
M&E integral components of measuring performance:
Monitoring
• Systematic follow-up and reporting of achievements in relation to
goals and objectives set out in the strategic plan.
• Includes follow-up of inputs, activities and outputs, outcomes and
impacts.
• Continuous function that uses the systematic collection of data on
specified indicators to provide the management and main
stakeholders with indications of the extent of progress and
achievement of objectives and progress.
• Results-based monitoring compares the performance of a project,
program or policy with expected results.
Monitoring & Evaluation (M&E) Evaluation
• Measurement of results compared to strategic goals and
objectives and analysis of consistency and adequacy of such goals
and objectives.
• Builds upon monitoring data but the analysis goes deeper, taking
into account contextual changes and addressing attribution.
• Systematic and objective assessment of an on-going or completed
project, program or policy, including its design, implementation and
results.
• Aim of evaluation is to determine the relevance and fulfilment of
objects, development efficiency, effectiveness, impact, and
sustainability.
Critical Success Factors • Competence and capacity of providers to collect data on performance indicators
• Competence and institutional capacity for data collection, analysis, performance review,
quality assessment, conducting audits on the process of data collection, managing M&E
work effectively
• Clear institutional mechanisms for conducting M&E processes:
• Leading monitoring and evaluation
• Producing M&E reports within the scope of the strategic plan
• Proposing remedial actions
• Build context into the analysis to empower decision-makers to take action
• Dialogue-based approach with stakeholders to overcome resistance
• Set priorities for action without taking drastic positions that could alienate actors
Framework for M&E
Monitoring & performance evaluation
1. Monitor indicators of inputs, activities, outputs, outcomes & impacts
2. Evaluate data for efficiency, effectiveness, impact & sustainability
3. Analyze and compare performance with expected results
4. Provide data collected to providers, management and main stakeholders with indications of the extent of progress and
achievement of objectives and collect their feedback
5. Provide suggestions for improving implementation
6. Assess monitoring & evaluation procedures yearly to ensure quality & validity of
indicators
Reporting, Dissemination, and Communication Framework
Ensuring continuous monitoring and effective
evaluation of healthcare performance
1. Monitor healthcare performance through continuous follow-up on
indicators.
2. Evaluate measurement of results compared to strategic goals &
objectives:
• Analyze consistency and adequacy of such goals and objectives
through systematic and objective assessment of KPIs.
• Build upon monitoring data but take into account contextual
changes.
Framework for M&E
3. Analyze data and compare performance with expected results and targets
4. Provide performance reports to providers, management and key stakeholders
5. Recommend improvements for measuring KPIs including recommendations for
improving data gaps, standards and quality issues, consistency of measurement of
indicators
Framework for M&E
6. Conduct regular planned assessments of the M&E system on a yearly basis with
providers to ensure:
• Indicators are measuring what they are meant to and that they are relevant to
the strategic goals and objectives. Indicators are responsive to the information
needs for monitoring progress and performance.
• Indicators cover each domain along the results chain: inputs/ structure &
processes, outputs, outcomes, and impact.
• Data generated according to standards, and critical data gaps are identified
and addressed.
EXERCISES & GROUP
WORK
Top Related