Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies...

44
Monitoring, Analyzing & Evaluating HRH Policies in the Region of the Americas Lima, Peru November 1416, 2012

Transcript of Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies...

Page 1: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Monitoring, Analyzing & Evaluating HRH Policies in the Region of the Americas

Lima, PeruNovember 14‐16, 2012

Page 2: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Overview

• Background & Context• Caribbean Case Study• A Regional Collaborative Approach • The Process of HRH Analysis & Evaluation• Challenges & Lessons Learned• First Steps: A Regional Example• Moving Forward

Page 3: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Background & Context

• Second Regional Goals Measurement• Develop Regional HRH policy monitoring and evaluation models

• While progress has been made a number of HRH goals may not be reached by 2015

• Aging populations, aging workforces• Reduction in country resources and capacities• Out migration, rural distribution, slow growth

Page 4: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Regional HRH Forecast

Page 5: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Joint Learning 

Laying the Foundation

Joint Learning Initiative (2004)                Toronto Call to Action (2005) 

WHO World Health Report (2006)      Health Agenda for Americas (2007)

Global Health Alliance (2007)  HRH Data Management Project (2007)

Twenty Regional Goals for Health  (2007)

Kampala Declaration (2008)   CARICOM CCH III (2008)

High Level Meeting, Caribbean HRH Road Map (2012)

Road Map Implementation, 2012‐17ROAD MAP IMPLEMENTATION, 2012‐17

RTRREGIONAL HRH POLICY 

MONITORING & EVALUATION

Page 6: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Caribbean Case Study

• 20 HRH Goals Review• Roadmap for Strengthening the Caribbean Health Workforce

• Interview Survey and Inventory on HRH Governance

Page 7: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

0

20

40

60

80

100

120

Challenge 1 Challenge 2 Challenge 3 Challenge 4 Challenge 51   2    3    4    5            6    7    8    9                 10  11  12                      13  14  15 16                17 18  19  20

Caribbean 20 HRH Goals Summary

Percentage of Goals achieved by Region 

High4 ‐ RNs to MDs9 ‐ Recruit PHC

1 – Supply3 – PHC teams12 ‐ Foreign  cred.16 ‐ Labour negot.20 ‐ Accreditation

Low6 – Urban/rural gap11 ‐ Self‐sufficiency13 ‐ Secure posts

Page 8: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

CARIBBEAN HRH REGIONAL GOALS

SUMMARY

Goal 2 ‐ Proportion of PHC physicians will exceed 40%  of the total medical 

workforce

Goal 3  ‐ Develop Primary Health Care teams with broad  competencies

60%

88%

53%

Goal 4  ‐ Ratio of Nurses: Physicians is at least 1:1

Goal 5 ‐ Establish a HRH unit

100%

95%

Goal 1 – HRH density ratio of  25 

per 10,000CHALLENGE ONE

Page 9: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

The primary thrust of the Road Map is to strengthen HRH planning, management and provider skills in the region in order to create a more stable, efficient and effective health workforce that can meet the needs of an aging population with a growing incidence of chronic diseases by broadening regional access to PHC and public health services.

Caribbean HRH Roadmap

Page 10: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

National Regional

HRH ROAD MAP

Caribbean HRH Road Map Context

Health                 Information                ServiceFinancing                 Systems                   Delivery

Page 11: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Caribbean Health Workforce:Renewal Process

Retain Re-recruit

7 8Renew Re-invest

6 1

5 2Re-vitalize Re-educate

4 3

Re-deploy Re-employ

Strengthening the Foundation

Page 12: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

CaribbeanHealthWorkForce

GOVERNANCE

EDUCATION&  TRAINING

First Steps:  Enhancing Workforce Capacity

Page 13: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Capacity & Skills Development

EDU

GO

ERNANCE

HRH Planners‐ Legislation & regulation‐ Policies & programs‐ Data systems & monitoring‐ Communications & partnerships‐ Research & evaluation‐ Evidence‐based planning‐ Funding ‐H R H

Personnel Managers‐ Health, safety & security‐ Labour relations‐ Assessment & deployment‐ Professional development‐ Performance management‐ Succession planning‐ Incentives & compensation‐ Recruitment & retention

HealthCare  Providers‐ Primary health care‐ Public health‐ Chronic care‐ Health promotion‐ Disease prevention‐ Community‐based‐ Integrated teams‐ Task‐shifting‐ Competency‐baseddeployment

EDUCATION

TRAINING

INFORMATIONSUPPORT

GOVERNANCE

MANAGEMENT

SERVICE         PROVISION

Page 14: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

ROAD MAP FOR STRENGTHENING THE CARIBBEAN HEALTH WORKFORCE, 2012‐2017 

            Context                                Components/Milestones                                                       Outcomes                                Impacts 

                                                                                                                                                                                                                           L           

                                                 

                                                                                                                                                                 

 

 

 

 

 

   

                              2012                                                                      2013 ‐  2017                                                                                                              2020+                        

POPULATION HEALTH  ISSUES  Epidemiology Aging Population Child Health Mental Health Education Poverty/Education Violence/Crime 

 

  

 

 

 

 

 GOVERNANCE  

Develop & Implement     HRH Plans      HRH Policies      Workplace           Improvements      HRH Inventories  

EDUCATION & TRAINING 

Managerial Skills     HIS Training     Policy Development    Staff Management Provider Competence 

STABILITY of Health Workforce  EFFICIENCY of Service Delivery  EFFECTIVENESS of Health Outcomes  

 

 ENABLERS 

Legislation Programs  Policies Management Funding Planning  Partnerships Communications Evaluation/Monitoring  

HRH Database 

 HRH 

Observatory 

Health Learning  Network 

HEALTH SYSTEM  ISSUES  

 Inefficiency Costs & Capacity Poor Access  Service Quality Leadership Management  

WORKFORCEOptimal Competent Aligned with Needs Stable Distributed Flexible Accountable   HEALTH  SYSTEM Efficiency Quality Equity Access Administration Sustainability   

POPULATION Improved Access Appropriate Access Better Quality Better Informed Better Health 

HRH PLANSCommunity Based Integrated PHC Teams – Enhanced  Skills ‐ Task Shifting – Competency Based 

Deployment ‐ Needs Based Planning – Health Promotion – Disease Prevention – Chronic Care 

HRH  ISSUES  HRH Shortages Rural Supply Growing Needs Attrition Emigration Health & Safety Training Productivity  

Road Map – The Journey

Page 15: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Road Map Final Destination

To achieve and maintain a stable and optimally trained, deployed and distributed health workforce with the capacity to provide  responsive,  timely, efficient and effective health care  services  to  meet  the  identified  health  needs  of  the population in a way that is flexible and innovative as well as affordable and sustainable.

Page 16: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Planning & Management Survey & Inventory

OBJECTIVESIn support of the Caribbean HRH Roadmap implementation process:

Short Term•To determine the role and scope of ‐ and capacity for ‐ HRH planning and management of Ministries of Health within the Caribbean region;•To identify legislative and regulatory mechanisms that govern HRH within the countries of the CARICOM;•To identify current gaps, challenges and areas requiring priority attention;Medium Term•To help identify HRH areas that would most benefit from additional skills and training;•To help identify areas were the country governance mechanisms might be developed or strengthened  through regional collaboration; and,•To support the exploration of regional planning mechanisms to best support national HRH planning activities.

Page 17: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Survey Results: “Single Biggest Challenges”

• HRH Database• HRH Unit• Redefine roles & skill mix• Capacity & resources• Inability to implement plans• Qualified staff• Program Evaluation

• Recruitment & retention schemes

• Identify training needs• Link HRH to Health plan• Salaries & Working Conditions• Inter‐Ministry HRH Management

• Rural distribution• Staff burnout• Chronic vacancies

Page 18: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

A Regional Collaborative Approach

More effective HRH policies will achieve:•More stable, optimal workforce•Greater efficiency in health care delivery•Improve geographic, professional, intra‐professional and public‐private distribution•Enhance quality of care•Health supportive motivating work environment that maximize productivity

Page 19: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Monitoring & Evaluation

Page 20: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Governance 

"GOVERNANCE is about the rulesthat distribute roles and responsibilities among government, providers and beneficiaries and that shape the interactions among them.  

It encompasses authority, power, and decision‐making in the institutional arenas of civil society, politics, policy, and public administration." 

DEFINITION

Page 21: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

LEGISLATION

POLICIES 

RULES  &  REGULATIONS

MANDATED  ROLES  &  RESPONSIBILITIES

Role of Governance

HRH PLANNING & MANGEMENT

AUTHORITY                             ENGAGEMENT

DECISION‐ POWERMAKING                        

OVERSIGHT LEADERSHIP  

Practices & Principles

Standards & Guidelines

Page 22: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH       Policy

HRH Policy Support Framework

Legislation FinanceRegulations IncentivesPrograms Education

Strategic Plans LeadershipOperational Plans ManagementPartnerships VisionCommunications MandateKTE ResearchData Systems Evaluation

Page 23: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

?     ?         ?        

Program Commitments Shared/Mandated Responsibilities Competing Priorities

Fiscal Space Sustainability

Allocation of Resources Timing

Stakeholder Interests Long‐Term

Labour Negotiations  Party Platform

Strategic Plan  Issue Profile

Government  Priorities Other Areas

Other Departments Deficit(s)

Data / Research / Information /  KTE

HRH Decision‐Making Arena 

Page 24: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Policy Development & Implementation

• Lack of explicit HRH policies hinders progress• Need comprehensive policies & strategies• Need to be integrated, and pro‐active• Involve stakeholders in all components• Long‐term political support critical• Social & cultural environments representation• Assessment of financial implications of policies• Capacity of country to mobilize needed resources• 45 of 57 HRH crisis countries HRH poorly implemented

Page 25: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

.

HRH Action Framework (GHWA)

Page 26: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Program Evaluation Framework

Page 27: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

EVALUATION: check and verify

What do you want to know? How will you know it?

Logic Model in Evaluation Logic Model in Evaluation 

Page 28: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Logic model and common types of evaluation 

Needs/asset assessment: What are the characteristics, needs, priorities of target population?What are potential barriers/facilitators?What is most appropriate to do?

Process evaluation:How is program implemented? Are activities delivered as intended? Fidelity of implementation?Are participants being reached as intended? What are participant reactions?

Outcome evaluation: To what extent are desired changes occurring? Goals met?Who is benefiting/not benefiting? How? What seems to work? Not work?What are unintended outcomes?

Impact evaluation: To what extent can changes be attributed to the program? What are the net effects?What are final consequences? Is program worth resources it costs?

Page 29: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Outcome Mapping

• Defines outcomes as changes in partner’s behavior• Focuses on facilitating change, not causing it• Recognizes complexities of developmental contexts• Looks for logical links with outcomes, not attributed• Requires program staff and partners in all stagesLogic Models – less utility with unequal distribution of power, complexities of the design and terminologies and self‐contained nature that assume causal links between program components are not as viable.

Page 30: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Applied HRH Planning & Evaluation

• Human Resources for Health Action Framework• Handbook on Monitoring and Evaluation of HRH• Rapid Assessment of the Effectiveness of the Country Coordination & Facilitation Process

• Evidence‐Informed Policy Network

WHO/GHWA/CCF

Page 31: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Rapid Assessment Lessons Learned

• Review progress of stakeholder participation• Create Observatories, improve HRH information• Translate strategies into costed operational plans• Enhance professional and financialmanagement• Incentives & working conditions crucial to recruitment• Rural shortages require task shifting /innovative roles• Create a national migration inventory • Improve planning capacity of HRH committees • Scale‐up stakeholder and government commitment• Sustain collaboration through annual progress meetings

Page 32: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

U K

Silcone Paint & PolishTire Pressure, Tread & SizeAir and Gas FiltersGas Octane LevelGearingAccelerationCity DrivingWarm‐Ups/Start & StopExhaust & Catalytic ConverterSpark PlugsEngine Valves & RadiatorSynthetic OilTop & Windows UpAir Conditioning & Power AssistsAltitude, Humidity & Termperature

$  $  $

Page 33: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Regional Goal #6 – Reduce the gap in Urban‐Rural distribution in health personnel.

Linked Policy Areas –•Recruitment and Retention Schemes•Educational preparation•Return‐in‐Service commitments•Needs identification/Rural quotas•Rural salary and benefits incentives•Working conditions/health and safety•Immigration/Temporary Workers•Bursaries•Task Shifting / Teams

Page 34: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Educational Bursaries (1)CONTEXT INVEST ( INPUT) ACTIVITIES (OUTPUT)

Local HRH Priorities        Budget Allocation       Defined Targets20 Goals/Targets              Staff Assigned             Strategy/PrioritiesHealth Plan                       Accountabilities          TrainingHRH Plan                           Equipment                   CommunicationsMoH Values                      Office Space                 Planning Groups Political Priorities             Time                            Budget Plan/TargetsPlanning Committees      Admin Support            Incentives

Selection CriteriaRecruiting PlanTracking/Data Support 

Page 35: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Educational Bursaries (2)OUTPUT (WHO)    OUTCOMES (LEARN)  OUTCOMES (ACTION)

#’sPartners Awareness                    UptakePlanning Groups             Competencies              Targeted/ApprovedUniversity                        Opinions                       Public/Private Mix   Community Colleges      Aspirations                   Responded/AcceptedMinistry of Health          Acceptance                   Inter‐Professional MixMinistry of Education     Interest   Urban/Rural RatiosCivil Service   Budget UtilizedFinance   Student RetentionHealth Professions Geographic DistributionStudents                                                         Professional Mix

Page 36: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Educational Bursaries (3)

IMPACT•Dollars  saved•Health Outcomes improved•Access to service enhanced•Improved rural supply•Improved workforce stability•Reduced dependence on Migrant Workers/Temporary Staff•Better working conditions•Improved teams and staff support

QUALITATIVE AND QUANTIATIVE INDICATORS

Page 37: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

Key questions…Next StepsAre Regional HRH goals being met?  If not, why not?   How do we know?Is the rate at which goals are being achieved reasonable and adequate?Are current policies, practices, processes and partnerships adequate to support reaching the 20 Regional HRH goals?

What is the simplest and most affordable evaluation and monitoring mechanism that:• Can be implemented in a timely and credible fashion?•Produce credible and implementable results?•Be acceptable to and supported by key partners, stakeholders, donors and government decision‐makers?

Page 38: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

To monitor progress…

…but with clearly defined goals

Page 39: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

To encourage Innovation…

…but with Accountability

Page 40: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

To adopt key partnerships…

…but with clear roles and authority

Page 41: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

To seek out the best evidence…

…and then Act on it!

Page 42: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH Policy Evaluation Process

Page 43: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HRH POLICY EVALUATION :  One Option…

Page 44: Monitoring, Analyzing & Evaluating HRH Policies in the ... · • Lack of explicit HRH policies hinders progress • Need comprehensive policies & strategies • Need to be integrated,

HERE’S ANOTHER…