Developing a country RMNCH scorecard - ALMA 2030 · evidence-based, country-led RMNCH...

Developing a

country RMNCH

scorecard:

Facilitator’s guide

April 2014

PROCESS DOCUMENT

RMNCH Management Tool | 1

Objectives of this facilitator’s guide

▪ Introduce the RMNCH scorecard tool

▪ Consolidate a set of templates and presentation

materials to support facilitation of this process

▪ Offer guidance to facilitators on how to launch

the tool in countries

▪ Define initial approach to support and sustain

improvements in RMNCH management


Contents

▪ Introduction

– Objectives of a country RMNCH

scorecard tool

– Functionality of the scorecard tool

▪ Developing and launching the RMNCH

scorecard tool

▪ Continuous strengthening and

facilitation

▪ Appendix 1: Pre-work materials


The RMNCH Scorecard supports country and partner collaboration in the

RMNCH landscape

Global

initiatives

Ministry /

health

districts

Country

profiles

Development

partners

Triangulate & validate

data

Track outcome and

coverage indicators

Focus national priorities

Provide structure &

frameworks

Convene multiple

stakeholders

Facilitate national process

Provide technical support

Identify service bottlenecks &

national priorities

Gather readily available data

Track intervention

implementation


Country-led RMNCH management scorecards help to enhance

transparency, accountability and action around RMNCH

▪ Enable better profiling and monitoring of high-impact

RMNCH interventions to help decision-makers identify and

prioritise gaps

▪ Facilitate and strengthen evidence-based action plans to

improve internal management of programmes and policies

▪ Serve as an advocacy tool for external partners and civil

society

▪ Align with existing national review processes and timing,

▪ Validate and triangulate the implementation of actions

intended to improve the health of women and children

National RMNCH profiles and management scorecards

RMNCH Management Tool |

A Promise Renewed: Collective action on 3 fronts

▪ Analyse the distribution of U5MR

▪ Identify barriers and bottlenecks

▪ Sharpen government-led action plans with high-

impact strategies

▪ Track and sustain progress against 5 year milestones

▪ Align cross-sectoral support for maternal, newborn

and child mortality

▪ Tell citizens about the political commitments made on

behalf of women and children

▪ Encourage civil society and other groups to advocate

and take action

▪ Compile and disseminate country score cards

▪ Celebrate national progress▪Promoting public

accountability for

results

▪Educating and

mobilizing citizens

▪Sharpening and

scaling-up high-impact

country plans

1

2

3

RMNCH Management Tool | 6SOURCE: Countdown to 2015

The Country Countdown process is an example of an

evidence-based, country-led RMNCH strategy/planning dialogue

INTRODUCTION

Objectives of Country countdown

“[Bring] old and new partners

together around a rigorous,

country-led process that can lend

new energy to national efforts to

improve women’s and children’s

health”

1 Engage full spectrum of RMNCH stakeholders

2 Gather & analyse data from a wide variety of

sources

3 Develop evidence-based action plans

4 Influence policy and programmes by holding

events and publicising reports

5 Track implementation progress


Country-led RMNCH dialogues and scorecard tools are complementary

and can be implemented together

▪ Serve as overarching

framework for global and

national RMNCH

initiatives

▪ Track country-specific status

against global objectives

▪ Develop standardised indicator

definitions

Global initiatives

▪ Defined process to

report country’s progress

and needs

▪ Significant civil society

involvement for a

balanced perspective

▪ Lead time (e.g., up to a

year) needed to collect

and validate data

▪ Broader set of high-

quality data

▪ Tool to help Ministry

manage day-to-day

RMNCH activities with

partner support

▪ Fully owned by national

and regional health

policy makers

▪ Immediate feedback with

regional reports

▪ Action tracking with

regional status updates

RMNCH scorecard

management tool

Evidence-based country-

led RMNCH strategy /

planning dialogue (e.g.

Country Countdown)

Country-led RMNCH toolkits

▪ Helps drive progress toward RMNCH

targets

▪ Provides online platform to integrate

different global initiatives

Focus on progress and validation

Focus on managerial needs

▪ Triangulates and validates data

▪ Provides input on indicator selection

▪ Publicises policy and programme

status


ALMA’s malaria scorecard has been adapted to serve the

managerial needs of RMNCH health officials

SOURCE: ALMA; UN Commission on Life Saving Commodities

Objectives ALMA launches Scorecard for

Accountability and Action

targeting national malaria

performance including tracer

MNCH metrics

2011

An online version of tool has been

developed and is being rolled out

in several countries

UN Commission on Life Saving

Commodities will adopt ALMA

scorecard to track RMNCH

commodities

2013

Ethiopia adapts tool to

track 19 RMNCH

indicators and integrates

into existing performance

management process;

Nigeria adapts tool for

“Saving One Million Lives”

initiative

2012

Support COIA cyclical process (i.e., monitor,

review, and act) to drive accountability

3

▪ Enable better monitoring and management

of key indicators regionally

– Full performance transparency

2

▪ Drive action and accountability to improve

awareness and performance

– Country-led with strong support from

H4+ partners

1


The RMNCH management tool drives tangible action at the national and

regional levels to improve maternal and child health

Feedback

and data

Political support

and recognition

Political leadersCivil society

▪ Conducts independent analysis

& validate data

▪ Advocates for recommended

actions to political leaders

▪ Made aware of progress, gaps and

best practices

▪ Use political power and resources

to support health programs where it

matters most

Performance

transparency

Action items

Managerial tool Region & other health programmes

▪ Made aware of progress, gaps

and best practices

▪ Held accountable before leader-

ship and peers for performance

▪ Use guidance and tracking of

action items to change

programming and focus on

improving outcomes


The RMNCH scorecard tool is a customisable, dynamic management tool

for Ministries of Health to strengthen accountability and drive action

The RMNCH scorecard tool is not …The RMNCH scorecard tool is …

▪ Management tool for the Ministry to

track national and subnational

performance, strengthen

accountability and drive action

▪ Prioritized set of high-impact

RMNCH indicators selected by the

Ministry that reflect country priorities

▪ An aggregation of existing data

▪ Customizable to align with National

Health Sector Strategic Plans and

existing management processes,

including national reviews

▪ Comprehensive inclusion of all

available RMNCH indicators

▪ A static scorecard that cannot be

easily updated to reflect changing

strategic priorities

▪ A one-size-fit all approach with pre-

defined set of indicators agnostic to

country context

RMNCH Scorecard Workshop | 11

The scorecard integrates into the Ministry’s existing

accountability mechanism e.g. Liberia

Owner

Review scorecard at

quarterly county

health team review to

develop action items

4

Review scorecard and

circulate with

management

3

Populate scorecard

with data from HMIS,

SLMIS, and EPHS

1

Collect / monitor

recommended action

from counties

2

Activity

Review scorecard

nationally to develop

strategic goals and

action items

5

Jan AprMarFebDecNov

▪ Minister &

program

directors

▪ CHSWT,

partners

▪ County

Health

Services

▪ County

Health

Services

▪ Monitoring &

Evaluation

ILLUSTRATIVE


Contents

▪ Introduction

– Objectives of a country RMNCH

scorecard tool



scorecard tool


facilitation


INTRODUCTION


The tool summarises RMNCH and health system information in

customised national and regional views and tracks actions

Input Output

A web-based online

tool captures all

information and

automatically produces

reports for

accountability and

action

Scorecard

showing

performance at

national and

regional level for

indicators

Action item

tracker listing

action items and

progress

against them

Regional reports

showing

performance on

each indicator

National data sources

Regional data sources

Strategic & operational plans

Outputs from working sessions

Tool builds on existing data

sources, strategic and

operational plans and outputs

from the working sessions to

gather:

▪ Baseline data

▪ Targets

▪ 3-5 national priority indicators

▪ 15-20 actionable regional

indicators


Outputs are targeted to national and regional stakeholders to support

accountability and action

Output Intent AudienceContent

Scorecard ▪ Heads of State and

Government

▪ Minister(s) of health

▪ Parliamentarians

▪ National / regional

health officials

▪ Partners

▪ National visibility,

transparency, and

accountability on key

performance

indicators

▪ Scorecard showing

performance at

national and regional

level for indicators

Regional report ▪ National health officials

▪ Regional health officials

▪ Partners – especially

local or regional

▪ Regional visibility,

transparency, and

accountability on key

performance

indicators and actions

▪ Regional reports

showing performance

on each indicator

Action item tracker ▪ National health officials

▪ Regional health officials

▪ Partners – especially

local or regional

▪ National and regional

visibility,

transparency, and

accountability

▪ Action item tracker

listing action items

and progress

against them


The scorecard shows the performance of selected RMNCH

indicators nationally and regionally

SOURCE: Screenshot of RMNCH scorecard tool

The scorecard captures regional

performance on 15-20 indicators

selected by the country

Indicators are highlighted based on

performance against target

The Scorecard captures progress on 2-6

key national priority indicators that:

▪ Measure outcomes or impact (e.g.,

maternal mortality rate)

▪ Don’t change quarterly (e.g., health

expenditure as % of total)

ILLUSTRATIVE

RMNCH Management Tool | 16SOURCE: Screenshot of Excel tool

The regional report provides an overview of

regional performance relative to goals

Each region’s report

captures:

▪ Value of each

indicator

▪ Red / Yellow /

Green highlighting

based on customis-

able thresholds

▪ The target

performance for

each indicator

▪ Changes in each

indicator’s

performance

ILLUSTRATIVE

RMNCH Management Tool | 17SOURCE: Screenshot of RMNCH scorecard tool

The action item tracker supports accountability

and drives progress

ILLUSTRATIVE

The action item tracker captures:

▪ Related category (often linked back to

scorecard categories, but may include

others

▪ Responsible person/persons

▪ Deadline set for action item

▪ Current progress

▪ Additional comments


The action item tracker enables national and regional

administrators to communicate about action items

The action item tracker enables national and regional administrators to

communicate about progress directly through the online interface, e.g.:

▪ Status updates about implementation progress

▪ Intermediate data updates

▪ Exchange ideas / discuss problems

ILLUSTRATIVE


The scorecard shows the performance of all regions across all selected

high impact priority RMNCH indicators

SOURCE: Screenshot of RMNCH management tool

ILLUSTRATIVE


Contents

▪ Introduction


scorecard tool

– 0. Pre-work and planning

– 1: Kick-off

– 2: Category and indicator selection and

prioritisation

– 3: Data collection and validation

– 4: Accountability and action

– 5. Capacity-building


facilitation



Developing the scorecard tool requires several components

Module LeadFacilitator ObjectivesModules

▪ Launch process for creating core working team in-country

▪ Support the implementation of pre-work requirements

▪ Ensure stakeholder alignment on objectives and approach,

including development of preliminary schedule

▪ Country leadPre-work and

planning

▪ Build understanding and excitement

▪ Review process for developing and launching

▪ Introduce scorecard tool

▪ Country lead

▪ FacilitatorKick-off

▪ Select appropriate categories based on strategic priorities,

continuum of care, levels of delivery, and cross cutting issues

▪ Review / develop initial indicators list and prioritise

▪ Sort indicators into waves based on data availability

▪ FacilitatorIndicator

selection and

prioritisation

▪ Country

team

▪ Complete indicator checklist (e.g., display type, and thresholds)

▪ Collect and input data into the scorecard tool

▪ Validate and refine data (e.g., compare to DHS / MICS)

▪ Country

teamData collection

and validation

▪ Present overview of objectives for accountability mechanism

▪ Understand existing routines for accountability and action

▪ Embed scorecard tool in existing routines

▪ FacilitatorAccountability

and action

▪ Identify candidates for training sessions (administrative,

technical, and managerial)

▪ Conduct training and gather feedback to enhance the tool

▪ Facilitator

▪ Country leadCapacity-

building


Developing the scorecard is a ~3-4 week processLed by country team

Led by facilitator

Facilitator in-country

1 week1-2 weeks

Pre-work

▪ Align on scorecard tool

objectives and approach

▪ Create core working team

▪ Identify stakeholders and

invite to workshops

▪ Prepare potential

indicators list

Scorecard tool adjustment

▪ Adjust scorecard tool to reflect country

preferences, feedback

▪ Update based on changing data sources

1 week

Scorecard tool design

▪ Kick-off with stakeholders

▪ Review and prioritise

indicators

Accountability Framework

▪ Understand existing

accountability and support

routines

▪ Define how to use the tool

for continuous action

▪ Build administrative and

technical skills to manage

the tool

▪ Build skills in tool analysis

and synthesis

Data validation

▪ Identify data sources

▪ Collect and validate data to populate

first scorecard

▪ Set up ongoing data collection

mechanisms

This schedule should

be adapted to specific

country needs

Training & roll out


Top 10 rules of creating a country RMNCH scorecard

▪ Do not begin without explicit commitment from the Minister of Health1

▪ Focus on accountability and action – they are more important than the indicators2

▪ No new data collection, make use of what the country has already3

▪ Do not be a data purist – directional data can prompt action in the right direction4

▪ Rely on the MoH team for data collection and validation, but also engage

development partners to triangulate and validate data and findings

5

▪ Ensure that data owners are engaged and data collection is done early, to

avoid chasing down indicators that don’t exist

6

▪ Be flexible, and change indicators once new priorities or new data emerge7

▪ Rely on a push from the top – let senior management rally workshop participants8

▪ Ensure a country owner for every step of the process9

▪ Capture the country vision and priorities through open dialogue and discussion10


Contents

▪ Introduction


scorecard tool


– 1: Kick-off


prioritisation





facilitation



Pre-work and planning: Overview of modulePRE-WORK AND PLANNING

Launch

process for

creating core

working team

in-country

▪ Conduct introductory call with country

focal point to give initial tool overview

▪ Outline core roles on country team to

ensure success of scorecard development

▪ Identify a dedicated MoH person with

extensive knowledge on data sources from

beginning

Support the

execution of

pre-work

requirements

▪ Discuss expectations and constraints

around pre-work

▪ Tailor pre-work guide and share with

focal point 2 weeks before country visit

▪ Schedule pre-work check-in calls at the

end of each week leading to country visit

▪ Country provides initial list of categories

and indicators

▪ Start identifying data sources during the

pre-work phase

Ensure

stakeholder

alignment on

objectives

and approach

▪ Identify key stakeholders (MoH and

partners) with input from country focal

point

▪ Focal point to schedule key working

sessions (e.g., kick-off, indicator

selection, accountability problem-solving)

▪ Align on who will set up bilateral

discussions with development partners

▪ Country vision for the scorecard and

main scorecard functions and uses are

aligned

▪ Don’t restrict the process to only HMIS or

MoH owned data, but leverage data from

partners

Steps required Key success factorsObjectives


The country team will be responsible for the majority of the pre-work,

supported by the Facilitator

PRE-WORK AND PLANNING

Length of time required for pre-work will vary based on the

level of consolidation vs. fragmentation in data systems

Country team

▪ Explain and share pre-work guide, with templates and starter

materials to assist countries with pre-work

▪ Conduct pre-work check-in calls to ensure completion of pre-work

and to answer questions

▪ Clarifying country team’s objectives and approach for

scorecard tool

▪ Prepare for workshops, building upon slides found in this document

▪ Structure and schedule joint work (e.g. schedule kick-off,

workshops)

▪ Identify relevant stakeholders

▪ Prepare initial categories indicators list and identify data

sources (if time, start to gather data)

Facilitator


The way that the scorecard will be used by different groups needs to be

clearly defined

Who?

▪ Who will use the scorecard?

▪ Who will they share it with?

▪ Will it be shared more broadly?

How often?

▪ In what fora and meetings will the scorecard be reviewed, and

how often?

What?

▪ What outputs will be used to facilitate the different discussions?

▪ What topics will different discussions cover?

How?

▪ What is the existing performance management system that we

could strengthen through the scorecard tool?

▪ How will corrective action be initiated?

▪ How will these actions be tracked and enforced?

Why?

▪ What would you see as the purposes of the scorecard tool?

▪ What are the country’s reproductive, maternal, newborn, and

child health goals and priorities and strategies?

PRE-WORK AND PLANNING Clarifying country team’s objectives and approach for scorecard tool


A pre-work guide will support the country team’s initial efforts

Pre-work materials included in the appendix

Create core working team (select

team to take ownership of the score-

card and define roles and responsibi-

lities, including tool owner, program

manager, data manager(s), etc.

Pre-work components Sample guidance

Plan timeline and agenda for

facilitated scorecard development

(e.g., Identify relevant stakeholders,

schedule schedule kick-off,

workshops)

Prepare initial indicators list and

identify data sources (if time, start

to gather data)

Explain and share pre-work guidePRE-WORK AND PLANNING

Countries benefit most when they have completed essential pre-work

▪ Hold a call with the facilitator team 2 weeks before the visit

▪ Create a preliminary list of indicators 1 week before the visit (by

holding an in-country working session with the technical committee)

▪ Schedule a kick off meeting for Day 1 of facilitator presence

▪ Identify Ministry of Health focal person responsible for leading the

score card development

▪ Identify Ministry of Health point person responsible for identifying

and compiling data sources

▪ Create a list of stakeholders and form the technical committee


Multiple stakeholders will be needed to ensure the success of the

scorecard

PRE-WORK AND PLANNING

Planning/M&E

RMNCH

Other health departments

(e.g., HIV, Malaria, Nutrition,

Immunisation, PHC, PSM)

Finance

Regions/states/districts

Partners

Potential role(s)

▪ End-to-end ownership of development and

implementation

▪ Thought leadership on accountability mechanism

▪ End-to-end ownership of development and

implementation

▪ Input on accountability mechanism as needed

▪ Thought leadership on categories and indicators

▪ Identification and validation of required data

▪ Defining actionable Health Finance indicators


▪ Identification and validation of subnational data sources

▪ Input on accountability mechanism as needed

▪ Thought leadership on categories and indicators


▪ Input on accountability mechanism

Identify relevant stakeholders


Country teams can develop the scorecard

in workshops, or through bi-lateral meetings

For each objective, what format seems best for you?

PRE-WORK AND PLANNING Structure and schedule joint work

Indicator

selection and

prioritisation

▪ Country pre-work is limited or incomplete

▪ Country team expertise is spread across large

group

▪ Pre-work has generated a solid

preliminary indicator list

▪ Expertise is concentrated, so cross-

functional discussions are less critical

Data collection

and validation

▪ Data sources not fully identified in pre-work

▪ Country partners gather and own much of the

data in-country

▪ Data sources successfully identified in

pre-work

▪ HMIS or MoH Programs gather and own

most of the data in-country

Accountability

and action

▪ There is a large stakeholder group involved in

designing the accountability mechanism (e.g.,

Director of Planning, Subnational Medical

Officers, Partners)

▪ Only 2-3 stakeholders involved in

designing the accountability mechanism

(e.g., top-down system from Director of

Planning)

Capacity-

building

▪ Broad group of tool administrators and users,

including partners, subnational M&E, etc.

▪ Narrow pilot roll-out focusing on National

M&E in a “train-the-trainer”model

Objective Workshops work best if… Bi-lateral meetings work well if…

Kick-off

▪ N/A▪ Country team needs to convene with facilitator

for the first time

▪ Program Champion (e.g., Minister of Health)

needs to introduce aspiration


Countries typically choose 6-8 categories to track on the scorecard

Select categories from around the RMNCH

continuum of care…

Post-

partum &

neonatal

care

Infancy &

childhood

Adol-

escence &

pre-preg-

nancy

Pregnancy

Childbirth

MNCH life

cycle

Maternal healthChild health

Newborn

health

… reflecting the top causes of mortality in the

country …

… across the spectrum of care delivery …

… including cross-cutting categories and na-

tional priorities to form a list of 6-8 categories

Questions for discussions:

▪ What categories across the

continuum of care make

sense for your country?

▪ What cross-cutting

categories?

▪ What are your 3-5 main

goals for MNCH? What’s

the order of priority?

– Which bottlenecks have

you experienced?

– What initiatives have you

undertaken to address

issues?

Cross-cutting categories

▪ Health systems

– Infrastructure

– Work force

– Supply chain

– Information

▪ Policies

▪ Financing

Continuum of care categories

▪ Family planning & contraceptive

▪ ANC

▪ PMTCT

▪ Delivery/Child birth

▪ PNC

▪ Breast-feeding

▪ Nutrition

▪ Pneumonia

▪ Diarrheal diseases

▪ Immunization

▪ Malaria

▪ Reproductive health

▪ Maternal health

(including PMTCT)

▪ Newborn health

▪ Child health

▪ Nutrition

▪ Health financing

▪ Health systems

Sample MNCH

category list:






categories?





you experienced?



issues?


▪ Health systems

– Infrastructure

– Work force

– Supply chain

– Information

▪ Policies

▪ Financing



▪ ANC

▪ PMTCT


▪ PNC

▪ Breast-feeding

▪ Nutrition

▪ Pneumonia


▪ Immunization

▪ Malaria


▪ Maternal health

(including PMTCT)

▪ Newborn health

▪ Child health

▪ Nutrition


▪ Health systems

Sample MNCH

category list:

Prepare initial categories and indicators listPRE-WORK AND PLANNING


Countries should create a list of potential categories and indicators based

on health system priorities and data availability

A starter list of indicators is available in

the appendix for countries to consider

Indicator

Update

Frequency Data SourceCategory Data available?

Reproductive Indicator 1 Monthly Source 1 Yes

Reproductive Indicator 2 Quarterly Source 2 Yes

Reproductive Indicator 3 Quarterly Source 3 No

Reproductive Indicator 4 Annually Source 4 Partially

Maternal Indicator 5 Monthly Source 5 Yes


Maternal Indicator 7 Quarterly Source 7 No

Initial list should

have at least 5

indicators for

each of the 6-8

categories

chosen

PRE-WORK AND PLANNING Prepare initial categories and indicators list


Contents

▪ Introduction


scorecard tool


– 1: Kick-off


prioritisation





facilitation



Kick-off: Overview of moduleKICK-OFF

Build under-

standing and

excitement

▪ Senior champion of scorecard tool

(e.g., Minister of Health) chairs the

session, setting aspiration and vision for

impact

▪ Kick-off session is attended by all key

stakeholders involved in scorecard

development (e.g., directorate heads,

programme managers, M&E, HMIS)

Review

process for

developing

and launching

▪ Introduce Programme Champion,

Scorecard Manager, and Tool Owner

identified in pre-work as focal points for

ongoing workstreams

▪ Review timeline for scorecard

development

▪ Stakeholder and facilitator introductions

given appropriate time to ensure name &

role recognition for ongoing interactions

▪ Country tool owner (e.g., Director of

Planning) to present final product in

plenary

Introduce

scorecard

tool

▪ Provide context and objectives

▪ Preview functionality

▪ Introduce accountability mechanism

▪ Present materials in subsequent pages



Kick-off discussion provides context and objectives for scorecard devel-

opment, and previews tool functionality and accountability mechanism

KICK-OFF

▪ Preview general scorecard structure1 (national and subnational)

▪ Preview subnational reporting capabilities

▪ Illustrate action items and communication platform Preview

functionality

▪ Reinforce country team’s understanding of scorecard tool objectives and

approach

▪ Clarify relationship between RMNCH scorecard tool and country-led

RMNCH dialogues / priorities

▪ Outline the role of the scorecard tool in facilitating dialog and driving

accountability and action

▪ Share lessons learned from scorecard development in other countries

Provide context

& objectives

▪ Introduce accountability mechanism to integrate scorecard into regular

monitoring & evaluation routinesIntroduce accountability mechanism

1 Country-specific version to be developed in subsequent sessions


The way that the scorecard will be used by different groups needs to be

clearly defined

KICK-OFF Reinforce country team’s understanding of scorecard tool objectives and approach

Who?

▪ Who will use the scorecard?

▪ Who will they share it with?

▪ Will it be shared more broadly?

How often?

▪ In what forums and meetings will the scorecard be reviewed,

and how often?

What?

▪ What outputs will be used to facilitate the different discussions?

▪ What topics will different discussions cover?

How?

▪ What is the existing performance management system that we

could strengthen through the scorecard tool?

▪ How will corrective action be initiated?

▪ How will these actions be tracked and enforced?

Why?

▪ What would you see as the purposes of the scorecard tool?

▪ What are the country’s maternal, newborn, and child health

goals and priorities and strategies?


The country team should organise to fulfill the key roles to make the tools

effective in practice


Tool owner

ResponsibilitiesPreferred experience

Programme

champion

▪ Responsible for the success

of the scorecard

▪ Ensures full stakeholder

participation

▪ Senior, respected member

in the FMOH

(M&E/planning or RMNCH)

Scorecard

manager

▪ Day-to-day data collection,

meeting set-up, and

stakeholder coordination

▪ Experience with health

performance management

system and with partners

▪ Mastery of Microsoft Excel

and internet skills

▪ Familiarity with

data sources

▪ Ability to train others

▪ Adjusts tool to country needs

▪ Regular data input

▪ Trains others on tool usage

▪ Output finalisation


M&E/planning

RMNCH

Other health

departments

HIV

Malaria

Nutrition

Immunisation

PHC

PSM

Other stakeholders will be invited to the workshops and play roles in the

success of the scorecard Potential role(s)

Tool owner

Thought

leader on

indicators

Data provider Scorecard

manager

Thought

leader on

accountability

The relevant program

stakeholders should

shape indicator choices


Workshops should have 10-15 attendees

Finance

Regions/states

Partners






national RMNCH

initiatives




definitions

Global initiatives



and needs


involvement for a




and validate data


quality data


manage day-to-day


partner support


and regional health

policy makers


regional reports



RMNCH scorecard

management tool




Country Countdown)



targets








status


The RMNCH scorecard tool drives tangible action at the national and

subnational levels to improve maternal and child health


and validates data

▪ Advocates for action items to political leaders

Development

Partners

▪ Made aware of progress, gaps and best practices

▪ Use political power and resources to support health

programs that matter most

Political

leaders


▪ Held accountable before leadership and peers for

performance

▪ Use guidance and tracking of action items to shape

programming

MoH

Directors &

Managers

KICK-OFF Outline the role of the scorecard tool in facilitating dialog and driving accountability and action


Lessons learned from RMNCH scorecards to date

gather quotations

KICK-OFF

Selected challengesKey success factors

▪ Insufficient attention to leading indicators that provide early

signals of progress (e.g., HIV testing of pregnant women and

ARV treatment of HIV+ pregnant women vs. HIV+ prevalence in

under-5 children)

▪ Balance – track multiple aspects of RMNCH to

ensure a holistic view

▪ Focus – Drive decisions by focusing on a small

subset of indicators

▪ Alignment – Align stakeholders by linking

strategic priorities to indicators

▪ Over-emphasis on being perfectly precise and complete

rather than using directionally correct, “good-enough” indicators

to prompt action now (e.g., survey-based indicators from DHS

and MICS vs. best available HMIS indicators)

▪ Action-oriented – select indicators that suggest

mitigating actions

▪ Choosing indicators that are “too difficult” to measure – and

therefore remain untracked

▪ Measurable – Focus on measurable and

available indicators

▪ Adaptable – Change indicators over times

▪ Lack of accountability, where responsibility for providing data

as well as improving outcomes is not assigned to specific

people, or is not enforced

▪ Integrated – Integrate scorecard into existing

performance management system

Share lessons learned from previous scorecard development efforts


The scorecard shows the performance of all regions across all selected

high impact priority RMNCH indicators

SOURCE: Screenshot of RMNCH management tool

ILLUSTRATIVE


The scorecard shows the performance of selected RMNCH

indicators nationally and regionally

SOURCE: Screenshot of RMNCH scorecard tool

The scorecard captures regional

performance on 15-20 indicators

selected by the country

Indicators are highlighted based on

performance against target

The Scorecard captures progress on 2-6

key national priority indicators that:

▪ Measure outcomes or impact (e.g.,

maternal mortality rate)

▪ Don’t change quarterly (e.g., health

expenditure as % of total)

ILLUSTRATIVE

RMNCH Management Tool | 45SOURCE: Screenshot of Excel tool

The regional report provides an overview of

regional performance relative to goals

Each region’s report

captures:

▪ Value of each

indicator

▪ Red / Yellow /

Green highlighting

based on customis-

able thresholds

▪ The target

performance for

each indicator

▪ Changes in each

indicator’s

performance

ILLUSTRATIVE



and drives progress

ILLUSTRATIVE




others






The action item tracker enables national and regional


The action item tracker enables national and regional administrators to





ILLUSTRATIVE


Objectives for Accountability Mechanism

▪ Build on existing governance routines

to ease adoption of the tool and avoid

creation of parallel processes

▪ Focus management attention on

priority areas, e.g.,

– Subnational regions with

performance below target

– Categories and indicators

progressing slower than expected

– High impact interventions

▪ Drive and monitor actions to address

bottlenecks and challenges

▪ Continuous monitoring and

feedback

– Ensures rapid issue identification

– Accelerates problem-solving

– Expedites corrective action

▪ Quarterly Review

– Formalises performance review

and enhances accountability

– Enables sharing of best-practices

across subnational regions

– Sets ongoing priorities based on

successes and challenges from

last cycle

Accountability Mechanism for the

RMNCH scorecard tool aims to

To achieve this, 2 key elements

are required

KICK-OFF Introduce accountability mechanism


Action items with clearly defined ownership enable continuous monitoring

and feedback

▪ Monitor

Continuous

Feedback

Cycle

▪ Act

1

2

▪ Management has real-time visibility on

status of indicators and action items

▪ Changes in indicators over time help

refine focus and generate action

▪ Communication history clarifies status of

each action item, and enables updates

using comments mechanism

▪ Scorecard reports inform quarterly reviews

and lead to targeted action items

▪ Action items can be assigned in the tool to

address challenges and bottlenecks

▪ Owners of action items can provide status

updates, flag bottlenecks and challenges,

and request clarification

▪ Clear assignment of ownership provides

clarity around expectations and supports

timely execution

Processes enabled by scorecard tool



Each country adopting the RMNCH scorecard tool should decide how to

implement it


Continuous

monitoring

and

feedback

Quarterly

review

Take stock of management

practices today …

… then define how the tools

will strengthen them

▪ Who will use them? How and when

will they use them?

▪ How will the scorecard tools support

monitoring and feedback?

▪ How will the country get the tools

into practice?

▪ How will the tools support current

reviews?

▪ Who will support the use of the tools

in the reviews?

▪ How will the use of the tools in

reviews and for continuous action fit

together to support improvements?

▪ What are your day-to-day

practices to identify issues, solve

them, and track progress?

▪ What is working well? What

needs change and

strengthening?

▪ Who leads and participates in the

review? When are they? What

are the inputs and outputs?

▪ What are the current reviews and

what objectives do they serve?


needs change and

strengthening?


Contents

▪ Introduction


scorecard tool


– 1: Kick-off


prioritisation





facilitation



Category and indicator selection and prioritisation: Overview of module

Select

appropriate

categories

▪ Select categories by considering

priority areas across:

– Continuum of care

– Causes of maternal and child mortality

– Levels of service delivery

– Cross-cutting enables

▪ Only select categories and indicators that

really reflect your priorities and that have

available data today

Review /

develop initial

indicators list

and prioritise

▪ Review initial list from pre-work, or

develop with country team if needed

▪ Prioritise the indicators that correspond

to current initiatives and national strategy

▪ Eliminate indicators that do not meet

SMART criteria

▪ Align on definition (numerator and

denominator) for prioritised indicators

▪ Correctly and accurately label

indicators – be clear on the units,

numerators and denominators

▪ Seek technical support from WHO and

other partners for supranational indicators

Sort into

waves based

on data

availability

▪ Identify data sources for prioritised

indicators

▪ Assign indicators to waves based on

expected data availability and future

priorities

▪ Clearly identify focal points for data

collection

▪ Have clear commitment and action plan

to improve indicators and data sources

over time to make the tools most useful


CATEGORY & INDICATOR SELECTION AND PRIORITISATION


Countries typically choose 6-8 scorecard categories across RMNCH



Post-

partum &

neonatal

care

Infancy &

childhood

Adol-

escence &

pre-preg-

nancy

Pregnancy

Childbirth

MNCH life

cycle


Newborn

health


country …


… including cross-cutting categories and national

priorities to form a list of 6-8 categories






categories?





you experienced?



issues?


▪ Health systems

– Infrastructure

– Work force

– Supply chain

– Information

▪ Policies

▪ Financing



▪ ANC

▪ PMTCT


▪ PNC

▪ Breast-feeding

▪ Nutrition

▪ Pneumonia


▪ Immunization

▪ Malaria


▪ Maternal health

(including PMTCT)

▪ Newborn health

▪ Child health

▪ Nutrition


▪ Health systems

Sample MNCH

category list:






categories?





you experienced?



issues?


▪ Health systems

– Infrastructure

– Work force

– Supply chain

– Information

▪ Policies

▪ Financing



▪ ANC

▪ PMTCT


▪ PNC

▪ Breast-feeding

▪ Nutrition

▪ Pneumonia


▪ Immunization

▪ Malaria


▪ Maternal health

(including PMTCT)

▪ Newborn health

▪ Child health

▪ Nutrition


▪ Health systems

Sample MNCH

category list:

Selecting appropriate categoriesCATEGORY & INDICATOR SELECTION AND PRIORITISATION


Categories should cover the entire RMNCH continuum of care

SOURCE: WHO; The Partnership for Maternal, Newborn & Child Health. 2011. A Global Review of the Key

Interventions Related to Reproductive, Maternal, Newborn and Child Health.

Post-

partum &

neonatal

care

Infancy &

childhood

Adol-

escence &

pre-preg-

nancy

Pregnancy

Childbirth

RMNCH

continuum

of care

Child

health

Newborn

health

Maternal

health

CATEGORY & INDICATOR SELECTION AND PRIORITISATION Selecting appropriate categories


Categories should reflect the major causes of RMNCH mortality in the

country

Causes of under-5 mortality

Sub-Saharan Africa, 2011

SOURCE: WHO / CHERG

Causes of Maternal mortality

Sub-Saharan Africa, 2007-2010

2345

10

11

MeaslesHIV/AIDS

Injuries

Congenital

anomalies

Sepsis5

Birth asphyxia

& trauma

Diarrhoea

Prematurity

13Malaria

14

ARI17

Other

17

Globally more than one third of

child deaths are attributable to

under nutrition

Regional estimates for sub-

Saharan Africa

17

34

19

Sepsis

9Unsafe

abortion9

Other

direct 11

Indirect

Hypertension

Haemorrhage

Embolism

1



Categories and indicators should cover the spectrum of care delivery

levels

Tertiary

level of care

Health Centers

Health Clinics /

Posts

Community

based delivery,

e.g. CHWs

Hospitals

Secondary level

of care

Primary level of

care

Community

based care



Select from categories that are in line with national priorities across the

continuum of care and from cross-cutting categories


▪ Health systems

– Infrastructure

– Work force

– Supply chain

– Information

▪ Policies

▪ Financing



▪ ANC

▪ PMTCT


▪ PNC

▪ Breast-feeding

▪ Nutrition

▪ Pneumonia


▪ Immunisation

▪ Malaria


▪ Maternal health

(including PMTCT)

▪ Newborn health

▪ Child health

▪ Nutrition


▪ Health systems

▪ Monitoring &

Evaluation

▪ Purchasing and

Supply management

Sample RMNCH

category list:






categories?


goals for RMNCH? What’s



you experienced?



issues?



Proposed two-step approach to selecting indicators

Review/develop initial indicators list

and prioritise

Sort into waves based on data

availability and intervention timeline

▪ Review initial list, if developed in pre-

work. Otherwise, build with country team

▪ Prioritise the indicators that are national

priorities or correspond with current

initiatives


SMART criteria sufficiently



▪ First wave: indicators with data

currently available

▪ Second wave:

– indicators with data expected in the

near future

– indicators for interventions to be

rolled out in the near future

▪ Third wave:

– Indicators with data not expected in

the short term

– Indicators to support emerging

priorities

This process will identify 2-3 indicators per

category for each wave

Review initial indicators and prioritiseCATEGORY & INDICATOR SELECTION AND PRIORITISATION

RMNCH Scorecard Workshop | 59

Indicators in each category should be prioritised based on a series of

questions

Key questions to answer in developing and refining initial indicator list

Select indicators Develop waves Populate scorecardDevelop scorecard Select categories

Question Example answer

▪ What are our most important priorities in

each category? Ensuring PNC within 2 days of

birth for maternal health

▪ What actions are needed (or being done) to

achieve those priorities?

Increase access to PNC services

in health care facilities

▪ What data do we have available today to

measure our progress on those actions?

% of mothers and babies

receiving PNC within 7 days

▪ What data will we have in the future to

measure our progress on those actions?

% of mothers and babies

receiving PNC within 2 days


Initial indicator list should then be tested against the SMART criteria

Questions

Timely▪ Is the data source updated quarterly? Could it be?

▪ Is the data available quickly enough to support continuous action and quarterly

review and action planning?

Measurable▪ Is it easy to measure? Can it be tracked against a specific target?

▪ Is data consistently available today?

▪ Is the data source accurate and complete (i.e. for all locations and individuals)?

Achievable

and agreed

▪ Can the person/group responsible for it actually influence it?

▪ Do we understand the drivers that are behind it?

▪ Are the relevant stakeholders aligned on the inclusion of the indicator?

Results-oriented

▪ Is it relevant to the performance of RMNCH?

▪ Does it support the high-level priorities for the country?

▪ Is it aligned with the subnational strategy and objectives?

Simple and

specific

▪ Does it have a clear definition?

▪ Is it straightforward to understand?

▪ Can it be easily generated without complex calculations?

Review initial indicators and prioritiseCATEGORY & INDICATOR SELECTION AND PRIORITISATION


Remaining indicators should be organised into waves

based on data availability and country priorities ILLUSTRATIVE

EXERCISE

Final list of Wave 1 indicators should contain no more

than 20 indicators across all RMNCH categories

CATEGORY & INDICATOR SELECTION AND PRIORITISATION

Wave 2 indicators:

▪ Data expected in the

near future

▪ Interventions not yet

rolled out

Wave 3 indicators:

▪ Data not expected in

the short term

▪ Emerging priorities

for future tracking

Wave 1 indicators:

▪ Data available today

Sort into waves based on data availability


Contents

▪ Introduction


scorecard tool


– 1: Kick-off


prioritisation





facilitation



Data collection and validation: Overview of module

Complete

indicator

checklist

▪ Confirm basic information is captured

for each indicator

▪ Define targets and performance

thresholds (red / yellow / green) Define

up / down threshholds

▪ Select display type for each indicator

▪ Be flexible about indicators given data

availability– i.e. do not be afraid to

switch indicators if accurate, timely,

complete data cannot be found

Collect and

input data

into tool

▪ HIS or M&E team to:

– Pull data from identified sources and

log any data gaps

– Populate initial scorecard for

prioritised indicators which have data

available

▪ Ensure the process is owned by the

Ministry of Health, with light facilitator

support

Validate and

refine data

▪ Pressure-test the data by triangulating

against other sources (e.g., DHS, MICS)

▪ Validate data with MoH leads and

partners

▪ Consult with all potential stakeholders

to find the best data sources available


DATA COLLECTION AND VALIDATION


The country team must detail the relevant information

for each prioritised indicator

Indicator check-list

Basic information (Frequency of collection, data

source, contact person)

Wave: 1, 2, or 3

Performance type: progression toward a target or

absolute value

Display type: national, triangle, or regular

Green / yellow / red thresholds

Up / down arrow thresholds

{Progression targets only} National or region-specific

targets



The team should assign each indicator green, yellow, and red performance

thresholds

Target achieved or on track

Progress but more effort required

Not on track

Yellow

Green

Red

Red-yellow threshold

Yellow-green threshold

Thresholds can

be different for

each indicator

if desired



There are 2 options on how these colors will be assigned and performance

measured for each indicator

Example of indicator performance

Situation: Contraceptive prevalence

rate (CPR) improves from 10% to 15%

over the last time period

CPR, percent

There are two basic options for

measuring performance

2 Measures the progress made

over the last time period

against a pre-defined target

Progress against target:

1 Measures the actual

performance, with color

defined relative to thresholds

Absolute performance:


Current

performance

15%

Baseline

10%


Option 1: Absolute performance

Description

Notes

▪ Measures the actual

performance, and assigns red,

yellow, or green based on

thresholds.

▪ E.g. CPR of 15% receives

“Yellow” based on thresholds

illustrated

▪ Using absolute performance

thresholds, the baseline value is

not needed to determine color

▪ Typically the scorecard is easiest

to interpret if thresholds for a

given indicator are consistent

across regions

15

25%

10%

Green

Yellow

Red


CPR, pct


Option 2: Progress against target

Description

10%

15%

25%

Baseline Current

performance

+67%

+50%

Target

Formula

[15% - 10%] / [25% - 10%]

5% / 15%

Progress against target = 33%

Example

Performance gap

▪ Measures the progress made toward

a pre-defined target

▪ Color based on progression

thresholds set

[Current performance – baseline]

[target – baseline]Progress

CPR improves from 10% to 15%

against a 25% target


▪ Only show this page if specifically asked about progress against target as a means of assessing performance


Arrows can be used to illustrate trends and have customisable thresholds

Description

The arrows indicate the delta (%) from previous period and are used to illustrate trend

in absolute performance

Formula: [1 - (Previous period / current period)]

Description Threshold

No significant change in

absolute performance of

indicator

Change of <= 5% from previous

period▼

Reduction in absolute

performance of indicator

Decrease of >5% from previous

period▼

Improvement in absolute

performance of indicator

Increase of > 5% from previous

period▲

The criteria can be

tailored for each indicator



National and subnational indicators have different display options

▪ National: displays at top of

scorecard in a priority area

▪ Triangle: two closely related

indicators in one cell; highlights

relationship and provides

snapshot of a topic (e.g., PMTCT

testing and treatment can be

paired)

▪ Subnational: regular format for

majority of indicators, one

indicator per cell displayed in the

subnational portion of the

scorecard

National

Subnational



Contents

▪ Introduction


scorecard tool


– 1: Kick-off


prioritisation





facilitation


ACCOUNTABILITY AND ACTION


Accountability and action: Overview of module

Overview of

objectives for

accountability

mechanism

▪ Understand aspirations for Accountability

Mechanism through interview with MoH

leadership (e.g., Director of Planning)

▪ Share presentation materials in this section

with Ministry of Health leadership

▪ Emphasise scorecard tool aims to enhance

existing accountability framework, rather

than create a competing framework

▪ Move quickly to action: operationalise the

scorecard as quickly as possible:

▪ Management should review scorecard

when launched to familiarise themselves with

the tool and how it can drive insight and action

▪ MoH leadership to identify other systems

and processes to integrate the scorecard into

Understand

existing

routines for

accountability

and action

▪ Review published accountability

frameworks (e.g., from strategic plans)

▪ Clarify routines through interviews with

Ministry leadership (e.g., Director of Planning)

▪ For current review infrastructure, understand:

– Frequency of reviews,

– Participants in each

– Preparatory activities and follow-up

▪ Correctly and accurately label indicators –

be clear on the units, numerators and

denominators

▪ Seek technical support from partners for

supranational indicators

▪ Ensure the process is owned by the

Ministry of Health, with light facilitator

support

Embed score-

card tool in

existing

routines

▪ Determine what additional preparation is

required to incorporate scorecard into reviews

▪ Problem-solve whether action items should

be generated top-down or bottom-up

▪ Design process to feed scorecard into

reviews and use action items to drive

accountability day-to-day

▪ Share early and broadly with the broader

RMNCH community, including MoH officials,

subnational officials, and partners

▪ Key leaders understand how the scorecard

tool fits into key meetings (e.g., national

leadership, key partners, other Ministries)




Objectives for Accountability Mechanism

▪ Build on existing governance routines

to ease adoption of the tool and avoid

creation of parallel processes

▪ Focus management attention on

priority areas, e.g.,

– Subnational regions with

performance below target

– Categories and indicators

progressing slower than expected

– High impact interventions

▪ Drive and monitor actions to address

bottlenecks and challenges

▪ Continuous monitoring and

feedback

– Ensures rapid issue identification

– Accelerates problem-solving

– Expedites corrective action

▪ Quarterly Review

– Formalises performance review

and enhances accountability

– Enables sharing of best-practices

across subnational regions

– Sets ongoing priorities based on

successes and challenges from

last cycle

Accountability Mechanism for the

RMNCH scorecard tool aims to

To achieve this, 2 key elements

are required



Each country adopting the RMNCH scorecard tool should decide how to

implement it

Continuous

monitoring

and

feedback

Quarterly

review

Take stock of management

practices today …

… then define how the tools

will strengthen them

▪ Who will use them? How and when

will they use them?

▪ How will the scorecard tools support

monitoring and feedback?

▪ How will the country get the tools

into practice?

▪ How will the tools support current

reviews?

▪ Who will support the use of the tools

in the reviews?

▪ How will the use of the tools in

reviews and for continuous action fit

together to support improvements?

▪ What are your day-to-day

practices to identify issues, solve

them, and track progress?


needs change and

strengthening?

▪ Who leads and participates in the

review? When are they? What

are the inputs and outputs?

▪ What are the current reviews and

what objectives do they serve?


needs change and

strengthening?



Action items with clearly defined ownership enable continuous monitoring

and feedback

▪ Monitor

Continuous

Feedback

Cycle

▪ Act

1

2


▪ Management has real-time visibility on

status of indicators and action items

▪ Changes in indicators over time help

refine focus and generate action

▪ Communication history clarifies status of

each action item, and enables updates

using comments mechanism

▪ Scorecard reports inform quarterly reviews

and lead to targeted action items

▪ Action items can be assigned in the tool to

address challenges and bottlenecks

▪ Owners of action items can provide status

updates, flag bottlenecks and challenges,

and request clarification

▪ Clear assignment of ownership provides

clarity around expectations and supports

timely execution

Processes enabled by scorecard tool


Using the scorecard is a three-step process

InvestigatePrioritize Act

Identify categories and

indicators requiring

attention

Determine which

counties are

performing best /

worst

Evaluate whether

performance is

improving or getting

worse

Gather information

about root causes of

performance

Determine which

factors impact

performance the most

Consider whether

performance is likely

to persist

Develop strong,

actionable action

items to improve

performance

Input action items

into scorecard and

monitor

1 2 3


Three analyses help identify and prioritize counties

and indicators for intervention

LONGITUDINAL (ARROWS): Down-arrow

indicates declining performance from previous

scorecard, and potential need for action item to

address

BY COUNTY: Low performing

indicators in each county highlight

areas of focus for action items

BY INDICATOR: Low performance of a given

indicator across most counties highlights

systemic bottleneck to achieving objectives

Prioritize Investigate Act


Low performance of a given indicator across

most counties identifies an area of national

priority

▪ Counties performing significantly above or below average identify particular areas for

attention

– Above average counties may serve as models to help create action items

– Below average outliers indicate specific counties requiring action items

Comparing indicators vertically across counties

helps identify areas of national priority

Indicators can be sorted by clicking on

indicator name



Low performing indicators

highlight county-specific areas for

action

Comparing indicators horizontally across a single

county helps prioritize local need for action items



Down-arrows indicate declining

performance from previous reporting

period, and potential need for action

items

Up-arrows indicate improving performance, which

may suggest that existing action items are working

and no new action is required

Arrows help determine whether new action items are

needed to address performance



Action items are monitored both by action owners and

by management


Visual indicator of status allows manager to see

quickly which action items are at risk (e.g.,

yellow for in progress and on track)

Actions are assigned at the national level,

based on a bottleneck analysis to ensure

targets are met and indicators move on upward

trajectory

List of action items

displays action type,

description, status, due

date, and owner

Missed deadlines are

highlighted in red,

prompting immediate

management attention


Online commenting functionality enables real-time feedback and

engagement between action owners and management

Each action has an associated comment list,

allowing direct communication between action

owner and management

Action owners monitor their assigned actions on

a regular (e.g., weekly) basis, and respond to

new comments directly within the scorecard tool

Up-to-date record of relevant

communications is always accessible

from the action items page




and drives progress

INTRODUCTION

ILLUSTRATIVE




others






The action item tracker enables national and subnational


The action item tracker enables national and subnational administrators to





INTRODUCTION

ILLUSTRATIVE


Actions should be easy to understand, and adhere to 4 guiding principles

Actions are discrete, specific, and practical

▪ Install 3 new computers with

Channel LMIS software in each

Subnational Store

▪ Improve quality of LMIS data

Owner is capable of implementation

▪ Secure funding for 5 million LLINs

for national campaign through the

Global Fund (owner = Malaria

Program Manager)

▪ Ensure LLIN coverage is

maintained (owner = DHMT)

Actions have clear owners in country

▪ Director of Maternal & Newborn

health to establish regular forum to

engage partners, identify funds and

improve funding allocation

▪ Increase total donor contribution to

Health Sector in Sierra Leone

Responsiveness can be tracked affectively

▪ Engage international NGO to

identify ~5 additional schools per

subnational region providing

teenage family planning info

▪ Improve access to teenage family

planning information

Actions should look like this… …and not like this



Responsibility for action should be assigned both broadly and also

specifically


Broad responsibility

Political

leaders

▪ Performance transparency

▪ Using political power and resources to improve health

where it matters most

Policy/

Program

▪ Identifying key gaps in programming

▪ Changing policy to foster improvements in RMNCH

▪ Re-allocate resources to solve problems requiring

funding

Development

Partners

▪ Identifying and providing financial and/or human (e.g.

technical assistance) resources for programming

needs

Subnational

regions

▪ Supporting implementation

▪ Benchmarking performance

▪ Feedback on implementation effectiveness


All stakeholders can contribute to and benefit from the tool,

but the ultimate responsibility resides within the country

SOURCE: ALMA

Contributions Benefits

Political

▪ Head of State

▪ Subnational

leaders

▪ Public

▪ Investment

▪ Credibility

▪ Focus on key strategic

initiatives

▪ Transparency

▪ Awareness

▪ Accountability

Health

System

▪ Programs &

Interventions

▪ Ministries of

Health

▪ Data

▪ View of issues and

obstacles

▪ Robust performance

dialogs

▪ Reward & recognition

▪ Open communication

Partners▪ Civil Society

▪ Development

partners

▪ Data gathering and

validation

▪ Input determining action

items

▪ Improved data for program

evaluation

▪ Prioritised programs and

initiatives

Stakeholders



Example: Existing reviews can use the scorecard tool to enhance

accountability and performance

Combined

District and

National

review

National

review

District

review

Q4Q3Q2Q1 Participants

▪ Minister / Deputy

▪ Permanent Secretary

▪ CMO/DMI

▪ DPI Director and staff

(e.g., planning, M&E)

▪ Districts (e.g., DMO)

▪ Directors

▪ Program Managers

▪ Partners



▪ CMO / DCMO

▪ Directors


▪ Partners

▪ DHMT

▪ District Council

▪ Local Council

▪ DPI (e.g., Planning,

M&E)

▪ Local partners

Review routine

National and District reviews inform

discussion in combined semi-annual reviews

District Reviews inform discussion

in National Reviews



Example: The scorecard tool can enhance accountability and action, and

ensure focus on priority indicator categories

FrequencyObjectivesKey inputs Stakeholders

National

review

▪ Quarterly▪ Overall tracking and evaluation of Health

Sector progress

▪ Problem solve to identify new action items to

address bottlenecks and challenges

▪ Drive accountability and action at national

level

▪ National

Scorecard

▪ All District

Reports and

Action item

sheets



▪ CMO / DCMO

▪ Directors


▪ Partners

District

review

▪ Quarterly▪ Monitor performance of Health District

▪ Review status of action items

▪ Identify bottlenecks and potentially propose

additional mitigating actions

▪ National

Scorecard

▪ Own District

Report and

action item

sheet

▪ DHMT

▪ District Council

▪ Local Council

▪ DPI (e.g., Planning,

M&E)

▪ Local partners

Combined

District and

National

review

▪ Semi-

annually

▪ Tracking progress at all levels against

national plans and targets

▪ Drive accountability and action by defining

and assigning ownership of action items at

national and district level

▪ Problem solve mitigating actions to address

challenges at national and district level

▪ National

Scorecard

▪ District Reports

▪ Action item

sheets



▪ CMO/DMI

▪ DPI Director and staff

(e.g., planning, M&E)

▪ Districts (e.g., DMO)

▪ Directors


▪ Partners



Inputs from the scorecard tool should be used to answer several key

questions during the quarterly reviews


Subnational

scorecards

Action items

from

previous

quarter

Key questions to address

National

scorecard

Performance

▪ Where is performance improving? Where is it

declining?

▪ Where is performance above / below target?

▪ For the areas of weakness, are there lessons

from select subnational regions which suggest

possible solutions?

▪ For the areas of strength, can we identify

lessons learned and best practices to share

across subnational regions?

Implementation

▪ What critical issues do the insights from the

tool suggest?

▪ Has there been good progress on the plans

and actions agreed at the last review meeting?

▪ What new action items could impact the key

indicators needing attention? Are the required

capacities and resources in place?

Light-touch facilitator support could assist with quarterly check-in calls


All scorecard processes and outputs should have clear timelines

and owners to ensure effective implementation

Scheduled

updates

Owner

Review quarterly

report

▪ Database

managers

1

Quarterly report

released with recom-

mended actions

▪ Policy &

Planning

team

2

Update action items

status and comments

▪ Scorecard

lead and

action owners

3

Prepare and update

quarterly report~3 weeks ▪ Technical

team5

Month 1 Month 2 Month 3

Scorecard lead

conducts 1-on-1 calls

with subnational focal

points

~3 weeks ▪ Scorecard

lead (e.g.,

Director of

Planning)

4



Contents

▪ Introduction


scorecard tool


– 1: Kick-off


prioritisation





facilitation



Capacity building: Overview of module

Identify

candidates

for training

sessions

▪ Identify tool owner to drive uptake and

integration of the scorecard tool into

management routines

▪ Identify scorecard manager to lead

configuration and data capture & entry

▪ Identify 3-5 tool admin technicians

from HIS or M&E within the Ministry or

Health

▪ Ensure there is sufficient user

knowledge within the Ministry of Health

to maintain the tool when key team

members are absent

▪ Decide with the Ministry of Health

whether to include partners, who can

provide ongoing input and assist with

future training

Conduct

training

sessions

▪ For each training session:

– Confirm participants

– Reserve training room with

projector

– Arrange workstations and internet

connectivity if needed

– Create user accounts for

participants

– Configure training scorecard

(separate from live environment)

▪ Be sure to address mindsets as well

as capacities; users should also

– Know why they are using the tool

– Have required resources (e.g.,

laptop)

– See that leadership values the

process

▪ Gather user feedback on the training

materials and the scorecard tool; new

users are often the best source of

improvements


CAPACITY BUILDING


The RMNCH scorecard tool has three types of users, all of whom will need

to be trained

CAPACITY BUILDING

Responsibilities Skills required

National

administrator

▪ Review scorecard and develop

action items

▪ Manage the scorecard, action

items, and data on a continuing

basis

▪ Access the online tool and review

the scorecard and action items

▪ Basic Internet and Microsoft Excel

skills, and ability to access and

analyse HMIS / LMIS data

Subnational

administrator

▪ Basic computer skills to access

and review the scorecard and

action items, and report on

progress

▪ Data collection from facilities and

entering into HMIS

▪ Review action items and report on

progress

Partner

▪ Review scorecard and action items

▪ Triangulate scorecard and actions

with surveys and other data

sources

▪ Basic computer skills to access

and review the scorecard and

action items


There are three core training sessions

▪ Does the Ministry have individuals with the technical background to manage the tool?

▪ Who should run each training? Who will run ongoing training after launch?

▪ How can partners facilitate / assist with user training and management of the tool?

▪ Review of tool access, user

management

▪ Advanced review of indicator

creation

▪ Hands-on configuration of

the scorecard from scratch

▪ Calculating indicators, and

uploading results into tool

Content ▪ Functional overview of

scorecard tool features

▪ Detailed instructions on how

to:

– Manage users

– Configure scorecard

– Input data

– Manage action items

▪ Entry of HMIS data into tool

for indicators selected by

country team

▪ Setting up and accessing a

user account

▪ Interpreting the scorecard,

subnational report, action

items

▪ Tracking action items,

commenting and providing

updates to MoH

▪ Overview of accountability

framework

Data management training

for HMIS specialists

Basic training

for users of tool outputs

Advanced training

for tool administrators

CAPACITY BUILDING

Materials ▪ User guide (pp 1-15)

▪ Facilitators guide

▪ User guide (in-depth) ▪ User guide (in-depth)

▪ MoH training on HMIS

Audience ▪ Users of tool outputs (e.g.,

health sector management,

partners)

▪ Users who will configure and

manage the tool (e.g.,

national / subnational admin)

▪ National M&E / planning

specialists

Instructor ▪ Country team or facilitator ▪ Facilitator ▪ Facilitator


Contents

▪ Introduction


scorecard tool


facilitation



Focus Areas Short-term (e.g., quarterly) Long-term (e.g., annually)

Accountability

mechanism

▪ Has the scorecard tool been launched /

adopted nationally and subnationally?

▪ Are the action items SMART? Are they

being completed on time?

▪ Has the scorecard tool been integrated

into all management routines?

▪ Do the action items address the issues /

bottlenecks identified in the scorecard?

▪ Is data analysis generating insight and

driving appropriate action?

Data & reporting

▪ Is data gathering effective and efficient?

▪ Is data quality satisfactory?

▪ Is there sufficient technical capacity to

gather and input date into the tool?

▪ How can data gathering be improved?

▪ How can data quality be improved?

▪ What additional training is needed?

▪ How can we ensure sustainable

technical support?

Stakeholder

alignment

▪ Are the stakeholders aligned on the

scorecard objectives and approach?

▪ Are the stakeholders aligned on the

scorecard objectives and approach?

▪ How can we increase use of the tool

and enhance stakeholder alignment

Scorecard design

▪ Do all categories and indicators align

with national priorities?

▪ Do they support key management

decisions with clear insight?

▪ Can any Wave 2 or Wave 3 indicators

be introduced based on data availability

or current priorities?

▪ Are other new indicators necessary to

capture national priorities?

Key questions

Continuous strengthening ensures rapid issue identification and ongoing

effectiveness of the scorecard tool

▪ Typically conducted internally, e.g.,

after quarterly reviews

▪ Can be facilitated with check-in calls

▪ Can be supported more extensively,

e.g., with in-person facilitator


Annual strengthening exercise should inform a detailed report to be

shared with scorecard tool owner and senior national leadership

▪ Communicate intention conduct

interviews and surveys to relevant

stakeholders

▪ Conduct interviews and surveys with

national and subnational

stakeholders, e.g.,

– Political leadership

– Minister of Health

– MoH directors / managers

– Subnational health sector

leadership

– Development partners

▪ Assess the scorecard tool on several

key dimensions, e.g.,

– Scorecard design

– Accountability mechanism

– Data & Reporting

– Stakeholder alignment

▪ Conduct analyses based on

interviews and surveys, e.g.,

– Identify issues / bottlenecks

within each key dimension

– Compare perceptions of

scorecard tool across

stakeholder groups

– Synthesize qualitative

stakeholder feedback

▪ Identify indicators requiring

adjustment, e.g.,

– Green for 90+ days requires

increased targets

– Red for 90+ days requires

bottleneck analysis

▪ Synthesize analyses into actionable

insights and recommendations

▪ Produce a report for senior political

and MoH leadership to inform next

steps for the scorecard tool,

including:

– Identify indicators requiring

adjustment, e.g.,

– Evaluation of management

insight offered by the tool

– Assessment of data collection

and reporting capacity and

processes

– Assessment of alignment

between indicators and national


▪ Report should include proposals for

how to address issues identified, to

be developed in collaboration with

MoH and national leadership

Annual strengthening exercise Analysis and Synthesis Report

Annual strengthening exercise and report-creation can be supported by external facilitator


Annual strengthening exercise gathers input from all key stakeholders

Key areas to address

Political leadership▪ Does the scorecard provide meaningful insights into RMNCH performance?

▪ How has issue identification and accountability been impacted by the scorecard tool?

Minister of Health

▪ How does the scorecard tool support your engagement with RMNCH programs?

▪ How do you use the scorecard tool during quarterly and annual reviews?

▪ How has the scorecard tool changed your management routines?

▪ Does it adequately capture national priorities? Does it lead to actionable insight?

MoH routine

management

(Directors and

Programme

Managers)

▪ How does the scorecard tool support engagement with national leadership?

▪ How does the scorecard tool support engagement with subnational leadership?

▪ How do you use the scorecard to develop actionable insight on interventions?


▪ What issues undermine its effectiveness as a management tool?

Subnational Health

Sector leadership


▪ How frequently do you review the scorecard, subnational report and action items?

▪ Do the scorecard indicators adequately capture performance within your region?

▪ Do you have the required resources and capacities to use the scorecard tool?

▪ What issues undermine its effectiveness as a management tool?

Partners

▪ Do you have access to the scorecard?

▪ Do the scorecard indicators capture RMNCH priorities that you focus on?

▪ Is the scorecard data satisfactory? Is it the best available for each indicator?

▪ How frequently do you review the scorecard and action items?

▪ Are you included in quarterly and annual reviews?

Stakeholder


Annual surveys to be tailored to each stakeholder group

Indicate the extent to which you agree or disagree with the following statements

Strongly

disagree

Strongly

agree

Somewhat

agree

Somewhat

disagreeIndicators Comments

The selection of indicators helps me

evaluate key interventions


identify drivers of performance


determine appropriate action items

The scorecard tool is fully integrated into

my management routine

The action items assigned to me are

easy to understand

Management routines

The action items assigned to me are

likely to improve RMNCH performance

I have the necessary resources to

complete my action items

The scorecard tool has enhanced my

management capacity

The scorecard tool is easy to use

Overall assessment

I would encourage my peers to adopt

the scorecard tool

Subnational health sector leadership survey

EXAMPLE

The selection of indicators help me

prioritize across interventions


Analysis of stakeholder interviews and scorecard trends can

inform targeted interventions to improve uptake

Effectiveness of action itemsInput to management routines

Qualitative feedbackOverall satisfaction

ILLUSTRATIVE


Survey question

The scorecard tool is fully integrated into my management routine

While integration of the scorecard tool into management

routines is good overall, some districts require attention

3.3

3.9

3.8

3.6

District

management

Director

Minister

Political

leadership

3.2

Partner

Integration into management routine

1 = strongly disagree, 4 = strongly agree

3.2

3.9

2.4

2.7

4.0

District 4

District 5

District 3

District 2

District 1

Integration into management routine (by district)


Further

investigation

needed

SOURCE: Annual scorecard tool strengthening exercise

ILLUSTRATIVE


33%44%

29%

22%

71%

23%

41%36%23%

38%48%14%

46% 31%

Districts with greatest number of action items are most likely

to be delayed, indicating a need to prioritise and filter actions

Status of action items per district

Action items

On track

Complete

Delayed

N=22

21

11

9

7

Survey question

I have the necessary resources to complete my action items

Average response to “I have the necessary

resources to complete my action items”


4.0

3.6

3.4

2.0

1.5

D4

D3

D2

D1

D5


ILLUSTRATIVE


Scorecard design and accountability mechanism led to strong launch –

effort needed to improve data quality and stakeholder alignment

Scorecard

design

Accountability

mechanism

Data &

reporting

Stakeholder

alignment

Political

leadership

3.7

3.8

3.4

3.2

Minister of

health

3.6

3.7

3.1

2.9

District

managers

3.3

2.9

3.1

3.0

Directors &

programme

managers

3.5

3.4

3.0

3.1

Partners

3.8

3.5

2.9

3.0

Average

3.6

3.5

3.1

3.0

District Managers are typically

less satisfied with the scorecard

tool than other stakeholders

Data & reporting and stakeholder

alignment require ongoing

attention


ILLUSTRATIVE


Directors and programme managers are satisfied with the

scorecard, but some have low confidence in the source data

Areas of success Area for improvement

ILLUSTRATIVE

Scorecard design“Scorecard is easy to read and

helps identify issues that we need

to address”

“Would be helpful if we could view historical

data on a chart”

Data & reporting

“The indicators in child health

identify those areas where we

need to make the most

improvements”

“Data coming out of some districts is almost

useless”

Accountability

mechanism

“The scorecard provides a fact

base, which keeps my decisions

grounded in reality”

“We’re still working out the kinks in the

quarterly review. Right now, there is too

much lag between the end of the quarter and

scorecard discussions”

Stakeholder

alignment

“The scorecard provides clear

targets for me to shoot for”

“Management places too much pressure on

getting indicators to green. There needs to

be more discussion around setting realistic

targets”



Final report summarises findings on key dimensions for national

leadership audience

Subsections

Accountability Mechanism

▪ Overview of management routines

▪ Adoption of the scorecard tool (National and Subnational)

▪ Assessment of action items (design and implementation)

Scorecard design

▪ Alignment of indicators and national priorities

▪ Assessment of management insight offered by the scorecard tool

▪ Introduction of next wave indicators & additional indicators needed

▪ Indicators and targets requiring adjustment

Report should be

tailored to national

leadership audience:

▪ Head of State

▪ Minister of Health

▪ MoH Directors

and Programme

Managers▪ Summary of recommendations

▪ Resources required

▪ Next stepsConclusion

Introduction

▪ Executive Summary

▪ Context and objectives of the scorecard tool

▪ Summary scorecard for last 12 months

▪ Narrative report

Chapter

Data & Reporting

▪ Data Quality

▪ Data gathering and input processes

▪ Analysis and Synthesis

Stakeholder alignment

▪ Perceptions of the scorecard tool

▪ Engagement among stakeholders


Contents

▪ Introduction


scorecard tool


facilitation


– Introduction and objectives of a

country RMNCH scorecard tool


– Process for developing the scorecard

tool


Country-led RMNCH management scorecards help to enhance

transparency, accountability and action around RMNCH

▪ Enable better profiling and monitoring of high-impact

RMNCH interventions to help decision-makers identify and

prioritise gaps

▪ Facilitate and strengthen evidence-based action plans to

improve internal management of programmes and policies

▪ Serve as an advocacy tool for external partners and civil

society

▪ Align with existing national review processes and timing,

▪ Validate and triangulate the implementation of actions

intended to improve the health of women and children

National RMNCH profiles and management scorecards






national RMNCH

initiatives




definitions

Global initiatives



and needs


involvement for a




and validate data


quality data


manage day-to-day


partner support


and regional health

policy makers


regional reports



RMNCH scorecard

management tool




Country Countdown)



targets








status


The RMNCH scorecard tool drives tangible action at the national and

subnational levels to improve maternal and child health


and validates data

▪ Advocates for action items to political leaders

Development

Partners


▪ Use political power and resources to support health

programs that matter most

Political

leaders


▪ Held accountable before leadership and peers for

performance

▪ Use guidance and tracking of action items to shape

programming

MoH

Directors &

Managers

PRE-WORK


Contents

▪ Introduction


scorecard tool


facilitation






tool


The RMNCH scorecard tool is a customisable, dynamic management tool

for Ministries of Health to strengthen accountability and drive action

The RMNCH scorecard tool is not …The RMNCH scorecard tool is …

▪ Management tool for the Ministry to

track national and subnational

performance, strengthen

accountability and drive action

▪ Prioritized set of high-impact

RMNCH indicators selected by the

Ministry that reflect country priorities

▪ An aggregation of existing data

▪ Customizable to align with National

Health Sector Strategic Plans and

existing management processes,

including national reviews

▪ Comprehensive inclusion of all

available RMNCH indicators

▪ A static scorecard that cannot be

easily updated to reflect changing


▪ A one-size-fit all approach with pre-

defined set of indicators agnostic to

country context


The tool summarises RMNCH and health system information in

customised national and regional views and tracks actions

Input Output

A web-based online

tool captures all

information and

automatically produces

reports for

accountability and

action

Scorecard

showing

performance at

national and

regional level for

indicators

Action item

tracker listing

action items and

progress

against them

Regional reports

showing

performance on

each indicator

National data sources

Regional data sources

Strategic & operational plans

Outputs from working sessions

Tool builds on existing data

sources, strategic and

operational plans and outputs

from the working sessions to

gather:

▪ Baseline data

▪ Targets

▪ 3-5 national priority indicators

▪ 15-20 actionable regional

indicators


Contents

▪ Introduction


scorecard tool


facilitation






tool


Developing the scorecard is a ~3-4 week process, with

ongoing adjustments made based on country needs

Led by country team

Led by facilitator

Facilitator in-country

Ongoing1-2 weeks

Pre-work

▪ Align on scorecard tool

objectives and approach

▪ Create core working team

▪ Identify stakeholders and

invite to workshops

▪ Prepare potential

indicators list

Scorecard tool adjustment

▪ Adjust scorecard tool to reflect

country preferences, feedback

▪ Update based on changing data

sources

2 weeks

Scorecard tool design

▪ Review and prioritise indicators

with stakeholders

Accountability Framework

▪ Understand existing accountability

and support routines

▪ Define how to use the tool for

continuous action

Data validation

▪ Set up ongoing data collection

mechanisms

This schedule should

be adapted to specific

country needs

PRE-WORK

Data validation

▪ Identify data sources

▪ Collect and validate data to

populate first scorecard

Training & roll out

▪ Build administrative and technical

skills to manage the tool

▪ Build skills in tool analysis and

synthesis

Countries benefit most when they have completed essential pre-work

▪ Hold a call with the facilitator team 2 weeks before the visit

▪ Create a preliminary list of indicators 1 week before the visit

(by holding an in-country working session with the technical

committee)

▪ Schedule a kick off meeting for Day 1 of facilitator presence

▪ Identify Ministry of Health focal person responsible for

leading the score card development

▪ Identify Ministry of Health point person responsible for

identifying and compiling data sources

▪ Create a list of stakeholders and form the technical

committee

Length of time

required for pre-

work will vary

based on the

level of

consolidation

vs.

fragmentation in

data systems


Countries typically choose 6-8 categories to track on the scorecard



Post-

partum &

neonatal

care

Infancy &

childhood

Adol-

escence &

pre-preg-

nancy

Pregnancy

Childbirth

MNCH life

cycle


Newborn

health


country …


… including cross-cutting categories and national

priorities to form a list of 6-8 categories






categories?





you experienced?



issues?


▪ Health systems

– Infrastructure

– Work force

– Supply chain

– Information

▪ Policies

▪ Financing



▪ ANC

▪ PMTCT


▪ PNC

▪ Breast-feeding

▪ Nutrition

▪ Pneumonia


▪ Immunization

▪ Malaria


▪ Maternal health

(including PMTCT)

▪ Newborn health

▪ Child health

▪ Nutrition


▪ Health systems

Sample MNCH

category list:






categories?





you experienced?



issues?


▪ Health systems

– Infrastructure

– Work force

– Supply chain

– Information

▪ Policies

▪ Financing



▪ ANC

▪ PMTCT


▪ PNC

▪ Breast-feeding

▪ Nutrition

▪ Pneumonia


▪ Immunization

▪ Malaria


▪ Maternal health

(including PMTCT)

▪ Newborn health

▪ Child health

▪ Nutrition


▪ Health systems

Sample MNCH

category list:

PRE-WORK


Countries should create a list of potential categories and indicators based

on health system priorities and data availability

A starter list of indicators is available in

the appendix for countries to consider

Indicator

Update

Frequency Data SourceCategory Data available?

Reproductive Indicator 1 Monthly Source 1 Yes

Reproductive Indicator 2 Quarterly Source 2 Yes

Reproductive Indicator 3 Quarterly Source 3 No

Reproductive Indicator 4 Annually Source 4 Partially



Maternal Indicator 7 Quarterly Source 7 No

Initial list should

have at least 5

indicators for

each of the 6-8

categories

chosen

PRE-WORK


Initial indicator list should then be tested against the SMART criteria

Questions

Timely▪ Is the data source updated quarterly? Could it be?

▪ Is the data available quickly enough to support continuous action and quarterly

review and action planning?

Measurable▪ Is it easy to measure? Can it be tracked against a specific target?

▪ Is data consistently available today?

▪ Is the data source accurate and complete (i.e. for all locations and individuals)?

Achievable

and agreed

▪ Can the person/group responsible for it actually influence it?

▪ Do we understand the drivers that are behind it?

▪ Are the relevant stakeholders aligned on the inclusion of the indicator?

Results-oriented

▪ Is it relevant to the performance of RMNCH?

▪ Does it support the high-level priorities for the country?

▪ Is it aligned with the subnational strategy and objectives?

Simple and

specific

▪ Does it have a clear definition?

▪ Is it straightforward to understand?

▪ Can it be easily generated without complex calculations?

PRE-WORK


Proposed two-step approach to selecting indicators

Review/develop initial indicators list

and prioritise

Sort into waves based on data

availability and intervention timeline

▪ Review initial list, if developed in pre-

work. Otherwise, build with country team

▪ Prioritise the indicators that are national

priorities or correspond with current

initiatives


SMART criteria sufficiently



▪ First wave: indicators with data

currently available

▪ Second wave:

– indicators with data expected in the

near future

– indicators for interventions to be

rolled out in the near future

▪ Third wave:

– Indicators with data not expected in

the short term

– Indicators to support emerging

priorities

This process will identify 2-3 indicators per

category for each wave

PRE-WORK


The country team should organise to fulfill the key roles to make the tools

effective in practice

PRE-WORK

Tool owner

ResponsibilitiesPreferred experience

Program

champion

▪ Responsible for the success

of the scorecard

▪ Ensures full stakeholder

participation

▪ Senior, respected member

in the FMOH

(M&E/planning or RMNCH)

Scorecard

manager

▪ Day-to-day data collection,

meeting set-up, and

stakeholder coordination

▪ Experience with health

performance management

system and with partners

▪ Mastery of Microsoft Excel

and internet skills

▪ Familiarity with

data sources

▪ Ability to train others

▪ Adjusts tool to country needs

▪ Regular data input

▪ Trains others on tool usage

▪ Output finalisation

Developing a country RMNCH scorecard - ALMA 2030 · evidence-based, country-led RMNCH...

Documents

Transcript of Developing a country RMNCH scorecard - ALMA 2030 · evidence-based, country-led RMNCH...