Managing for Results Programme Cycle Management

Managing for Results Programme Cycle Management

Managing for Results Programme Cycle Management

Trine Eriksen and Inger K. StollFinancial Mechanism OfficeNovember 2012

Trine Eriksen and Inger K. StollFinancial Mechanism OfficeNovember 2012

1

RESULTS BASED MANAGEMENT

What is Results Based Management (RBM)RBM is a management strategy by which all actors, contributing directly or indirectly to achieving a set of results, ensure that their processes, products and services contribute to the achievement of desired results (outputs, outcomes and impact)

In the context of the Grants: A result is the output, outcome or impact of a development intervention

A risk is an event that may occur and impede the objective

2

What is a result and a risk?

KEY CONCEPTS

Results achievement

Beneficiary oriented

Managing for results and reduced risks

3

RESULTS BASED MANAGEMENTIN PRACTICE

1. Clearly identifying program beneficiaries and designing programs to meet their needs.2. Defining realistic expected results, based on appropriate analysis.3. Identifying and managing risks.4. Monitoring progress towards results with the use of appropriate indicators.5. Increasing knowledge by learning lessons and integrating them into decisions, and6. Reporting on results achieved and the resources involved.

4

Part I Quality at entry - Basis for good management

5

The Programming Cycle and Continuous Feedback

Planning

Programming

Budgeting

ImplementationMonitoring

Evaluation

Reporting

PROGRAMMING CYCLE

6

Present situation

Baseline

Activities

Outcome

Impact--------------------------------

--------------------------------Programme area

Programme area

Inputs Outputs

Future situationTargets

National/ sector area


The Programme and the Context

RESULTS CHAIN

Outcomes Intermediate effects on target groups and systems Medium term

Impact Widespread improvement in society/sector Long term

Outputs Products and services delivered Short term

Activities Tasks transforming inputs to outputs Ongoing

Inputs Financial, human & material resources Ongoing

8

Identify main programmeelements of EEA/N GrantsObjective (impact)Widespread improvements in societ or a sector

32 predefined objectives

Outcome(s)Intermediate effects ontarget groups and systems

160 predefined outcomes In DoRISdata base

OutputsProducts and servicesdelivered

Self-defined (at least two outputs per outcome)

In DoRISdata base

ActivitiesTasks transforming inputs to outputs

Self-defined Not in DoRIS

InputsFinancial, human and material resources

Budget + Budget in DoRIS

9

WHAT IS AN OUTCOME?

• Outcomes can occur on multiple levels:

• Individual level;• Family or household;• Community or population group;• Organization; • System (involving multiple institutions or agencies); or• State.

10

DIFFERENT TYPES OF OUTCOMES

• The nature of the changes can vary as well. Outcomes can involve different types of changes:

• Changes in perceptions, attitudes, feelings, beliefs, or values;• Knowledge;• Skills or level of functioning;• Behaviours or practices;• Conditions;• Status (such as socio-economic status, and status can reflect formal or

informal status); or• Policies.

11

Defining outcomes and outputs

• Express an outcome and an outputas a positive condition.

• A measurable, desired result: Wait list for a cancer operation has been reduced from 16 moths to 10 months at hospital x in y province

• Avoid «double»objectives as these blur what will be measured.in order to through

12

Cross-Cutting issues asoutputs and targets• Good governance• Gender equality• Sustainable development: Environmental, Economic

and SocialCross-cutting issues are either reflected in the outputs

or the indicators (targets), or assumptions. If the programme substantially addresses cross-cutting

issues, it is advisable to define outputs and targets that reflect the cross-cutting issues. This strengthens follow-up.

13

Programme logic

Inputs

Activities

Outputs

Expectedoutcomes

Objective

Overall objectives

if the inputs are available, then the activities will take place

which will contribute to the two overall objectives

if the activities take place, then the outputs will be produced

which will contribute to the fulfilment of the objective

if the outputs are produced, then the expected outcome is likely to be achieved

Should be able to guarantee that this will happen

Likely to happen

Contribute to

Contribute to

14

ASSUMPTIONS

• The intervention logic is an aim-means chain and a cause-effect chain.

• It is assumed that “certain events” need to occur at one level in order to achieve the desired effect at the next level.

• It is not certain that these events happen; they represent uncertainties.

15

There areAssumptionsbetween eachlevel:

IdentifyAssessMonitorManage and

Mitigate

Impact/Objective

Outcome

Outputs (deliverables)

Activities

Inputs

16

Identify important assumptions

• Assumptions are worded as positive conditions (see objectives)• Assumptions are linked to the different levels of the

Programme Matrix. Each level must contain the necessary and sufficient conditions for the next level above.

• Start from bottom of matrix and work upwards• Are weighted according to importance and probability

Example: Scholarship recipients return to assigned jobs

17

Check the significance of the assumptions

1. Eliminate the obvious factors which are:

• Not important for the outcome• Very likely to occur

2. Assess the probability of occurrence for the remaining factors:

a) Quite likely but not certain:Include the factor into the PM and make sure to:•Monitor it•Report changes•If possible influence it

b) Not likely to occur (killing factor!)

Redesign the programme

Reject the programme proposal

c) If this is not possible:

RISKS

• If important assumptions do no occur, they might represent a risk.

• A risk is any event that could occur and adversely impact the achievement of results at all levels

• Risks are an expression of uncertainty• Assumptions are expressed as a positive condition (a

necessary event), whilst risks are expressed as negative conditions (un-wanted events)

19

Identify main programmeelementsObjective (impact)Widespread improvements in societ or a sector

Assumptions and risks

Outcome(s)Intermediate effects ontarget groups and systems


OutputsProducts and servicesDelivered


ActivitiesTasks transforming inputs to outputs


InputsFinancial, human and material resources


20

Cross-Cutting issues asassumptions• Good governance• Gender equality• Sustainable development: Environmental, Economic

and SocialCross-cutting issues are either reflected in the outputs

or the indicators (targets), or assumptions. If not taken care of they can represent a risk to the sustainability of the programme results. Therefore cross-cutting issues should be monitored in the same way as assumptions and risks.

21

The basic elements of theProgramme Matrix

Impact (Objective)

Outcome

Outputs (deliverables)

Activities

Inputs

Assumptions describe the necessary conditions that must exist for the cause and effect relationships between the levels to occur. If an assumption does not occur, it might represent a risk

22

The Matrix is the point of reference throughout the life of the programme

Present situation

Baseline

Programme area

Programme area

Future situation

Programme implementation

Programme



PART II Risk Management

24

PART III

25

Programmes and projects

Programmes and projects

Programmeproposal

Strategicapproach

Projects Pre-defined projects

Pre-defined projects

26

Programme design

OBJECTIVE (INDICATORS) (ASSUMPTIONS)

OUTCOME INDICATORS ASSUMPTIONS

OUTPUTS INDICATORS ASSUMPTIONS

OBJECTIVE (INDICATORS)

(ASSUMPTIONS)



ACTIVITIES INPUTS ASSUMPTIONS


(ASSUMPTIONS)





(ASSUMPTIONS)




PROGRAMME OUTCOME

PROGRAMME OUTPUTSProg

ram

me

Proj

ects

27

Example – Renewable Energy

Increased share of renewable energy in energy use

Increased renewable energy production

Increased awareness of and education in renewable energy solutions

More energy efficient RES in place

X policies at local and regional level to stimulate RES developed

100 MW capacity RES constructed and in operation

X awareness raising programmes at local level carried through

X training courses in RES provided to officials at local and regional level

Objective (pre-defined)

Expected outcome(s) (pre-defined)

Outputs

Types of projects

•Modernised RES infrastructure

•R&D on RES

•Feasibility of RES mapped out in relation to local conditions

•

•

•Training in RES planning competence

•Plans/policy development

•

•

•Windmills

•Solar systems

•Hydropower

•Bioenergy

•

•

•Awareness raising campaigns at local level

•Train the trainers

•

•

•Training courses for officials at regional level

•Training courses for officials at regional level

•

•

28

EXAMPLE Gender-basedviolence

29

Gender-based violence prevented and tackles

Gender-based violence reduced

Domestic violence reduced

Services for women at risk of violence designed

A national coordination centre for service provision established

Awareness around gender based violence increased

Services for victims of domestic violence improved

Procedures for work in crisis center standardised

• Support to existing centers

• Support to start new centers

• Support to provide services for women at risk such as counseling, phone-lines

• Training

• New coordination center

• Support for coordination activities

• Training

• Methodologies and guidelines development

• Studies

• Media campaign

• Education activities

• Curriculum development

• Training of police and medical personnel at local and regional levels

• Training for counselors

• Support to start new counseling centers

• Support to develop new standards and methodologies

• Training

• Manual developed

Objective (pre-defined)

Expected outcome(s) (pre-defined)

Outputs

Types of projects

Call for proposals

Be clear about which outcomes and outputs you want the projects to contribute to.Which projects are necessary and sufficient?

Project template in DoRIS:http://www.eeagrants.org/id/3192

30

http://www.eeagrants.org/id/3192

Logical relationship

31

PART IV

32

Basis for results measurement

BASIS FOR RESULTS MEASUREMENT

Result based management

33

"The starting point for a results-oriented approach is ex ante setting of clear and measurable targets and outcome indicators. Indicators must be clearly interpretable, statistically validated, truly responsive and directly linked to policy intervention, and promptly collected and publicised."

From the conclusions of the 5th Cohesion Report, adopted by the European Commission in November 2010

QUALITY AT ENTRY!

30

INDICATOR

• A means to measure achievement, or to help assess a development

35

HOW SHOULD INDICATORS BE FORMULATED?

•Once indicators have been identified, they should be formulated to reflect measurable change. Indicators frequently utilize the following metrics to depict measurable amounts:•Number; •Ratio;•Percentage; or•Rate;•TimeIndicators shall whenever relevant, be broken down per gender

In DoRIS, always specify unit of measurement!

36

LESS IS MORELESS IS MORE

EXAMPLES OF INDICATORS Example 1: Outcome: Improved health status of young children in region/district X

Indicator: Percentage of children under 6 years who are immunized against x y z diseases in district A

Example 2: Outcome: Increased participation of worker’s

organisations in policy development

Indicator: Number of cases in which the submitted comments of worker’s organisations on draft legislation are reflected in final legislation

37

INDICATORS

To describe how results will be measured, indicators must be specific, observable and come from reliable data. •Does the indicator provide the most direct evidence of the condition or result to be measured?•Is the indicator important, understood, and accepted by important stakeholders?•Is the indicator defined in the same way over time?

38

• Are data for the indicator collected in the same way over time?

• Will data be available for the indicator? • Will data be collected frequently enough?• Are data currently being collected? • If data are not currently collected, can cost effective

instruments for data collection be developed quickly?

39

Indicators related to Bilateral Relations

• All programmes should contribute to strengthened Bilateral Relations

• Use the Bilateral guideline.Four types of bilateral results expected:Extent of cooperationShared resultsWider effectsKnowledge and mutual understanding

40

Indicators related to «Horizontal issues»

• A human-rights based approach and concerns for horizontal issues like fighting discriminiation, racism, hate speech etc. should guide defining the outputs and the indicators.

Example: X% of members of Parliament are minorities.

Example: Incidences of hate speech has decreased in Y school

41

Programme matrix - indicators

Sources of Verification

• When defining the indicators, consider the data sources. Where can you easily get data that are reliable and timely?

• Specify your sources of information

• Assess where you can get easy access to dataExample Ministry of Health’s statistics on cancer (health status of population) and various statistics on cancer detection and treatment services

Programme matrix – Sources of Verification

WHAT ARE BASELINES?

• Baselines provide context for the setting of targets and capture the situation before a development intervention begins, or at the beginning of a time period that will be monitored and assessed.

• Baselines describe the conditions prior to programming efforts. The present situation!

• Essential to make credible and meaningful assessments on progress towards outcomes.

45

Programme matrix – Baseline data

WHAT ARE TARGETS?

•Targets set the level for the magnitude of change expected by the end of a predetermined time period. They reflect what organizations commit themselves to achieve by the end of some duration of time.

•Targets provide tangible and meaningful points of discussion with beneficiaries, stakeholders, and partners, and allow us to add further specificity to the outcomes from the results logic.

47

TARGETS

• When you have defined the indicators and know the baseline situation, you set the targets.

• What do you want the situation for the beneficiaries to be in 3 or 5 years?

• Examples:• 1. 60% of W 40+ called in for cancer test in Southern Region by year

2016. We want to increase the % of W 40+ called in from 30% in 2010 to 60% in 2016.• 2. Time lag between detection of cancer to operation is not more than 3 months in 2016.

48

BASELINES AND TARGETSSUMMARY• A baseline is the value of the indicator before the

implementation of the project starts• Targets orients stakeholders to the tasks to be

accomplished• Targets help establish clear expectations• Targets serve as the guideposts for monitoring

whether progress is being made• Targets promote transparency

49

Programme matrix - Targets

43

Experience: The EU

In 2010 the EU evaluated the 4,131 impact indicators under the ERDF programmes. Among the programmes;•94% had final achievements;•58% had targets;•6% had baselines;•55% had targets and achievements;•5% had baselines, targets and achievement; and finally•0.5% had baselines but no targets and no achievements.

How can we assess impacts if we have no idea of the starting point?Conclusion: Much of the data collected are not meaningful and do not represent the impact of the Structural Funds.

51

Part IV Quality at implementation•Monitoring•Reporting on results and risks•Evaluation

52

The Programming Cycle and Continuous Feedback

Planning

Programming

Budgeting

ImplementationMonitoring

Evaluation

Reporting

PROGRAMMING CYCLE

53

PERFORMANCE MONITORING• Define processes and systems for performance

measurement;• Decide the intervals at which measurement will occur;• Assign roles and responsibilities; • Set standard requirements for project promoters• Build in adequate time and resources for analysis of

performance measurement data so that progress is reviewed and assessed.

• Use web site for communication with project promoters and other stakeholders

54

Performance monitoring

• Status regarding the output and outcome indicators is measured and reported towards the baseline situation (“where we were”), and also towards the future, desired situation (“the targets”).

The Matrix is the point of reference throughout the life of the programme

Present situation

Baseline

Programme area

Programme area

Future situation

Programme implementation

Programme



Programme performance framework

Reporting requirements in annual report from Programme Operators

Programme outcome• Report and analyse status of the selected indicators for the expected programme outcome(s)• Analyse how the projects and the programme outputs have contributed to the expected

programme outcomes.

Programme outputs• Report and analyse status of the selected indicators• Summerize and analyse how projects have contributed to the planned programme outputs

Outcome and outputs of bilateral relations• Report and assess achievemenet of outcome and outputs of bilateral relations

Assess risks and risk mitigation, cross cutting issues, sustainability, and outreach to target groups

58

EVALUATIONS

• Evaluations have three key functions:(1) Learning: As an input to provide decision-makerswith knowledge and evidence about performance and

good practices;(2) Accountability: To donors, funders, political

authorities, stakeholders and the general public, and

(3) Communication: Results for the beneficiaries

THANK YOU!59

Managing for Results Programme Cycle Management

Documents

Transcript of Managing for Results Programme Cycle Management