Real Time Evaluation of UNICEF’s response to the Mali crisis

Final report

Enrico Leonardi Ricardo Solé Arqués

July 2013

ACKNOWLEDGMENTS ......................................................................................................... 3

ACRONYMS ........................................................................................................................... 4

EXECUTIVE SUMMARY ....................................................................................................... 5

CONTEXT AND BACKGROUND ........................................................................................ 18

RATIONALE, OBJECTIVE and SCOPE of the RTE .............................................................. 20

METHODOLOGY ................................................................................................................. 22

Methodology Limitations .................................................................................................... 24

FINDINGS ............................................................................................................................. 25

General aspects ................................................................................................................... 25

PROGRAMMATIC AREAS............................................................................................... 29

HEALTH AND HIV/AIDS .......................................................................................................... 29

NUTRITION ............................................................................................................................... 34

WATER, SANITATION AND HYGIENE (WASH) .................................................................... 39

EDUCATION .............................................................................................................................. 41

CHILD PROTECTION ................................................................................................................ 44

COMMUNICATION FOR DEVELOPMENT (C4D) ................................................................... 48

PROGRAMME INTEGRATION ................................................................................................. 50

MONITORING AND EVALUATION (M&E) ................................................................... 51

HUMANITARIAN PERFORMANCE MONITORING (HPM) .................................................... 51

THIRD PARTY MONITORING (3PM) ....................................................................................... 55

SITREP ....................................................................................................................................... 56

OPERATIONS ................................................................................................................... 57

HUMAN RESOURCES ............................................................................................................... 58

SECURITY ................................................................................................................................. 59

SUPPLY AND LOGISTICS ........................................................................................................ 60

COMMUNICATION ................................................................................................................... 62

COORDINATION .............................................................................................................. 64

TRANSVERSAL ISSUES .................................................................................................. 66

GENDER ..................................................................................................................................... 66

ACCOUNTABILITY TO BENEFICIARIES ................................................................................ 67

ETHICAL ISSUES ...................................................................................................................... 67

CONCLUSIONS .................................................................................................................... 68

RECOMMENDATIONS ........................................................................................................ 71

Annex: REFERENCES ........................................................................................................... 79

Annex: ITINERARY OF THE MISSION ............................................................................... 81

Annex: LIST of INTERVIEWEES ......................................................................................... 82

Annex: TERMS OF REFERENCE ......................................................................................... 84

Annex: CLUSTER SURVEY: Questionnaire and Results ....................................................... 96

ACKNOWLEDGMENTS

During the exercise, the consultant received greatly appreciated support from many

people in the UNICEF Regional Office in Dakar and in the Mali Country Office. In

Dakar, Inoussa Kabore, Toby Wicks and Cairn Verhulst dedicated much of their busy

time to facilitate the work of the Real-Time Evaluation team, organise numerous

meetings and supported the planning of Mali part of the mission. The whole Emergency

Team was behind this RTE and greatly contributed to the implementation of the

exercise. The consultants would also like to thank Manuel Fontaine, Regional Director,

for his support, important inputs, and encouragement.

In Mali, the RTE team benefited from the support of several staff. Shannon Strother

assisted in the planning of the exercise and the field visits, and was frequently the

driving force behind occasional problem-shooting. Noriko Kominami helped the team

with the organisation of numerous meetings in the UNICEF office and with external

counterparts in Bamako. Additional help for the daily work in the office was kindly

provided by Jessica DuPlessis and Jeremy Shusterman. Assistance from Anne-Celine

Delinger to translate into French questionnaires and materials for the Cluster survey was

essential; merci! The consultants would like to thank Francoise Gruloos-Ackermans,

Representative of Mali Country Office, for her support, encouragement and the

information provided.

ACRONYMS

AAP Accountability to Affected People IR Intermediate Result

ANC Antenatal care ITN Insecticide treated net ARV Antiretroviral IYCFP Infant and young children feeding practices

ASACO Community health association (French acronym)

KII Key informant interviews

AWP Annual Work Plan MAM Moderate acute malnutrition

C4D Communication for development MCO Mali Country Office

CAAFAG Children associated with armed forces and armed groups

M&E Monitoring and Evaluation

CAAP (IASC) Commitments on Accountability to Affected People

MINUSMA United Nations Integrated Mission for the Stabilisation of Mali (French acronym)

CAP Consolidated appeal process MoE Ministry of Education

CBO Community based organization MoH Ministry of Health

CCC Core Commitments for Children MoU Memorandum of Understanding

CFS Child friendly spaces MRE Mine risk education CHW Community health workers MRM Monitoring and Reporting Mechanism

CIDA Canadian International Development Agency NatComs National Committees

CLTS Community level total sanitation NFI Non-food items

CO Country Office NGO Non-governmental Organisation

CP Child protection OCHA Office for the Coordination of Humanitarian Affairs

CPD Country Program document ORE Other resources emergency

CRS Catholic Relief Services ORR Other regular resources CSCOM Community health centre (French acronym) PBA Programme Budget Allocation

CSRef Reference health centre (French acronym) PBR Programme Budget Review

DHR Division of Human Resources PCA Project Cooperation Agreement

EMOPS Office for emergency programs PDM Post distribution monitoring

EPI Expanded program of immunisation PMTCT Prevention of mother to child transmission

ESARO Eastern and Southern Africa Regional Office PSI Population Services International

FGD Focus groups discussion PTF Financial and technical partners (French acronym) FGM/C Female Genital Mutilation/Circumcision RO Regional office

FO Field office RTE Real time evaluation

FTRP Fast Track Recruitment Process S/UAC Separated/unaccompanied children

FTS Financial Tracking System SAM Severe acute malnutrition

GAM Global acute malnutrition SIDA Swedish international development agency

GBSV Gender-based and sexual violence SMART Standardized Monitoring and Assessment of Relief and Transitions

GBV Gender-based violence SRSG Special Representative of the Secretary General

GoM Government of Mali SSA Special Service Agreement

GPE Global partnership for education TA Temporary Assignment

HAC Humanitarian action for children TOR Terms of Reference

HCT Humanitarian Country Team 3PM Third party monitoring

HDI Human development index UNDP United Nations Development Programme

HI Handicap International UNDSS United Nations Department of Safety and Security HPM Humanitarian performance monitoring UNFPA United Nations Population Fund

HQ Headquarters URENA Ambulatory units for nutritional recovery

HR Human Resources URENI Inpatient units for nutritional recovery

IASC Inter Agency Standing Committee UXO Unexploded ordinance

ICT Information and Communication Technology WASH Water, Sanitation and Hygiene

IDP Internally Displaced Person WCARO Western and Central Africa Regional Office

IM Information management WFP World Food Programme IOM International Organization for Migration WG Working Group

IP International Professional WHO Wold Health Organisation

EXECUTIVE SUMMARY

Context and Background Mali, one of the poorest countries in the world, already affected by the Sahel nutritional

crisis, went through a destabilization process in 2012 when armed rebel groups took

control of wide areas of the North and triggered a political crisis in the capital.

Subsequently, an armed intervention by French troops in January 2013 restored control

and access to the North. A cease-fire was established in June and the deployment of a

12,600 strong UN peace-keeping force (MINUSMA) was planned for July. The

situation should be further stabilised by the celebration of elections in July and the

consolidation of a peace accord with rebel groups.

The consequences of the turmoil have been twofold: on one hand, the displacement of

populations to neighbouring countries (some 175,000 refugees) and other regions of

central and southern Mali (around 350,000 IDPs), with the related humanitarian

consequences thereof, as well as the stretching of the coping mechanisms of both those

displaced and of host families (a total of 2.8 million persons are estimated to be directly

affected by the conflict). On the other hand, the ramifications of the conflict have

entailed the freezing of development programs and budgetary support to the

government, on which many basic social services were dependent.

Challenges created by the situation include access to areas in the North, threats to

humanitarian space by the presence of armed personnel, new and increased protection

challenges affecting children and women, and the new role to be played by UNICEF in

channelling donor funding to assure basic services. The current trend towards

stabilization remains fragile and different scenarios (from the worst to the best case)

have to be taken into account. While the on-going nutritional crisis was declared an

“informal” Level 2 emergency during most of 2012, the Mali conflict and consequent

complex crisis were officially declared a Level 2 emergency by the Executive Director

of UNICEF on the 28th

of January, 2013, which has been extended until September

2013.

UNICEF’s 2013 humanitarian response plan aims to ensure a fully integrated approach

to the overlapping components of the Mali Crisis and is articulated through

interventions ranging from direct service delivery, capacity development initiatives,

coordination support and advocacy.

RTE Objectives and Scope

This Real Time Evaluation is carried out in order to provide recommendations to

enhance the scale-up of UNICEF planning and response for the Mali humanitarian

crisis.

The criteria of effectiveness and coverage, efficiency, appropriateness, relevance,

adequacy and coordination are addressed in the framework of the RTE.

The scope of the exercise covers the period from January to May 2013. It aims at

assessing response in the entire territory of Mali and focuses on both programme and

operations’ areas of UNICEF’s intervention.

Methodology

The methodological approach has been straightforward and participative, utilising a

number of qualitative and quantitative techniques and tools. The different processes

employed included an extensive desk review, quantitative analysis of available hard

data, Key Informant Interviews (KII), field visits, observations and group discussions,

as well as a simple survey. The evaluation mission took place between the 2 and 23 of

June 2013.

Quantitative approaches have been utilised for the analysis of health and nutrition

information management systems, for expenditure and performance analysis, and for

supply procurement and distribution analysis. Information from VISION was especially

consistent and useful for parts of this exercise. A quantitative approach was also used to

analyse the results of the Cluster survey carried out in Mali’s capital, Bamako.

The RTE team carried out 22 KIIs with WCARO staff (13 males, 9 females), 2 (both

males) with EMOPS staff at HQ level and, in Mali, the consultants met with 32

UNICEF staff members (18 males, 14 females) in Bamako and Mopti. Additionally,

they interviewed staff from other UN agencies, NGOs, the government and local

administration, as well as donor representatives. In total, the RTE team undertook 90

KIIs (60 males, 30 females).

Field visits were organised in Kangaba (Koulikoro region), Mopti and Nampala (Segou

region). These allowed the team to meet NGO and governmental counterparts and to

visit certain project sites covering all sectors of UNICEF intervention. Such visits

afforded the opportunity for direct interaction with beneficiaries and end users.

With the support of key Cluster staff, a small-scale, self-administered and confidential

Cluster survey was carried out in Bamako to investigate the efficiency and effectiveness

of the UNICEF-chaired Clusters. This exercise helped identify strengths and

weaknesses in the system, as well as problems and bottlenecks.

The information collected has been analysed and compared to identify pertinent issues,

trends and divergences. Specific findings from different methodologies have been

triangulated to corroborate the evidence they provide.

Methodological limitations

The team faced challenges and difficulties in the planning and organisation of field

visits. For a number of reasons, the team was not able to visit any location in the North

of the country. Planning further in advance would have allowed broader coverage of

project sites, including in Northern Mali. The consultants compensated for these

limitations by multiplying the amount of interviews in order to gain sufficient elements

to triangulate and confirm findings.

The quality and scarce availability of data also proved a limitation, especially in terms

of Government service providers. Whenever possible, data from alternative sources

(mainly NGO partners) has been used to confirm or challenge some of the information

obtained.

Findings

The humanitarian crisis in Mali has exacerbated a reality characterised by poor

infrastructure and services, high rates of poverty and exclusion, low development

indicators, and recurrent ethnic and political tensions.

The main challenges of humanitarian response remain access, security, humanitarian

space limitations related to the deployment of armed peace-keeping forces, and the

difficulties in defining the limits between humanitarian response and addressing

structural problems associated with inequities and poverty.

During 2012, all organisations (both UN agencies and NGOs) struggled with the shift

from development to emergency response, and most are still working on the revision of

their strategies, scope and operational structures. The Cluster system was rolled out in

2012 and a first Consolidated Appeal Process (CAP) launched half-way through the

year.

In the complex environment created by the crisis, several factors contribute to opening a

window of opportunity for all involved actors – the UN, NGOs, the Government and

donors – to recognise and address not only immediate humanitarian needs, but also

some of the chronic weaknesses and ethnic and political tensions affecting the country.

In light of the fact that many donors diverted part of their development funding to UN

agencies, UNICEF’s ORR budget was greatly increased by these contributions. The

current context, wherein most key actors and structures in place are positioned to

comprehensively address the emergency-to-transition agenda, offers the Organisation

the opportunity to reshape its approach and become an essential player in the future of

Mali.

The RTE found that support from the RO to MCO throughout the humanitarian

response has been effective and appreciated overall. Specific issues are addressed in

each related section.

Health: Mali suffers from some of the worst health indicators worldwide. At present, a

number of structural problems detrimentally affect access and the quality of services.

The health sector is highly dependent on external support; the political crisis of 2012

prompted certain key donors to suspend budgetary support to the Government and to

instead channel significant funding to the health system through other partners,

including UNICEF.

The humanitarian response was designed to provide basic services to the North, where

only 10% of the population lives, and assist IDPs, host families and other vulnerable

groups in the rest of the country, while ensuring and reinforcing the basic capacities of

Mali’s health system. The response to the crisis is entangled with the various structural

problems that affect the health system at large.

The RTE found that most actors in Mali are not vocal enough to take advantage of the

humanitarian crisis in order to strengthen, promote and advocate for measures to allow

free access to basic services. As such a critical aspect in determining health outcomes

and affecting equity, a sound advocacy strategy from UNICEF could be envisaged in

this regard.

HIV interventions are related with PTCMT services and the availability of ARV. In the

North, the system was severely disrupted and PTCMT services are slowly being

resumed. Patients under ARV have been displaced, and only some have been tracked to

assure continuity of treatment. HIV testing in nutrition centres, furthermore, appears to

be extremely limited.

Regarding efficiency in the use of resources, as in other sections, the RTE was able to

verify an adequate rate of expenditure, but a very low level of translation into progress

of HPM indicators. Even allowing for delays in reporting, these low implementation

rates are a cause for concern. Perhaps employing alternative indicators could better

inform on progress and help to relate it with the use of resources.

Nutrition: Malnutrition affects Mali countrywide. A SMART survey carried out in 2012

in southern Mali confirmed the prevalence of acute malnutrition below the 15%

emergency threshold. The highest Severe Acute Malnutrition (SAM) rates were found

in Segou (3.8%), while a SMART survey carried out in the region of Gao in April-May

2013 showed SAM levels of 2.4%.

The agreement on caseload and the target to be reached for case management purposes

is a key issue in Mali. Based on the surveys, and with correcting factors applied, a

caseload of around 210,000 children with SAM is to be expected yearly. This year, the

target has been established at 125,000, a figure which represents an estimation of the

number of patients to be assisted according to the current capacity of partners on the

ground.

The consultants could not evaluate directly the response in the North, but indirect

information indicates that supplies are available and activities are on-going in some

CSCOMs. Outreach services are also provided by certain NGOs. In the areas visited in

the South, the programme, as integrated within the basic offer of health services, is

affected by issues of quality and access. Moreover, Community Health Workers’

(CHW) involvement, awareness and general interest in the programme appears

relatively weak.

While the measures already taken to improve effectiveness, including assuring the

presence of NGOs in the areas of highest prevalence, seem to assure better results than

last year, the progress in admissions in mid-June does not permit the assumption that the

numbers at the end of 2013 will be close to the 125,000 target. An additional concern

lies in the low quality of data available regarding cured and defaulter rates.

Of additional concern is the poor integration of other sectors in the response. WASH in

Nut seems to be implemented only in a limited way at CRENI level, HIV testing is

mostly not carried out, child stimulation at CRENI level is not occurring in the areas

visited, and C4D activities seem absent in the North and poor in the areas visited in the

South.

Water Sanitation and Hygiene (WASH): Access to improved water supply and

sanitation in Mali is extremely low and approximately 2.2 million people still practice

open defecation. The situation at the school level is similarly poor. Diarrhoea

contributes to ±20% of child mortality. UNICEF’s pre-crisis WASH programme

focused on increasing access to safe water, the promotion of Community Level Total

Sanitation (CLTS), hygiene promotion, and assistance to schools. The programme’s

coverage, however, was limited to selected districts. The emergency response added a

new result area to the programme, focusing on assisting affected populations –

including SAM children admitted to the nutritional programme – through the provision

of safe water and hygiene supplies, as well as on WASH interventions in schools

affected by the crisis, and on the preparedness and response to sudden crises (e.g.

cholera and floods, etc.).

The emergency response is sound and well planned, and recent UNICFE-led

interventions for sudden cholera outbreaks have been both successful and praised.

However, specific challenges remain. Most notably, coverage and effectiveness are

limited in relation to needs. Activities involving the engagement and participation of

local communities are extremely weak. There are discrepancies between expenditure

and implementation rates, only partially justified by supply and service delivery

bottlenecks, as well as reporting delays. In terms of all other sectors, there are concerns

that unless implementation rates improve, UNICEF will be under-performing by the end

of the year. WASH section has started addressing some of these concerns.

Education: Even before the crisis, more than 1 million children in Mali were out of

school and classes are frequently extraordinarily over-crowded, with up to 300 children

per class. UNICEF’s collaboration with the Ministry of Education (MoE) focused on

capacity building, as well as on the basic rehabilitation of, and material support to,

selected schools. The crisis in the North had major repercussions: 75,000 children and

85% of teachers were displaced and schools were severely looted and damaged. Limited

community initiatives allowed classes to continue running even during the occupation

of the North and the imposition of Sharia approaches to education.

UNICEF was effective in sending emergency supplies to the North during the

occupation period. The current response, however, focused on the reactivation of the

school system in the North, including “catch-up” classes and support to both IDPs and

host communities in the rest of the country, appears insufficient. Limited numbers and

capacity of implementing partners and Government counterparts, coupled with certain

internal weaknesses of the UNICEF Education section, affected the response. Although

unclear targets and indicators do not allow for a proper assessment of the effectiveness

of the response, major differences between expenditure and implementation rates

indicate efficiency problems.

The excellent collaboration between UNICEF and the MoE, in addition to the ongoing

strengthening of the Education section and its review of its strategic partnerships and

work plans, offers grounds for optimism. The organisation of a “Back-to-School”

campaign later this year, fine-tuned to local challenges and needs, would be an ideal

opportunity for UNICEF to raise its profile in this sector, as expected by both

Government stakeholders and donors.

Child Protection: Prior to the crisis, UNICEF’s CP programme was relatively small.

The conflict in the North and consequent displacements created new vulnerabilities

among women and children, requiring drastic changes in the section’s approach and a

dramatic increase of its capacities. An extensive period was required to start

accomplishing these changes, and the challenging process remains on-going.

The work of the section and its partners is affected by a general lack of capacity to

address technical issues and almost non-extant baseline data. This is a major problem in

a context wherein new protection issues are affecting an unknown number of women

and children. The deployment of MINUSMA adds another layer of complexity and

offers both challenges and opportunities. UNICEF has specific responsibilities in terms

of interacting constructively with the Mission’s protection branch. The recent inclusion

of Mali on the list of countries mandated by the UN Security Council to implement the

Monitoring and Reporting Mechanism (MRM) for violations of children’s rights in

conflict (UN Security Council Resolution 1612) is the latest challenge added to the long

list already faced by the CP section in Mali.

The work of the section has thus far focused on increasing the capacities of

governmental counterparts and NGOs, as well as on developing an effective programme

for Mine Risk Education (MRE). Good advocacy has contributed to the likely signature

of a national protocol for the identification, management and protection of children

associated with armed forces and armed groups (CAAFAG). Responses to other issues

and challenges have been more sporadic and undertaken on an ad hoc basis. The section

requires an in-depth review of its strategies, priorities, targets and indicators, as well as

its work plans and partnerships. This will be essential to maintain the momentum

created by the arrival of the new section chief, and support from both RO and HQ

should be assured.

Communication for Development (C4D): Communication for development is a key

aspect of UNICEF’s interventions. C4D activities are included in all section response

plans, and the C4D section has further articulated its own specific plan. The strong pre-

crisis focus on development has favoured community-based approaches in order to

achieve results, even when Government capacity is limited. The RTE did not assess

these aspects, but evidence gathered indicates significant weaknesses in the community-

level components of different sectors’ emergency programmes.

At present, C4D faces a number of challenges in Mali. Crucially, C4D activities tend to

be neglected whenever UNICEF experiences a shortage of funds. Moreover, the

aligning of C4D activities with concrete results in emergency situations must occur in

shorter timeframes, which might add another challenge for partners. It seems that

traditional partners are not well-suited to deliver activities in insecure or unstable

environments.

Building upon existing tools and experiences to involve beneficiaries in the planning

phases of the programme sectors’ response, in the framework of broader C4D activities,

would be a potentially interesting initiative. This is a field where C4D could markedly

contribute to improving the humanitarian response.

Programme integration: Programme integration is embedded in numerous planning

documents produced by MCO and several NGOs have praised the multi-sector PCA

approach utilised since 2012. However, field visits and interviews with various partners

and local administration at the field level indicate weaknesses in the way in which

programme integration is implemented. A number of NGO staff confirmed that this

situation is common in most of the rest of the country. At the Field Office (FO) level,

UNICEF staff indicated that joint, multi-sector assessment, planning or monitoring

missions hardly ever occur, thereby limiting chances to strengthen an integrated

perspective into response.

Multi-sector assessment and trip report tools exist in the context of UNICEF’s Office,

with a new tool recently applied by the WASH sector. Senior managers should

consistently stress the importance of programme integration, while all sectors should

find ways to enhance this approach both at the Bamako and the FO level.

Humanitarian Performance Monitoring (HPM): The HPM framework has been

evolving and adapting to specific contexts. The process in Mali seems more

participative and appropriated by the various programme sectors than it was during the

2012 Sahel crisis. Some questions remain, however, in terms of the clarity, accuracy,

feasibility and appropriateness of the proposed targets. An additional problem has been

the simplification of HPM reporting, which sometimes neglects important information

necessary for understanding performance (SAM cases cured, frequenting of health

centres, etc.).

The RTE team feels that the current use of HPM targets and indicators in the Situation

Report (Sitrep), may negatively affect the public perception of the performance of

UNICEF’s response. Some of the targets and indicators would be easy to clarify, and

should become useful tools to understand achievements, trends and challenges, while

others would require a more in-depth review.

Third Party Monitoring (3PM): UNICEF has been proactive in addressing the

monitoring challenges that the lack of access to the North have presented since early

2012. The system has been progressively fine-tuned, and in 2013 new tools have been

developed to further improve monitoring.

3PM already provides useful information on the implementation of projects and

bottlenecks in areas where access is difficult. Challenges remain in the context of

harmonising and integrating the information provided into the response plans of

programme sections. Field Offices (FO) have an important role to play in collaborating

with 3PM partners towards achieving adequate monitoring standards.

Accountability to beneficiaries is a conspicuous gap in the response of the humanitarian

community in Mali, one which UNICEF, with its C4D strengths, is in a privileged

position to address. With specific adaptations in current 3PM tools, such as Post

Distribution Monitoring (PDM), additional valuable information on beneficiary

satisfaction, the adequacy of the response, and on complaints, could be collected and

addressed.

Situation Report (Sitrep): The format and content of the Sitrep currently utilised in Mali

does not satisfy expectations. Notably, its utilisation as a monitoring or communication

tool seems to be contentious.

As has been indicated, the HPM technical component of the Sitrep requires clarification.

The origin of most targets – both for UNICEF and for the Clusters – is not explained

and, in some cases, where they are the sum of various categories of beneficiaries, they

are not indicated. Donors specifically stressed the need to provide clarifications on these

numbers so as to be able to evaluate trends.

The current bullet-point arrangement used to provide information is of limited use in

terms of providing an understanding of UNICEF’s response and performance. A

narrative that explains achievements and drawbacks from previous Sitreps, thus

maintaining some level of continuity, would help facilitate an understanding of the

response’s trends.

If the Sitrep must target different audiences – e.g. technical internal groups/offices,

donors and National Committees (NatComs), other organisations and the Government,

and, ultimately, Senior Management in NYHQ – with different needs in terms of

conciseness, the communications’ angle, technical analysis and programme details,

certain changes should be considered.

Operations: As in other emergencies where a sudden increase of staff is required, MCO

initially focused on increasing programme staff, partially neglecting support staff for

increased workloads in finance, administration and human resources. This discrepancy

has only recently been compensated for, with new positions established in the

Operations team.

The RTE found that the scale-up challenges faced by Operations are proportional to

those encountered by programme sections. In order to cope with a higher number of

staff (from ±100 to ±180), the restructuring and strengthening of the Field Offices, the

increasing numbers of partnerships and collaborations, and interactions related to

logistics, administrative and security issues with MINUSMA, it is essential to maintain

and eventually strengthen the capacity of all Operations units.

Human Resources: The Office faced problems in recruiting quality staff to respond to

the humanitarian crisis and work in an increasingly more complex environment. Major

challenges were faced in terms of identifying French speaking Child Protection and

Nutrition specialists, problems further exacerbated by the conversion of Bamako into a

non-family duty-station. Since 2012, a number of staff – national and international –

have been recruited, making good use overall of the Level 2 activation and the Fast

Track Recruitment Process (FTRP). Some delays were experienced more with regard to

national positions than with IPs. Certain key positions (e.g. Chief of Field Operations

and Chiefs of Field Offices) remain vacant, while other important posts (e.g. Deputy

Representative and Emergency Chief) will shortly become vacant. The Office would

benefit from having access to the FTRP until all important positions are filled, and from

improving the utilisation of this process for national staff. It is expected that the

restructuring of the Field Offices will raise several staffing issues, which should be

managed in a fair and transparent manner throughout.

Security: The main responsibility for security arrangements and set-up for the UN in

Mali lies with UNDSS, and this area is beyond the scope of the Real Time Evaluation.

Nonetheless, the consultants feel that there is room to increase the efficiency and

flexibility of security regulations, consequently increasing the capacity of UN agencies

to better implement their programmes.

Supply and Logistics: The supply and logistics component of the response has been

effective overall. Emergency supplies were shipped and delivered to Mali’s northern

regions even during the occupation, and no major pipeline disruptions took place.

Analysis of supply delivery in terms of quantities, both by region and by sector,

confirmed an even approach between the North and South and reasonable differences

between sectors. Support from the RO was limited due to adequate in-country capacity,

and focused on testing and applying new monitoring and tracking tools.

The discrepancies identified in all programme sectors between expenditure and

implementation rates indicate delays in supply distribution, once supplies were given to

the partners, as one of the contributing factors. It would be important for the Supply

section to work closely with all programme sections to review the details and

bottlenecks of post-delivery distribution.

Communication: The communication capacities of UNICEF MCO are in the process

of being upgraded to adequately face the challenges ahead. The process will be finalised

by the end of July, with new staff positions covered and report-lines revised. Currently,

an AWP is being agreed upon with the Government and will define the communication

strategy of UNICEF Mali for the 2013-2014 period. While communication largely

depends on the strategies and performance of the sections to build and share success

stories, it is also anchored to the priorities of the organisation and can become a potent

tool for advocacy.

An additional challenge will be the “occupation of the communication space” by

MINUSMA, which will require a sound positioning by UNICEF and a clear strategy

and adequate capacity to interact with the Mission in a complementary and constructive

way.

With respect to the RTE’s comments of the communication angle of the Sitrep, more

clear and concise information about implementation trends seems advisable in a product

targeting donors and NatComs and which conveys UNICEF’s successes and challenges

in the humanitarian response.

Coordination: In the context of Mali, where coordination structures with different

histories and backgrounds are progressively bridging their differences and considering

how to facilitate the transition from emergency to development, the UN family is in a

key position to accompany the process.

As of May 2013, most of the Financial and Technical Partners’ (PTF) thematic groups

(traditional development oriented coordination instruments) have resumed political and

technical dialogue with local authorities and their collaboration with the Clusters has

significantly improved during the past months. The latter regularly participate in the

activities of specific PTF technical working groups.

In the framework of the RTE, the evaluators carried out a small-scale survey of the

UNICEF-led Clusters and sub-Cluster. The results from the survey indicate that broadly

all Clusters are performing well and according to IASC standards. UNICEF

commitment to recruit long-term staff for both coordination and information

management (IM) positions also proved to be a sound approach.

Gender: The consultants found few instances of gender breakdown in the information

management systems at the country level, and no consolidation at the higher level, to

enable better planning. The gender awareness in project documents is weak and is not

apparent in implementation. This is probably an area to which UNICEF should also pay

specific attention in light of the upcoming complexities of MINUSMA deployment in

the country.

Accountability to beneficiaries: This RTE has found that community mechanisms seem

dysfunctional in the areas visited in Mali. Although challenging, the humanitarian crisis

could represent an opportunity to resume paying attention to the beneficiary dimension

with new approaches. Beneficiary participation is currently broadly absent at the

humanitarian coordination level, as well as when defining strategies of response.

UNICEF is potentially in a favourable position to address accountability to

beneficiaries. This is mentioned in the context of sections of the report dealing with

comments on 3PM and C4D, and specific recommendations are proposed in this regard.

Conclusions

The RTE was able to verify the positive reputation gained by UNICEF vis-à-vis

humanitarian partners thanks to the role played in the initial phases of the humanitarian

response in 2012, as well as due to its capacity to raise awareness on humanitarian

challenges and principles. This strong role in the humanitarian response seems partially

side-lined at present. In light of existing humanitarian challenges, the possibility of

worst-case scenarios, and issues and opportunities created by the deployment of

MINUSMA, it seems that there remains a need to address the issue of promoting the

high profile of UNICEF within the emergency context once again.

The process of switching from development to emergency in Mali is built on qualified

and dynamic technical staff, a well-articulated country strategy, commendable

interaction with the Humanitarian Country Team (HCT) and Government counterparts,

and strong support from the RO. It would appear, however, that the Office is facing an

overall challenge to the translation of technical capacity and adequate expenditure into

results on the ground. The impression given is that the last step of the response - the

materialization of results - seems weak in some cases. This suggests that certain internal

processes might not be adequate in terms of emergency response, and thus UNICEF

sections should look at identifying ways to overcome bottlenecks that, while not critical

for a development or institutional response, affect the delivery of timely humanitarian

aid.

There are high expectations among donors, NGOs and Governmental counterparts for

UNICEF leadership in key areas of the strategic approach to transition from emergency

to development. UNICEF Mali has already put in place many elements to respond to

this challenge and take advantage of this window of opportunity. The organisation has

the potential to become a driving force behind some of the structural changes this

country, its people, and particularly its children, sorely need.

Clustering of Recommendations

In the following table, we propose a thematic clustered arrangement of the

recommendations from the main recommendation matrix at the end of this report (page

70). We include the cumulative average score, based on importance and urgency, of

each cluster as a priority criterion, and list the clustered recommendations accordingly.

Clustered Recommendations Findings Scoring & Comments

Implementation Rates (Recs 17, 18, 53) All programme sections in collaboration with Supply section to revise discrepancies between expenditure and implementation rates, looking at processes and structural delays of the

supply delivery systems of implementing partners and Government counterparts, and at service provision of private contractors.

- Limited effectiveness of WASH in Nut strategy and programme (Para 109) - Discrepancies between expenditure and implementation rates (Paras 111, 74, 81, 99,

128) - Supplies distributed confirm the low progress towards targets in comparison to expenditure rates (Para 200)

(8,5+8,5+9) / 3 = 8,66 The RTE team considers this as one of the most important recommendations,

due to the reputational risk for UNICEF if under-performing.

Field Offices (Recs 4, 15, 51) MCO, with support from RO, to speed-up re-shaping of Field Office structure, taking into consideration HR and staff well-being issues. Focus should be on strengthening existing FOs, revising reporting lines, responsibilities and accountabilities, and speeding up the opening of FOs in the North to expand programme implementation and monitoring

capacities.

- Importance of decentralisation and increased capacity at field level, requiring the speeding-up of Field Offices’ restructuring process (Para 54) - Limited capacity of UNICEF staff (indicated for Nutrition, but valid for all sectors) at FO level for planning, monitoring and information management (Para 100) - Importance of taking into consideration specific staff well-being issues and problems

to ensure successful restructuring of Field Offices (Para 188)

(9,5+8+8) / 3 = 8,5 This aims at improving UNICEF’s capacity to deliver. While it is urgent to address, it will take some time to be effective. It links with Staffing and HR

recommendations.

Education – Back-to-School campaign (Rec 23) Education section, in coordination with senior managers, Communications and Supply section to urgently assess the suitability and feasibility of a context-specific “Back to School” campaign, shaped around the complexities and needs of the country, and eventually utilise the campaign for communications purposes and to raise the profile of UNICEF as

leading agency for Education.

- “Back-to-School” campaign: an opportunity to advocate for and encourage enrolment, quality of education, and girls’ scholarisation. Discussions about the opportunity to carry out this campaign, or a lower-scale form of it, are on-going in the MCO. Concerns on capacity to carry it out seem relevant. (Para 126)

8,5 Extremely urgent if UNICEF intends to build (reputation, leadership, etc.) from such a campaign.

M&E for Health and Nutrition (Recs 7, 12) Health and Nutrition sections, in coordination with M&E section and respective partners, to establish a regular information system based on sound indicators aiming at measuring utilisation of services, coverage and quality of performance in priority areas (both North and South), and to enhance supervision capacities (UNICEF FOs, NGOs, local administrations) at field level.

- Weak information management system for the provision of health services country-wide. Unclear, incorrect targets and indicators proposed by UNICEF to monitor its performance. (Paras 73, 74, 78, 81) - Poor quality of information management system for SAM programme at CSCOM level. Insufficient indicators for UNICEF programme performance monitoring (Paras 89, 97)

(7,5+9) / 2 = 8,25 This is an on-going process, needs immediate attention to find solutions to the weaknesses identified.

Strategy (2, 20, 24, 29) MCO and all programme sections, with support from the RO, to undertake a strategic planning exercise to develop/refine clear and comprehensive multi-sector and sector-specific strategies, as well as a resource-mobilisation strategy, to bridge the transition from emergency to development, implement them, and be vocal about them both internally and externally.

- Window of opportunity for UNICEF to become leading agency in the emergency-to-development transitional agenda. (Paras 55, 56) - Education section’s need and opportunity to finalise the revision of its strategy. (Paras 118, 123) - Chronic weaknesses of CP section and patchiness of its response calling for an urgent development of a consistent strategy, aiming at addressing numerous contextual challenges. (Paras 130, 131, 132, 137) - Challenges of C4D and its partners in implementing activities supporting programme

sectors at the pace required by the emergency response (Paras 146, 147, 149)

(7,5+7,5+9,5+8,5) / 4 = 8,25 Perceived as very relevant, addresses next steps in the response. A strategic recommendation to be addressed now if UNICEF wants to be ready for the next phase. Partially addressed through the new recently adopted AWP

Partnerships (Recs 11, 16, 21 30) All programme sections to review partnerships networks (both international and national NGOs, as well as other implementing counterparts), aiming at expanding number and scope/coverage of the PCAs/SSAs, and consequently of the response, and at improving their monitoring and reporting capacities.

- Limited coverage of nutritional response based also on limited numbers and capacities of implementing partners (Paras 92, 94, 98). - Similar situation, starting to be addressed, for the WASH programme (Paras 105, 109, 113). - Limited number of active PCAs for Education, and limited capacities of Governmental counterparts (Paras 119, 120) - Limited number and capacity of C4D partners to work in a more challenging

humanitarian context (Paras 145, 146, 147, 149).

(9,5+9+7,5+7) / 4 = 8,25 This might be a complex problem, the RTE believes it could be addressed more proactively from UNICEF side, approaching and understanding partners concerns and challenges.

Child Protection (Recs 24, 25, 26, 27) MCO senior managers, RO and HQ to strongly support the activities of the Child Protection section, including in the design of its strategy, expansion of its partnership network and collaborations with the CP sub-Cluster, hiring of key staff, and interaction with MINUSMA.

- Challenges faced by the CP section throughout 2012-13 and dramatic increase of number and complexity of issues to be addressed by the CP section (Paras 129, 130, 136, 141). - Importance of challenges faced by CP section and consequent reputational risk for UNICEF if these are not addressed consistently (Paras 131, 132). - Relevance of work with the CP sub-Cluster as the main forum for technical expertise and joint advocacy efforts (Para 138).

(9,5+8+7,5+7,5) = 8,12 Critical issue, it needs immediate attention given the fast-developing protection scenario.

- CP section staffing weaknesses and gaps, in parallel to rapidly expanding sector-specific responsibilities and challenges (Paras 129, 132, 137).

HIV (Rec 9)

Health section to address HIV related gaps, assuring testing at URENI level and functional PMCTC in the North. Partners should be aware of the protocol to apply for patients in ARV that discontinued treatment.

- HIV testing at nutritional centre level apparently not occurring. Protocol for patients in ARV therapy not widely known and adopted (Para 80)

8

The HIV in Nutrition programme is only one aspect of weaknesses of HIV management in Mali, an issue beyond this RTE’s scope

Clusters (Rec 58) MCO to consolidate already decentralised clusters under UNICEF responsibility with adequate leadership and planning and monitoring capacities, and advocate for the activation of clusters in Timbuktu and Gao.

- Weaknesses of existing Clusters decentralised at FO level, and importance of activation of decentralised Clusters in Gao and Timbuktu (Para 219).

8 Complex problem of leadership and capacity at decentralised level. Related with recommendation 2 on Field

Offices.

Communications & Advocacy (Recs 54, 55) Communications section, with the support of senior manager and RO, to finalise the development of MCO communications and advocacy strategy, including on interacting constructively with MINUSMA, and raising the profile of MCO in the emergency to development transitional agenda.

- New opportunities and challenges for a stronger Communications team in light of humanitarian crisis and MINUSMA deployment. Importance of responsibility and role played by senior management in making effective use of Communications team and its strategy in Mali context (Paras 204, 206, 207)

(8+7,5) / 2 = 7,75 Partially already being addressed by MCO Communications team

Humanitarian Performance Monitoring (Recs 36, 37, 38, 39, 40, 41)

HPM teams, locally and regionally, in coordination with programme sections, should further refine and adapt HPM targets and indicators for a sound monitoring of the humanitarian response. Specifically, they should: i) clarify operational and aspirational targets; ii) avoid integrating different objectives in the same target/indicator; iii) utilise performance (cured SAM children) instead of input-oriented indicators (enrolled SAM children). HPM reports should include some analysis of trends and feature explanations.

- Generalised lack of clarity on HPM targets and indicators included in Sitrep format

(Para 158). - Confusion between aspirational and operational targets prevents clear understanding of trends (Paras 159, 162) - Targets composed by the sum of different categories of beneficiaries are confusing (Paras 160, 161) - Utilisation of unclear and innacurate HPM targets and indicators does not allow trends to be understood t (Para 165) - Better performance indicators exist to monitor specific programmatic responses (Para

163) - Inclusion of HPM in PCA is positive but need some revision (Para 164)

(7+8+8+8+7,5+7) / 6 = 7,58

The RTE feels that the HPM team is in the position to address the identified challenges. This Rec relates to number 4 above, on M&E for Health and Nutrition.

Staffing (Recs 1, 22, 27) MCO to speed up recruitment process for remaining key positions (IP and National) at Bamako and Field Office level. Special focus should be put on senior managers also responsible for strategic planning and policy interactions (Deputy representative and Emergency Chief). Develop strategy for the identification of specific profiles and set

priorities in collaboration with RO and FTRP unit in DHR.

- Need for MCO to maintain strong capacity at senior management level to address key humanitarian policy issues and challenges. (Para 46). - Education section’s opportunity to increase its profile in interactions with MoE and with coordination structures for both humanitarian and development work. (Paras 123, 124)

- CP section staffing weaknesses and gaps, in parallel to rapidly expanding sector-specific responsibilities and challenges. (Paras 129, 132, 137)

(9+6+7,5) / 3 = 7,5 Critical to maintain the leadership role of UNICEF and further increase its capacities.

Human Resources (Recs 49, 50) MCO and RO to advocate with HQ for the extension of the L2 or the use of the FTRP, and to ensure proper application of the recruitment process also for national staff.

- Urgent need to fill up key positions in the Bamako office and in the Field Offices (Paras 46,185) - L2 authorised use of FTRP also for national staff, but MCO and RO did not do it effectively. (para 186)

(8+7) / 2 = 7,5 Process-oriented and directly linked to the previous recommendation

Programme Integration (Recs 13, 33, 34, 35)

MCO senior managers and all programme sections to enhance capacities and opportunities for programme integration at all levels by strengthening integration in programme planning and coordination, increasing capacities at Field Office level, and testing and adopting new practical tools (WASH trip report format). Special urgent attention should be given to the integration of other sections’ components (WASH in Nut, child stimulation, HIV testing) in the nutrition programme.

- Poor integration of other sectors in the nutritional response (Paras 98, 153)

- Limited capacities for programme integration at FO level (Para 154) - Importance of role to be played by senior managers to strengthen programme integration at all levels of UNICEF programme (Paras 153, 154) - Availability of different trip report formats (M&E, WASH) to optimise multi-sector visits, reporting and follow up (Para 155)

(8+7+7+7) / 4 = 7,25

A recurrent issue in UNICEF response planning. Some progress is evident, but should be strengthened

Security (Rec 52) MCO to advocate with UNDSS for one security office in each FO location, authorised to

issue security clearances, and attuned to the reality of the area covered. The criticality exercise, defining critical outputs or activities by the whole UN country team, should be used

- Security arrangements and set-up at FO level hinder flexibility of movement and

consequently planning and monitoring capacities of UN staff (Paras 189, 190, 191)

7 Security constraints are affecting

UNICEF operational and monitoring capacity. This is to be addressed from a

as baseline document for UNDSS staff to find acceptable solutions to implement key field activities.

practical perspective, on the assumption that UNICEF can advocate with UNDSS

Leadership (Recs 3, 59) MCO, in collaboration with RO, to identify ways to strengthen its leadership at general programme and sector level, with the aim of facilitating and taking a leading role for the implementation of a transitional agenda from emergency to development.

- A window of opportunity has been created by several concomitant factors for UNICEF to become a key leading and coordinating actor in the development and implementation of the emergency-to-development agenda. Its role of Cluster lead for four sectors, and its increasing role in the PTF working groups and other fora reinforce the foundations to take this leading role (Paras 49, 50, 51, 52, 55, 56, 212, 213)

(7+7) / 2 = 7 Critical aspect for the next phase, should be addressed from now on to assure adequate UNICEF traction in the upcoming processes.

Community Participation (Recs 6, 14, 19, 31, 32, 43) All programme sections, in close coordination with C4D section, to look at innovative ways to consistently integrate C4D initiatives and approaches in their strategies, to expand

community level outreach, participation and feedback (accountability to beneficiaries). Senior managers and RO should encourage and support this process, and ensure funds are

made available.

- Wide-spread weaknesses of the role played by communities in UNICEF planning and monitoring activities among all sectors (Paras 77, 94, 98, 110, 144) - Although funding for C4D is planned for, when there are shortages, the related

activities are the first to be neglected (Para 143) - Accountability to beneficiaries is a major gap in the whole humanitarian response in Mali. UNICEF has the expertise and the favourable position to start filling this gap (Paras 151, 173)

(6,5+7+7+7+7,5+7) / 6 = 7 This is critical to assure aspects of accountability to beneficiaries in

humanitarian action. Weaknesses of community aspects of the programmes need to be addressed as well in the context of each programme with C4D

1. Third Party Monitoring (3PM) (Recs 8, 42, 43) All programme sections and their respective Field Office staff to enhance and fine-tune engagement with 3PM partners, in order to streamline their information collection system

into the programme cycle of the sections. By involving also C4D, programme sections and 3PM partners to articulate accountability to beneficiaries (AAP) approach and test a beneficiary satisfaction tool.

- Collaborations between 3PM partners and sectors (not only Health) have improved considerably since 2012; there is still room to further optimise this collaboration and integrate 3PM feedback into work plans (Para 74, 170, 171, 172)

-3PM partners’ work offers an opportunity to expand UNICEF work on accountability to beneficiaries (Para 173)

(6,5+7+7) / 3 = 6,83 Related with previous Rec., 3PM needs specific attention to be incorporated in

the programme cycle, but has potential as well for AAP (especially PDM)

Gender-based Violence (GBV) (Rec 28) Child Protection section, in collaboration with CP sub-Cluster and UNFPA, to review status of implementation and coordination of GVB initiatives in the country, with the aim of re-starting operations of the GBV sub-Cluster and developing a joint strategy with relevant actors. Other sections, under guidance of CP section, should start including GBV issues and

activities in their respective emergency responses.

- Limited initiatives for GBV and of limited quality among UNICEF and its partners, and a dormant GBV sub-Cluster are reasons for concern (Para 139)

6,5 GBV is a UNICEF responsibility in emergency, is included in the CP CCCs, and it is in line with the latest Global CP in Emergency Guidelines, published

by CP working group.

Long-term Advocacy (Recs 5, 10, 56) MCO, and especially its Health section, to advocate for access to health services for women and children, revitalising the working group to propose specific recommendations. With support from RO, MCO should build an integrated institutional position towards a problem that is based on inequities affecting women and children.

- Disparities created by the detrimental effects of the Bamako initiative underlined the rationale for free access to basic health and nutrition services. International community not taking advantage of the opportunity created by the humanitarian crisis to advocate for this. Potential important role for UNICEF to play, developing a sound advocacy and communication strategy (Paras 61, 62, 63, 64, 68, 85, 208)

(6+6+6,5) / 3 = 6,16 Equity issues and access challenges are beyond the humanitarian response, but UNICEF should take advantage of the context in order to increase advocacy

Sitrep (Recs 44, 45, 46, 57)

MCO, RO Emergency Unit and EMOPS to consider revising structure and types of Sitrep documents based on target audiences and utilisation: i) a main document with communications angle and advocacy potential, targeting donors and NatComs; ii) a quarterly HPM-focused document for technical audiences (RO, EMOPS, M&E staff); and iii) a one-page brief distilled by EMOPS for the senior managers in NYHQ.

- Targets and indicators included in the Sitrep are generally unclear and do not offer

the possibility to evaluate trends and bottlenecks (Paras 176, 177) - In light of the different audiences for the information included in the Sitrep, documents with different characteristics should be produced without excessively increasing the workload for those involved (Para 179) - A Sitrep format with a specific communication angle and synthetic and clear technical information targeting donors and NatComs should be considered (Paras 178, 209)

(7,5+6+5+6) / 4 = 6,12

Mali could be a pilot for a new reporting, monitoring and communication Sitrep model.

Gender (Rec 60) All sections in MCO to review and assure inclusion of gender in their planning tools and in the implementation of their programmes. The office might look into having a gender specialist assisting in this exercise.

Limited presence of gender issues and perspectives in planning documents and programme implementation (paras 221, 222).

6

RO Operations (Rec 47) RO senior management to advocate with HQ (DHR, EMOPS) for the Operations team in the RO to have deployable staff to directly support COs facing new emergencies.

Limited capacities of the RO Operations section to deploy staff in support of new crises (Para 181)

5

Level 2 issues (Rec 4 bis) EMOPS to assess the suitability of including in the L2 and L3 standard operating procedures (SOP) a link to the emergency PBR approach,

Option to use Emergency PBR should be offered/reminded to COs for which an L2 response is activated (Para 56.2)

5

18

CONTEXT AND BACKGROUND

1. Mali is one of the poorest countries in the world (ranking 182 out of 186 in the

HDI). Is a landlocked state of an area of 1,2 million sq Km (over twice the area of

France) with 65% of its territory covered by desert. Mali has a population of around

15 million people, with an annual demographic growth rate of 3,6%, which strains

further the limited available resources. Most of the population lives in the southern

regions, while in the three regions in the North lives only less than 10% of the

population.

2. After being considered for a long time a success story of democracy in West Africa,

where donors were eager to support the Government and its development agenda,

the political crisis in the north acted as a “wake-up call”. The humanitarian crisis

triggered by the occupation of the northern regions, the following coup d’état and

the consequent displacement of hundreds of thousand inside and outside the

country, opened the lid on, and exacerbated a reality characterised by poor

infrastructures and services, high rates of poverty and exclusion, low development

indicators, and recurrent ethnic and political tensions.

3. The attempted offensive to the South by the rebel forces, supported by jihadist

groups in January 2013, prompted the intervention of the French army (Operation

Serval), which helped to secure most of the northern regions. Around 2,8 million

people are estimated as directly affected by the conflict, 353.000 IDPs live mainly

with host families (and only a few hundred in camps), and 174.0001 refugees are

displaced to neighbouring countries (Burkina Faso, Niger, Mauritania). It should be

noted that the displacement waves triggered by the two main violent episodes (the

rebellion of the North and the takeover by French and Malian troops) affect different

groups, depending on affiliation with and support to the various players.

4. The beginning of a transitional phase, aiming at new elections in July 2013, and the

restoration of the control of most of the territories of the North, has created the basis

for some stabilization. Following these events, the Security Council authorized the

deployment of the United Nations Integrated Mission for the Stabilisation of Mali

(MINUSMA), 12,600 strong, and planned to arrive in July 2013.

5. As a consequence of the events of 2012, most donors put on hold development

projects in the country, stopping budgetary support to the Government of Mali

(GoM), and channelling basic support to social services through UN and NGOs

present in the country. The recent constitution of a transitional Government and the

calling for elections, led developmental actors to come back to some extent. By mid-

2012 though, the humanitarian community arrived in the country, addressing

pressing needs in the North, as well as the consequences of the deterioration of basic

services in the rest of the country (not always related to the crisis). The humanitarian

Cluster system was rolled out, and a first CAP was launched in September 2012

(requesting 214 USD million, of which 151 or 70% were contributed). This was

followed by the CAP 2013, requesting a total of 410 USD million, currently in the

Mid-year Review (MYR) process, and that has received so far 135 USD million

(±33%).

1 OCHA Situation Report, 5 July 2013 figures.

19

6. The situation in Mali during the period covered by this Real Time Evaluation (RTE)

(January to May 2013) is defined as a complex emergency. While only some limited

armed activity remains (at the time of writing) in the region of Kidal, the overall

humanitarian situation is worrisome, adding to the rampart nutrition crisis, which is

estimated will affect approximately 660.0002 children with acute malnutrition. The

humanitarian consequences are a mix of effects from the conflict in the North and

from an extremely fragile social situation for the rest of the country, exacerbated by

rampant poverty, difficult access to services for the poor and widespread corruption

and mismanagement.

7. The institutional instability advises to be cautious regarding possible future

scenarios. While there are hopes for positive developments, with a progressive

stabilization of the North and the return of refugees and IDPs, most involved actors

are aware of the concrete chance of worse scenarios, and work on contingency and

preparedness planning is ongoing.

8. Access to the North has improved, although many agencies are still hindered by

security restrictions, and some return movements of refugees and IDPs have been

reported. It is becoming crucial to better understand the dynamics of the affected

population and the conditions for return that IDPs and refugees would require, while

assuring security and not creating expectations (pull factors). The deployment of

MINUSMA is providing additional complexities to issues of humanitarian space,

independence and neutrality of humanitarian action. The Mission’s responsibilities

with regard to peace-building might on the other hand represent an opportunity.

9. The following humanitarian figures are proposed in the CAP 2013:

2,5 million people in need of health services (500.000 children under five)

1,5 million exposed to risk of cholera outbreaks

670.000 children with compromised access to education

2 million people in situation of food insecurity

210.000 expected cases of severe acute malnutrition (SAM)

10. In addition, the conflict has triggered the need to assure the necessary protection

mechanisms, especially for children, and to enhance the provisions to prevent and

deal with cases of violence. Protection needs are based on the following numbers

and issues:

100.000 children exposed to UXO and other war-related remnant dangers

Recruitment of young people in armed groups

Family separation

Gender-based violence, including sexual violence and abuse (GBSV)

Neglect and psychosocial distress

11. UNICEF was already present in the country as an important development partner for

the Government. Its presence facilitated the reaction to the deteriorating nutritional

situation of 2011-12 and to the challenges triggered by the more recent political

turmoil. Through the 2013 HAC appeal, UNICEF is requesting for its programs

2 The figure of 660,000 indicates GAM children (210,000 SAM, 450,000 MAM) as per various

documents, including the Mid-Year Review of the CAP and latest UNICEF Mali Sitreps.

20

81M USD3 (this funding is included in the global Mali’s CAP mentioned above).

UNICEF has the responsibility of leading two of the Clusters (Nutrition, WASH)

and the Child Protection sub-Cluster; it is also the co-chair with Save the Children

for the Education Cluster. UNICEF also participates in the Health, Logistics and

other Clusters.

12. Following the activation in early 2012 of an “informal” Level 2 emergency response

for the nutritional crisis, which was phased out between September and December

2012, the Mali conflict and consequent complex crisis was officially declared again

a Level 2 emergency by the UNICEF Executive Director on 28th

January 2013. This

designation has been further extended till September 2013, when it will be reviewed

again. This implies additional support to the CO from other parts of the

Organization, (HQ, RO), and raises specific accountabilities.

13. UNICEF’s response aims to ensure that the benchmarks set out in the Core

Commitments for Children (CCCs) in Humanitarian Action are met or exceeded in

Mali. UNICEF’s contribution towards the CCC benchmarks varies from sector to

sector, and is articulated through interventions ranging from direct service delivery,

capacity development initiatives, coordination support and advocacy. The UNICEF

2013 humanitarian response plan aims to ensure a fully integrated approach to the

Mali Crisis.

RATIONALE, OBJECTIVE and SCOPE of the RTE

Rationale

14. Programming in humanitarian crises is high-risk, and requires close monitoring and

evaluation. The RTE of the UNICEF response to the Mali crisis will feed into

results-based management for both UNICEF programmes and Clusters under

UNICEF’s responsibility. The urgency for this UNICEF-focused RTE to support

operational management means a rapid, light exercise is necessary. The aim of the

RTE will therefore be to produce feasible and implementable recommendations to

improve the current UNICEF planning and response in Mali.

Objective

15. As per the Terms of Reference, the main objective of the RTE is: To provide

recommendations to enhance the scale-up of UNICEF planning and response for

the Mali humanitarian crisis.

16. The following evaluation criteria are addressed in the framework of the above

objective: i) Effectiveness and coverage, ii) Efficiency, iii) Appropriateness &

Relevance, iv) Adequacy, v) Coordination.

17. The RTE team addresses the articulation of this report in order to fulfil the

formulated objective, and particular attention paid to extract coherent and

implementable recommendations, in order to maximize the potential for utilization.

3 This figure is subject to revision. By the time this report has been finalized, UNICEF is looking at a

ceiling of about 92M for the 2013 HAC appeal

21

Scope

Timeframe

18. This RTE focuses on the Mali CO’s humanitarian response between January and

May 2013, following the launch of the Humanitarian Action for Children (HAC) on

23 January. It aims at taking stock of progress, learning what has or has not worked

well, identify areas that require strengthening or adjusting, and map out any gaps in

UNICEF’s response.

Geographical scope

19. This RTE assesses the response in all the territory of Mali, taking into account the

different nature of the elements of the crisis affecting different areas of the country.

Questions of access and security, as well as of humanitarian space are relevant in the

North and affect the delivery of humanitarian aid. Most of the malnutrition caseload

is in the South, affected also by population movements and the pressure of the IDPs

on the already fragile coping mechanisms of the more vulnerable. The response

involving assistance to refugees in neighbouring countries is excluded from this

RTE. Security and transport constraints prevented field visits to the North, but the

scope of the evaluation remains the same, and measures have been taken to assure

enough indirect evidence as to sustain conclusions.

Programmatic areas

20. This RTE focuses on UNICEF key programmatic areas, and on how the response is

in line with meeting the CCCs, specifically for Nutrition, WASH, Health,

HIV/AIDS, Child Protection, Education, Communication for Development (C4D).

Additionally, the RTE looks at how the existing programmes have shifted into

emergency mode, at the changes adopted, at the support provided by the RO, and at

the mobilization of resources (financial, human and supplies).

21. Special attention is paid to the data-collection, humanitarian indicators, monitoring

and reporting systems established by UNICEF in collaboration with its partners to

keep track of the implementation of the different programmatic responses.

Disaggregation of data, as well as coverage of marginalized areas and categories of

beneficiaries is investigated.

22. The consultants assessed if the design and formulation of the programme

components of the response are adequate to the existing needs and scenario at the

time of the RTE. This would allow confirming the strategy or suggesting changes

accordingly.

23. The RTE team has as well been assessing how UNICEF has been able to build from

its privileged position of long-term partner of the Government in the country, and

from its access to the beneficiaries involved in its core activities. Gender and

HIV/AIDS issues are checked against benchmarks of humanitarian response.

Operations areas

24. The efficiency of the response is directly linked to the capacity to mobilize

resources (human, financial, supplies) and speed up administrative processes. It is

critical to assess how operations sections (supplies, admin/finance and human

resources) have been able to cope with the shift into emergency mode and the need

to find solutions to adequately support the response effort timely and efficiently.

22

25. The activation of a Level 2 emergency response for Mali in January 2013 should

have offered the opportunity to speed up processes and – inter alia – utilise fast-

track channels for supplies and human resources. To what extent this has happened

is as well addressed in the framework of the RTE.

METHODOLOGY

26. As per Real Time Evaluation standards, the methodological approach was light and

participative. The evaluation team applied a number of techniques and tools, and a

methodology based on both qualitative and quantitative approaches. Data was

collected in a variety of direct and indirect ways, and preliminary findings have been

triangulated and validated in order to strengthen the basis of evidence. The

evaluation criteria for the exercise (paragraph 16, above) have been used

systematically to frame the findings, and are embedded in the text of specific

programmatic chapters.

27. The evaluation mission took place between the 2 and the 23 of June 2013. Several

data collection tools have been utilised, aiming at expanding the range of

information gathered, as well as analysis and triangulation possibilities. In light of

the “real time” nature of the exercise and the challenges faced in the Malian current

context (security, access, logistics, see also Limitation chapter below), the main

focus has been on qualitative data. KIIs, observations tools and group discussions,

even if not formal FGD, have been the main source of data, and efforts have been

exercised to assure reliability through cross-checking and triangulation.

28. A simple self-administered, voluntary and confidential questionnaire was distributed

to all Cluster members, and got 35 responses. Other quantitative information has

been analyzed regarding shipment of supplies to different areas of the country, as

well as administrative data on management of contracts and related to program

performance (Health Management Information Systems, Nutrition program data,

population-based data, and quantitative data related with the HPM approach).

29. Quality of data has been uneven sometimes, especially when obtained from

Government of Mali sources, but when concerns were identified triangulation and

cross-checking was employed for validation. Data on UNICEF’s disbursement and

liquidation has been considered of high quality and the source (VISION) very

consistent.

Informal Desk Review

30. The team undertook a desk review of approximately 400 background documents,

provided by UNICEF or its partners, and in some cases posted in a shared Drop-

Box. Different categories of documents have been reviewed, providing background

information, which eventually contributed to the interviews, analysis and

triangulation phases of the exercise. A list of key reference documents and of the

categories of materials reviewed in included in the Annexes.

Key Informant Interviews (KII)

31. The consultants carried out (either jointly or separately) a number of KIIs at

different levels. In order to abide by ethical considerations, all interviewees

23

(UNICEF and external) were offered the opportunity to decline their participation

without consequences, and informed of the confidentiality of the interviews. In the

Regional Office in Dakar, the team interviewed programme and operations staff

directly or indirectly involved with the humanitarian response in Mali, as well as

senior managers with broad responsibilities and involved with the activation of the

Level 2 approach. These KIIs provided information about the dynamics of the

response, and the role played by the RO in its planning and implementation. A total

of 22 KIIs (13 males, 9 females) took place in the Regional Office. Two EMOPS

HQ staff, both males, were also interviewed on Skype.

32. In Mali, 32 KIIs (18 males, 14 females) were carried out with UNICEF staff in the

Bamako Office and in the Mopti Field Office. In addition, the evaluation team met

with 34 staff (27 males, 7 females) from other UN Agencies, implementing partner

organisations (NGOs, CBOs, donor agencies and Embassies, and Governmental

departments). Due to logistic, time and security constraints, the consultants had only

limited chances to interact with beneficiaries at field level.

33. In total, the evaluation team interviewed 90 people (30 females, 33%, and 60 males,

67%). A complete list of interviewees is included in the Annexes of this report.

Findings from different types of KIIs have been analysed and clustered to strengthen

evidence of specific findings and trends, and utilised during the analysis and

triangulation phase.

Field visits

34. The RTE team had the opportunity to visit some locations and projects, all in the

southern regions of the country, to assess different aspects of the UNICEF response.

Field visits took six days of the RTE mission and allowed to visit 6 nutrition and

health centres, 4 schools, and 4 child-friendly spaces (CFS) in Kangaba (Koulikoro

region) and Mopti. A daily trip to Nampala (northern Segou region), offered the

opportunity to visit a planned water intervention, assess the situation related to water

supply, education sector and health and nutrition, and allowed to meet some of the

local administration involved. Albeit limited, these visits offered the chance to gain

some understanding of UNICEF’s and its partners’ operations in the field, and meet

and interview key counterparts. Active observation was carried out during field

visits and while in UNICEF premises (MCO and FO). Observation forms were used

as appropriate at some project-sites. Some voluntary informal discussion groups

were carried out with groups of social workers in Mopti (2), and with community

leaders in Nampala (1), but proper FGD did not happen. All participants were

informed about the confidentiality of the approach.

Mini Cluster Survey

35. With the endorsement of the MCO senior managers, approval from OCHA, and

support from key Cluster staff, the consultants organised a small-scale survey to

gauge the efficiency and effectiveness of UNICEF-chaired Clusters (WASH,

Education – as co-chair –, and Nutrition) and sub-Cluster (Child Protection). Cluster

members were asked to voluntarily fill up a simple self-administered and

confidential questionnaire focusing on the performance of UNICEF as chair or co-

chair, against IASC benchmarks and Cluster-lead ToR. The analysis, both

qualitative and quantitative, of the 35 questionnaires received, together with

information obtained from a similar exercise carried out by the WASH Cluster to

24

investigate its own performance, helped identifying strengths and weaknesses in the

different Clusters, as well as problems, issues and bottlenecks.

Analysis, Debriefing and Reporting

36. Information collected through the various tools has been analysed and compared to

identify trends, divergences and patterns. Specific findings and issues from the

different methodologies have been triangulated and further analysed to corroborate

and strengthen their evidence. The initial analysis took place already during the

mission in Mali, in order to identify preliminary findings, conclusions and

recommendations for a debriefing and discussion session in Bamako. A Power-Point

presentation was shown during the debriefing to MCO, RO and NYHQ staff

(EMOPS and Evaluation Office). Comments to the presentation and its preliminary

recommendations were provided to the evaluation team.

37. A draft report – structure agreed with the Management Group – was finalised soon

after the end of the mission. Comments to it provided by the expanded Reference

Group and programme and operations sections from WCARO and the MCO were

taken into consideration by the consultants in the elaboration of this final report. A

matrix has been built with the comments received and the actions taken by the

evaluation team regarding each comment.

Methodology Limitations

38. The team faced some challenges in the organisation and planning of the field visits.

Security constraints affected mobility around Mopti, and prevented the evaluators to

visit some projects in Konna. Security constraints and limited transport options did

not allow the consultants to visit any location in the North of the country (Timbuktu,

Gao or Kidal), where most of the humanitarian response is taking place and the

complexities of the political crisis are more evident. While these are expected

limitations in a RTE, more advanced planning would have allowed for a broader

coverage of project sites in more locations.

39. The consultants tried to compensate for these limitations by multiplying and fine-

tuning interviews in order to reach the necessary saturation of findings and indirect

evidence. Adequate triangulation allowed reaching “remote-built” and sustained

conclusions.

40. The RTE came across interesting data through the VISION system. With more time

available, a more in-depth analysis could have further strengthened some

conclusions. While the RTE favours field based information and evidence based on

external sources, VISION offers a potential excellent source of quantitative

information aiding triangulation and specific trend analysis. As a lesson learned for

future exercises, RTE consultants should require VISION-based information and

reports before the field visits, and office staff should provide support for additional

VISION-based analysis when required.

41. The quality and scarce availability of data has also been a limitation, especially from

Government service providers. Whenever possible, data from alternative sources

(mainly NGO partners) has been used to confirm or challenge some of the

information obtained. Still, this limited availability and quality of data affected the

adequate assessment of efficiency in the context of the RTE. Current reporting

25

systems (including those HPM-related) are still suffering from inaccuracies and lack

of timeliness, even if progress can be sometimes noted.

FINDINGS

General aspects

43. The humanitarian response to the crisis started slowly in 2012. All organisations

(UN and NGOs) faced a challenging shift of gear from development to emergency,

and many are still struggling. The roll out of Cluster system started in April 2012

and the CAP was launched by mid-year. Access to beneficiaries (IDPs and

population of the North) remains challenging. The level of poverty of the Mali

population (50% under poverty level) makes it difficult to single out specific needs

for crisis-affected people: IOM reports that 80% of IDPs have been contacted to

assess their needs, but up to 60% have not received any specific support4.

45. The humanitarian crisis is ongoing and its magnitude is quite consistent, with

353,455 IDPs (the majority of whom living with host families) and 174,3945

refugees mainly in Mauritania, Niger and Burkina Faso. These add to the

approximately 660,000 children affected by the nutritional crisis. As described in

this report, UNICEF has been overall successful in the immediate response to the

crisis in 2012, and specifically, donors and other actors have been praising

UNICEF’s capacities for humanitarian policy and advocacy, in a moment where

other organisations were still focused on development issues.

46. There is a strong feeling among actors interviewed in Mali that a positive scenario is

likely, and that chances for stabilisation and return of the majority of IDPs and

refugees are concrete6, in light of MINUSMA’s deployment and especially if the

4 IOM survey results, not published

5 Figures for both IDPs and refugees from OCHA Mali: Complex Emergency Situation Report No. 36 (as

of 5 July 2013)

6 “Returns of internally displaced people (IDPs) to the North have continued to increase since May.

According to UNHCR, 580 Malian refugees in neighbouring countries have also returned to the North

between 8 May and 16 June.” - OCHA Situation Report No 36, 5 July 2013

26

elections planned for July will be peaceful. Nonetheless, important work on worse

case scenarios is ongoing, and it is essential for UNICEF to maintain strong

capacities to respond to new crises and to keep playing a key role as advocate for

humanitarian principles and approaches, especially in the interactions with the

protection and military branches of MINUSMA7.

Recommendation

1) In light of the need of maintaining the acquired strong emergency response and

humanitarian advocacy capacities at senior manager level, especially in the event of

worse case scenarios, MCO should prioritise, aside other key positions, the recruitment

of a Deputy Representative versatile on emergency and transition, and a strong Chief of

Emergency.

47. Several donors, UN Agencies and NGOs stressed the idea that many years of

development in Mali greatly underperformed, and that the humanitarian crisis lifted

the veil on the real, chronic problems of the country. In this very complex

environment, several factors contribute to creating a window of opportunity to

recognise and address not only the immediate humanitarian needs, but also some of

the chronic weaknesses and the ethnic and political tensions affecting the country.

48. The Technical and Financial Partners (PTF) coordination structure developed by

donors about ten years ago has been in place throughout the crisis. The works of its

various working groups slowed down during 2012 though, and most donors

interrupted funding to Government for fear of lack of financial management

capacity. Many donors diverted part of this development funding to UN Agencies.

As shown below, UNICEF ORR budget was greatly increased by these

contributions.

7 An “Integrated Mission Support Cell”, composed by EMOPS, RO Emergency staff and MCO

Emergency staff, has been active and regularly meeting to discuss and strategise about the risks and

opportunities offered by the deployment of MINUSMA, including in the peace-building areas, and MCO

position vis-à-vis the mission.

2008 2009 2010 2011 2012 2013

RR 11,082,904 12,822,763 12,253,670 13,380,704 13,430,038 16,160,249

ORR 10,667,665 11,838,099 11,433,133 8,759,739 12,498,630 58,466,671

ORE 3,622,158 2,144,267 6,301,038 4,309,600 15,211,264 30,396,058

TOTAL 25,372,727 26,805,129 29,987,841 26,450,043 41,139,932 105,022,978

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000

USD

UNICEFMALI:FundsAllocated2008-2013

27

49. While the CAP is still relatively poorly funded (33% as per June 2013), the

exceptional ORR funding, spanning a few years into 2015, offers UNICEF some

budget flexibility to fund projects and initiatives linking the emergency to a sound

development agenda, addressing some of the chronic weaknesses of the country.

50. A “National Economic and Social Stabilisation Fund” was established in early 2012

by some PTFs to support priority transition activities and projects, and UNDP

provides trustee services for the Fund. Amongst others, contributions to the Fund

have been provided for Education, Social Protection and WASH. While it cannot

access funding through this structure, UNICEF’s participation to the Fund’s works

is an opportunity for leveraging and advocacy in the transition period.

51. By 2013, most of the PTF working groups were reactivated, and progressively

strengthened their collaboration with the Clusters to look at ways to bridge the gap

between emergency to development. Recognising the specific role of UNICEF and

its mandate, donors offered UNICEF the chair of the WASH working group, which

was accepted, and of the Education and Vocational Training working group, which,

as per June 2013, has not been accepted. Several donors and governmental

counterparts underlined to the evaluation team the expectation of seeing UNICEF

taking the lead in some of the sectors of its mandate, notably Education, Nutrition

and Child Protection.

52. A new PTF working group, the Commission for the Rehabilitation of Post-Conflict

Zones, has been created in April 2013 and is currently chaired by Germany. Its

ToRs indicate among the specific objectives the following: “Assurer la cohérence et

la complémentarité (passage de relais) entre l’aide humanitaire et l’aide au

développement”8. UNICEF is already an active player in this Commission, where it

has been represented so far by the Deputy Rep, and now by the Chief of Social

Policy. Also the Clusters are represented at this forum, by respective coordinators.

There are expectations for UNICEF to play an even bigger role in this Commission.

53. In addition to the above, it is worth noting that even OCHA is planning to open a

field office in Kayes – far from the core of the emergency in the North – specifically

to “explore” the humanitarian dimensions of the chronic crises affecting the country.

54. Recognising the need to address both emergency and development issues, and based

on a strategic assessment, UNICEF MCO has already started to re-shape the

structure of its Field Offices. Offices in Koulikoro and Segou will be closed, and

specific region needs will be covered from Bamako. New offices will be opened in

Sikasso, Gao, and most likely in Timbuktu. These last two would represent a return

to effective programme reach in the North, stopped by UNICEF in 1995. FOs in

Mopti and Kayes will be strengthened. There is a lot of pressure from donors, NGOs

and Government for UNICEF and other UN Agencies to open FOs in the north, in

light of the humanitarian needs and the progressive expansion of the humanitarian

space. It is expected that the latter will be further expanded by the deployment of

MINUSMA.

8 “To assure coherence and complementarity between humanitarian and development aid”

28

55. This context offers the organisation a window of opportunity to reshape its approach

and strategy and become an essential player in the future of Mali. Some RO senior

managers share this view and would ensure a strong support to MCO. UNICEF

MCO is already moving to re-shape its agenda, strategy and programme, including

with the extension of its Country Programme and the production of new Annual

Work Plans spanning the emergency-development transition. In addition to more

specific recommendations included in this report under sector-chapters, the

consultants would like to propose some more general recommendations to

strengthen and speed up this re-shaping process.

Recommendations

2) MCO, with support from RO, should undertake a strategic planning exercise to

develop/refine a clear and comprehensive multi-sector strategy to bridge the transition

from emergency to development. The strategy should include a resource mobilisation

component, to address the CAP underfunding and additional ORR fund-raising. MCO

to implement this multi-sector strategy credibly, and be vocal about it internally and

externally.

3) MCO, with support from RO, to review its current chairing and coordination

responsibilities, evaluate new ones, and identify best possible ways to play a leadership

role at both general programmatic and sector level.

4) MCO, with support from RO, to speed-up re-shaping of Field Office structure. Focus

should be on strengthening existing FOs, revising reporting lines, responsibilities and

accountabilities, and speeding up the opening of FOs in the north to expand programme

implementation capacities and counterbalance MINUSMA’s presence.

56.1. The activation of the Level 2 emergency response, "informally" first, in January

2012, to support the response to the nutritional crisis, and formally in January 2013, to

respond to the complex emergency, was overall beneficial for the MCO. It allowed to

speed up recruitment processes for international staff through the application of the Fast

Track Recruitment Process (FTRP), facilitated supply procurements at local and

international level and other financial transactions, and increased flexibility in the

process of establishing and extending PCAs with implementing partners and other

financial processes. The L2 also activated the Regional Emergency Management Team

(REMT)9.

56.2. In 2013, the CO had to get through a Programme Budget Review (PBR) process

to extend the Country Programme and revise several programmatic and operational

aspects of it, including the restructuring of the Field Offices and consistent changes in

the organisation chart. Senior managers interviewed underlined that the PBR process

was successfully supported by the RO, but suggested that the faster, more agile, existing

emergency PBR should have been applied in the case of Mali. This would have likely

made it easier and faster for MCOs to adapt to the sudden changes and scale-ups

required by the crisis.

Recommendation

4 bis) EMOPS to assess the suitability of including in the L2 and L3 standard operating

9 The REMT is chaired by the Regional Director and composed by the MCO Representative, emergency

staff of the RO and MCO, and staff from EMOPS and relevant HQ Divisions. The level of the

consultations is normally high, and in the case of Mali a lower level technical REMT is about to be tested.

This seems a positive initiative, whose effectiveness and efficiency should be reviewed in the future.

29

procedures (SOP) a link to the emergency PBR approach, so that COs for which the

Level 2 response is activated would be aware of it.

57. Through interviews with MCO staff, the RTE found that support from the RO to

MCO throughout the humanitarian response has been overall effective and

appreciated. Direct assistance and back-up from the RO Emergency Section – in one

instance jointly with EMOPS –, as well as from the Regional Director a.i. has been

deemed very important by MCO senior managers. Programme and Operations

sections in Mali benefited by different degrees of assistance. While in some cases

(described in respective specific chapters) personality issues influenced the

effectiveness of the support provided, the overall approach was successful.

PROGRAMMATIC AREAS

HEALTH AND HIV/AIDS

Health sector situation

58. When the crisis broke up in 2102, the health situation in Mali was among the worst

in the region. The country has one of the highest under five mortality rates in the

world (176/10,000, in comparison to the region’s average of 107/10,000). Maternal

mortality is around 540/100.000 live births, higher than the regional average as well,

as it is malaria incidence and mortality. Main causes of mortality among under 5 are

respiratory infections (20% of deaths), malaria (16%), and diarrhoea (14%).

Additionally, with a fertility index of over 6 children per woman, needs for

antenatal, postnatal and obstetric care are huge.

59. With a HIV prevalence of around 1,1%10

, the risks associated with HIV

transmission are largely confined to specific groups, as sex workers and army

personnel, and to mother to child transmission. The coverage of ARV is of 50% of

estimated cases (around 29.000 patients are under ARV), and the prevention of

mother to child transmission reaches less that 40%11

of estimated cases. The conflict

in the North has caused around 600 patients to suspend treatment while moving to

other areas of the country, of these 300 have been localized and were able to resume

treatment.

61. The Bamako Initiative adopted in 1988 has favoured the for-profit service providers,

a strong pharmaceutical sector (the basis for the cost recovery schemes) and a

primary health care (delivered at self sustained community health centres, CSCOM)

based on contributions by user. The initial potential benefits of the Initiative have

been surpassed by the detrimental effects in terms of access to services. Available

research demonstrates some pernicious effects, especially in low frequentation

figures and poor public health results. It is sometimes difficult to disentangle these

effects from additional contributing factors, as quality of services offered,

willingness versus inability to pay and geographical and cultural barriers. In Mali,

additionally, an inefficient administrative division of the country further impairs a

rational distribution of services and its access.

10 Among adults of age 15 to 49 years, higher for women (1,5%), source:

http://apps.who.int/gho/data/view.country.13100, last visited June 27 11 http://apps.who.int/gho/data/view.country.13100, last visited June 26

30

63. Nevertheless, policy-makers within donor organizations and governments, both in

the West and in the developing world, have bought into the World Bank’s mid-

1980s arguments in favour of cost-sharing, and still retain a widespread belief that it

is a necessary component of healthcare financing in the developing world – largely

because there are no other options, since taxation or insurance systems are too

complex to operate in these environments. Evidence, though, support the abolition

of user fees as beneficial in terms of health and welfare outcomes12

.

64. The issue of user fees has been raised as well in the context of emergencies and

humanitarian aid, and from an initial approach of not interfering with locally

established systems the tide has turned into an overarching approach of assuring free

access in contexts of humanitarian action. This approach is widely adopted now by

NGOs and donors as a valid policy arrangement in emergencies and complex

situations13

.

65. Donor support to the sector is important in Mali. For the period 2006-2009, sector

budget support accounted for approximately 10% of sector expenditure and external

(donor funded) projects for approximately 23%14. The political crisis of 2012 led

some key donors to suspend budgetary support and to channel significant funding to

support the health system through other partners. UNICEF has received significant

funding in 2013 from CIDA and SIDA including for the health sector

66. Claims of possible mismanagement by Mali authorities in 2011 induced the Global

Fund to re-channel allocations through alternative partners: HIV/ARV funding

though UNDP, TB through Catholic Relief Services (CRS) and malaria through

Population Services International (PSI).

UNICEF response

67. The health section of UNICEF MCO enjoys a relatively strong team and good

technical capacity. The section has been a recognized player within the coordination

mechanism before the crisis and has participated actively in the WHO-led health

cluster since its rollout in 2012.

68. The RTE found that most actors in Mali are not vocal enough to take advantage of

the humanitarian crisis to strengthen and promote measures to allow free access to

basic services and to raise donor commitment for this. The complexity of issues to

be addressed (poverty, geographical distribution of services, adequate financing of

the sector to address inequities, cultural and other type of barriers, general poor

quality of the services in the poorest areas) makes the endeavour very challenging,

but being such a critical aspect determining health outcomes and affecting equity, a

sound advocacy strategy from UNICEF could be envisaged. Unfortunately, so far

12

Economic and Welfare Effects of the Abolition of Health User Fees: Evidence from Uganda, Klaus

Deininger, Paul Mpuga, World Bank Policy Research Working Paper 3276, April 2004 13

Cost-sharing in complex emergencies: an example of inappropriate policy transfer, by Tim Poletti,

Egbert Sondorp, Olga Bornemisza and Austen Davis, Humanitarian Exchange, ODI March 2004 Issue 14 Documenting Results of efforts to Improve health aid effectiveness: MALI Case Study, October

2011; Elisabeth Paul, WHO

31

UNICEF advocacy for free access for children and pregnant women, both in the

normal programming and in the emergency response, has obtained limited results15

.

69. The humanitarian response was designed to provide basic services to the North,

where only 10% of the populations lives, and granting access to those services to

IDPs, host families and other vulnerable categories in the rest of the country, while

ensuring and reinforcing basic capacities of the Mali health system. Given the

specificities of UNICEF support to the health system, its main focus is on

immunization both in the North and South. Other aspects of the health response

addressed by UNICEF are access to anti-natal care (ANC) and prevention of

mother-to-child transmission (PMTCT). Nutrition is also included as part of the

basic services to be provided by the health system, but through its own response

planning. Response to cholera outbreaks is a recurrent problem in Mali. It is

embedded in the humanitarian response plans, and is part of contingency planning

initiatives involving WASH and C4D sections, as well as the health cluster, led by

WHO.

70. The response to the crisis is entangled with the structural problems that affect the

health system. Access to basic services in the North implies establishing a parallel

system through NGOs, and this includes specific information and supply systems.

This affects a wide territory with sparse population and with some damage of

already limited infrastructure. The response has to be implemented keeping in mind

that the return and capacity of local health administrations has to be promoted and

facilitated, and reconstruction by development actors coordinated.

71. Access to services for the rest of the country suffers from the same limitations that

affected the implementation of health programs before the crisis. Reinforcing local

capacities to ensure immunization coverage was already a challenge in a “non-

crisis” context. Official data on immunization coverage is challenged even by

UNICEF/WHO estimates, significantly lower; and even these estimates show a low

grade of confidence16

. The emergency response intends to keep the objectives for

polio and EPI countrywide (even yellow fever has been included in some planning

documents). The assumption that measles immunization has been assured for all the

under 5 population (mentioned by UNICEF to explain that measles mop-up

campaigns are not part of the response plan) can be judged as optimistic17

and the

risk of outbreaks exists. Information that the outbreak in Gao in 2013 is affecting

only population over 9 years could not be confirmed.

72. The response plan of the health section is articulated through 4 main objectives

related with the CCC18

:

15 UNICEF has tried (with WHO and other partners) to get free access for children and pregnant women,

this was established during 2012 countrywide and it seems extended indefinitely in the North. In practical

terms has little impact though, as there are no budgetary provisions to cover the fees to service providers.

Only directly supported UNICEF (or ECHO) partners can cover some of the expenses involved in

accessing care. 16 http://www.gavialliance.org/country/mali/#, and http://apps.who.int/gho/data/view.country.13100, both

last visited June 27, 2013 17 WHO data for under 1 year of age is of 56% (2011), http://apps.who.int/gho/data/view.country.13100 18 Mali Response Plan 2013 Mar13, provided by M&E section

32

contribute to a coordinated response, supporting the cluster system, contribute to

fundraising for the sector, and advocating for improving emergency

preparedness and response

integrated vaccination of polio (with and vit A and de-worming), aiming at 95%

coverage in 13 districts of the North and supplies to cover the needs of women

and children in these 13 districts and 3 additional ones; distribution of ITN to

children under 1 and pregnant women in the 13 districts of the North

EPI immunization carried out countrywide, aiming at 90% coverage in 50

districts, assuring as well essential medicines to address outbreaks (cholera),

obstetric care and ARV therapy. HIV testing to be provided at nutrition priority

referral centres

Health promotion and health education countrywide. Related with nutrition

priority areas, in districts of the north and countrywide (integration of activities

of C4D).

73. Such a plan reflects the challenges described above: a mix of targets that are

difficult to reach even in a normal development context, and some not related to

humanitarian response (polio, yellow fever). The integration of some activities with

other sectors (WASH for cholera, HIV testing for nutrition, C4D for promotion and

education), and the commitment with coordination structures have been noted as

positive approaches. Implementation in the field seems weak though. Overall,

immunization targets and objectives on health services availability and access is

unrealistic. Even in the North, where NGOs have been mobilized with specific

PCAs and have taken the responsibility of providing direct care while supervising

and supporting community health services, there is limited progress towards targets,

and the 3PM reports have highlighted a number of shortfalls in terms of availability

of supplies and functioning of the centres, as well as deficiencies in the cold chain.

The RTE could not access the area and a more direct assessment has not been

possible.

75. It has to be acknowledged that thanks to UNICEF’ support a continuation of basic

services has been maintained in the North, but the RTE could no evaluate the

dimension of actual adequacy, effectiveness and coverage. The involved NGOs

reckon that they cover adequately a vast region with sparse population and their

presence assures at least the gratuity of basic services, through direct care by NGO

staff, or through incentives and fees to ASACOS and health staff.

76. In the rest of the country, even if in some cases partner NGOs are active with

integrated nutrition and health activities, the information is incomplete and the RTE

could verify weaknesses of supervision and data reporting in the areas visited, as

well as deficiencies in the cold chain in the CSCOMs. It should be pointed out that

these weaknesses are not specific of the humanitarian response.

77. As in the Nutrition response, the community component seems largely neglected,

and community participation and involvement very limited. In the areas visited

CHW and community “relais” were quite de-motivated and apparently not engaged

in their expected activities and roles. The RTE cannot ascertain how generalised is

this situation, but additional information from partners allows to raise a possible

issue of de-motivation and lack of adequate supervision. This is not specifically

related to the humanitarian response, but can be particularly indicative of a general

33

dysfunction of the system, too focused on a for-profit activity and not able to

involve and incentivize critical community aspects. Assuming that CSCOMs and

ASACOS assure a community dimension of health care is not straightforward19

.

78. Regarding the effectiveness of the sector response, the current set of indicators and

targets (input oriented and related to immunization coverage) do not allow

evaluating performance. The HPM indicators that appear in the Sitrep are limited to

immunization coverage and to the distribution of supply kits to the north. An

additional HPM not included in the Sitrep is about the number of ANC consultations

conducted in health facilities supported by UNICEF, potentially more adequate to

gauge performance.

79. The response plan defines additional indicators that could be useful to better

understand progress towards targets, like ITN distributed, % of health facilities

provided with functioning cold chain, % of facilities supported by cash to assure

outreach activities, number of children receiving appropriate case management for

malaria, respiratory infections and diarrhoea, number of cholera kits prepositioned,

number of children resistant to nutrition treatment that receive HIV testing. The

response plan defines targets for all these indicators, and although the suitability of

these in relation with a humanitarian response can be discussed, they would be

informative and help to better judge the effectiveness of the response. Information,

as it can be found in the narrative of the Sitreps and provided on a three monthly

basis, does not allow following progress.

80. HIV testing in nutrition referral centres seems not established systematically, and

the consultants could verify this during the field visits. The absence of reagents is

one of the reasons, and reports available show very limited availability of HIV test

supplies in the North, but it seems that lack of awareness is also a significant factor.

As explained in the trip report of the HIV specialist from WCARO in March 2013, a

research on the prevalence of HIV + among malnourished children should be carried

out, also to raise awareness about the problem. The fact that some PMTCT centres

in the North were reported again active could not be verified, nor the actual capacity

to carry out its functions. In Mali the protocol for ARV therapy seems not uniformly

known by health workers, and the disruptions, in staff and availability of supplies,

created by the crisis is likely to weaken further this aspect of health care.

81. Regarding efficiency in the use of resources, like in other sections, the RTE could

verify an adequate pattern of liquidation of expenditures, but a very low translation

into progress of the HPM indicators. Even allowing for delays of reporting, maybe

by using alternative indicators as explained above could better inform on progress

and relate it with the use of resources.

Recommendations

5) Health section and senior managers to advocating for access to care for the unable to

pay, revitalising working group to come up with practical recommendations.

6) Health section, in coordination with C4D, to reactivate community aspects of the

19 CSCOMS are managed by community health associations (ASACOS), but their role is basically related

with the administrative and financial management of the service provision of the health centres, and has

little relevance on aspects of health promotion and education, other than managing funds for related

programs.

34

health programme currently neglected in many areas

7) Health section to establish a regular information system based on indicators included

in the RP, oriented to measure utilization of services, coverage and quality in priority

areas (in the North and in particularly vulnerable areas of the South)

8) Health section to strengthen collaboration with 3PM partners in order to capture

relevant information for program monitoring and management.

9) Health section to address HIV related gaps, assuring testing at URENI level and

functional PMCTC in the North. Partners should be aware of the protocol to apply for

patients in ARV that discontinued treatment.

NUTRITION

82. The Sahel is a region with frequent episodes of food insecurity triggered by climate

and market determinants, rampant population growth that stretches already limited

coping mechanisms, and increased poverty for wide segments of the population. The

combination of these factors with cultural ones and deficient food practices leads to

a chronic situation of high malnutrition rates. Health systems show limited capacity,

are generally underfunded to provide adequate case management, and have so far

demonstrated poor effectiveness in preparedness and resilience approaches. Mali

experiences a chronic situation leading to a permanent high prevalence of chronic

malnutrition (around 30%) and a more or less stable caseload of acute, severe and

moderate malnourished children with occasional incidence peaks (between May and

September).

83. UNICEF early call for action at the end of 2011 in light of warnings of a possible

nutritional crisis in the Sahel included also Mali; following the regional approach

supplies were mobilized and response scaled up during 2012. Mali, however, went

into a political and humanitarian crisis caused by the occupation of the North by

armed groups and the subsequent displacement of population to neighbouring

countries and to the South. Health infrastructures in the North were damaged, and

staff of government health centres left the area, adding complexity to an already

fragile situation.

84. Malnutrition affects Mali countrywide. Most of the population lives in the South

where food is basically available but where most cases of malnutrition are prevalent.

In the North food security is threatened more often due to the arid nature of the

territory and insecurity causing displacement of herds, but it is sparsely populated

and the nomad and pastoralist character of the population, feeding on a more

nutritious diet based on milk and diaries, limits the impact of malnutrition.

85. Malnutrition case management is part of the essential services provided free of

charge by health centres in Mali (therapeutic food, routine medicines, medical

complication medicines) but access of patients to care is hindered by the distance

involved in some cases to reach a health centre, the need to cover costs of transport

and the associated costs of care (medical consultation), and the low quality of

services offered. This is coincident with the challenges of access to health services

(see also the Health chapter of this report), and as well influenced by inequities

between segments of population.20

20 See : Résume de l’étude « Pauvreté des enfants et inégalités au Mali » Ministry of Finance, published by UNICEF

35

86. A SMART survey carried out in 2012 in the South of the country showed moderate

figures of acute malnutrition, below the 15% emergency threshold, with the highest

prevalence of GAM in Segou (12.2 %) and Kayes (10.1 %), while SAM reached

3.8% in Segou, 2.9% in Mopti and 2.5% in Kayes. In the North the survey could not

be carried out due to security constraints. A SMART survey has been carried out in

the region of Gao in May this year; preliminary results show that prevalence of

GAM in Gao region reached 13.5% (serious level) with SAM levels of 2.4%.

UNICEF response

87. UNICEF approaches malnutrition problems in Mali through the support to

Government capacities to deal with surveillance and case management in the health

structures and with additional activities aiming at sensitization and awareness at

community level in order to change inappropriate behaviour for maternal and child

nutrition through Communication for Development. Since the 2012 scale up of its

nutritional programme, UNICEF Mali has provided all the necessary therapeutic

food for the treatment of malnutrition, as well as most of the additional elements of

related curative assistance (antibiotics, vitA, de-worming drugs, micronutrients).

88. Coverage of care is an important issue in Mali. Due to limitations of the health

system (see health section) support by an external partner is envisaged to assure

delivery of services and adequate information management. Currently, only 47% of

nutrition centres are supported by an external partner supported by UNICEF, but a

coverage analysis shows that the presence of those partners is likely to cover around

90% of the areas with high malnutrition rates (information from the section). This

analysis is useful, and should be consolidated by a coverage survey, planned in the

Response Plan 2013.

89. The agreement on the caseload and the target to be reached for case management

purposes is a key issue in Mali. From the surveys, and with correcting factors

applied, an annual caseload of around 210.000 children with SAM is to be expected

in 2013. The target for 2012 was established at 175.000 children to be admitted to

treatment; apparently, no more than 68,764 were reached by the end of the year21

.

This year, the target has been established at 125.000. From the data available the

caseload has not changed significantly, and the target figure represents an estimation

of the number of patients to be reached according current capacity of partners on the

ground.

90. The nutrition section of UNICEF Mali CO is composed by a team of 14 staff and

has a strong technical capacity. It is very much involved in development programs

and in policy development with the GoM partner institutions. During 2011 and

2012, after the UNICEF regional call for action, Mali CO stated to increase

operational capacity through partners.

91. The response plan elaborated in 2012 is the basis for the current one and is anchored

in the CCCs. Main components of the RP are:

21 55,029 annual new admissions and 13,735 in treatment at the beginning of year (2013); source, Nutrition section’

comments to draft report

36

support to coordination: including extension of clusters to regional and district

level

establishment of quality assessments and surveillance system

reinforcement of Infant and Young Children Feeding Practices (IYCFP) and

awareness and counselling in critical areas

effective management of acute malnutrition, including active screening and

referral

meeting the needs for micronutrients

communication and C4D activities

92. The plan in itself is adequate and addresses the overall challenges. But given the

baseline in treated children from previous years a more ambitious approach

addressing the problems of access to services could have been drafted. UNICEF has

in fact made an effort improving coverage and presence of NGOs supporting

nutrition centres. New PCAs related with the emergency have been agreed upon, in

some cases integrating health, nutrition and WASH activities. In the North, NGO

partners are in charge of keeping some nutrition services and referral hospitals

running, even if some facilities have been damaged (especially URENIs).

93. The RTE consultants could not evaluate the response in the North, but indirect

information allows concluding that supplies are basically available and activities are

on-going in CSCOMs where the population is likely to show up and, in general,

services have been available in Gao and Kidal. Outreach services are as well

provided by NGOs. 3PM have reported higher disruption of services in Timbuktu

area, at least one URENI closed in Gourma-Rharous22

and six CSCOMs/URENAS

closed in Niafunke and six in Dire/Goundam. Lack of staff and/or supplies, and

sometimes damage of the facilities seem the main reasons.

94. Regarding the South, the RTE field visits could identify weaknesses of some of the

elements included in the response plan. The coverage of NGO support to

CSCOMs/URENAS is still only 50% in and around Mopti, and non-existing in

Douentza, Tenekou and Youwaraou (3PM report). In addition the Community

Health Workers (CHW) involvement, awareness and general interest in the program

seems quite weak in the areas visited. The community based active screening does

not seem to work adequately from the information gathered, including interviews at

community level. Lack of supervision, follow up and motivation could contribute to

this situation. This problem is not related to the humanitarian response; it is rather

linked to the general inefficiencies of development programs.

95. Other activities, as Vitamin A distribution and de-worming, are activated in the

context of specific campaigns (Child Health Days) coupled with poliomyelitis

events conducted twice a year. Results apparently show high coverage of these

activities, but they run in parallel to the emergency response, and cannot be

considered humanitarian response’s related activities. The presence of additional

capacity and humanitarian coordination systems might, though, become an

opportunity to boost results in these types of activities, including EPI and polio

immunisation rates. Vitamin A and de-worming tablets are currently being

distributed in the three North regions of Mali. Micronutrient availability seems still

22 UNICEF has a new PCA with International Medical Corps to implement nutrition activities in this district.

37

not achieved (we do not know which strategy will be put in place to reach 500.000

children and 350.000 pregnant women in the context of an emergency response).

The plan of creating a surveillance system with SMS messaging conceptually

intends to by-pass the difficulties of the information systems in place, but will

require important investments and intense follow up. This one-year plan is

supported and funded by ECHO.

96. The effectiveness of the response will be very much dependent on UNICEF and

partners’ capacity of increasing the performance of active case finding strategies and

assuring coverage through the support of CSCOMs by NGOs, assuming that this

involvement would provide an adequate quality in terms of cured rates. The

measures already taken (although still limited in terms of coverage and active

surveillance) seem to assure better results than last year (see figure). With only

45.000 admissions at mid-June though, it is difficult to assume that the numbers at

the end of the year will be close to the 125.000 target. An improvement of the

situation in the North will undoubtedly improve access to services there, but still

this would have a minor influence in total numbers. Underreporting and delays from

GoM centres can contribute to explain to some extent the low rate of

implementation so far, but the RTE could verify low program enrolment in the areas

visited mainly due to a weak community and social mobilization and no removal of

user and transport fees.

97. An additional concern lies in the low quality of data regarding cured and defaulter

rates. This information seems to be collected, and aggregation is made at district

level. The RTE has not seen any analysis or compilation of that data in a way that

-

2.000

4.000

6.000

8.000

10.000

12.000

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

2013 2010 2011 2012

Mali-Number of new SAM admissions

38

could inform program managers23

. While mortality rates do not seem to be high,

defaulter rates seem to be a problem in all areas visited (with low frequentation in

most cases). This is related sometimes to lack of some components of the case

management, like antibiotics or other medicines, which will have to be purchased by

the patient’s family, or low quality of the services, or additional user fees to pay.

There is no system in place as to capture clients’ satisfaction or demands to correct

dysfunctions.

98. Evidence gathered indicates poor integration of other sectors in the response.

WASH in Nut seems to be implemented only in a limited way with specific

agreements with NGOs, HIV testing is mostly not carried out, child stimulation at

URENI level is not happening in the areas visited but is carried out in some URENIs

with specific partners, and C4D activities are absent in the North (according to 3PM

partners reports) and very poor in the areas of the South visited. Additional

supervision of partners, close monitoring and follow-up, enhanced community

participation and beneficiary feedback need to be addressed.

99. Regarding the efficiency in the use of resources, the rates of liquidation are coherent

with the time of implementation, although results obtained are still short of what

they should be by this time of the year (considering only SAM cases admitted into

program, not taking into account other shortcomings as referred above). Possibly,

underreporting and delayed reporting contribute to this discrepancy, and an

improved information flow would provide better expenditure/results ratio. The

volume of shipments of inputs, however, indicates that the admission figures are not

far from reality, given the inputs requested (see supplies section of this report)

100. Regarding Coordination, UNICEF has appointed a dedicated Cluster lead and an

IM staff in the Cluster. The cluster has succeeded in bringing together all actors

involved, and the role of UNICEF is in general praised. However, the strong

technical capacity of the team is not translated to the current set up of Field

Offices24

, and the decentralization of the Clusters has not provided an added value

of coordination. Many partners indicated that the handover to local administrations,

whose capacity is quite limited, of the leadership of regional Clusters has affected

performance in terms of coordination and guidance. This was confirmed by the RTE

regarding the situation in Mopti, where nutrition Cluster meetings are no longer

taking place and leadership is absent. The strengthening of UNICEF Field Offices

and the revision of their functions and responsibilities should improve the

coordination leadership of UNICEF at field level.

101. UNICEF has recently signed a local MoU with WFP (in line with their global

MoU), clarifying competencies, approaches, criteria and responsibilities in the field

of malnutrition, where WFP addresses the MAM caseload and UNICEF the SAM

caseload. A number of synergies are possible, since both programs are based at

CSCOM level and rely on community engagement for education and surveillance

activities. This type of agreement is an example of good practice.

23

A monthly statistics report format with quality indicators at district and regional level was developed

three months ago allowing potentially to have a better estimate of IMAM quality indicators. 24 New national officers (for Koulikoro, Mopti and Sikasso) are on board since May 2013

39

Recommendations

10) Advocate for access to health services for women and children in line of what is

recommended for the health section. UNICEF should build an integrated institutional

position towards a problem that is based on inequities affecting women and children

11) Establish an operational plan with enlarged base of NGOs in order to cover

adequately the needs for support

12) Enhance supervision capacity at field level (this RTE would favour strengthening

the functions of FO on this aspect) and health/nutrition information systems, in order to

allow for a systematic analysis of performance indicators of nutrition program

13) Integrate and reinforce currently weak interactions of other sections (child

stimulation, WASH in Nut, HIV testing, C4D)

14) Establish mechanisms of beneficiary feed back as users of nutrition centres, and

strengthen beneficiary participation in the community process

15) Strengthen technical and coordination capacity at FO level.

WATER, SANITATION AND HYGIENE (WASH)

102. 2010 coverage estimates25

for water and sanitation in Mali provide a challenging

picture. National coverage for improved water is 64%, with 87% in urban areas and

51 in rural areas. National coverage for improved sanitation is only 22%, with 35%

in urban areas and 14% in rural areas. 14% of the total population still resorts to

open-defecation. Overall numbers are consequently staggering: ±5.5 million people

do not access improved water supplies, and ±12 million do not have access to basic

sanitation. As much as 2.2 million still practice open-defecation.

103. A UNICEF-led survey indicates that the situation at school level shows similar

weaknesses. Only 12% of schools have improved, sex-separated, functioning

latrines and only 7% of them have access to an improved water point. 60% of

schools visited by the survey show signs of open-defecation.

104. Diarrhoea is responsible for ±20% of child mortality in Mali, and cholera

outbreaks have increased in frequency, with six epidemics in the last ten years. The

number of cases has remained constant, indicating that no real progress has been

made in containment, once the outbreak has started.

UNICEF Response

105. UNICEF had a WASH programme before the political crisis in the North,

addressing access to safe water in Guinea worm endemic areas, promotion of

Community Level Total Sanitation (CLTS) in selected regions and districts, hygiene

promotion through inter-personal and mass communication, and WASH in school,

progressively improving water and sanitation facilities in selected school facilities.

The coverage of these components was limited to selected priority areas, and has

expanded in 2012-13, also thanks to the increase of ORR funding. Still, complete

coverage of the country is a long-term objective.

106. With the humanitarian crisis, a new Intermediate Result for emergency was

added to the WASH work-plan. Its focus is three-fold: on the conflict in the North,

25 WHO-UNICEF Joint Monitoring Programme 2012.

40

on the ongoing, chronic nutritional crisis, and on cholera response. The main

activities are: provision of safe water to affected populations, distribution of hygiene

kits, distribution of non-food items (NFI) kits, distribution of hygiene kits/education

to children admitted to the SAM programme (WASH in Nut strategy), and WASH

interventions in schools and health centres located in regions affected by the crisis.

Since 2012, UNICEF has been implementing this programme also by funding

approximately 70% of the WASH cluster interventions.

107. Since 2012, UNICEF has been chairing the WASH Cluster, funding also its

Information Manager, and has recently accepted the chair of the newly established

PTF working group on WASH. In the UNICEF Mali strategy and Country

Programme Document (CPD) for 2013-14, WASH is included as a main support

component to other priority areas (Education, Nutrition and Child Protection),

strengthening the integrated approach of the office. A good example of programme

integration is also offered by a trip report format utilised by WASH staff visiting

potential or current project locations.

108. A more in-depth review of the emergency component of the programme

revealed a sound approach with evident challenges. Evidence gathered confirms that

the response to the cholera epidemics has been very effective, quickly arresting the

spread of the outbreak and limiting mortality, demonstrating good coordination with

NGOs and local administrations in the affected areas; several partners praised these

interventions.

109. Looking at the WASH emergency programme activities from the evaluation

criteria perspective, information collected allows concluding that their coverage and

effectiveness are limited in relation to the needs. This is partially explained by the

still limited number and capacities of implementing partners as well as of local

administrations staff in key areas, especially in the North. The WASH in Nut

strategy has been less effective than expected in 2012, and it has recently started to

scale-up, but there are specific efficiency concerns – based on expected

implementation rates, see below – that even once up-to-scale, some gaps will

remain.

110. All activities requiring community-level involvement and action have been

affected by challenges. With the exception of the villages where the CLTS strategy

has already been implemented, the WASH programme, as other UNICEF sector

programmes, has been struggling with community participation. Distance, limited

capacities of implementing partners, chronic weaknesses of local administrations

and communities themselves, have been mentioned to explain this problem.

111. The RTE looked at the efficiency of the WASH emergency programme, and

specifically at the discrepancy between the low achievement rates, as described in

the May Sitrep, and the quite good expenditure rates of the section (>50%). Various

factors influence this discrepancy. Expenditure rates include disbursement to private

contractors and to procure supplies as soon as payments are done upfront; it can

then take a long time before the services are completely delivered by the private

contractors (as the RTE team had the chance to see in Nampala), and before supplies

are received and distributed by implementing partners. In addition, common late

reporting by partners, contractors and local administration staff, contribute to the

41

delays. These reasons are most likely explaining similar discrepancies between

implementation and expenditure rates of other programmatic sectors.

112. The WASH targets ad indicators utilised for the Sitrep suffer from lack of clarity

on their origin and breakdown of different categories of beneficiaries summing up to

the total target.

113. According to MCO WASH Chief, several initiatives are already in place to

address some of the challenges mentioned above. Newly signed PCAs, once fully

implemented, will increase implementation rates, especially for some activities. The

WASH in Nut component will also benefit from these additional PCAs, but even

expected implementation rates seems still low in relation to the needs. More

partnerships with NGOs and local administrations, also aiming at increasing their

capacities at field level (vehicles, fuel, office equipment, etc.), are under discussion

and linked to availability of funding. Distribution and post-distribution monitoring is

considered a priority by the section. It is normally included in new PCAs, is

addressed by 3PM partners and a consultancy firm has been under contract since

end-2012 to monitor rehabilitation initiatives in the North.

Recommendations

16) WASH section, based on availability of funding, to keep increasing number and

scope of PCAs with NGOs progressively scaling up, and further expanding capacity

building of local administrations. Work with partners and counterparts should also aim

at reducing reporting delays.

17) WASH section, in coordination with Nutrition section, to investigate bottle-necks

and delays in the implementation for the WASH in Nut programme aiming at improving

efficiency and effectiveness of the project.

18) WASH section to further investigate and optimise post-delivery, distribution and

utilisation phases of key supplies, and service provisions of contracted companies,

aiming at mitigating bottle-necks and improve efficiency and effectiveness of the

response.

19) WASH section, in coordination with other sections and C4D, to devise and improve

strategies to enhance community participation in support to the programme.

EDUCATION

114. In Mali, the education system has been affected by major weaknesses for a long

time. More than one million children are out of school, and class over-crowding is a

wide-spread phenomenon; the RTE team visited classes with more than 200 children

and heard about classes with up to 300.

115. Before the crisis, UNICEF has been working in collaboration with the Ministry

of Education (MoE) to progressively increase capacities of teachers, directors, as

well as management bodies and education authorities. Supplies and training were

provided, and basic infrastructure rehabilitation targeted selected schools; more

infrastructural work was done in collaboration with the WASH section to build or

improve water and sanitation facilities in targeted schools.

42

116. The crisis in the northern regions of the country – Timbuktu, Gao, Kidal, but

also some northern districts in the Mopti and Segou regions – had major

repercussion on the education system. Approximately 75,000 children were

displaced internally or to neighbouring countries, together with 85% of the teachers.

Many schools were damaged and equipment was extensively looted, bringing school

activities to a halt. According to NGO staff interviewed, some of these damages are

precedent to the recent crisis.

117. Local communities tried to organise themselves in order to ensure some sort of

temporary scholarisation to the ±450,000 children who remained in the North, but

had to contend and negotiate with the occupying forces which tried to impose a

Sharia approach to education (separation between boys and girls, mandatory veil for

girls, teaching of Arabic, etc.). The French intervention (Operation Serval) started in

January 2013, liberated most of the northern regions from the occupying forces, and

teachers and students slowly started to come back.

UNICEF Response

118. An emergency response plan crafted on the CCCs for education was developed

in 2012, and progressively partially refined and revised. The latest version made

available to the RTE team is dated March 2013; it is very complex and ambitious,

and contains major budgetary gaps. Some respondents indicated that it does not

reflect properly the changes in the situation in the north after January 2013. In

parallel, the section has been working on the development of the revised Annual

Work Plan (AWP) for the extended programme 2013-14. This document integrates

the various components of the emergency response into the four Intermediate Result

(IR) areas related to the regular programme. This could be a good approach to

integrate the emergency response in the sector work plan and engage the whole

Education team.

119. UNICEF was effective in sending some emergency education supplies to the

North throughout the most difficult periods. Some interviewees indicated though,

that the response in favour of IDPs in the South – where their arrival further

exacerbated the over-crowding problem – was insufficient. The emergency

education programme is currently supporting the reactivation of the school system

in the North with basic rehabilitation, supplies, equipment and training, and

supporting “catch-up” classes for thousands of children who missed school for

months. This is mainly done through implementing partners, and progressively in

coordination with returning local authorities and administration. In addition to the

challenges caused by the conflict in the North, the emergency programme is also

aiming at supporting IDP children in the South and children of school age affected

by floods in several parts of the country; this is done mainly by building on existing

collaborations with the education authorities.

120. Discussions with various UNICEF education staff, both in the RO and in MCO,

indicate that the section faced some challenges in 2012, including “lack of

leadership and risk-aversion”. The RO was ready and fast to provide some support,

but the work in-country for the RO Education in Emergency specialist was difficult,

also due to personality issues. Issuing PCAs was apparently more cumbersome than

43

for other sections; there are currently only two PCAs active and ten under discussion

(extensions and new).

121. The RTE team had the chance to visit some education projects in Mali - schools

and child-friendly spaces (CFS) in Mopti - and discuss with staff from two

implementing partners, but did not have the opportunity to go to the northern

regions. The perception gained, further confirmed by discussions with the UNICEF

education staff in Bamako, is that the emergency programme is affected by notable

inefficiencies at project level (disorganised CFS with insufficient resources) and

lacks a comprehensive strategy to connect apparently patchy interventions. A school

supported by UNICEF with supplies, WASH facilities and capacity building was

found deserted by the RTE during a morning visit; the school director was not able

to provide any valid justification. This raises further concerns about the

effectiveness of the capacity building intervention and the interactions with local

administrations.

122. UNICEF has recently signed a MoU with WFP covering many programmatic

areas. For the Education sector, the main component is school feeding. The RTE

team tried to have a clear understanding of the effectiveness of this programme, but

obtained somehow contradicting information (very high numbers of children

covered, from WFP, and indication of major gaps and poor efficiency from NGOs

and UNICEF), and did not have the chance to visit any school with a running

canteen.

123. The section has now a new Chief and several new staff; there is a clear intention

to revise the emergency response strategy and integrate it with a broader transitional

agenda, linking emergency to development. In order to increase capacities to

interact with the MoE and influence its agenda, the section is planning to hire a

senior education policy specialist, to be embedded in the Ministry.

124. UNICEF has a very good collaboration with the MoE, and the latter underlined

great expectations about the role UNICEF should play in the education sector. This

has been stressed also by two donors. When offered the chair of the PTF working

group for Education and Vocational Training, UNICEF refused, apparently because,

according to senior management, it was not the right moment. It seems that it was a

decision based also on the recognition of limited capacities.

125. The PTF Education WG is currently chaired by the World Bank, and among its

initiative, it is revising the plans for the use of 41.7M$ for school rehabilitation and

institutional support. The funds are part of a Global Partnership for Education (GPE)

contribution, put on hold because of the crisis. The Education Cluster is already

contributing to the discussion to revise the use of the funds and direct some of them

to the North. This is an area where UNICEF could and should express its leadership

role.

126. UNICEF has a tradition of utilising the opening of the school year to organise a

“Back-to-School” campaign: an opportunity to advocate for and encourage

enrolment, quality of education, and girls’ scholarisation. It is a logistical challenge

and a great communications opportunity to show UNICEF capacities and leadership

in this sector. In the context of Mali, it is even more challenging, given the political

44

and ethnical complexities in the North, the need to maintain a balanced approach in

all the country, the importance of maintaining the MoE “on the driving seat”, and

finally due to the limited time available to organise such an initiative. At the time of

the RTE visit, discussions about the opportunity to carry out this campaign, or a

lower-scale form of it, were ongoing in the MCO.

127. The RTE evaluators faced some problems in assessing the effectiveness of the

emergency programme. The origin and breakdown of most targets proposed on the

May 2013 Sitrep is unclear; and these do not match figures proposed in the AWP or

in the CAP 2013 or its MYR. If, as it seems, the first target on the Sitrep includes a

mix of beneficiaries from the humanitarian response in the North and other

categories, like children affected by flood, the RTE would argue that separated

targets and indicators would be more useful for programmatic purposes.

128. In terms of efficiency, the consultants compared the achievement rates indicated

in the Sitrep with the expenditure rates (from SM PBAs contributions, only; not able

to get comprehensive picture of possible ORR fund allocated on the emergency

response). As in the WASH section, there is a discrepancy, with very low

achievement rates (17% or lower) and quite higher expenditure rates (44%, although

these are lower than the average of the programme sections). Various factors

partially justify this discrepancy, including delays in reporting and poor efficiency

in supply distribution beyond delivery to partners and local administrations. Further

analysis is required to fully understand poor achievement rates.

Recommendations

20) Education section to finalise the process of revising priorities, strategy, plan of

action, targets and indicators for the emergency response, integrating transition to

development.

21) Education section to engage more consistently with partners and MoE to increase

number and scope of PCAs and other partnerships, aiming at increasing coverage and

consistency.

22) Education section to identify and hire senior staff for Education policy work and

coordination, aiming at strengthening UNICEF leadership role in the sector, and, if

feasible, reconsider the offer to lead Education PTF working group.

23) Education section, with senior managers, Communications and Supply section to

urgently assess the suitability and feasibility of a context-specific “Back to School”

campaign, shaped on the complexities and needs of the country, and eventually utilise

the campaign for communications purposes and to raise the profile of UNICEF as

leading agency for Education.

CHILD PROTECTION

129. Before the crisis, UNICEF had a relatively small Child Protection programme

focusing on birth registration, early marriage, female genital mutilation/

circumcision (FGM/C), child trafficking, “Talibe” children26

and capacity building

of Governmental counterparts. At the onset of the crisis in the North, in light of the

26 Talibe children are children who are entrusted to a maître coranique/marabout for religious education

which can lead to abuse, exploitation, and violence, family separation and a range of other child

protection concerns

45

new child protection issues caused by the conflict and the consequent

displacements, the section decided to activate a response based on CCC-related

issues, and to aim at having one specialist per CP issue: mine risk education (MRE),

gender-based violence (GBV), children associated with armed forces and armed

groups (CAAFAG), and separated and unaccompanied children (S/UAC).

130. Between March and November 2012, there was some reluctance from the CP

section in Mali to receive a technical support mission from the RO. Then, by the end

of 2012 and in the beginning of 2013, several missions took place (CP in

Emergency specialist for four months, MRE and GBV specialists, senior specialists

from the CO in Senegal and HQ)27

. These missions struggled with structural

weaknesses of the section and limited experience on CP in emergency issues.

Several sources (UNICEF RO, MCO, NGO partners) stressed that the section was

the most affected by the always-difficult shift from development to emergency.

Scaling it up was challenging: the needs for CP in emergencies staff were initially

underestimated, and it was not easy to find high calibre CP specialists speaking

French, especially after Bamako became a non-family duty station. A newly

selected CP Chief withdrew the candidature at the last moment, further

compounding the problems of the section. The new CP Chief arrived in Mali during

the RTE mission, in June 2013.

131. The consequences of the conflict in the North required a completely new

approach to address the new CP challenges. Generalised lack of baseline data

affected, and still affects, the development of different programme components.

Only recently, some information started becoming available, provided by NGO

partners working on MRE and CAAFAG. There are indications that gender based

violence issues (rapes, forced marriages) have been exacerbated by the crisis. While

numbers of identified and supported CAAFAG are still low, there is a concern this

is only due to the fact that information and programme response are not available

and access to many areas in the North still limited, and that figures might be much

higher.

132. The deployment of MINUSMA adds another layer of complexity, challenges

and opportunities. As in all countries with a UN Mission presence, an increase of

protection issues is a concrete risk. The UN Agencies, UNICEF and specifically the

CP section will have to find suitable and constructive ways to interact with both

political and protection branches of the Mission, ensuring the preservation of

humanitarian principles and space. On the other end, the Mission will likely bring

some stability, expand access to the North, allowing for better programming, and be

the driving force behind peace-building initiatives. Recently, Mali has been included

in the list of countries mandated by the UN to implement the Monitoring and

Reporting Mechanism (MRM) for grave violations of children’s rights in conflict

(UN Security Council Resolution 1612); UNICEF will have a major role to play in

this context. The RTE team was informed that the incoming SRSG leading the

Mission has solid experience of protection issues, and this could facilitate UNICEF

CP work, as well as the work of the CP sub-Cluster in the coming months. In light

of these extraordinary challenges, a joint RO/HQ CP technical mission to Mali in

November 2012 concluded that the MCO should develop a large and complex CP

27 Several of these missions have been from the RO or facilitated by the RO Emergency and CP sections.

46

programme to address them. Expectations from the UNICEF regarding CP

programme performance are high, and it is felt that the reputation of UNICEF would

be at risk if these expectations are not fulfilled.

UNICEF Response

133. Notwithstanding the challenges and problems faced by the section, some

initiatives were started during 2012. UNICEF managed to invite several times to

Bamako many members of “comité de crise” from the northern regions and sensitise

them about CP issues and vulnerabilities, and started training them on the

identification of cases and information collection.

134. The CP section has been successful in advocating for the discussion and

adoption of a national protocol for the identification and handover of CAAFAG

from the National Gendarmerie, Malian forces, and allied troops to the Government

and UNICEF. The protocol should be signed by July 2013. In parallel, the section

accompanied the process of demobilisation of the first small group of CAAFAG

identified by the French-Malian troops, and a dedicated transit centre has been set

up.

135. Partnership with Handicap International and other partners, and the support from

a MRE specialist (Stand-by Partner) allowed developing and strengthening the MRE

component of the CP programme, including awareness, education and sensitisation,

as well as information management and support to victims and families. HI, as other

INGOs in Mali, is undergoing an exercise to revise its short- and long-term strategy,

and it plans to expand both scope and geographical coverage of its operations.

136. In addition, the section has a number of other ongoing partnerships with

international and national NGOs; some of these are under negotiation for extension.

Some of these partners, as HI, are planning to expand their programmes in the

country. During the brief field visit, the consultants had the chance to meet some of

these partners and visit some UNICEF-funded projects. While the efficiency and

effectiveness of some of these projects is questionable (poor coordination and

resource planning), others (i.e. those implemented by Save the Children and the

International Rescue Committee) seem more effective and their extension and

expansion should be considered.

137. The recently developed AWP for CP broadly covers all key CP issues and

categories stemming from the crisis. Numerous capacity building activities seem

well thought, in light of the very limited capacities of Government and partners to

deal with delicate protection issues. Activities are included to ensure constructive

coordination with MINUSMA and sound implementation of the MRM. Data and

information collection and management represent the largest part of one of the

Intermediate Result areas. Several informants expressed concerns though, that

capacities to implement this work plan are not yet available in the CP section.

According to the latest version of the organisation chart, two international

specialists positions (P3) are still vacant, and the MRE specialist provided by a

Stand-by Partner is leaving soon.

47

138. The section has been coordinating its activities with and within the CP sub-

Cluster, for which UNICEF has chairing responsibilities. The sub-Cluster went

through a lot of changes of coordination and IM staff, and that only recently it

managed to start working more consistently. Sub-Cluster staff confirmed that CP

capacities – both for programming and information management – are still

extremely low in the country, among Government departments, NGOs and civil

society groups, and that the macroscopic lack of baseline data prevented the

development of clear strategies for the country so far. The sub-Cluster represents the

only forum where CP in emergency actors can regularly meet and issues can be

discussed. A new sub-Cluster coordinator has recently arrived; she will have the

responsibility to lead the group in the development of the sub-Cluster work-plan and

in managing interactions with MINUSMA.

139. Through some of its partners, MCO has been implementing several GBV

training and referral initiatives in the south of the country. Although welcome, these

activities are considered patchy, often ineffective and not integrated in a sound

strategy. There has been no GBV capacity in the CP section since September 2012.

According to informants from NGOs and other UN Agencies, the sub-Cluster for

GBV has been activated by UNFPA, in collaboration with UNICEF, only “on

paper”; it met a few times but its effectiveness is far from optimal. This is an area of

concern and concrete reputational risks exist for UNICEF, UNFPA and the broader

UN.

140. Based on the documents provided and accessed, the RTE team had limited

chances to properly assess efficiency and effectiveness of the CP programme. As

much as possible, the consultants compensated by comparing information gathered

at regional, country and field level from many different actors, and triangulating this

information with the available documentation.

141. Similarly, the complete lack of HPM targets for CP on the Sitrep prevents any

type of trend and effectiveness analysis. The HPM indicators proposed, are unclear,

and seem to reflect the dramatic lack of data in the whole sector. The section shows

the lowest expenditure rate based on SM PBAs (36%) in the office.

Recommendations

24) CP section to urgently develop a comprehensive prioritized strategy and plan of action

addressing new vulnerabilities (S/UAC, CAAFAG, MRE, GBV, etc.) and linking them to the

pre-crisis approach; focus should be maintained on capacity building of governmental

counterparts and partners, and enhanced on service delivery, as well as data and information

collection and management for all programme components.

25) MCO senior manager, RO and HQ to strongly back up the activities of the CP section,

facilitating technical assistance, supporting staff recruitment process and supporting high-level

protection discussions and negotiations with MINUSMA and other actors.

26) CP section to intensify and strengthen collaboration and coordination with CP sub-Cluster

for programmatic, policy and advocacy issues.

27) CP section to urgently identify and deploy high calibre staff (IPs, national staff, TAs) for

key positions (liaison/interactions with MINUSMA; sensitive issues).

28) CP section, in collaboration with CP sub-Cluster and UNFPA to review status of

implementation and coordination of GVB initiatives in the country, with the aim of re-starting

operations of the GBV sub-Cluster and developing a joint strategy with relevant actors. Other

sections, under guidance of CP section, should start including GBV issues and activities in

their respective emergency responses.

48

COMMUNICATION FOR DEVELOPMENT (C4D)

142. Communication for development is potentially a key aspect of UNICEF

interventions. The concept itself and the relevance it has gained in the Organization

make C4D approaches instrumental to achieve programmatic objectives.

143. The problem of this type of activities is how attributing the contribution of C4D

to sector results that require massive investments in “hardware” (facilities,

equipment and supplies, like the Health and WASH sectors). This is a problem

beyond this exercise but affects the overall ability to raise C4D resources. In the

budgetary structure of the sector programming, a part of the budget is normally

allocated to C4D activities and intended results, contributing to the broader sector’s

results. What normally happens, is that when funds are limited, C4D are among the

first budget lines to be curtailed. Even if C4D actions are clearly embedded in all

programmatic sectors’ CCCs, their implementation during the humanitarian

response seems challenging. Country-specific quick design and operational

solutions for C4D should be included in the preparedness activities of UNICEF

COs.

144. In Mali, C4D approaches and activities are well consolidated. The strong pre-

crisis presence of developmental actors and donor interest has favoured community-

based approaches as strategies to achieve development results, even when

government capacity is limited. The assessment of this approach and its

achievements is well beyond this RTE scope, but the consultants were

systematically challenged by the low level of community involvement in initiatives

and projects related to UNICEF’s response. The role of Community Health Workers

seems quite devaluated, and community dynamics and sessions seem languishing,

probably due to a number of contributing reasons. Poor progress of development

indicators, rampant poverty and exclusion, and increase in disparities among regions

and segments of society, probably contribute to a sense of disillusion among

communities, and limit participation.

145. C4D coverage in Mali has been increasing and is reaching currently 38 out of 60

districts in the country. Priority districts are defined by the programme sections,

with which C4D engages to contribute to the intended results. Besides collaborating

with the programme sectors, the section works with three C4D partners

countrywide, each focusing on specific areas of activity (traditional communicators,

nutrition related community activities, and radio networks)

146. Maybe the main drawback for C4D in the complex context of Mali (and

elsewhere) is the lack of a developed concept and practice of C4D in emergencies,

which would facilitate quick design and specific operational solutions in the face of

crises. Even if a conceptual agreement exists on the role C4D should play in

emergencies, practical application and allocation of resources is weak.

Understanding of C4D potential in emergencies by other sections is still limited.

147. C4D faces a number of challenges in Mali. It seems that traditional partners are

not suited to deliver activities in insecure or unstable environments; additionally, the

aligning of C4D activities with concrete results in emergency has to happen in

shorter timeframes, and this might add another challenge for the partners.

49

UNICEF Response

148. C4D activities are included in all response plans of the sections, and C4D

section has also articulated its own plan, including what they are providing to the

sectors plus a number of transversal issues. The focus on confidence building and

reconciliation initiatives is very interesting, and judged by all informants as very

relevant in the context of Mali, especially when a positive scenario of stabilization

in the North allowing for the return of IDPs and refugees is likely.

149. The section tried to adapt C4D partners’ approaches to the crisis situation in the

north and six months agreements were signed, including specific targets. 3PM

partners noted that C4D activities were not carried out in the North, due to access

problems and difficulties to deal with armed groups. While the section is looking for

suitable solutions, the RTE found that there is a problem of capacity of partners in

the North, and that finding alternatives in the short term for emergency related

activities will be challenging.

150. In the South, even if C4D is formally part of other sectors’ response, the RTE

could verify the poor performance of community instruments in the areas visited

and other informants (especially partners working in areas not visited) confirmed

this low level of community initiatives in Mali. While budgetary constraints – as

explained above – contribute to this weakness of UNICEF C4D response, there

seem to be broader problems, which deserve attention by the C4D section, and by

other development actors.

151. Accountability to beneficiaries is one of the main gaps of the broad humanitarian

response, and a number of initiatives, tools and indicators are being tested at IASC

working group level. Of particular interest, it would be to develop within C4D

adequate tools to involve beneficiaries in the planning phases of programmatic

sectors’ response. Where UNICEF has established reliable C4D activities through

community networks, it would be possible to capture beneficiary inputs and refine

participation in the response, while Post Distribution Monitoring (PDM) and other

monitoring and client satisfaction tools would provide information on the final

adequacy of the response. This is a field where C4D could contribute to improve the

humanitarian response.

Recommendations

29) C4D to revise strategy defining specific emergency related activities that could provide

clear contributions to the achievement of sections’ targets

30) Capacities of partners should be assessed in light of the needs for the emergency response,

and training of traditional ones or search for more adequate ones pursued.

31) Senior managers to ensure that funding for C4D is made available, including from other

sectors, in order to allow implementing key strategic priorities, also in the framework of the

emergency-to-development agenda

32) Build up from current experience and tools into an Accountability to Beneficiaries

framework in humanitarian action, from the angle of involving beneficiaries in the planning

and monitoring of the response, coordinating with 3PM and PDM the final capture of

satisfaction.

50

PROGRAMME INTEGRATION

152. The RTE team was able to notice consistent improvements in the way MCO

addresses programme integration, in comparison with the situation met in other

Sahel COs during the Real Time Independent Assessment of 2012. In several

documents, ranging from strategies to planning tools (emergency response plans,

country programme extensions, annual work plans, etc.), from multi-sector PCAs to

assessment mission reports, integration between sectors is frequently described,

underlined and theoretically pursued. Some implementing partners acknowledged

the consistency of UNICEF integrated approach and praised the multi-sectors PCAs.

153. Field visits and interviews with various partners and local administration at field

level indicate though, that there are weaknesses in the way programme integration is

implemented. The consultants visited one of the most successful – according to

MCO – nutrition projects, in the district of Kangaba, Koulikoro region. Here they

could find no trace of WASH in Nut kits, supposedly being distributed to families of

SAM children admitted to the programme, neither of the stimulation component (a

recreational area with toys and other stimulation materials at nutrition centre level),

supposedly being provided to these SAM children. Several NGO staff confirmed

that the same situation is common in most of the rest of the country. In the same

tone, HIV testing is absent from CRENIs on a systematic way.

154. At FO level, UNICEF staff indicated that joint, multi-sector assessment,

planning or monitoring missions hardly ever happen, limiting chances to strengthen

the integrated perspective of the response. This might be due also to the limited

accountabilities of FO staff and their dependency from Bamako for programme

implementation.

155. The consultants have been informed of the existence of a field trip report format,

developed by the M&E section, incorporating a section for feedback to other

sections. They also had the opportunity to review a trip report format utilised by the

WASH section, with very clear and concise findings bullet-points and recommended

follow-up actions for other sections.

Recommendations

33) MCO to enhance capacity for multi-sector assessment, planning and monitoring at

FO level, including fine-tuning 3PM partners

34) MCO senior managers to ensure that program integration is a regular item on the

agenda of programme management, emergency management, and UNICEF inter-

Cluster meetings. These should be also the fora to encourage, plan and organise multi-

sector field missions.

35) M&E section to review and compare the standard trip report format and the WASH

trip report format, aiming to finalise and adopt an agreed tool including a follow-up of

action points system.

51

MONITORING AND EVALUATION (M&E)

HUMANITARIAN PERFORMANCE MONITORING (HPM)

156. Humanitarian Performance Monitoring (HPM) is an important initiative

bolstered by UNICEF to harmonize a monitoring system that could provide an

adequate overview of the humanitarian response’s performance. The attempt intends

to build a regional/global approach to the main challenges of humanitarian response

monitoring.

157. The HPM framework has been evolving and adapting to specific contexts. While

during the response to the nutritional crisis in the Sahel in 2012 some COs felt that

the HPM indicators were imposed on a top down process, without proper

consideration of contextual aspects, the process in Mali, as assessed during the 2013

RTE, seems more participative and appropriated by the various programme sectors.

It seems evident that an effort has been made to align the HPM with the CCCs

benchmarks, where the response plans are in principle anchored. However, this

approach seems somehow limited in practical terms. While coherence with the

CCCs needs to be assured, a direct link does not seem to offer an added value in

order to make the indicators more intuitive and operational.

158. HPM indicators are proposed as a means to monitor progress towards agreed

upon targets that would disclose the effectiveness of the response as planned.

However, targets seem in some cases not clearly justified, and lack of progress

becomes evident when analyzing the reports. The format of the presentation

(indicator tables in the monthly Sitrep) varies as well over time, not facilitating the

understanding of trends. HPM need in many cases to better define targets coherent

with the intended response and capacity to deliver.

159. For example, the decision this year to target only a proportion of SAM children

can be accepted as an operational target, based on coverage and capacity in the field,

but UNICEF has to make clear that the aspirational target is assuring that all

children with SAM can access adequate treatment. An analysis of the factors

preventing this and the measures taken to address them has to be explained while

setting up a realistic operational target. If even the operational target seems difficult

to achieve, as is the case in Mali this year, then a judgment of poor performance can

follow if no adequate explanation is provided.

160. The same would apply to other indicators of other sectors, as in WASH,

Education, and CP. It should be a standard to include what is the objective/result

related with the indicator presented: sometimes they measure intended results of

cholera response or of distribution to conflict affected areas, but in many cases are

composite measurements (including conflict affected and flood affected) as well as

other combinations. The reader cannot infer what is to be achieved, hence measured.

161. Some questions remain as the accuracy, feasibility or appropriateness of the

targets defined. The construction of the targets is often an aggregation of different

dimensions. For example; “Number of students (total, girls, boys) of pre-school and

primary age benefitting from learning materials (IDPs; Children in host

communities; Children in flood affected schools; Children remaining in the

52

North”.28

While it is true that when defining the targets a disaggregation of the

components was envisaged: “Disaggregated - Children in the north = 120,000; x, y,

z”, this is not reported in the Sitrep and confuses the interpretation of performance29

.

162. Other indicators are formulated in a way that makes achieving the target

unrealistic. The following one for WASH is formulated against a target of 1.549.672

people. “Number of people who received (temporary) access to potable chlorinated

water (with a rate of residual chlorine between 0.5 and 1 mg / litre) and of sufficient

quantity (≥ 5 litres water / day / pers., in cholera affected health areas ≥ 10 litres

water / day / pers)”. If conflict related, it would equal to all the population of the

North, if only addresses cholera affected areas this should be specified. It apparently

includes various categories of beneficiaries, but this is not understandable. And, in

any case the intention of carrying out residual chlorine test for such a large target

does not seem feasible in the Mali context, other than in urban centres with piped

water supply.

163. An additional problem has been the trend of simplifying the reporting of HPM,

sometimes neglecting important information necessary for the understanding of

performance. For example, the indicator on SAM children admitted to nutrition

centres does not provide information on rates of cured children and of abandonment

of the program. Similar gaps in basic information could be mentioned for

frequentation for health centres, gender disaggregation, de-worming and

distributions of ITN.

164. HPM indicators have been included in the PCAs agreements related to the Mali

response. However, the evaluation team found that the integration of this approach

in the PCAs is weak: sometimes it is not clear if the indicator relates to the HPM

indicator definition, sometimes the indicator is input-oriented and there is no

clarification on reporting frequency and format. Rather than refining the reporting

requirements of partners it seems that an additional burden has been added, and the

compliance of the partner (especially the frequency of reporting) will depend to a

great extent on the capacity of supervision of the technical staff concerned.

165. The RTE team feels that the current utilization of HPM for all sectors in the

Sitrep can negatively affect the public perception of performance of UNICEF

response. Some of the targets and indicators would be easy to clarify, and become

useful tools to understand achievement trends and challenges. Others would require

a more in-depth review.

166. The fact that HPM are presented jointly measuring UNICEF and Cluster

performance is in principle a very good initiative, integrating the response of

UNICEF’s led Clusters. The data management and sources can create some

28 The target is defined at 403.000 children, which coincides with the total of children between 3 and 12

years of the North before the conflict. An estimation of IDPs in school age could be calculated, and the

number of refugees in neighbour countries is as well known. The target could be refined, even

geographically. 29 According to information collected, a breakdown of categories of beneficiaries approach has been

successfully utilised in Pakistan in 2012. This could be used as a reference. Please note also

Recommendation 38, below.

53

inconsistencies among the two levels, but does not seem critical. However,

explanations needed to justify targets are as well relevant for the Clusters.

167. The following table addresses the specific HPM indicators, as in the May 2013

Sitrep, with the RTE comments:

Sector Indicators UNICEF

target

Cumulative

result (#)

Achieved

target (%)

RTE comments

WASH

Number of people who received

temporary access to potable

chlorinated water of sufficient

quantity

1,549,672 262,169 17%

Target to be clarified, and

breakdown provided

(according to CCCs target

should be all affected

population). Explain why

affected population is

different in UNICEF’s and

cluster’s calculation

Number of people who received

access to an improved potable water source

136,000 23,200 17% Same as above

Number of households received a

hygiene kit & a community

hygiene awareness session (in

conflict affected areas)

200,000 12,870 6%

The target equals all

households in conflict

affected areas (North?).

Realistic?

Health / nutrition centres received

WASH interventions (latrines /

safe water/ chlorine products /

hygiene promotion)

110 11 10%

Target: only health centres in

conflict affected areas? Why

low progress? If lack of

funding could be made clear,

might be a fund raising

strategy

Health

Number of people benefitting

from Inter-Agency Health Kits

(IAHK)

1,400,000 195,000 14%

This is an input indicator, not

measuring performance other

than of the supply chain

Number of children under –five

vaccinated against Polio in each

round in the three conflict affected

regions

307,214 15,415 5%

Not easy to accept as an humanitarian performance

indicator, even if this is a

global and urgent goal,

means and strategies are not

humanitarian related

Number of children under – five

provided Vitamin A in each round

in the three conflict affected

regions

276,493 13,627 5%

This can be a proxy of

immunisation coverage, but

is basically related with polio

vaccination, not informative

here.

Number of children and adults

vaccinated against measles in the conflict affected north

To be defined

123,066 Data not available

Even if the vaccination is

going to be reactive to

outbreaks, there is an

outbreak already in Gao. Maybe defining a target and

monitor implementation

would be useful

Child

Protection

Number of people reached by

community level Mine Risk

Education activities

To be

defined 68,337

Data not

available

Target should be defined,

areas of risk are mapped and

known. If the intervention is

reactive the indicator should

be redefined

Number of personnel trained to

conduct Mine Risk Education

activities

To be

defined 188

Data not

available

Not clear why target is not

defined. This personnel

should be framed in some

sort of plan for awareness

Number of children released from No target 7 Data not The indicator should not be

54

armed groups provided with care /

support

available public as such, especially if

not strategies in place to

identify children in armed

groups. Seems that the 7

children are anecdotic

Education

IDP children in host schools/

conflict or flood affected children

benefitting from Education

Supplies

403,800 36,014 9%

Would be good to refine the

target; flood and conflict

areas should not share the

same approach/indicator.

Overlapping, when exists,

should be clarified.

Teachers trained to provide

psychosocial support to students 6,880 1,157 17%

Are those teachers from the

North? What proportion of

teachers in conflict affected

areas?

Temporary learning spaces

installed 200 10 5%

Reasons for low

implementation? No need?

Early childhood development

centres installed 350 18 5%

This is covering a gap in

many areas of Mali, why this

target?, are we targeting

conflict affected areas or

areas where IDPs are

numerous?

C4D

Number of concessions visited by

local communicators 3,672 Not available

Not

available

In April sitrep, figures were

provided in results. This

inconsistency affects credibility of sitrep

Number of group discussions on

EFP 9,546 Not available

Not

available

Same as above, but in

addition the narrative of May

sitrep (p10) discloses 1,875

group discussions carried out

Number of displaced persons

educated on EFP No target

Not

available

In April sitrep 1.789 were

reported in the results column

Nutrition

New admissions to programmes

for the treatment of children with

SAM with no complications

(URENAS) 125,000

24,496

22%

HPM should inform on cured

children. There are concerns

on high rates of defaulters

New admissions to programmes

for the treatment of children with

SAM with complications

(URENI)

2,476 Same as above, cured and

mortality rates matter here

New admissions to programmes for the treatment of children with

MAM (URENAM)

100% of new admissions Target is 450,000 for Cluster, but only 8% reached. Unclear

why so low.

Percentage of health areas

supported by NGOs for treatment

of acute malnutrition

??????

Cluster target ±70%. No

target for UNICEF. Being

one of the causes of low

quality coverage should be

addressed as well by

UNICEF.

WASH in

Nut

Number of severely malnourished

children in treatment programmes

who receive hygiene kits and

hygiene promotion

26,972

2.100 8%

Low results/implementation

has to be explained, if target

will be limited to URENI

admissions then indicator

should be clarified

Education in Nut

Number of stimulation and

recreational centres for

malnourished children functional

52 15 29%

Target seems to intend all URENI, with only one

partner (IRC, according

sitrep) seems difficult to

attain.

55

Recommendations

36) HPM teams locally, regionally or globally, in coordination with programme

sections, should further refine and adapt HPM indicators.

37) Clarify targets in a way that operational and aspirational ones are understood (i.e.

total and operational SAM coverage).

38) Avoid integrating different objectives in the same indicator (conflict affected and

flood affected, cholera prone areas and conflict areas, etc): strategies are different and

measurements of effectiveness should differ.

39) HPM reports should include some analysis of trends and explanations, especially in

case of low progress to target.

40) Performance indicators - as the cured rate of SAM and activity and main conditions

treated in health centres - should be included in the HPM instead of input oriented

indicators (SAM children enrolled in malnutrition programme).

41) When introducing HPM indicators in the PCAs, simplification leading to a more

agile and accurate reporting should prevail over adding new requirements to an already

cumbersome procedure.

THIRD PARTY MONITORING (3PM)

168. UNICEF has been proactive in addressing the monitoring challenges that the

lack of access to the North presented since early 2012. After the retreat of

Government officials some NGOs took the burden of basic service providers and

some community associations kept running some basic education activities; this

raised the need to monitor the implementation of UNICEF supported activities and

programs in the North, as well as the delivery and use of supplies.

169. The first Special Service Agreements (SSA with an NGO and a private sector

company) to provide information as “third party monitors” were signed around mid

2012. The challenges involved were and still are huge, ranging from logistic and

operational, to data standardization, analysis and compliance with each programme

needs.

170. The information provided during 2012 was useful, but to some extent not

systematic enough and not tailored to inform on program performance or

monitoring. This has been addressed in 2013 and new tools developed, creating

already the conditions for a useful monitoring. However, the initial weakness of

being a monitoring not integrated in the program cycle of the projects prevail.

Efforts have been made to fine-tune the monitoring to specific needs of the

programs, and a number of meetings and trainings have been carried out. In

addition, a new post-distribution monitoring (PDM) questionnaire, has been recently

introduced and a comprehensive assessment of adequacy of what has been

distributed is now possible.

171. The potential of the tool is promising, but challenges remain to tailor the

reporting in a way that could be used by programme and supply sections to correct

dysfunctions and address shortfalls. The M&E unit plays a key role, as they can

actually breakdown and present information in a way that could become directly

useful to UNICEF sections. 3PM teams have already information on the active

56

PCAs in the areas to visit and can feedback useful monitoring information, but have

to be channelled adequately in time and format.

172. One example is the finding that C4D partners in the North were not carrying out

the activities foreseen. This brought together the UNICEF C4D team, the 3PM

monitors and the partners that should have been active in the North, to address

relevant issues and to revert the situation. Information regarding the conditions and

use of nutrition and health centres, availability of supplies, and state and supplies of

education facilities are also relevant and captured through the 3PM. Challenges

remain in harmonizing the information provided to the needs and structure of the

program cycle of sections in a way that could be easily taken up by UNICEF staff.

FO have as well a potential role to play in fine-tuning 3PM partners towards

adequate standards of monitoring

173. An additional aspect of the 3PM is the potential to provide feedback from

beneficiaries in a systematic way. With specific adaptations in the current tools,

valuable information on beneficiary satisfaction, adequacy of the response provided,

and complaints could be collected and eventually addressed. Accountability to

beneficiaries is a conspicuous gap in the response of the humanitarian community in

Mali, and UNICEF enjoys a privileged position to address it. Information is already

gathered, needs completing and being managed in a way that would inform the

beneficiary dimension of the response. Accountability to beneficiaries’ framework is

possible to be defined with contributions from the C4D and PDM.

Recommendations

42) UNICEF to keep refining 3PM tools in order to streamline the information into the

program cycle of the sections

43) UNICEF to articulate an accountability to beneficiaries approach though the

inclusion of specific tools in the 3PM (PDM) and C4D (see discussion section)

SITREP

174. Although not formally included in the scope of the RTE, the evaluators reviewed

the MCO Sitrep format and contents, discussing its monitoring (HPM) and

communications components with various interviewees (UNICEF and donors). A

synthetic description of findings and few recommendations are proposed.

175. The document is overall of better quality of those reviewed in 2012 for other

COs in the Sahel. Still, there are quality discrepancies between programmatic

chapters, somehow reflecting the strength and weaknesses of the respective sectors.

176. As indicated in the chapter on HPM, the main technical component of the Sitrep,

there is need for some clarification. Most targets – both for UNICEF and for the

Clusters – are not clear: their origin is not explained, and in some cases they are the

sum of various categories of beneficiaries, which are not explained. The consultants

tried to link to these targets to other available documents, like the CAP 2013 and its

Mid-Year Review, but were not successful. In some cases (Health, CP, C4D) targets

are not indicated at all.

57

177. Donors interviewed specifically stressed the need to provide some clarifications

on these numbers, in order to be able to evaluate trends. The recommendations

proposed in the HPM chapter would contribute to improve the technical part of the

Sitrep.

178. Finally, the general perception about the communications component of the

Sitrep, is that the bullet-point structure offers in most cases “snippets” of

information unrelated to broader strategies. While interesting, these bullet-points are

of limited use for getting an understanding of UNICEF response and performance.

A narrative or short analysis that explains achievements and drawbacks form the

previous Sitreps, keeping some sort of continuity, would help to understand the

response and its trends. The consultants have also been informed that the Executive

Director and senior managers in NYHQ are increasingly asking for one-page

syntheses of different countries Sitreps.

179. The RTE team understands that the format of the Mali Sitrep is decided at CO

level following extensive consultations. They feel that if it has to target different

audiences – technical internal groups/offices, donors, other organisations and

Government, and ultimately Senior Management in NYHQ – with different needs in

terms of conciseness, technical analysis and programmatic details, some changes

should be considered.

Recommendations

44) MCO to include in the Sitrep only sound targets and indicators, providing

clarifications in foot-notes about the origin and breakdown of targets. To avoid

heaviness of the main pages, foot-notes might be condensed at the end of the

document. This could be the document with a communications angle and advocacy

potential targeting donors, NatComs and eventually the general public.

45) MCO and RO Emergency and HPM teams to consider the production of a more

technical HPM-focused document for technical audiences (RO, EMOPS, M&E

colleagues), and, in light of wide-spread reporting delays, consider its production on

a quarterly basis only.

46) EMOPS to consider the “distillation” of one-pager briefs from the monthly Sitreps,

for the Executive Director and other senior managers in NYHQ.

OPERATIONS

180. During the shift from development to humanitarian programming in 2012,

UNICEF Mali was affected, as it very often still happens in these circumstances, by

a focus of the surge on programmatic positions. It took longer to start filling up new

positions to strengthen Operations, Finance, Administration and Human Resources.

By mid 2013, the gap seems filled up, with new staff hired to strengthen the

Operations units of the office and the response. According to the latest version of

the proposed organisation chart, only three positions remain vacant in the

Operations section. According to some interviewees, it seems that although the L2

response was activated already in 2012, MCO had for some time a limited

understanding of the potential of the approach and the opportunities it offers,

especially in terms of easier and more agile financial and administrative processes.

58

181. Several staff members, both in the RO and MCO, underlined the limited

capacities of the RO Operations Section to deploy staff in support of new crises; a

capacity that is well embedded in all other sections, including Supply and Logistics,

ITC, and Communications.

Recommendation

47) The Operations team in the Regional Office should have some deployable staff to

directly support COs facing new emergencies.

182. At RO level, the consultants were informed that the Chief of Operations in the

approved PBR is a P4, but that discussions took place to evaluate if the support from

a P5 during 2012 to kick-start operations was sufficient. Given the macroscopic

scale up of the MCO (from ±100 staff to ±180 by end 2013) and the challenges

posed by the augmented number of partners, as well as the inevitable complex

interactions with MINUSMA, there is a strong rationale – and consensus among

some of the interviewees in the MCO – for increasing the level of the post to P5.

Recommendation

48) In light of the scale-up of the office, RO and MCO to consider raising the level of

the Chief of Operations position to P5 level.

HUMAN RESOURCES

183. MCO benefited since 2012 from the activation of the unofficial Level 2 response

to the nutritional crisis, and later of the official L2 response to the political crisis.

The L2 implied the authorisation to utilise the Fast Track Recruitment Process

(FTRP) for international staff and national staff. The role played by the RO in

support to the MCO in this area is considered effective whenever a specific HR

Surge Office has been in place: during the first half of 2012, and more recently,

since the former Chief of Operations in Mali joined the RO Emergency team to

support it on HR in emergency issues.

184. Broadly, the office made a good use of the FTRP for international staff, and, as

long as the approach was backed up by fast processes at country level, recruitments

of IPs were quite fast. As in the rest of the Sahel francophone countries, the main

challenges have been encountered in identifying good calibre, French-speaking

Nutrition and Child Protection staff.

185. According to the proposed organisation chart, which does not include the

planned office in Timbuktu, there are still nine IP positions to be filled; some of

them are key (Chief, Field Operation; Chiefs, Field Offices). Additional important

IP positions are likely to become vacant in the near future (Deputy Representative;

Chief, Emergency; Chief, Nutrition).

Recommendation

49) In light of need to speed up recruitment for key positions and FO positions, MCO

and RO should advocate with DHR and EMOPS to extend either the L2 or at least the

authorisation to use the FTRP till end 2013.

59

186. A review of some HR matrixes for national staff though, show a different

picture, with many recruitment cases spanning up to nine, ten and even twelve

months. The proposed organisational chart indicates at least 16 national staff

positions still to be filled, and it does not include the Timbuktu office. In the case of

Mali, the FTRP could be utilised also for the recruitment of national staff. It appears

though, that the collaboration between MCO and RO, with the latter having a key

role to play in these recruitments, was not effective. This has been stressed by some

participants in the RTE exercise. UNICEF’s credibility and reputation is also linked

to its capacity to recruit right people for the right place at the right time, it is

essential to speed up this process and find ways to eliminate, or at least mitigate,

existing bottlenecks. During the Horn of Africa crisis, ESARO piloted a FTRP-type

of approach, in support to COs affected by the crisis (Somalia, Kenya, Ethiopia),

focusing on national staff. The review of this approach considered it very

successful.

Recommendation

50) MCO and WCARO HR sections should undergo a review of national staff

recruitments in 2012 and 2013 and identify bottlenecks affecting the process, especially

in relation to the FTRP framework. They should also assess the FTRP-type of approach

piloted by ESARO for national staff with the aim of replicating it in WCARO for Mali,

and potentially other crises facing HR challenges.

187. In light of the scale up and the change in scope of the programme, MCO has

already obtained the authorisation to close two of its previous, small-scale Field

Offices in Koulikoro and Segou, to strengthen those in Kayes and especially Mopti,

and open new ones in Sikasso and Gao. Processes to authorise also the opening of a

Field Office in Timbuktu are ongoing.

188. This decentralisation of capacities seems an important move, especially if

coupled with a consistent review of the responsibility, accountabilities and report

lines of these FOs; it comes though, with some HR challenges. Several staff

members are seeing their post abolished in one place, and offered an alternative one

in a different location. While this is inevitable, it requires very careful management

and complete transparency of processes, in order make everyone involved

understand risks and opportunities, and to take as much as possible into

consideration personal issues and constraints.

Recommendation

51) MCO, in close coordination with the MCO Staff Association to ensure complete

transparency of the FO structure reshaping processes and, with RO support, ensure that

personal issues are taken into consideration to the maximum possible extent.

SECURITY

189. While any type of security assessment was beyond the scope of this RTE

exercise, the consultants had the chance to gauge how security arrangements and set

up are influencing the delivery of programmes at field level. Following the field

visit to Mopti, Segou and Nampala, and after talking to several programme people

from UNICEF and other UN Agencies, some conclusions have been drawn.

60

190. The set up at field level requires UN staff to obtain security clearance from

Bamako whenever they need to go to the field (project/monitoring visit, assessment,

etc.) even if very close to the FO location. The RTE team was informed by UNICEF

staff that given the limited number of UNDSS security staff in Bamako, and their

increasing workload due to the deployment of MINUSMA, field staff are required to

submit the request for security clearance 48 hours in advance. Exceptionally, the

process can take 24 hours. According to OCHA, the process should and could be

less cumbersome.

191. These constraints seem to affect the agility of movements and consequently the

effectiveness of the response. Some interviewees indicated also that UNDSS has a

risk-adverse attitude in Mali, especially in the south, where risks are considered

smaller then in the north.

Recommendation

52) MCO to advocate with UNDSS for having one security office in each FO location,

authorised to issue security clearances, and attuned to the reality of the area covered.

The criticality exercise, defining critical outputs or activities by the whole UN country

team, should be used as baseline document for UNDSS staff to find acceptable solutions

to implement key field activities.

SUPPLY AND LOGISTICS

192. Discussions with Supply and Logistics staff both in the RO and in the MCO,

triangulated with a review of shipments and delivery records, and with impressions

from implementing partners and donors, indicate that the performance of UNICEF

in this sector has been overall successful. MCO had reasonable capacities in-country

at the beginning of the political crisis with prepositioned contingency stocks and

draft supply plans, and was benefitting from the unofficial L2 activation for the

nutritional crisis. Support from the RO was limited and focusing more on testing

and optimising monitoring tools than on increasing capacity. Some challenges were

caused by the roll out of VISION at the beginning of 2012, and by some personality

issues, now solved.

193. Supplies have been properly planned, prepositioned and mobilized in time

according to response plans. Supplies have been stored in adequate warehouses of

the right capacity and stock management procedures, and no distinction is made

between emergency and regular programme ones, which is consistent with the

nature of the UNICEF response in the country.

194. During the occupation of the North in 2012, the logistic section was able to keep

shipping trucks with supplies to partners by using local contractors and maintaining

a low profile. Practically no breaks in the pipeline were reported, other than one

truck damaged. In the North however, 3PM partners have identified lack of supplies

in some facilities, which were out of reach for the RTE (see Limitations) and should

be further investigated. The preliminary conclusion is that these problems lie more

in the management capacity of the partner to adequately place orders to UNICEF in

time and in the complex distribution to some remote areas, than in an actual failures

from UNICEF supply and logistics.

61

195. In the South, UNICEF delivers supplies up to the district level. Given the large

extension of most districts, further dissemination to CSCOMs is challenging and

sometimes not effective, counting on local capacity for final distribution. Local

capacity is weak, and in some cases results in extra costs for CSCOMs or

beneficiaries. Ad hoc solutions are occasionally found, like adding small quantities

of supplies to administration vehicles traveling for other reasons, or taking

advantage of WFP distributions for the MAM programme.

196. The RTE noted that Supply Division in Copenhagen has developed a software

called UNITRAK to track supplies from partners to the end-users and beneficiaries.

UNICEF Mali seems ready to volunteer as a pilot country to implement this new

tracking system.

197. The records indicate an adequate proportion (between North and South) in the

delivery and distribution of multi-sector supplies. Approximately 70% of the

supplies (in metric tones, MT) shipped to the South reflect the heavy weight of food

for the nutritional programme, targeting much higher numbers there than in the

North.

198. The breakdown by sector reflects also the nature of the intervention and the high

proportion of nutrition related supplies to the South, reflecting programmatic needs:

Total MT to North: 337 Total MT to the South: 729

199. A simple comparison of the value of the above supplies (for Health, Nutrition

and WASH) compared with the planning figures in the Response Plan matrixes is

proposed:

NORD SUD Total 2013 RP planned

NUT $488.489,50 $1.347.096,50 $1.835.586,00 $ 6.830.705

WASH $863.903,56 $ 680.219,44 $1.544.123,00 $ 4.000.000

SANTE $150.282,88 $ 203.029,75 $ 353.312,63 $ 1.325.118

200. The matrix suggests that the supplies distributed confirm the low progress

towards targets in comparison to expenditure rates that the RTE has raised for all

sectors (with this approach WASH seems better off than nutrition and health). The

fact that the expenditure rates are higher and in line with the period of the year that

12%

83%

4% 0%

MT shipped to the South Section's breakdown

Wash

Health

Nutrition

Education

Protection

39%

6%

21%

34%

0%

MT shipped to the North Section's breakdown

Wash

Health

Nutrition

Education

Protection

62

has already elapsed can be explained by the fact that supply orders have been

already placed and accounted for, while actual distribution will take place in the

future, following the demands of program implementation.

201. An additional aspect that the supply value suggests, is the higher unit costs for

nutritional treatment in the North (cost of the goods - F100, F75 and Plumpy Nut -

supplied per beneficiary, not considering staff and other associated costs):

Beneficiaries

North South Total

URENAS 5526 26457

URENI 459 2715

5985 29172 35157

202. The analysis indicates that SAM treatment for each patient in the North costs

81,6 $USD ($488.489,50 to treat 5.985 patients), while in the South the cost is of

52,2 $USD ($1.347.096,50 to treat 29.172 patients), assuming all supplies have been

used already for the registered number of children admitted. This is coherent with

higher costs associated in reaching beneficiaries in the North.

203. A lot of caveats lie within these calculations, but are offered here to provide a

glimpse of the challenges and differences of the responses in the two regions. As

expressed in the limitations of the methodology, lack of accuracy and timeliness of

data affect an adequate calculation. Denominators (number of patients active in the

program, total numbers for a period of time, etc) can only be taken as estimations.

Recommendation

53) Supply Section to work with programme sections to revise discrepancies between

expenditure and implementation rates, looking at processes and structural delays of the

supply delivery systems of implementing partners and Government counterparts.

COMMUNICATION

204. The communication capacities of UNICEF MCO are in the process of being

upgraded to face adequately the challenges ahead. While it seems that during 2012

UNICEF was not vocal enough in advocacy and communications, and lacked

capacity (staffing challenges) and an adequate strategy, these aspects have been

addressed in 2013. From March this year a new responsible has been appointed,

commutation is now separated from C4D and encompasses resource mobilization,

advocacy and partnerships. It is anchored with the MCO priorities, and the Chief of

the section reports directly to the Representative in the new organizational chart.

The scale-up process will be finalized by end July, with new staff positions covered

and report-lines revised. Currently, an AWP is being agreed upon with the

government and will define the communication strategy of UNICEF Mali for the

period 2013-2014.

205. UNICEF has become the main operational UN agency with a significant

communication capacity. OCHA has by mandate a communication and information

role, but confined to the humanitarian response and with little operational

63

implications. Other Agencies communications sections are small in size and scope,

in comparison with UNICEF.

206. Communication largely depends on the strategies and performance of the

sections to build and share success stories; it is anchored also in the priorities of the

organization and can become a potent tool for advocacy. The new communication

team faces some important challenges, starting with the need to better communicate

UNICEF priority lines of action in the country and build communication stories

around these. Cholera awareness campaigns, protection campaigns related to

CAAG, nutrition initiatives, and other elements of UNICEF response can be

supported by press releases, fact-sheets and success stories.

207. The additional big challenge for UNICEF Communications will be the

deployment of MINUSMA, bringing to Mali dozens of international

communications staff and a senior (D1) Chief of Communications. This “occupation

of the communications space” by MINUSMA will require a sound positioning by

UNICEF, and a clear strategy and capacity to interact with the Mission in a

complementary and constructive way. This interaction will be critical not only for

UNICEF, but also for the broader UN family, as well as the other humanitarian

actors.

208. Advocacy is going to be a key aspect for the new communication team. As

explained in the report, huge advocacy challenges are relevant in Mali (as in the

region), and not always sufficiently addressed. Making the case for free access to

health and nutrition services for women and children, the equity challenges between

different segments of the population, are only two of the potential areas of concern

and responsibility for UNICEF as will be ensuring that humanitarian space is also a

major priority issue in the context of MINUSMA

209. The RTE explored to what extent the Sitrep is perceived as a communication

tool. This seems a complex issue, as the current format appears oriented towards

monitoring of the response, with little attention on how the information is

communicated. A stronger communication angle with very clear and concise

information about implementation trends seems advisable in products informing

about UNICEF success in the humanitarian response, targeting donors and

NatComs.

Recommendations

54) Communication team to quickly define its niche before full deployment of

MINUSMA, and to find ways to interact constructively with Mission’s

Communications team.

55) Representative and RO to strongly support the section in its dealing with

MINUSMA and in raising the profile of UNICEF in the emergency-to-development

agenda.

56) MCO to build key advocacy strategies to address access to services and equity

concerns in Mali.

57) The communication dimension of the Sitrep should be taken into account when

addressing a review and/or renovation of the tool.

64

COORDINATION

210. For the ten years prior to the current humanitarian crisis, major Development

Partners in Mali have coordinated through “Technical and Financial Partners” (PTF)

groups, aimed at harmonising assistance and aligning it to national development

strategies and plans, and at improving effectiveness. Two levels of coordination are

in place under this structure: the political and strategic level (Collective PTF), and

the technical and sector-level, through 9 Thematic Groups30

. The mechanism is

chaired by a Troika, whose presidency rotates between two bilateral and one

multilateral partner. Germany is currently the leading PTF, with the World Bank

and Denmark.

211. After the coup d’etat in March 2012, the PTF members jointly agreed to suspend

cooperation with Mali, concerned by the Government capacities to ensure sound

financial management. Several donors decided to divert part of their contributions to

UN Agencies31

.

212. At the beginning of 2012, in the context of the deployment of humanitarian

assistance mechanisms and to strengthen coordination of humanitarian assistance,

seven IASC Clusters and few additional sub-Clusters were activated. UNICEF got

the responsibility to lead the Nutrition and WASH Clusters, the Child Protection

sub-Cluster, and to co-lead with Save the Children the Education Cluster32

. As

indicated by several respondents, during 2012, collaborations between the PTFs and

the Clusters were far from optimal, due mainly to difference in agenda, approach

and mentality.

213. As of May 2013, most of the PTF’s Thematic Groups have resumed political and

technical dialogue with local authorities, and during the last months their

collaboration with the Clusters has significantly improved. UNICEF-led Clusters

regularly participate to the works of specific PTF technical working groups.

UNICEF has been asked and has accepted to chair the newly established PTF sector

working group for WASH. UNICEF was also asked to lead the PTF sector working

group on Education and Vocational Training, but has refused the offer. The recently

established Commission for Post-Conflict Rehabilitation in the North – including

“facilitation and support of transition from emergency to development” among its

specific objectives – and the possible future one on Resilience-Strengthening are

both fora where UNICEF could play a key role.

214. Some of the Clusters are already looking at how to strategically fold into the

PTF working groups, if and when the situation in the country will stabilise and the

30 The nine thematic working groups and their leads are: 1. Agricultural Economics and Rural Development- Coordinator: United States; 2. Economy and Finance - Coordinator: European Union

Delegation; 3. Decentralization and Institutional Development - Coordinator: Germany; 4. Justice and

the fight against corruption - Coordinator: Canada; 5. Democratic Process and Civil Society -

Coordinator: DUE; 6. Education and Vocational Training - Coordinator: World Bank; 7. Health

(including HIV AIDS, Water and Sanitation) - Coordinator: Canada; 8. Gender and Women's

Empowerment - Coordinator: UNFEMME; 9. Environment and Climate Change - Coordinator: Sweden 31 UNICEF received significant ORR contributions from Canada; Netherlands; France and EC during the

crisis. 32 Other Clusters are Food Security, Health, Protection, Logistics, and Emergency Telecoms.

65

humanitarian component of the response will be downsized. The process will vary

among Clusters/sub-Clusters, depending on capacity and role, and especially at

regional level: transitional structures will likely appear in the southern regions first.

According to OCHA, some of the cluster-specific coordination arrangements, such

as participation of lower level actors (NGOs, CBOs) as well as decentralisation to

field offices, might be incorporated in the PTF working groups.

215. In this environment, where coordination structures with different history and

background are progressively bridging their differences and looking at how to

facilitate the transition from emergency to development, the UN family should be in

a key position to accompany the process. According to several interviewees,

coordination between UN Agencies is not ideal. While personal relations ensure

some good collaboration (UNICEF and WFP), the arrival of MINUSMA is

diverting some actors’ attention from the current response.

216. In the framework of the RTE, the evaluators carried out a small-scale survey of

the UNICEF-led Clusters and sub-Cluster. For WASH, it was also possible to build

on a similar internal survey recently carried out by the Cluster’s members. Members

were asked to fill up a questionnaire with 19 questions related to different aspects

and responsibilities of the Clusters; each question could be scored from 1 to 5 (from

very bad to very good) and comments could be added. 35 questionnaires were

received from members of the fours Clusters or sub-Cluster chaired by UNICEF33

.

217. The results from the survey indicate that broadly all Clusters are performing, up

to the commitments agreed upon for cluster coordinator and that UNICEF

commitment to recruit long-term staff for both coordination and information

management (IM) positions – instead of counting only of short-term Stand-by

Partners support in need of continuous replacement, as it happens in several other

countries – is a sound approach. The matrix below shows a synthesis of the best and

worst scoring areas of responsibility of the Clusters.

Areas of Responsibility Average Score

Planning and strategic planning 4.48

Coordination with Government and local administration 4.48

Cluster internal coordination 4.45

Monitoring and Information Management (IM) 4.39

Technical expertise 4.37

Leadership 4.37

Capacity building 3.70

Participatory approach 3.70

Cross-cutting issues 3.59

Fund raising 3.10

218. While this RTE approach does not offer the detailed insight provided by the

formal IASC Cluster surveys methodology, it indicates that the main responsibilities

of the Clusters are fulfilled, while some challenges remain in areas traditionally

33 Questionnaire format and the broad results for the survey are included in the Annexes of this report.

66

more difficult to work on. The lowest score for fund-raising reflects the limited

success of the Consolidated Appeal, so far only 29% funded34

.

219. All UNICEF-led Clusters have been to different extent decentralised at field

level (Education in Segou and Mopti; WASH in Segou; Nutrition in Kayes, Segou,

Kuolikoro and Mopti; Child Protection in Mopti). According to several interviewees

(UNICEF, UN Agencies, NGOs), it seems that decentralisation is not effective and

many of these field level structures exist only on paper. Among the problems, some

respondents indicated that giving responsibility to chair the decentralised Clusters to

local administration is premature, and that UNICEF and NGO staff involved have

still limited understanding of the Cluster approach. Nonetheless, most interviewees

stressed the importance of strengthening the structures, and underlined the urgent

need to establish Clusters arrangements also in Timbuktu, Gao and eventually Kidal.

Recommendations

58) Mali CO to consolidate already decentralised Clusters under UNICEF responsibility

with adequate leadership and planning and monitoring capacities, and advocate for the

activation of Clusters in Timbuktu and Gao.

59) Engage in linking coordination with development partners instrument, in the areas

of responsibility of UNICEF, and design specific tools for harmonizing humanitarian

and development strategies and coordination

TRANSVERSAL ISSUES

GENDER 220. The RTE tried to investigate the gender perspective of the humanitarian response

by looking at how planning documents and tools take this aspect into consideration,

and by assessing its implementation at field level in the framework of UNICEF-

supported projects and initiatives.

221. The findings have been limited. Aside few planning documents and the

emphasis on gender found in the GBV approach – in itself still quite small –, the

consultants found few instances of gender breakdowns in the information

management systems at CSCOM level, and no consolidation at higher level to

consent better planning.

222. While gender is an issue in the context of the Sahel, and especially in Mali in the

context of a humanitarian crisis compounded by an armed conflict, the gender

awareness in project documents is weak, and not apparent in implementation. This

is probably an area where UNICEF has to pay specific attention in light of the

upcoming complexities of the MINUSMA deployment in the country.

Recommendation

60) All sections in MCO to review and assure inclusion of gender in their planning tools

and in the implementation of their programmes. The office might look into having a

gender specialist assisting in this exercise.

34 As per OCHA Financial Tracking Service (FTS), 4 July 2013, in relation to revised needs of 476M$ as

per Mali Consolidated Appeal mid-year review.

67

ACCOUNTABILITY TO BENEFICIARIES 223. This is a major gap in the humanitarian response system. This “accountability

deficit” in relief interventions is the subject of on-going concerns among

humanitarian organisations. Accountability toward beneficiaries entails empowering

individuals and communities at the field level to become active participants in the

relief and development processes, to assert their rights and demand for greater

transparency from humanitarian organizations, and partnering with them for

sustainable community recovery. This is a challenging aspect that cannot be

ignored, but ways to address it are still elusive. The IASC working group on

accountability to beneficiaries is developing tools and indicators to boost this

dimension within humanitarian operations.

224. This RTE has found that community mechanisms seem dysfunctional in the

areas visited in Mali, and this coincides with observations of partners on the ground.

Possibly, not enough attention had been paid to community participation and

initiatives in the pre-crisis development operations or some sort of fatigue has

occurred, and this may have contributed to the poor performances generally

reported. Although challenging, the humanitarian crisis could represent an

opportunity to resume paying attention to the beneficiary dimension with new

approaches. In the context of Mali, there might be opportunities to better collect and

utilise beneficiaries’ feed-back and perspectives to assist the bridging between

humanitarian and development interventions.

225. Beneficiary participation is currently broadly absent at the humanitarian

coordination level and when defining strategies of response. Adequacy of what is

provided is not checked, and satisfaction (or dissatisfaction) of beneficiaries is not

captured. Dozens of small scale needs assessments have been carried out (OCHA

manages a database of those), but little information is available on the level of

adequacy of the response, while some available information suggest that

beneficiaries have not received what they expected.

226. UNICEF is potentially in a privileged position to address accountability to

beneficiaries. The nature of some programs, with strong community components,

the need to interact with the community for education, health, nutrition

interventions, the relevance of C4D and the development of the 3PM approach with

specific tools to capture beneficiary satisfaction, could pave the ground for critical

developments in this domain. This is mentioned in the context of the comments on

3PM and C4D sections of the report, and specific recommendations proposed.

227. The RTE findings related to the weaknesses of the current community based

component of the nutrition programme, and the somehow minor attention (financial

and operational) paid to C4D, are of some concern. But the potential is there, and as

the identified weaknesses could be addressed, a real opportunity of addressing

accountability to beneficiaries within the work of UNICEF exists.

ETHICAL ISSUES

228. The team assured the necessary measures to take ethical dimensions of the study

into consideration. Confidentiality of the interviews was assured, and the adequate

information on the objectives and use of the information provided was conveyed to

68

both UNICEF and external interlocutors. Consent was always required, although

this type of exercise did not allow getting a signed consent document from

participants of interviews and discussion groups. All persons contacted were offered

declining their participation without any consequence and any kind of pressure was

carefully avoided. The Cluster survey was designed in a way to offer Cluster

members the option to participate or not, and they were informed that data collected

would remain confidential.

CONCLUSIONS

229. Notwithstanding the limitations described (paragraphs 38-41), the RTE has been

able to raise some critical issues of relevance to UNICEF. Building from the

findings of the exercise, we propose some overarching conclusions, followed by

comprehensive recommendations.

230. The consultants have been able to verify the complex process of change that the

MCO has been through since 2012, and that will culminate, hopefully by the end of

this year, with the completion of the new organizational chart, the filling up of

additional key positions with staff on TA contracts, and new, stronger Field Offices

with more responsibilities and decentralized supervision lines. This is coherent with

the challenges the MCO has experienced since the beginning of the nutritional crisis

at end of 2011, followed by the events of 2012 and 2013. More challenges and

opportunities lie ahead, as described in the report; these will require adequate

capacity and strategic approaches by MCO.

231. Proactivity of the senior management in addressing this change process has to be

praised. However, the process seems to go ahead with limited engagement of part of

the staff, and it is likely going to raise occasional resistances. The consultants did

not see a properly formulated change plan, aiming at involving all and sharing

vision and goals to motivate team spirit. Some interesting elements of team building

have been noted, as the “stand up meeting” format, but we believe this should be

inscribed within a broader type of measures to engage all into a new Mali CO

model. Change plans are challenging processes in complex organizations, and

require specific attention.

232. We noted as well the good reputation gained by UNICEF vis-à-vis humanitarian

partners thanks to the professional participation in the initial phases of the

humanitarian response in 2012. The important role played by the Deputy

Representative and the emergency team was acknowledged by partners and other

organisations, as their capacity to raise awareness on humanitarian challenges and

principles. The RTE had the impression that due to the macroscopic change the

MCO is going through, this high level leadership for humanitarian issues has been

recently partially sidelined. In light of the still existing humanitarian challenges, the

possibility of worse case scenarios and the deployment of MINUSMA, the RTE

believes that there is a need to address the issue of high profiling UNICEF within

the emergency context again.

233. The process of switching from development to emergency in Mali built on some

qualified and dynamic technical staff, a well-articulated country strategy, good

interactions with the Humanitarian Country Team (HCT) and with Government

69

counterparts, and on strong support from the RO. Quite a lot has been achieved in

terms of response-planning, of monitoring systems, and in expanding partnerships.

Notwithstanding the limited opportunities to visit field locations and projects, the

RTE could not avoid though, noting the limited impact of the response so far, and

that not enough is happening in the field given the resources and the capacity of

UNICEF MCO.

234. Emergencies require capacity to deliver in the short-term, when needs are

pressing. MCO knows what it wants to achieve, it has most capacity and means in

place, and even if the CAP is poorly funded, thanks to available ORR financial

resources do not seem a critical issue. It would appear though, that the office is

facing some sort of overall challenge to translate technical capacity and adequate

expenditures into results on the ground. The HPM provides some indication of these

low implementation rates, although possible delays in reporting and other

limitations of the information flow prevent to capture in real time what has been

achieved.

235. The sections provided partial justifications for these delays and discrepancies

between expenditure and implementation rates, but evidence from field visits and

numerous interviews (major delays in Nampala WASH project, low frequentation of

nutrition centers, very limited integration of other sectors’ components in the

nutrition programme, limited achievements of the CP section, failed C4D activities

in the north, etc.) indicate that while all the elements for a sound response are in

place, the last push is missing to make things happen. The impression is that the last

step of the response - the materialization of the results - seems in some cases

affected by “development inertia”. This suggest that some internal processes might

not be adequate to emergency response, and UNICEF sections should look at

identifying ways to overcome the bottlenecks that, while not critical for a

development or institutional response, affect the delivery of timely humanitarian

assistance.

236. Upcoming challenges and opportunities related to MINUSMA deployment will

require as well an organizational position. Attention should be paid to ensuring

capacity for institutional relations with the mission, as well as leadership and

coordination capacities for interactions with its political, protection and

communications branches. Advocacy and communication capacities will be critical,

and UNICEF senior management must be involved, both to support the

Communications section and to be the credible face of UNICEF. The RTE team

believes that the new communication and advocacy capacity being consolidated will

be of critical importance.

237. As indicated in the chapter on General Findings, Mali is in a very special

context. There is a sense that many years of development in the country did not

attained expected results; key development indicators are still extremely low, and

the activation of the humanitarian response somehow shed some light on the chronic

weaknesses still affecting social services. Currently, there is in Mali a concomitant

fortunate presence of strong development and humanitarian actors and

organisations, as well as funds available to flexibly cover both areas. After an initial

challenging communication between development and humanitarian coordination

structures, now Clusters and PTFs seem to collaborate constructively, and the

70

former are already looking at way to fold into the latter in the future, as soon as the

humanitarian crisis will recede.

238. UNICEF long-term presence and mandate make it a special actor in this context.

Its responsibilities for both humanitarian interventions – framed by the CCCs and

the Cluster responsibilities – and its institutional collaborations with the

Government, should allow the organisation to position itself strategically into this

environment. UNICEF has been already asked to chair some key technical working

groups (WASH, Education) or development oriented donors, and it is an important

player in other fora looking at, and working on, post-conflict transition and

reconstruction. The concept of bridging the short-term humanitarian response and

the long-term development needs of the country recurs in the ToR and work-plan of

all these coordination structures.

239. There is a lot of expectation among donors, NGOs and Governmental

counterparts for UNICEF leadership in key areas of this strategic approach to

transition. If it will be able to demonstrate this leadership and its implementation

capacities, UNICEF should face no problems in getting the back-up of key donors,

and the support of other organisations.

240. UNICEF Mali has already put in place many elements to respond to this

challenge and take advantage of this window of opportunity. It could become a key

driving force behind some of structural changes this country and its people and

children need. It would be a pity if this opportunity will be missed.

71

RECOMMENDATIONS

A consolidation of the Recommendations developed in the text of the report is proposed here with the explanation of the finding behind them and

the related number/s of paragraph/s in the text. Additional comments – whenever suitable – are included, as well as the consultants’ perspective

on Importance and Urgency. Numeric scoring from 1 to 5 (1 = very low; 5 = very high) – and the average between the two consultants – is

proposed for each Recommendation’s Importance and Urgency. Not having a sufficient knowledge of UNICEF systems and capacity of the

involved offices, the team did not attempt to score Feasibility. The scoring of these Recommendations offers the basis for the scoring and

prioritization of the clustered recommendations presented in the Executive Summary.

Sector Average

RTE

Action General Reference in text of the report (paragraph) Comments Imp Urg Total

1 MCO RO support

In light of the need of maintaining the acquired strong emergency response and humanitarian advocacy capacities at senior manager

level, especially in the event of worse case scenarios, MCO should prioritise, aside other key positions, the recruitment of a Deputy Representative versatile on emergency and transition, and a strong Chief of Emergency.

Need for MCO to maintain strong capacity at senior management level to address key humanitarian policy

issues and challenges. (Para 45, 46, 232).

5 4 9

2 MCO RO support

MCO, with support from RO, should undertake a strategic planning exercise to develop/refine a clear and comprehensive multi-sector strategy to bridge the transition from emergency to

development. The strategy should include a resource mobilisation component, to address the CAP underfunding and additional ORR fund-raising. MCO to implement this multi-sector strategy credibly, and be vocal about it internally and externally.

Window of opportunity for UNICEF to become leading agency in the emergency-to-development transitional agenda. (Paras 55, 56)

Partially already happening with new AWPs

4 3,5 7,5

3 MCO RO support

MCO, with support from RO, to review its current chairing and coordination responsibilities, evaluate new ones, and identify best possible ways to play a leadership role at both general programmatic and sector level.

A window of opportunity has been created by several concomitant factors for UNICEF to become a key leading and coordinating actor in the development and implementation of the emergency-to-development

agenda. Its role of Cluster lead for four sectors, and its increasing role in the PTF working groups and other fora reinforce the bases to take this leading role (Paras 49, 50, 51, 52, 55, 56, 212, 213)

Might require authorisation for TAs to embed in key fora

4 3 7

4 MCO RO support

MCO, with support from RO, to speed-up re-shaping of Field Office structure. Focus should be on strengthening existing FOs, revising reporting lines, responsibilities and accountabilities, and

Importance of decentralisation and increased capacity at field level, requiring the speeding-up of Field Offices restructuring process (Para 54, 172, 219, 188)

4,5 5 9,5

72

speeding up the opening of FOs in the North to expand

programme implementation capacities.

4 bis

EMOPS, RO, CO EMOPS to assess the suitability of including in the L2 and L3 standard operating procedures (SOP) a link to the emergency PBR approach, so that COs for which the Level 2 response is activated would be aware of it.

“Option to use Emergency PBR should be offered/reminded to COs for which an L2 response is activated" Para 56.2

3 2 5

Action Health Reference in text of the report (paragraph) Comments Imp Urg. Total

5 Health section and senior managers, policy advisor

Health section and senior managers to advocate for access to care for the unable to pay, revitalising working group to come up with practical recommendations.

Disparities created by the detrimental effects of the Bamako initiative underlined the rationale for free access to basic health and nutrition services. International community not taking advantage of the opportunity created by the humanitarian crisis to advocate for this. Potential important role for UNICEF to play, developing sound advocacy and communication strategy (Paras 61, 62, 63, 64, 68, 85, 208)

Structural problem; long-term strategy

4 2 6

6 Health section, C4D

Health section, in coordination with C4D to reactivate community aspects of the health programme currently neglected in many areas

Wide-spread weaknesses of role played by communities in UNICEF planning and monitoring activities among all sectors (Paras 77, 94, 98, 110, 144)

To coordinate also with other sectors

3,5 3 6,5

7 Health section and M&E

Health section to establish a regular information system based on indicators included in the RP, oriented to measure utilization of services, coverage and quality in priority areas (in the North and

in particularly vulnerable areas of the South)

Weak information management system for the provision of health services country-wide. Unclear, incorrect targets and indicators proposed by UNICEF to monitor

its performance. (Paras 73, 74, 78, 81)

4 3,5 7,5

8 Health section and M&E

Health section to strengthen collaboration with 3PM partners in order to capture relevant information for program monitoring and management.

Collaborations between 3PM partners and sectors (not only Health) have improved considerably since 2012; there is still room to further optimise this collaboration and integrate 3PM feedback into workplans (Para 74, 170, 171, 172)

Similar for other sectors

3,5 3 6,5

9 Heath section,

HIV unit and Nutrition

Health section to address HIV related gaps, assuring testing at

URENI level and functional PMCTC in the North. Partners should be aware of the protocol to apply for patients in ARV that discontinued treatment.

HIV testing at nutritional centre level apparently not

happening. Protocol for patients in ARV therapy not widely known and adopted (Para 80)

4 4 8

Action Nutrition Reference in text of the report (paragraph) Comments Imp Urg Total

10 Nutrition, senior managers, policy advisor

Advocate for access to nutritional services for women and children in line of what is recommended for the health section. UNICEF should build an integrated institutional position towards

a problem that is based on inequities affecting women and children

Disparities created by the detrimental effects of the Bamako initiative underlined the rationale for free access to basic health and nutrition services. International

community not taking advantage of the opportunity created by the humanitarian crisis to advocate for this. Potential important role for UNICEF to play, developing sound advocacy and communication strategy (Paras 61,

Structural problem, long-term strategy

4 2 6

73

62, 63, 64, 68, 85, 208)

11 Nutrition section Establish an operational plan with enlarged base of NGOs in order to cover adequately the needs for support

Limited coverage of nutritional response based also on limited numbers and capacities of implementing partners (Paras 92, 94, 98).

Very urgent if UNICEF wants to attain targets

5 4,5 9,5

12 Nutrition section, M&E

Enhance supervision capacity at field level (this RTE would favour strengthening the functions of FO on this aspect) and health/nutrition information systems, in order to allow for a systematic analysis of performance indicators of nutrition

program

Poor quality of information management system for SAM programme at CSCOM level. Insufficient indicators for UNICEF programme performance monitoring (Paras 89, 97)

5 4 9

13 Nutrition, WASH, education, C4D

Integrate and reinforce currently weak interactions of other sections (child stimulation, WASH in Nut, HIV testing, C4D)

Poor integration of other sectors in the nutritional response (Paras 98, 153)

Also a visibility issue 4 4 8

14 Nutrition, C4D Establish mechanisms of beneficiary feed back as users of nutrition centers, and strengthen beneficiary participation in the community process

Wide-spread weaknesses of role played by communities in UNICEF planning and monitoring activities among all sectors (Paras 77, 94, 98, 110, 144)

4 3 7

15 Nutrition, senior management

Strengthen technical and coordination capacity at FO level. Limited capacity of UNICEF staff (indicated for Nutrition, but valid for all sectors) at FO level for planning, monitoring and information management (Para 100)

4 4 8

Action WASH Reference in text of the report (paragraph) Comments Imp Urg Total

16 WASH WASH section, based on availability of funding, to keep

increasing number and scope of PCAs with NGOs progressively scaling up, and further expanding capacity building of local administrations. Work with partners and counterparts should also aim at reducing reporting delays.

limited coverage through partners, starting to be

addresses, for the WASH programme (Paras 105, 109, 113).

Indispensable to

expand coverage

5 4 9

17 WASH, Nutrition WASH section, in coordination with Nutrition section, to investigate bottle-necks and delays in the implementation for the WASH in Nut programme aiming at improving efficiency and

effectiveness of the project.

Limited effectiveness of WASH in Nut strategy and programme (Para 109)

4 4,5 8,5

18 WASH, M&E WASH section to further investigate and optimise post-delivery, distribution and utilisation phases of key supplies, and service provisions of contracted companies, aiming at mitigating bottle-neck and improve efficiency and effectiveness of the response.

Discrepancies between expenditure and implementation rates in WASH (Para 111). Discrepancies between expenditure and implementation rates in other sectors (Paras 74, 81, 99, 128)

4 4,5 8,5

19 WASH, Nutrition C4D

WASH section, in coordination with other sections and C4D, to devise and improve strategies to enhance community

participation in support to the programme.

Wide-spread weaknesses of role played by communities in UNICEF planning and monitoring activities among all

sectors (Paras 77, 94, 98, 110, 144)

4 3 7

Action Education Reference in text of the report (paragraph) Comments Imp Urg Total

20 Education , RO support

Education section to finalise the process of revising priorities, strategy, plan of action, targets and indicators for the emergency response, integrating transition to development.

Education section’s need and opportunity to finalise the revision of its strategy. (Paras 118, 123)

Happening; recommend to speed up process

4 3,5

7,5

74

21 Education Education section to engage more consistently with partners and

MoE to increase number and scope of PCAs and other partnerships, aiming at increasing coverage and consistency.

Limited number of active PCAs for Education, and

limited capacities of Governmental counterparts (Paras 119, 120)

4 3,5 7,5

22 Education Education section to identify and hire senior staff for Education policy work and coordination, aiming at strengthening UNICEF leadership role in the sector, and, if/when feasible, reconsider the offer to lead Education PTF working group.

Education section’s opportunity to increase its profile in interactions with MoE and in coordination structures for both humanitarian and development work. (Paras 123, 124)

3,5 2,5 6

23 Education, senior management, Communication, Supply

Education section, with senior managers, Communications and Supply section to urgently assess the suitability and feasibility of a context-specific “Back to School” campaign, shaped on the complexities and needs of the country, and eventually utilise the campaign for communications purposes and to raise the profile of UNICEF as leading agency for Education.

UNICEF has a tradition of utilising the opening of the school year to organise a “Back-to-School” campaign: an opportunity to advocate for and encourage enrolment, quality of education, and girls’ scholarisation. (Para 126)

3,5 5 8,5

Action Child Protection Reference in text of the report (paragraph) Comments Imp Urg Total

24 CP section CP section to urgently develop a comprehensive prioritized strategy and plan of action addressing new vulnerabilities (S/UAC, CAAFAG, MRE, GBV, etc.) and linking them to the pre-crisis approach; focus should be maintained on capacity building of governmental counterparts and partners, and enhanced on service delivery, as well as data and information collection and management for all programme components.

Challenges faced by the CP section throughout 2012-13 and dramatic increase of number and complexity of issues to be addressed by the CP section (Paras 129, 130, 136, 141). Chronic weaknesses of CP section and patchiness of its response calling for an urgent development of a consistent strategy, aiming at addressing numerous contextual challenges. (Paras 130, 131, 132, 137)

Essential to re-start raising UNICEF leadership role in this sector

5 4,5 9,5

25 MCO senior manager, RO and HQ

MCO senior manager, RO and HQ to strongly back up the activities of the CP section, facilitating technical assistance, supporting staff recruitment processes and supporting high-level protection discussions and negotiations with MINUSMA and other actors.

Importance of challenges faced by CP section and consequent reputational risk for UNICEF if these are not addressed consistently (Paras 131, 132).

4,5 3,5 8

26 CP, CP sub-Cluster

CP section to intensify and strengthen collaboration and coordination with CP sub-Cluster for programmatic, policy and

advocacy issues.

Relevance of work with the CP sub-Cluster as the main forum for technical expertise and advocacy joint efforts

(Para 138).

Especially important for CP to articulate

policies and strategies

4 3,5 7,5

27 CP CP section to urgently identify and deploy high calibre staff (IPs, national staff, TAs) for key positions (liaison/interactions with MINUSMA; sensitive issues).

CP section staffing weaknesses and gaps, in parallel to rapidly expanding sector-specific responsibilities and challenges (Paras 129, 132, 137).

4 3,5 7,5

28 CP section, in collaboration with CP sub-Cluster

and UNFPA

CP section, in collaboration with CP sub-Cluster and UNFPA to review status of implementation and coordination of GBV initiatives in the country, with the aim of re-starting operations of

the GBV sub-Cluster and developing a joint strategy with relevant actors. Other sections, under guidance of CP section, should start including GBV issues and activities in their

Limited initiatives for GBV and of limited quality among UNICEF and its partners, and a dormant GBV sub-Cluster are reasons for concern (Para 139)

3 3,5 6,5

75

respective emergency responses.

Action Communication for Development (C4D) Reference in text of the report (paragraph) Comments Imp Urg Total

29 C4D C4D to revise strategy defining specific emergency related activities that could provide clear contributions to the achievement of sections’ targets

Challenges of C4D and its partners in implementing activities supporting programme sectors at the pace required by the emergency response (Paras 146, 147, 149)

Partially included in response plan; needs implementation plan

5 3,5

8,5

30 C4D Capacities of partners should be assessed in light of the needs for

the emergency response, and training of traditional ones or search for more adequate ones pursued.

Limited number and capacity of C4D partners to work in

a more challenging humanitarian context (Paras 145, 146, 147, 149).

4 3 7

31 Senior management

Senior managers to ensure that funding for C4D is made available, including from other sectors, in order to allow implementing key strategic priorities, also in the framework of the emergency-to-development agenda

Although funding for C4D are planned for, when there are shortages, the related activities are the first to be neglected (Para 143)

4 3 7

32 C4D, M&E, RO Build up from current experience and tools into an

Accountability to Beneficiaries framework in humanitarian action, from the angle of involving beneficiaries in the planning and monitoring of the response, coordinating with 3PM and PDM the final capture of satisfaction.

Accountability to beneficiaries is a major gap in the

whole humanitarian response in Mali. UNICEF has the expertise and is the favourable position to start filling this gap (Paras 151, 173)

RO C4D team offered

some ideas for this

4 3,5 7,5

Action Programme Integration Reference in text of the report (paragraph) Comments Imp Urg Total

33 MCO, M&E MCO to enhance capacity for multi-sector assessment, planning and monitoring at FO level, including fine tuning 3PM partners.

Limited capacities for programme integration at FO level (Para 154)

4 3 7

34 MCO senior managers

MCO senior managers to ensure that program integration is a regular item on the agenda of programme management, emergency management, and UNICEF inter-Cluster meetings. These should be also the fora to encourage, plan and organise multi-sector field missions.

Importance of role to be played by senior managers to strengthen programme integration at all levels of UNICEF programme (Paras 153, 154)

4 3 7

35 M&E section M&E section to review and compare the standard trip report format and the WASH trip report format, aiming to finalise and

adopt an agreed tool including a follow-up of action points system.

Availability of different trip report formats (M&E, WASH) to optimise multi-sector visits, reporting and

follow up (Para 155)

4 3 7

Action M&E – Humanitarian Performance Monitoring (HPM) Reference in text of the report (paragraph) Comments Imp Urg Total

36 HQ, RO, Sections, M&E

HPM teams locally, regionally or globally, in coordination with programme sections, should further refine and adapt HPM indicators.

Generalised lack of clarity on HPM targets and indicators included in Sitrep format (Para 158).

4 3 7

37 HQ, RO, Sections, M&E

Clarify targets in a way that operational and aspirational ones are understood (i.e. estimated caseload and operational SAM coverage).

Confusion between aspirational and operational targets prevents clear understanding of trends (Paras 159, 162)

4 4 8

38 HQ, RO, Sections, M&E

Avoid integrating different objectives in the same indicator (conflict affected and flood affected, cholera prone areas and conflict areas, etc): strategies are different and measurements of

Targets composed by the sum of different categories of beneficiaries are confusing (Paras 160, 161)

4 4 8

76

effectiveness should differ.

39 HQ, RO, Sections, M&E

HPM reports should include some analysis of trends and explanations, especially in case of low progress to target.

Utilisation of unclear and wrong HPM targets and indicators does not allow to understand trends (Para 165)

4 4 8

40 HQ, RO, Sections, M&E

Performance indicators – as the cured rate of SAM,or activity and main conditions treated in health centers - should be included in the HPM instead of input oriented indicators (SAM children enrolled in malnutrition programme, distribution of kits).

Better performance indicators exist to monitor specific programmatic responses (Para 163)

4 3,5 7,5

41 M&E, SEctions When introducing HPM indicators in the PCAs, simplification leading to a more agile and accurate reporting should prevail over adding new requirements to an already cumbersome procedure.

Inclusion of HPM in PCA is positive but need some revision (Para 164)

4 3

7

Action M&E – Third Party Monitoring (3PM) Reference in text of the report (paragraph) Comments Imp Urg

42 RO, M&E UNICEF to keep refining 3PM tools in order to streamline the information into the program cycle of the sections.

Collaborations between 3PM partners and sectors (not only Health) have improved considerably since 2012; there is still room to further optimise this collaboration

and integrate 3PM feedback into workplans (Para 74, 170, 171, 172)

Linked to sector-specific recommendations

4 3 7

43 RO, M&E, C4D UNICEF to articulate an accountability to beneficiaries approach through the inclusion of specific tools in the 3PM (PDM) and C4D

Accountability to beneficiaries is a major gap in the whole humanitarian response in Mali. UNICEF has the expertise and is the favourable position to start filling this gap (Paras 151, 173) 3PM partners’ work offer an opportunity to expand UNICEF work on accountability

to beneficiaries (Para 173)

Linked to sector-specific recommendations

4 3 7

Action SITREP Reference in text of the report (paragraph) Comments Imp Urg Total

44 MCO, RO, Communication, M&E

MCO to include in the Sitrep only sound targets and indicators, providing clarifications in foot-notes about the origin and breakdown of targets. To avoid heaviness of the main pages, foot-notes might be condensed at the end of the document. This could be the document with a communications angle and

advocacy potential targeting donors, NatComs and eventually the general public.

Targets and indicators included in the Sitrep are generally unclear and do not offer the possibility to evaluate trends and bottlenecks (Paras 176, 177)

4 3,5 7,5

45 MCO and RO Emergency and HPM team

MCO and RO Emergency and HPM teams to consider the production of a more technical HPM-focused document for technical audiences (RO, EMOPS, M&E colleagues), and, in light of wide-spread reporting delays, consider its production on a quarterly basis only.

In light of the different audiences for the information included in the Sitrep, documents with different characteristics should be produced without excessively increasing the workload for those involved (Para 179)

4 2 6

46 EMOPS EMOPS to consider the “distillation” of one-pager briefs from the monthly Sitreps, for the Executive Director and other senior managers in NYHQ.

In light of the different audiences for the information included in the Sitrep, documents with different characteristics should be produced without excessively increasing the workload for those involved (Para 179)

More a food-for-thought point than a Rec

3 2 5

Action Operations Reference in text of the report (paragraph) Comments Imp Urg Total

77

47 RO The Operations team in the Regional Office should have some

deployable staff to directly support COs facing new emergencies.

More a food-for-

thought point than a Rec

3 2 5

48 RO, MCO In light of the scale-up of the office, RO and MCO to consider raising the level of the Chief of Operations position to P5 level.

Recommendation reflects the perception of RTE team

3,5 2 5,5

Action Human Resources Reference in text of the report (paragraph)

49 MCO and RO, DHR and EMOPS

In light of need to speed up recruitment for key positions and FO positions, MCO and RO should advocate with DHR and EMOPS to extend either the L2 or at least the authorisation to use the FTRP till end 2013.

Urgency to fill up key positions in the Bamako office and in the Field Offices. (Paras 46, 185)

4 4 8

50 MCO and WCARO

MCO and WCARO HR sections should undergo a review of national staff recruitments in 2012 and 2013 and identify bottlenecks affecting the process, especially in relation to the FTRP framework. They should also assess the FTRP-type of approach piloted by ESARO for national staff with the aim of replicating it in WCARO for Mali, and potentially other crises facing HR challenges.

L2 authorised use of FTRP also for national staff, but MCO and RO did not do it effectively. The Lessons Learned exercise for the Horn of Africa response in 2011 might offer an example of how to improve efficiency. Personally confirmed by ESARO Chief of HR to RTE

team. (Para 186).

4 3 7

51 MCO senior staff MCO, in close coordination with the MCO Staff Association to ensure complete transparency of the FO structure reshaping processes and, with RO support, ensure that personal issues are taken into consideration to the maximum possible extent.

Importance of taking into consideration specific staff well-being issues and problems to ensure successful restructuring of Field Offices (Para 188)

4 4

8

Action Security Reference in text of the report (paragraph) Comments Imp Urg Total

52 MCO, UNDSS MCO to advocate with UNDSS for having one security office in each FO location, authorised to issue security clearances, and

attuned to the reality of the area covered. The criticality exercise, defining critical outputs or activities by the whole UN country team, should be used as baseline document for UNDSS staff to find acceptable solutions to implement key field activities.

Security arrangements and set-up at FO level hinder flexibility of movements and consequently planning and

monitoring capacities of UN staff (Paras 189, 190, 191)

Based on the assumption that

UNICEF can advocate with UNDSS

3,5 3,5 7

Action Supplies Reference in text of the report (paragraph) Comments Imp Urg Total

53 Supply, sections, M&E

Supply Section to work with programme sections to revise discrepancies between expenditure and implementation rates,

looking at processes and structural delays of the supply delivery systems of implementing partners and Government counterparts.

Supplies distributed confirm the low progress towards targets in comparison to expenditure rates (Para 200)

Linked to similar Recs for programme

sections. Essential to implement jointly

5 4 9

Action Communication Reference in text of the report (paragraph) Comments Imp Urg Total

54 Communication Senior management CO

Communication team to quickly define its niche before full deployment of MINUSMA, and to find ways to interact constructively with Mission’s Communications team.

New opportunities and challenges for a stronger Communication team in light of humanitarian crisis and MINUSMA deployment. (Paras 204, 206, 207)

Already happening; should maintain speed and pressure

4,5 3,5 8

55 Representative and RO

Representative and RO to strongly support the section in its dealing with MINUSMA and in raising the profile of UNICEF in

Importance of responsibility and role played by senior management in making effective use of Communication

4 3,5 7,5

78

the emergency-to-development agenda. team and its strategy in Mali context (Paras 204, 206,

207)

56 MCO, Sections, Communication

MCO to build key advocacy strategies to address access to services and equity concerns in Mali.

Disparities created by the detrimental effects of the Bamako initiative underlined the rationale for free access to basic health and nutrition services. International community not taking advantage of the opportunity created by the humanitarian crisis to advocate for this. Potential important role for UNICEF to play, developing

sound advocacy and communication strategy (Paras 61, 62, 63, 64, 68, 85, 208)

Long-term strategy 4 2,5 6,5

57 MCO, RO, Communication, M&E

The communication dimension of the Sitrep should be taken into account when addressing a review and/or renovation of the tool.

A Sitrep format with a specific communication angle and synthetic and clear technical information targeting donors and NatComs should be considered (Paras 178, 209)

Links to SitRep Recommendations

3,5 2,5 6

Action Coordination Reference in text of the report (paragraph) Comments Imp Urg Total

58 MCO, OCHA,

UNCT

Mali CO to consolidate already decentralised Clusters under

UNICEF responsibility with adequate leadership and planning and monitoring capacities, and advocate for the activation of clusters in Timbuktu and Gao.

Weaknesses of existing Clusters decentralised at FO

level, and importance of activation of decentralised Clusters in Gao and Timbuktu (Para 219).

4 4 8

59 MCO, sections,, PTF, Clusters, Donors

Engage in linking coordination with development partners instrument, in the areas of responsibility of UNICEF, and design specific tools for harmonizing humanitarian and development strategies and coordination

A window of opportunity has been created by several concomitant factors for UNICEF to become a key leading and coordinating actor in the development and implementation of the emergency-to-development

agenda. Its role of Cluster lead for four sectors, and its increasing role in the PTF working groups and other fora reinforce the bases to take this leading role (Paras 49, 50, 51, 52, 55, 56, 212, 213)

4 3 7

Action Gender Reference in the text Comments Imp Urg Total

60 MCO, sections All sections in MCO to review and assure inclusion of gender in their planning tools and in the implementation of their

programmes. The office might look into having a gender specialist assisting in this exercise.

Limited presence of gender issues and perspectives in planning documents and programme implementation (parag 220,221, 222)

3,5 2,5 6

79

Annex: REFERENCES

A Real Time Evaluation is a quick exercise that allows limited time for academic research.

Notwithstanding, the consultants established a conceptual framework in order to provide a coherent reference where to anchor the exercise. Related to the RTE approach, references used have been:

The United Nations University. Real-time evaluations of humanitarian action. An ALNAP

Guide, Pilot Version, John Cosgrave, Ben Ramalingam, and Tony Beck, available at:

www.alnap.org

Inter-agency Real Time evaluations (IA RTE) of emergency humanitarian operations.

Procedures & methodologies, developed by the IA RTE steering group

Real time evaluation of UNICEF’S Response to the 2005 food and nutrition crisis in Niger,

UNICEF evaluation office, December 2006

Evaluation of humanitarian action, Pilot guide, Margie Buchanan-Smith, John Cosgrave,

available at www.alnap.org Evaluating humanitarian action using the OECD-DAC criteria, an ALNAP guide for

humanitarian agencies: www.alnap.org

In relation to the emergency response in Mali, the consultants had access to a significant amount of

relevant documents shared through a Dropbox folder, provided by MCO and WCARO. Around 400 documents were consulted during the exercise, belonging to the following categories:

Response plans, work plans, and contingency plans.

Internal and external strategy documents (programmatic, fundraising, humanitarian

monitoring, communications and advocacy, etc.).

Situation reports and other reports.

Monitoring tools, reports and analyses.

Partnership agreements, information and statistics.

Interagency strategies and planning documents, reviews, analysis and reports.

Human resources and supply statistics, shipment records, reports and monitoring documents.

Internal Audit reports relevant for the country and/or the context.

Administrative reports on disbursement and liquidation processes (VISION)

Dispatch reports, warehouse statistics

Statistical data from different programme areas, both from UNICEF partners and from GoM

services

In addition, the consultants found useful to review documents analyzing challenges affecting the

health and nutrition sectors in the Mali context, and used the following references:

Socio-economic differences in Health, Nutrition and Population in Mali, Davidson R.

Gwatkin, Shea Rustein, Kiersten Johnson, Rohini Pande, and Adam Wagstaff, for the HNP/Poverty Thematic Group of The World Bank May 2000

Economic and Welfare Effects of the Abolition of Health User Fees: Evidence from Uganda,

Klaus Deininger, Paul Mpuga, World Bank Policy Research Working Paper 3276, April

2004

Cost-sharing in complex emergencies: an example of inappropriate policy transfer, by Tim

Poletti, Egbert Sondorp, Olga Bornemisza and Austen Davis, Humanitarian Exchange, ODI

March 2004 Issue

Documenting results of efforts to improve health aid effectiveness: Mali Case Study,

October 2011; Elisabeth Paul, WHO

Résume de l’étude « Pauvreté des enfants et inégalités au Mali » Ministry of Finance,

published by UNICEF, consulted while at MCO

Évaluation des actions humanitaires financées par ECHO au Nord-Niger et au Nord Mali

(1998 - 1999) Secteur Santé, Ricardo Solé Arqués, Avril 2000, Available at:

http://ec.europa.eu/echo/files/evaluation/2000/mali_niger1.pdf

Statistical databases from the WHO, UNICEF, GAVI and the WB

80

In order to review the current state of affairs in the Cluster system and Humanitarian Reform, the RTE team examined the following documents:

Reference module for cluster coordination at the country level, IASC transformative

agenda reference document, November 2012 at http://www.humanitarianinfo.org/iasc/

IASC Cluster approach evaluation, 2nd

phase, April 2010, GPPI-URD at

http://clusters.humanitarianresponse.info

Regarding the subject of Accountability to Affected People (AAP), the consultants reviewed:

The good enough guide : Impact Measurement and Accountability in Emergencies,

Emergency Capacity Building project (ECB) (also in French) available at

http://www.ecbproject.org Cadre opérationnel, Comment assurer une plus grande redevabilité envers les populations

affectées en cas d’urgence humanitaire, IASC Taskforce on Accountability to Affected

Populations: Co-Chaired by WFP and World Vision International, available at

www.humanitarinainfo.org

Les engagements des responsables de l’IASC en matière de redevabilité envers les

populations affectés (CAAP) ; Outils permettant de respecter ces engagements, at

www.humanitarinainfo.org

81

Annex: ITINERARY OF THE MISSION

The RTE mission took place between the 2 and the 23 of June 2013, with the following

itinerary:

Day Location Main Activities

2 June, Sunday Dakar Travel (air). Arrival of consultants

3 June, Monday Dakar KIIs with WCARO staff

4 June, Tuesday Dakar KIIs with WCARO staff

5 June, Wednesday Dakar to Bamako KIIs with WCARO staff. Travel (air)

6 June, Thursday Bamako KIIs with MCO staff

7 June, Friday Bamako KIIs with MCO staff

8 June, Saturday Bamako Desk review

9 June, Sunday Bamako Desk review

10 June, Monday Bamako KIIs with MCO staff and external KIIs

11 June, Tuesday Bamako Day-trip to Kangaba, project visits, external KIIs

12 June, Wednesday Bamako to Mopti Travel (air). KIIs with partners

13 June, Thursday Mopti KIIs with partners and UNICEF Mopti staff

14 June, Friday Mopti to Segou Travel (car)

15 June, Saturday Segou Day-trip to Nampala, project visit, external KIIs

16 June, Sunday Segou to Bamako Travel (car). Desk review

17 June, Monday Bamako KIIs with MCO staff and external KIIs

18 June, Tuesday Bamako KIIs with MCO staff and external KIIs

19 June, Wednesday Bamako Preparation of debriefing

20 June, Thursday Bamako Preparation and delivery of debriefing

21 June, Friday Bamako

Bamako to Seville

EL – Work on preliminary recommendations

RSA – Travel (air) Bamako to Seville

22 June, Saturday Bamako to Addis Ababa EL – Travel (air) Bamako to Addis Ababa

23 June, Sunday Addis Ababa to Harare EL – Travel (air) Addis Ababa to Harare

82

Annex: LIST of INTERVIEWEES

Name Title Organisation Martin Dawes Chief, Communications UNICEF WCARO

Philip Van De Graaf Supply & Procurement Specialist, Emergency UNICEF WCARO

David Vautrin Pipeline Monitoring and Reporting specialist UNICEF WCARO

Fabio Frisca C4D Specialist UNICEF WCARO

Jennifer Hofmann Education in Emergency Specialist UNICEF WCARO

Robert Hanawalt HR in Emergency Specialist UNICEF WCARO

Grant Leaity Emergency Advisor UNICEF WCARO Valerie Serres HIV in Emergency Specialist UNICEF WCARO

Claudes Kamenga Chief, HIV/AIDS UNICEF WCARO

Francois Bellet WASH in Emergency Specialist UNICEF WCARO

Carole Vignaud Emergency Coordinator UNICEF Mali

Manuel Fontaine Regional Director a.i. [now RD] UNICEF WCARO

Cecile Marchand Child Protection in Emergency Specialist UNICEF WCARO

Marleen Korthals Altes Child Protection Specialist UNICEF WCARO Pascale Crapouse Chief, Supply UNICEF WCARO

Alex Mac Rae Chief, Operations UNICEF WCARO

Toby Wicks Emergency Specialist UNICEF WCARO

Cairn Verhulst HPM Specialist UNICEF WCARO

Patricia Hoorelbeke Nutrition in Emergency Specialist UNICEF WCARO

Inoussa Kabore Chief, Monitoring and Evaluation UNICEF WCARO

Herve Peries Planning Advisor UNICEF WCARO Donatella Massai Health in Emergency Specialist UNICEF WCARO

Frederic Sizaret Deputy Representative UNICEF Mali

Julie Gauthier WASH in Emergency Specialist UNICEF Mali

Ateeq Bashir Child Protection Cluster IM UNICEF Mali

Oumou Guindo Human Resources Assistant UNICEF Mali

George Fom Ameh Health Specialist UNICEF Mali

Ndeye Ngone Toure HIV/AIDS Specialist UNICEF Mali

Iain Murray M&E Specialist UNICEF Mali Jeremy Shusterman HPM Specialist EMOPS NY (on mission)

Denis Garnier Nutrition Manager UNICEF Mali

Alan da Cruz Chief, Operations UNICEF Mali

Hawa Samake C4D Specialist UNICEF Mali

Stephanie Kleshnitzky Resource Mobilisation Specialist, Consultant UNICEF Mali

Maddalena Bertolotti Resource Mobilisation Specialist a.i. UNICEF Tanzania (on mission)

Jessica Duplessis Information Manager/ Report Writer UNICEF Mali Mamadou Wane Chief a.i., Child Protection UNICEF Mali

Ben Harvey WASH Cluster Coordinator UNICEF Mali

Roben Picalli WASH Cluster IM UNICEF Mali

Albert Tshiula Lubanga Nutrition Cluster Coordinator UNICEF Mali

Anne-Celine Delinger Nutrition Cluster IM UNICEF Mali

Evans Atis Education Cluster Coordinator UNICEF Mali

Frederic Urlep Logistics Specialist UNICEF Mali Marina Muenchenbach Deputy Country Director ACF Spain

Abdias Ogobara Dougnon Chef de Projet Nutrition Bamako ACF Spain

Maman Saley Maman Sani Coordinateur Wash Mali ACF Spain

Tasha Gill Country Director IRC

Sebastien Lemaire General Coordinator MdM Belgique

83

Francoise Gruloos-Akermans Representative UNICEF Mali

Fernando Arroyo Chief of Office OCHA Mali

Guido Pizzini IM Officer OCHA Mali

Badreddine Serrokh Intercluster Coordination Specialist OCHA Mali

Andrea Berther Chief, Education UNICEF Mali Haleinta Thiam Education Specialist UNICEF Mali

Pierre Andriamampianina Education Specialist UNICEF Mali

Mariam Barakou Toure Education Specialist UNICEF Mali

Birama Toure Interim du Medecin Chef Kangaba CSRef

Shannon Strother Chief, Social Policy UNICEF Mali

Alpha Bamado Responsible Social Mobilization Aga Khan Found. Mopti

Pierre D. Kadet Food security expert FAO Francis Bjimtessem Chief of Field Office SCF Mopti

Moussa Thera Education Specialist SCF Mopti

Harouna Traore Child Protection Specialist SCF Mopti

Alhousseine Maija Education Project Officer Handicap Int. Mopti

Emma Massia Programme Specialist (MRE) Handicap Int. Mali

Adama Sanogo Head of Field Office Family Care Int. Mopti

Various Family care team Family Care Int. Mopti Billy Keita Chief of Field Office WFP Mopti

Bony Mpaka N. Field Coordinator OCHA Mali

Boussiratou Maiga Health Officer UNICEF Mopti

Moussa Sidibe CP Specialist – Chief of Field Office a.i. UNICEF Mopti

Thierno Boubacar Cisse Governor Gouvernatorat Segou

Djoouro Bocoum Director Dept. Hydraulics Segou

Local authorities meeting Mayor, Sub-governor, other principals Nampala Guillaume Woehling Chief de Mission Handicap Int. Mali

Nathalie Derrien Operation Coordinator, Emergency Programme Handicap Int. Mali

Asa Palmgren Head of Development Cooperation Swedish Embassy Mali

Desire Ballo Human Rights Program Officer Swedish Embassy Mali

Sally Haydock Country Representative WFP Mali

Shameza Abdulla Emergency Specialist UNICEF WCARO

Patrick Barbier Representative ECHO Mali Kat Echeverria Programme Officer (OFDA) USAID Mali

Boubacar Diallo Manager CEDREF

Anissa Badaoui Child Protection sub-Cluster Coordinator UNICEF Mali

Buenaventura Maiga Advisor Ministry of Education

Mono Ayouba Gonavel Directeur National Adjoint Ministry for the Promotion of Family and Children

Faneke Dembele Disaster Management Focal Point Civil Protection Amadou Diabate Chef de Projet RECOTRADE Mali

Hector Calderon Chief, Communications UNICEF Mali

George Tabbal Chief, WASH UNICEF Mali

Guillaume Sauval Emergency Specialist, HFSS UNICEF EMOPS, NYHQ

Akhil Iyer Deputy Director UNICEF EMOPS, NYHQ

84

Annex: TERMS OF REFERENCE

UNICEF West and Central Africa Regional Office

Real-Time Evaluation (RTE)

Of UNICEF’s Response to the Mali Crisis, 2013 TERMS OF REFERENCE – FINAL – 15 MAY 2013

I. BACKGROUND

In 2012, Mali experienced a coup d’état that led to massive displacement of Malians in both the northern and southern regions of the country, with almost 300,000 internally displaced people (IDP), 50% of which are children. Many of those displaced are living with host families in the south, with some living in an IDP camp. As a result of this displacement, IDPs are now facing an extremely difficult context. Such movements have placed significant burdens on individuals, families, and host communities, preventing access to social services while posing additional security threats amidst a backdrop of widespread malnutrition affecting the Sahel sub-region. The Mali Crisis was officially declared a Level 2 emergency by the UNICEF Executive Director on 28th January 2013.35 UNICEF’s response aims to ensure that the benchmarks set out in the Core Commitments for Children in Humanitarian Action (CCCs) are met or exceeded in Mali. In concrete terms, to achieve this, the overall figures for affected populations have been agreed upon and form the basis for the planning, prevention measures, treatment, preparedness and response. UNICEF’s contribution towards the CCC benchmarks varies from sector to sector, but is articulated through interventions ranging from direct service delivery, capacity development initiatives and coordination support to advocacy. The UNICEF 2013 humanitarian response plan aims to ensure a fully integrated approach to the Mali Crisis. This Real-Time Evaluation (RTE) seeks to assess UNICEF’s progress in achieving results for children through this response, as defined in the response plan. 36 The RTE will focus on the Mali CO’s humanitarian response during the three months following the launch of the Humanitarian Action for Children (HAC) (January 24, 2013-April 24, 2013). It is planned to be completed in a fairly short timeframe, in order to enable UNICEF to take stock of progress, learn what has or has not worked well, identify areas that require strengthening or adjusting, and map any gaps in UNICEF’s response. UNICEF Mali’s current humanitarian planning and response will be adjusted to include concrete recommendations from the RTE. II. RATIONALE

Humanitarian planning and response merits close monitoring and evaluation as it has a high risk–programme design. This RTE is intended to feed into results-based management both for UNICEF-

35

A Level 2 emergency is defined as a situation where the magnitude of the emergency is such that a Country Office needs

additional and prioritized support from other parts of the organization (Headquarters, Regional and Country Offices) to respond, and where the Regional Office (RO) must provide dedicated leadership and support. It is designed for the UNICEF Country Programme's response (sectors and cross-cutting areas) to be timely, appropriately assessed, and designed and executed according to the Core Commitments for Children in Humanitarian Action (CCCs). 36

The UNICEF Mali 2013 Response Plan with the most recent figures of the population affected, and outlining the specific programmatic elements to be assessed, will be made available to the consultants by the UNICEF team during the inception phase.

85

supported programmes and for Clusters where UNICEF is the designated Lead Agency at the country level. The urgency for a UNICEF-focused RTE to support operational management means a rapid, light exercise is necessary. The aim of the RTE will therefore be to produce feasible and opernationalizable recommendations to improve the current UNICEF planning and response in Mali with focus on the first trimester of humanitarian activity following the launch of the Humanitarian Action for Children (HAC)/Consolidated Appeals Process (CAP) 2013 for Mali (24 January 2013- 24 April 2013). In addition, the RTE will provide recommendations for scale up of the Mali CO’s humanitarian response. With Regional Director (RD) agreement, key RTE findings will be shared with later joint or UNICEF’s specific evaluation of results achieved for children in the response. III. PURPOSE AND OBJECTIVES

The main objective of the RTE is to provide recommendations to enhance the scale-up of UNICEF’s planning and response. The assessment will focus on the following criteria for analysis:

Effectiveness – the degree to which the planning and response meets the priority needs of the most vulnerable population groups affected communities including coverage: such as the extent to which the response has reached the affected populations, including the hardest-to-reach and most affected geographic areas. The RTE will examine UNICEF Operations and Programmes, planning and response, monitoring, Cluster Lead Agency role and operational support commitments, as defined by the CCCs, and contextualized and prioritized for the UNICEF response. The RTE will assess the intended, unintended, positive, negative and secondary effects of the UNICEF humanitarian response on the affected populations and other stakeholders. The RTE will assess barriers and bottlenecks influencing implementation of the response; and opportunities that may contribute to better targeting and make the response more effective.

Efficiency – how well UNICEF used resources to achieve the planned results.; The RTE will look at the utilization of available funds, measures put in place for adequate disbursement and liquidation, and partners’ reporting and management procedures, as well as rapid and effective supervisory mechanisms including partnership monitoring.

Appropriateness / Relevance – the degree to which the programmatic and operational planning and response has facilitated the Mali CO to meet the relevant CCCs, and to meet stakeholders’ and beneficiaries’ needs and expectations; and, whether the response is provided at the right place at the right time, including the timely establishment of partnerships (with a range of stakeholders) to meet the response. The degree to which the planning and response is adapted to and implemented in line with key stakeholders’ plans and objectives, as well as program integration across sectors, including the need to coherently plan for the humanitarian response, resilience, and an exit from the humanitarian crisis.

Adequacy - The RTE will look at the adequacy and coherence of the planning and response, with key factors including resource mobilization, response planning (including targeting), supply planning and management, human resources, coordination and humanitarian performance monitoring and reporting.

Coordination – the degree to which UNICEF and the UNICEF-led clusters have effectively met the needs for coordination within UNICEF itself (at country level and with WCARO/HQ), with the Government, UN agencies, NGOs, beneficiaries and other stakeholders; how well

86

the coordination has worked, what has worked well, not well; the opportunities for improving coordination.37

While the field work of the RTE will concentrate on the programmatic response carried out by the Mali CO, an organizational objective of the assessment will include an analysis of the appropriateness, effectiveness and timeliness of Regional Office and HQ support and guidance in light of L2 accountabilities. The assessment team will endeavor to interact with a broad range of stakeholders, including beneficiaries where possible under security restrictions, and where relevant to the UNICEF humanitarian programme. Support provided by the Mali CO to organize these consultations will not jeopardize the assessment team’s independence. Intended users and use of the RTE The intended use of RTE is primarily learning to inform corrective action when and where appropriate. The findings will contribute to strengthening the CO and RO response, including to best meeting the Mali CO’s needs and L2 accountabilities. Building upon the experience of the 2012 Sahel RTIA, this may involve a country level management response plan to respond to the key findings and recommendations of the RTE. The primary users of the assessment will be the Mali CO, WCARO and UNICEF HQ. A secondary tier of users include non-UNICEF partners, including implementing and, inter-agency partners, CBOs, donors and other critical stakeholders. The RTE report will be shared with these stakeholders at UNICEF discretion. The CO may consider creating a short narrative of the RTE report available for external partners. To contribute to global knowledge, and in line with UNICEF’s Evaluation Policy, the final report and management response will be uploaded into UNICEF evaluation database making it accessible throughout the organization. IV. SCOPE

The geographic scope of the RTE will include regions in both southern and northern Mali affected by both the armed conflict and the ongoing nutrition crisis. The RTE analysis and recommendations will focus on the UNICEF humanitarian response for the first trimester of activities in 2013 (24 January to 24 April 2013), following the launch of the Mali CAP/HAC 2013. The focus of the RTE will be on the ensemble of crises affecting Mali, and on how these affect the Mali humanitarian response (including the nutrition crisis in the south, militant occupation in northern regions, the military intervention, and the effects of the political instability on relations with stakeholders). The programmatic focus will be derived from the UNICEF CO response strategy to the Mali Crisis through the lens of the Core Commitments for Children (CCCs) relevant to the Mali CO response.38 As this RTE is intended to be primarily a learning-focused assessment, greater emphasis will be placed on identifying internal factors affecting UNICEF’s planning and response. In order to identify these factors, it is necessary to assess which planned outputs and outcomes were achieved, or not,

37

In Mali, UNICEF is cluster lead for Nutrition, WASH, Education, and co-lead with Save the Children from the Child Protection sub-cluster.

38

The UNICEF Mali CO response strategy for 2013 will be made available to the consultants as part of the inception phase.

87

or are on track. Accordingly, the RTE will focus on assessing the extent to which UNICEF’s planning and response is in line with meeting the CCCs, specifically for programmes, i.e. Nutrition, WASH, Health, HIV/AIDS, Child Protection, Education, Communication for Development (C4D), as well as Operations, i.e. Finance and Administration, Communications, Human Resources, Supply and Logistics, Monitoring and Evaluation (M&E), and Coordination. While focusing on the UNICEF response and internal factors, the RTE will take into account the implications for and contributions of other stakeholders, and external factors. In keeping with the purpose of the exercise to facilitate regional and country-level learning for action, internal UNICEF factors will be sought at these two levels rather than at HQ level. While HQ-level factors that have affected the planning and response will be captured where appropriate, these will be documented in a separate memo from the assessment team to HQ management for its consideration.39 In the case that there is any significant change in the humanitarian context or the UNICEF humanitarian response in the period following 24 April, this may be taken into account in the analysis. This RTE does not preclude a decision by UNICEF at a later stage to undertake more complete lessons learnt exercise, and/or to commission another independent evaluation of the UNICEF planning and response in Mali. Nor is it intended to substitute for UNICEF’s involvement in any future Inter-Agency Real-Time Evaluation (IA RTE) that might be undertaken.

V. RTE QUESTIONS

The following represents a provisional and illustrative list of broad questions to be fine-tuned by the consultants in the inception report and through the data collection tools. Effectiveness

How successfully has UNICEF at regional and COS level delivered and monitored results against the specific programmatic, cluster and operational commitments it has pledged to achieve, as rooted in the CCCs, the response plans and other planning documents? How optimal were these CCCs?

How did the Mali CO adapt programmes and operations to meet humanitarian requirements? What extra risks did that imply?

Have any outcome-level results been observed, as defined in the CCCs? What, if any, unintended consequences either positive or negative, has the response had on local populations and local institutions?

The effectiveness of the response and coverage

To what extent has the response reached the affected populations, including the hardest-to-reach, and most affected geographic areas? How were they impacted by the interventions? Have data been disaggregated by sex and age, and meaningfully analyzed so as to ensure equity of coverage?

39

Two large-scale exercises in 2010-11 and 2011-12, the Independent Review of UNICEF’s Operational Response to the January 2010 Earthquake in Haiti and the Lesson-Learning Exercise on UNICEF’s Response in the Horn of Africa, have already documented a substantial number of HQ-level recommendations that are currently being acted on. Thus, in order to use the resources for the present exercise more efficiently, the RTE will focus on the decentralized levels of the organization.

88

What role has remote programming played, and how well have beneficiaries in the unstable areas been reached by UNICEF supported programmes?

What role have the UNICEF zonal offices played in humanitarian preparedness and response?

Efficiency

How well and how adequately has the UNICEF programmatic planning and response been articulated, targeted and prioritized, in terms of coordination, human and financial resources, capacity to respond, and progress toward targets, including coverage of harder to reach, conflict-affected areas?

How well have funding, human resources, partners, supplies and equipment been used to achieve outputs and outcomes, and in scaling up to meet the needs of the affected populations?

To what extent have innovative or alternative modes of delivering on the response been explored and exploited to lower costs and/or maximize results?

How well and quickly has UNICEF performed in resource mobilizations (i.e., human resources, funding and supply) and scaling-up to meet the needs of the affected populations?

How timely has the Mali CO response to the crisis been? What, if any, significant delays have occurred in the preparation, scale-up and implementation of the response, and why?

To what extent did the Mali CO put capacity in place (i.e. human resources, admin processes, travel, etc.) to prepare for and respond to the crisis? What quality and quantity? Did the CO and RO have the capacity to respond to the needs?

How timely have UNICEF and its partners been in mobilizing resources, developing partnerships, getting resources to where they are needed, justifying and reporting on expenditures?

How long have partnership agreements taken to put in place? And what if any bottlenecks have there been in the process?

How timely has the support and guidance of the RO to the Mali CO been in preparing, planning, scaling-up and responding to the crisis?

Appropriateness / Relevance

Is UNICEF’s programmatic planning and response articulated, prioritized and targeted from the CCCs? Is it appropriate?

To what extent did UNICEF’s response meet the needs of the affected populations (as identified by the populations themselves and in systematic needs assessments)?

How evidenced-based has the decision-making process been for determining the response interventions?

To what extent has the initial response strategy remained relevant over time, responding appropriately to the evolving nature of the crisis?

How appropriate has the UNICEF finance, administration, supply and human resource procedures been to meet the needs of the humanitarian programme?

How well have gender, protection and human rights issues been addressed?

To what extent is the UNICEF humanitarian response aligned with international and national priorities including national planning frameworks such as the CAP, national development frameworks, and the implementation strategies of other clusters not led by UNICEF?

What was the role of the Government in planning for and responding to the humanitarian crisis following the 2012 coup d’état?

To what extent is the planning and response owned by the Government?

89

To what extent has the UNICEF planning and response aligned the concurrent needs to respond to the crisis at the same time as building resilience and supporting the country in a sustainable exit from the crisis?

To what extent has UNICEF supported partners to analyze and overcome key humanitarian bottlenecks?

Adequacy

Do UNICEF available40 supplies meet the needs of affected population? How timely is the delivery of supplies to end users41?

To what extent has UNICEF taken on additional responsibilities beyond its core commitments and its usual programmatic scope?

Coordination

To what extent has UNICEF met its Cluster/Sector Lead coordination commitments, and how functional has it been, within the framework of the CCCs and IA ToR for Country Cluster Lead?

How well has the Government’s capacity been strengthened as it has the primary responsibility to respond to its population’s needs (e.g. in needs assessments)?

What are the implications of the Level 2 emergency procedures on the humanitarian programme in the Mali CO?

VI. METHODOLOGY AND APPROACHES

As an assessment focused on facilitating early and rapid learning for improvement, it is essential that the RTE will follow a participatory, interactive approach with major stakeholders, including beneficiaries, in order to strengthen its validity, promote feedback and ensure ownership and buy-in and utilization of the results. At the same time, in light of the decisions that could hinge on the recommendations put forward in this assessment, it is essential that the RTE be rigorous and evidence-based. A mixed-method approach that triangulates various quality available data sources to reach conclusions and validate findings will be employed, and may include and not be limited to the following:

a. Review of existing documents for understanding of the planning and response and the context, and including desk review of key documentary evidence sources using document review tools, as necessary;

b. Systematic secondary analysis of programme data/statistics (including existing national monitoring outputs, CO section collated information and Humanitarian Performance Monitoring outputs);

c. In situ direct-observations (e.g., sites, service delivery points); d. Key informant interviews (by phones and face to face) and focus group discussions; and e. Rapid data collection among beneficiaries (i.e., semi-structured interviews and/or focus

group discussions among purposively sampled subset of the affected population) or other methods, as appropriate (where security restrictions permit).

The list of existing data and documents will be prepared and provided to the assessment team. The existing documents will include the CCCs, the Humanitarian Action Update, the UNICEF Mali CO

40

Available is defined as pre-positioned stocks. 41 For Nutrition, end user is defined as feeding centre treating SAM.

90

response plan, the timeline for the Mali CO compiled by EMOPS, the Sahel 2012 Real Time Evaluation report, the 2011 Audit Report, the CO (ERM) risk analysis matrix, Partnerships Matrix, Annual Report 2012, warehouse stock reports, supply distribution reports, reports from Third Party Field Monitoring and the briefing pack compiled for the December 2012 Executive Director’s visit. The key informant interviews will include RO and CO and zonal office staff playing a key role in the implementation of the response as well as interviews with key relevant Government, NGO and civil society stakeholders at national and local levels. The RO and CO will provide a draft list of key informant interviews to be conducted which the RTE team will provide feedback on. What about the focus, who do intent to do focus groups with? Beneficiaries, community members… How is gender and segmentation taken into account in the focus groups? With guidance from the Mali CO and in light of security restrictions, the assessment team is also expected to make a field visit to Mopti and possibly other sites in northern Mali, security permitting, to observe directly the effect and the systems in place to deal with the complex crisis, and to one or two additional sites in South Mali to compare the ongoing humanitarian programmes particularly as they relate to nutrition, IDPs and protection (including children released from armed groups). Direct beneficiaries’ viewpoints are critical to support the findings of the assessment. Therefore, the team will be expected to develop creative methodologies and approaches to collect data quickly, systematically and safely among representatives of the affected population. In Mopti and other sites, the team can also visit beneficiaries that received support from UNICEF programmes for IDPs, Schools and Separated/ Unaccompanied Children. While there are a large number of IDPs in Mali, the majority are living with host families with only one IDP camp in Sevare, Mopti region. The assessment team will develop further the overall details of the methodology and will describe these in their inception report prior to field work. It is expected that 12-14 days will be spent in-country. Data analysis should go concurrently with the data collection, in such a way that a summary debriefing is provided to Mali CO at the end the data collection. Any critical issues that may need immediate attention will be communicated to the RO and the CO along with the RTE management team to take the necessary action, and priority action should be taken on these by the Mali CO, WCARO or NY/HQ.

VII. MANAGEMENT AND GOVERNANCE, AND ENGAGEMENT OF STAKEHOLDERS IN RTE

As mentioned above, the assessment will be independent. All stakeholders and users described above will be engaged in the process to ensure their needs and feedback is taken into account in the design and implementation, and use of the findings. However, care will be taken by the management team to ensure their views and inputs do not jeopardize the independence of the assessment. To engage them, two levels of governance and management structures are proposed: Management Group: This group, led by the Regional Chief of Monitoring and Evaluation, with support from the RO Emergency Section, will convene weekly meetings to review progress and provide orientation to the implementation to the assessment, as needed. Their role will not lead to influencing the independence of the assessment, but rather foster the adherence to UNICEF’s standards, quality, policies and procedures. In summary the Management Group will be committed to:

Review the RTE ToR;

Take part in selection process of RTE external Evaluator(s);

Weekly meeting (max one hour) with RTE day-to-day manager – will include External Evaluators once on-board;

91

Review of preliminary and final reports;

Help frame key messages that need to go back to the RO and the Mali CO as results of the preliminary findings of the assessment in the field.

Reference Group: A slightly wider Reference Group made up RO M&E, RO Emergency Section, RO program section technical emergency focal points, a representative from EMOPS/NY and EO/NY, and a Social Policy focal point from the Mali CO will input into the development of the ToR and preliminary and final reports. The RTE Management Group will ensure their inputs do not interfere with the independence of the assessment. Executive Sponsor of the Assessment: The Acting Regional Director, Manuel Fontaine, is the executive sponsor and ensures clear communications to Mali, and will keep them engaged during the design and implementation phases. He will be responsible for ensuring a consistent management response to the findings and recommendations issued from the assessment, and with assistance from the regional Chiefs of Sections. He will coordinate the approval of the report with the assessment Management Group. Mali CO Social Policy and Monitoring and Evaluation Sections: Their role will be to facilitate field visits to intervention sites, be available for interviews, share necessary data, and attend the in-country debriefing. Ethical Considerations: This assessment is considered very minimal risk and will not be subject to an ethical review committee approval. However, the UNICEF Mali CO will seek the approval of relevant government entities before the data collection. Each individual to be interviewed will be asked to provide verbal consent to participating in interviews and focus groups. No name will betaken from any individuals during and after interviews. The interviewees’ confidentiality will be warranted.

VIII. REPORTING REQUIREMENTS

The quality of the Assessment will be checked by the Management Group and by the Reference Group, as described above. This RTE will go through GEROS at global level. Expected outputs/outcomes of the RTE

Inception Report with tools and materials

RTE Report for the Mali crisis including information on the methodology, timing of the RTE, persons met, best practice, recommendations for Mali country office and regional office on what is working well, what needs to be strengthened and to identify any gaps requiring immediate or medium-term corrective action.

Memo to HQ HQ-level factors will be documented in a separate memo from the assessment team to HQ management for its consideration.

Debriefing Presentation of lessons learnt and corrective actions, including a preliminary debriefing with the CO and the verification of the key findings with WCARO before report finalization.

Summary of Expected Deliverables:

Expected Deliverables Language Estimated Workdays Required

Tentative Completion

Date

Payment Schedule

92

Output 1: Draft inception report, following the review of literature provided

English 4 By 7 June 2013

30%

Output 2: Interviews, focus groups, direct observations, rapid data collection among beneficiaries, and other data collections modalities; discussion with stakeholders/ partners; review of existing documents and secondary analysis of programme data/statistics; summary debriefings.

English and French

12 By 21 June 2013

Output 3: Debrief with Mali CO with participation of the RO

English and French

3 By 26 June 2013 50%

Output 4: Draft report English 5 By 1st July 2013

Output 5: Final report English 10 By 11 July 2013 20%

Expected Contents of Preliminary and Final RTE Report: Below are the key elements that should be reflected in the report. However, the UNICEF evaluation report format will be provided to the consultants to prepare the preliminary findings and the final report including but not limited to:

Background, objectives

Methodology and limitations

Recommendations geared toward the RO and the Mali CO on what is working well, what needs to be strengthened and to identify any gaps requiring corrective action;

UNICEF inputs/supplies: availability (pre-positioning), timeliness and utilization42 in meeting needs of affected population;

Programmatic planning and response: appropriateness of the Mali CO response in meeting CCCs relevant to CO strategy;

Country Cluster/Sector Lead Agency: effectiveness of UNICEF as country cluster/sector lead for Nutrition, WASH, Education, Child Protection and Gender Based Violence (as relevant to country context);

Operational issues: effectiveness and timeliness in scaling-up CO operations to meet response needs including resource mobilization, response planning, human resources, finance and administration and monthly Humanitarian Performance Monitoring and reporting;

RO support: appropriateness, effectiveness and timeliness of RO guidance and support. Operationalization Based upon the recommendations and lessons learnt the Regional Office and CO will develop a management plan of action to respond and the actions taken should be monitored on a regular basis, including through inclusion in the revision of the CAP and HAC, and a description of actions in the RO and CO Annual Reports. IX. BUDGET AND TIMELINE

US Dollars Assume Salary (34 days) DSA* Flights Total

Senior external consultant (L5) $540/day 18,360 4,678 10,000 33,038

Mid-level external consultant (L4) $540/day 18,360 4,678 10,000 33,038

Total 66,076 *Assumes 14 days Mali, 6 days Senegal

42

‘Utilization’ defined as use by end user – not as ‘leaving UNICEF warehouse’ or ‘delivered to UNICEF partner’.

93

WCARO Mali+ RTE process Timeline Week beginning Monday

April May June July

Step Evaluation Team Location

Estimate of time required

Total Days

1 8 15 22 29 6 13 20 27 3 10 17 24 1 8 15 22 29

Draft concept note and ToR 5 days 5

Advertise and recruit consultants

15 days 15

Assume Start date of RTE 03.06.2013

Document review, orientation, develop data collection tools, finalise ToR and draft inception report.

Remote 5 days 5

Consolidate comments and feedback on inception report and data collection tools

5

Output 1: Finalised Inception report

Output 2: Finalised country data collection tools

CO Visit, data collection including in-country debriefing and presentation of preliminary CO findings.

Mali 10 days with CO

10

Preparation of Regional debriefing and presentation of preliminary findings.

Dakar, Senegal

4 days in RO 4

94

Output 2: Preliminary Report

Review of preliminary findings and draft RTE report

Remote 5 days 5

Integration of UNICEF feedback and preparation of final report

Remote 4 days 4

Output 3: Final Report

OK to release final report by RD 5 days 5

Dissemination and operationalization of corrective actions recommended in final report

Remote 5 days 5

Total RTE time

Use of Data Total consultant days

34

RO and CO corrective action Preliminary Findings

95

X. ASSESSMENT TEAM QUALIFICATIONS

Required qualifications and experience, Senior Consultant – International (L5):

Masters, PhD or Advanced degree in related subject area (Public Health, Social Services, Sociology, etc.);

At least 12-15 years progressively responsible experience including evaluative and analytical work on humanitarian performance at an organization or interagency level for an organization comparable to UNICEF;

Knowledge of current evaluative and analytical literature on the humanitarian sector in general;

Exposure to UNICEF’s humanitarian work;

Excellent English and French speaking and writing skills;

Excellent and proven research skills including development and application of analytical frameworks and tools and production of analytical papers;

Excellent interviewing capacity;

Excellent facilitation skills. Required qualifications and experience, Mid-Level Consultant – International (L4):

Masters or Advanced degree in related subject area (Public Health, Social Services, Sociology, etc.);

At least 12-15 years progressively responsible experience including evaluative and analytical work on humanitarian performance at an organization or interagency level for an organization comparable to UNICEF;

Significant knowledge of current evaluative and analytical literature on the humanitarian sector in general;

Excellent English and French speaking and writing skills;

Strong knowledge of results-based management and monitoring concepts;

Good knowledge of the local context, including institutions and people would be an asset;

Very good interviewing and facilitation skills.

96

Annex: CLUSTER SURVEY: Questionnaire and Results

QUESTIONNAIRE D’ENQUÊTE SUR LES CLUSTERS

DONT L’UNICEF DETIENT LE LEAD

MERCI DE REMPLIR UN QUESTIONNAIRE POUR CHAQUE CLUSTER

AUQUEL VOUS ÊTES MEMBRE

Cette enquête est destinée à des organisations (ONGs internationales et locales, OCB et services

gouvernementaux) qui sont parties prenantes de un ou plusieurs clusters coordonné par l’UNICEF au

Mali (WASH, NUT, EDUCATION, et Protection de l’Enfance). Un questionnaire doit être rempli par

organisation pour chaque cluster duquel elle est membre. Le questionnaire doit être rempli par la

personne qui participe habituellement aux réunions de cluster, ou par une personne qui peut le/la

substituer.

La collecte et le traitement des données sont confidentiels et impersonnels. Les informations seront

analysées de façon quantitative (sur la base du système de qualification numérique) et qualitative (sur la

base des commentaires). L’objectif principal de l’exercice est d’identifier les défis, faiblesses, et

dysfonctionnements liés au travail des clusters et de proposer des recommandations (vers le cluster et

vers l’UNICEF) en vue de solutionner et/ou corriger les contraintes identifiées. Les résultats de cette

enquête serviront à l’évaluation en temps réel de la réponse de l’UNICEF à la crise humanitaire au Mali.

Nom de l'Organisation

_____________________________________________________________________________________

____

Secteur/s d’intervention (Noter : N, E, WASH, ou

CP)_______________________________________________________

Zone/s et brève description des intervention/s

______________________________________________________________

Participation et rôle dans le Cluster (Noter : N, E, WASH, ou

CP)___________________________________________

Qualité du rôle de l’UNICEF en tant que leader ou co-leader (pour l’Education) du CLUSTER

Toutes les rubriques devront être notées de 1 à 543

(cochez ce qui vous semble approprié). Vous

pouvez ajouter des commentaires pour chaque rubrique

1 - Comment qualifieriez-vous l’expertise technique dont l’Unicef a fait preuve

dans le domaine du Cluster auquel vous prenez part?

Commentaires:

43 Notes 1-5: 1=très faible, 2=faible, 3=moyen, 4=bien, 5=très bien

1 2 3 4 5

97

2 - Dans quelle mesure l’UNICEF a fait preuve de leadership au sein du Cluster ?

Commentaires:

3 - A quel niveau noteriez-vous l’inclusion de toutes les parties prenantes du

secteur au sein du Cluster ?

Commentaires:

4 - Comment qualifieriez-vous la coordination mise en œuvre par le Cluster avec

ses membres ?

Commentaires:

5 - Coordination Intercluster: dans quelle mesure est-elle efficace?

Commentaires:

6 - Comment qualifieriez-vous le travail du Cluster dans la promotion et/ou mise

en pratique d’évaluations conjointes des besoins dans le secteur ?

Commentaires:

7 - Dans quelles mesures, le Cluster a développé et partagé des planifications

orientant les décisions stratégiques dans le Secteur ?

Commentaires:

8 - Quelle note donneriez-vous au Cluster pour son rôle de promoteur des normes

standards dans le secteur ? Et dans le suivi de leur application ?

Commentaires:

9 - Dans quelles mesures le Cluster a développé un ou des mécanismes de suivi,

reporting et gestion des données (désagrégation inclus)?

Commentaires:

10 - Quelle appréciation donneriez-vous sur la mise en place, par le Cluster,

d’outils et de mécanismes permettant des prises de décision rapides et précises ?

Efficacité du Groupe (décisions en temps opportun et points d'action clairs)

Commentaires:

11 - Dans quelles mesures, le Cluster a-t-il contribué à l’amélioration de la qualité

des activités dans le secteur, de la couverture géographique ainsi qu’à l’inclusion

des catégories les plus vulnérables ?

Commentaires:

12 - Le cluster a-t-il contribué et participé à promouvoir les questions

transversales prioritaires (par exemple l'âge, la diversité, l'environnement, le

genre, le VIH/SIDA et les droits de l'homme) ?

Commentaires:

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

98

13 - Comment pourriez-vous apprécier la promotion d’approches participatives et

communautaires des bénéficiaires par le Cluster?

Commentaires:

14 - Dans quelles mesures, le cluster s’est-il engagé dans des activités de

formation et de renforcement des capacités?

Commentaires:

15 - Comment qualifieriez-vous le rôle du Cluster en termes de plaidoyer ?

Commentaires:

16 - Dans quelles mesures, le Cluster a été un outil d’aide à l’obtention de

financements (y compris les allocations CERF) par les membres du Cluster ?

Commentaires:

17 - Dans quelles mesures UNICEF a agit en tant que responsable/opérateur de

l’aide en dernier recours ?

Commentaires :

18 - Comment noteriez-vous les relations mises en place par le Cluster avec le

Gouvernement et aussi les autorités locales et nationales ?

Commentaires:

19 - Dans quelles mesures les problématiques de préparation à l’urgence ont-elles

été gérées par le Cluster ?

Commentaires:

Autres commentaires - Quelle est selon vous la plus grande contrainte auquel le Cluster a été

confronté ? Y-a-t-il une voie de recours pour la surmonter? Autres.

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

99

CLUSTER SURVEY RESULTS

EDUCATION CLUSTER

Rubriques Gradings Average

1 Expertise Technique 3 4 5 4 4 4 4 5 4 5 4 4 5 5 4,29

2 Leadership 4 4 5 4 4 4 4 5 4 5 3 4 5 5 4,29

3 Inclusivité 3 4 5 3 3 4 3 5 4 1 3 4 3 5 3,57

4 Coordination avec les membres du Cluster 5 5 4 3 4 5 4 4 4 3 5 5 5 4,31

5 Coordination Intercluster 3 2 4 3 4 4 3 5 3 3 5 5 3,67

6 Évaluation des besoins 3 5 5 3 4 2 5 4 4 4 5 5 5 4,15

7 Planning et développement de stratégie 4 4 5 4 5 5 5 5 5 4 5 5 4,67

8 Promotion et application des standards 4 4 5 4 4 4 4 5 4 4 4 3 5 4,15

9 Monitoring et rapport, gestion de données (désagrégation inclus) 3 4 5 4 4 4 4 5 5 3 5 5 5 4,31

10 Efficacité du Cluster (décisions en temps opportun et points d'action clairs) 3 4 5 4 4 4 4 5 4 4 5 5 5 4,31

11 Efficacité du Cluster (contribue à améliorer le travail sur le terrain; le taux

de couverture de la réponse; les catégories marginalisées sont atteintes;

etc) 4 4 5 4 4 4 5 4 4 3 5 5 5 4,31

12 Le cluster aide-t-il et participe-t-il à promouvoir les questions transversales

prioritaires? 4 4 4 2 3 5 4 5 2 3 4 5 3,75

13 Le cluster prend-t-il en compte/ promeut des approches participatives et

communautaires? 3 4 4 2 4 4 4 4 4 5 3 3,73

14 Le cluster est-il engagé dans les activités de formation et de renforcement

des capacités? 3 5 4 2 3 4 4 4 1 4 3 3,36

15 Rôle de plaidoyer 2 4 4 4 3 3 5 3 5 3 5 5 3,83

16 Rôle pour la collecte de fonds (y compris les allocations CERF) 2 2 3 3 3 4 5 2 1 2 3 3 2,75

17 Dans quelles mesures UNICEF a agit en tant que responsable/opérateur de l’aide en dernier recours ? 3 3 5 3 4 4 5 3 3 5 3,80

18 Relations avec le Gouvernement et aussi les autorités locales et nationales 4 5 5 4 4 4 5 5 5 4 5 5 5 4,62

19 État de la préparation d’urgence: Dans quelle mesure ceci est géré par le

Cluster 2 4 5 4 4 5 4 4 3 5 5 4,09

100

NUTRITION CLUSTER

Rubriques Gradings Average

1 Expertise Technique 4 4 5 5 4 5 5 5 4,63

2 Leadership 5 5 4 5 4 5 5 5 4,75

3 Inclusivité 3 5 4 4 4 5 5 3 4,13

4 Coordination avec les membres du Cluster 5 5 5 5 5 5 5 5 5,00

5 Coordination Intercluster 2 3 4 4 4 4 5 3,71

6 Évaluation des besoins 4 3 4 4 4 4 5 5 4,13

7 Planning et développement de stratégie 4 4 3 5 4 5 5 5 4,38

8 Promotion et application des standards 5 4 4 5 5 5 3 4,43

9 Monitoring et rapport, gestion de données (désagrégation inclus) 5 5 4 5 5 5 5 5 4,88

10 Efficacité du Cluster (décisions en temps opportun et points d'action clairs) 5 3 4 4 4 5 5 5 4,38

11 Efficacité du Cluster (contribue à améliorer le travail sur le terrain; le taux

de couverture de la réponse; les catégories marginalisées sont atteintes;

etc) 5 4 4 5 5 4 4 5 4,50

12 Le cluster aide-t-il et participe-t-il à promouvoir les questions transversales

prioritaires? 4 2 3 4 4 4 4 4 3,63

13 Le cluster prend-t-il en compte/ promeut des approches participatives et

communautaires? 4 3 3 3 4 3 4 5 3,63

14 Le cluster est-il engagé dans les activités de formation et de renforcement

des capacités? 5 3 3 5 4 5 4 4 4,13

15 Rôle de plaidoyer 5 4 2 5 5 5 5 4,43

16 Rôle pour la collecte de fonds (y compris les allocations CERF) 4 2 2 5 3 5 3 3,43

17 Dans quelles mesures UNICEF a agit en tant que responsable/opérateur de l’aide en dernier recours ? 4 3 3 5 5 3 3 3,71

18 Relations avec le Gouvernement et aussi les autorités locales et nationales 5 4 4 4 5 5 5 5 4,63

19 État de la préparation d’urgence: Dans quelle mesure ceci est géré par le

Cluster 3 4 5 5 5 5 5 4,57

101

WASH CLUSTER

Rubriques Gradings Average

1 Expertise Technique 5 5 5 4 5 4 4,67

2 Leadership 5 3 5 4 5 3 4,17

3 Inclusivité 4 3 5 4 3 2 3,50

4 Coordination avec les membres du Cluster 5 4 5 4 5 4 4,50

5 Coordination Intercluster 5 5 4 5 3 4,40

6 Évaluation des besoins 4 1 4 4 5 4 3,67

7 Planning et développement de stratégie 5 4 5 5 5 4 4,67

8 Promotion et application des standards 3 3 5 4 3 3 3,50

9 Monitoring et rapport, gestion de données (désagrégation inclus) 5 5 5 4 5 3 4,50

10 Efficacité du Cluster (décisions en temps opportun et points d'action clairs) 5 4 5 4 5 3 4,33

11 Efficacité du Cluster (contribue à améliorer le travail sur le terrain; le taux

de couverture de la réponse; les catégories marginalisées sont atteintes;

etc) 3 4 4 4 5 3 3,83

12 Le cluster aide-t-il et participe-t-il à promouvoir les questions transversales

prioritaires? 4 2 4 3 4 3 3,33

13 Le cluster prend-t-il en compte/ promeut des approches participatives et

communautaires? 5 5 4 5 3 4,40

14 Le cluster est-il engagé dans les activités de formation et de renforcement

des capacités? 2 4 5 3 4 4 3,67

15 Rôle de plaidoyer 3 5 3 5 3 3,80

16 Rôle pour la collecte de fonds (y compris les allocations CERF) 3 4 4 3 3 3 3,33

17 Dans quelles mesures UNICEF a agit en tant que responsable/opérateur de l’aide en dernier recours ? 5 5 5 4 3 4,40

18 Relations avec le Gouvernement et aussi les autorités locales et nationales 4 3 4 4 5 4 4,00

19 État de la préparation d’urgence: Dans quelle mesure ceci est géré par le

Cluster 4 3 4 5 3 3,80

102

CHILD PROTECTION SUB-CLUSTER

Rubriques Gradings Average

1 Expertise Technique 5 4 4 4 3 4 4 4,00

2 Leadership 5 5 4 4 4 4 4 4,29

3 Inclusivité 5 5 3 2 3 4 4 3,71

4 Coordination avec les membres du Cluster 4 4 3 4 5 4 4,00

5 Coordination Intercluster 4 5 3 4 4 4,00

6 Évaluation des besoins 4 4 4 3 4 4 3,83

7 Planning et développement de stratégie 4 3 4 4 5 4,00

8 Promotion et application des standards 3 5 2 4 4 4 3,67

9 Monitoring et rapport, gestion de données (désagrégation inclus) 5 3 4 3 4 4 3,83

10 Efficacité du Cluster (décisions en temps opportun et points d'action clairs) 4 2 3 4 4 3,40

11 Efficacité du Cluster (contribue à améliorer le travail sur le terrain; le taux

de couverture de la réponse; les catégories marginalisées sont atteintes;

etc) 4 5 3 4 3 4 3,83

12 Le cluster aide-t-il et participe-t-il à promouvoir les questions transversales

prioritaires? 3 4 4 4 3 3 3,50

13 Le cluster prend-t-il en compte/ promeut des approches participatives et

communautaires? 3 3 2 3 4 4 3,17

14 Le cluster est-il engagé dans les activités de formation et de renforcement

des capacités? 5 4 3 4 3 3,80

15 Rôle de plaidoyer 5 4 4 3 4,00

16 Rôle pour la collecte de fonds (y compris les allocations CERF) 4 4 2 3 3 3,20

17 Dans quelles mesures UNICEF a agit en tant que responsable/opérateur de l’aide en dernier recours ? 4 4 3 3 4 3,60

18 Relations avec le Gouvernement et aussi les autorités locales et nationales 5 5 5 4 4 4 4,50

19 État de la préparation d’urgence: Dans quelle mesure ceci est géré par le

Cluster 3 3 4 2 4 4 3,33

103

ALL CLUSTERS

Rubriques Gradings Average

1 Expertise Technique 3 4 5 4 4 4 4 5 4 5 4 4 5 5 4 4 5 5 4 5 5 5 5 5 5 4 5 4 5 4 4 4 3 4 4 4,37

2 Leadership 4 4 5 4 4 4 4 5 4 5 3 4 5 5 5 5 4 5 4 5 5 5 5 3 5 4 5 3 5 5 4 4 4 4 4 4,37

3 Inclusivité 3 4 5 3 3 4 3 5 4 1 3 4 3 5 3 5 4 4 4 5 5 3 4 3 5 4 3 2 5 5 3 2 3 4 4 3,71

4 Coordination avec les membres du Cluster 5 5 4 3 4 5 4 4 4 3 5 5 5 5 5 5 5 5 5 5 5 5 4 5 4 5 4 4 4 3 4 5 4 4,45

5 Coordination Intercluster 3 2 4 3 4 4 3 5 3 3 5 5 2 3 4 4 4 4 5 5 5 4 5 3 4 5 3 4 4 3,86

6 Évaluation des besoins 3 5 5 3 4 2 5 4 4 4 5 5 5 4 3 4 4 4 4 5 5 4 1 4 4 5 4 4 4 4 3 4 4 4,00

7 Planning et développement de stratégie 4 4 5 4 5 5 5 5 5 4 5 5 4 4 3 5 4 5 5 5 5 4 5 5 5 4 4 3 4 4 5 4,48

8 Promotion et application des standards 4 4 5 4 4 4 4 5 4 4 4 3 5 5 4 4 5 5 5 3 3 3 5 4 3 3 3 5 2 4 4 4 4,00

9 Monitoring et rapport, gestion de données (désagrégation inclus) 3 4 5 4 4 4 4 5 5 3 5 5 5 5 5 4 5 5 5 5 5 5 5 5 4 5 3 5 3 4 3 4 4 4,39

10 Efficacité du Cluster (décisions en temps opportun et points d'action clairs) 3 4 5 4 4 4 4 5 4 4 5 5 5 5 3 4 4 4 5 5 5 5 4 5 4 5 3 4 2 3 4 4 4,19

11 Efficacité du Cluster (contribue à améliorer le travail sur le terrain; le taux

de couverture de la réponse; les catégories marginalisées sont atteintes;

etc) 4 4 5 4 4 4 5 4 4 3 5 5 5 5 4 4 5 5 4 4 5 3 4 4 4 5 3 4 5 3 4 3 4 4,18

12 Le cluster aide-t-il et participe-t-il à promouvoir les questions transversales

prioritaires? 4 4 4 2 3 5 4 5 2 3 4 5 4 2 3 4 4 4 4 4 4 2 4 3 4 3 3 4 4 4 3 3 3,59

13 Le cluster prend-t-il en compte/ promeut des approches participatives et

communautaires? 3 4 4 2 4 4 4 4 4 5 3 4 3 3 3 4 3 4 5 5 5 4 5 3 3 3 2 3 4 4 3,70

14 Le cluster est-il engagé dans les activités de formation et de renforcement

des capacités? 3 5 4 2 3 4 4 4 1 4 3 5 3 3 5 4 5 4 4 2 4 5 3 4 4 5 4 3 4 3 3,70

15 Rôle de plaidoyer 2 4 4 4 3 3 5 3 5 3 5 5 5 4 2 5 5 5 5 3 5 3 5 3 5 4 4 3 4,00

16 Rôle pour la collecte de fonds (y compris les allocations CERF) 2 2 3 3 3 4 5 2 1 2 3 3 4 2 2 5 3 5 3 3 4 4 3 3 3 4 4 2 3 3 3,10

17 Dans quelles mesures UNICEF a agit en tant que responsable/opérateur de l’aide en dernier recours ? 3 3 5 3 4 4 5 3 3 5 4 3 3 5 5 3 3 5 5 5 4 3 4 4 3 3 4 3,85

18 Relations avec le Gouvernement et aussi les autorités locales et nationales 4 5 5 4 4 4 5 5 5 4 5 5 5 5 4 4 4 5 5 5 5 4 3 4 4 5 4 5 5 5 4 4 4 4,48

19 État de la préparation d’urgence: Dans quelle mesure ceci est géré par le

Cluster 2 4 5 4 4 5 4 4 3 5 5 3 4 5 5 5 5 5 4 3 4 5 3 3 3 4 2 4 4 4,00

104