Document of The World Bankdocuments.worldbank.org/curated/en/316581468058523732/... ·...

Document of The World Bank

Report No: ICR00003391

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-4825-RO)

ON A

LOAN

IN THE AMOUNT OF €47.2 MILLION (US$58.5 MILLION EQUIVALENT)

TO

ROMANIA

FOR A

SOCIAL INCLUSION PROJECT

February 28, 2015

Social Protection and Labor Global Practice Central Europe and the Baltic Countries

Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 2014)

Currency Unit = New Romanian Leu (RON) RON 1.00 = €0.22 €1.00 = RON 4.48

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS ADP Annual Development Plan CBS Community-Based Services CFAA Country Financial Accountability Assessment CoA Court of Accounts CPS Country Partnership Strategy DPSSIP Direction for Policies, Strategies, and Social Inclusion Programs DPL Development Policy Loan ECA Europe and Central Asia ECD Early Childhood Development ECE Early Childhood Education EU European Union GDSACP General Direction for Social Assistance and Child Protection GOR Government of Romania GSG General Secretariat of the Government IDF Institutional Development Fund IFR Interim Financial Report JIM Joint Inclusion Memorandum JNAP National Action Plan for the Implementation of the JIM LTC Long-Term Care M&E Monitoring and Evaluation MER Ministry of Education and Research (later renamed, Ministry of

National Education) MIS Management Information System MLFSP Ministry of Labor, Family and Social Protection MLSSF Ministry of Labor, Social Solidarity and Family MPF Ministry of Public Finance NAFP National Agency for Family Protection NAPH NationalAuthority for Persons with Handicap NAR National Agency for Roma NASB National Agency for Social Benefits NGO Nongovernmental Organization NSIRS National Strategy for Improvement of the Roma Situation OM Operational Manual

ii

PAD Project Appraisal Document PBA Program-Based Approach PDO Project Development Objective(s) PPP Public-Private Partnership PWD Persons with Disabilities RAS Reimbursable Advisory Services RF Results Framework RSDF Romanian Social Development Fund SAFIR NASB’s MIS SASMP Social Assistance System Modernization Project SDFP Social Development Fund APL2 Project SIL Specific Investment Loan SIP Social Inclusion Project, the project SSDP Social Sector Development Project TA Technical Assistance TOR Terms of Reference

Vice President: Laura Tuck Country Director: Mamta Murthi Sector Manager: Andrew Mason

Project Team Leader: Richard Florescu ICR Team Leader: Ugo Gentilini

iii

ROMANIA Social Inclusion Project

CONTENTS

DATA SHEET ................................................................................................................... vi

A. Basic Information ...................................................................................................... vi B. Key Dates .................................................................................................................. vi C. Ratings Summary ...................................................................................................... vi D. Sector and Theme Codes ......................................................................................... vii E. Bank Staff ................................................................................................................ viii F. Results Framework Analysis ................................................................................... viii G. Ratings of Project Performance in ISRs .................................................................. xv H. Restructuring (if any) .............................................................................................. xvi I. Disbursement Profile ........................................................................................... xvii

Executive Summary ....................................................................................................... xviii 1. Project Context, Development Objectives and Design ............................................... 1

2. Key Factors Affecting Implementation and Outcomes ............................................... 8

3. Assessment of Outcomes ........................................................................................... 19

4. Assessment of Risk to Development Outcome ......................................................... 33

5. Assessment of Bank and Borrower Performance ...................................................... 34

6. Lessons Learned ........................................................................................................ 38

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ........... 42

Annex 1. Project Costs and Financing .............................................................................. 43

Annex 2. Outputs by Component...................................................................................... 45

Annex 3. Economic and Financial Analysis ..................................................................... 67

Annex 4. Bank Lending and Implementation Support/Supervision Processes ................. 68

Annex 5. Beneficiary Survey Results ............................................................................... 70

Annex 6. Stakeholder Workshop Report and Results ....................................................... 71

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 72

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders ........................ 113

No Comments ................................................................................................................. 113

Annex 9. List of Supporting Documents ........................................................................ 114

Annex 10. Results Framework ........................................................................................ 115

Annex 11. Outcome Rating for Project with Formally Revised Project Objectives ...... 127

Annex 12. Project photos ................................................................................................ 128

MAP ................................................................................................................................ 130

iv

DATA SHEET

A. Basic Information

Country: Romania Project Name: SOCIAL INCLUSION PROJECT

Project ID: P093096 L/C/TF Number(s): IBRD-48250,TF-90485 ICR Date: 01/15/2015 ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVERNMENT OF ROMANIA

Original Total Commitment:

USD 58.50M Disbursed Amount: USD 49.56M

Revised Amount: USD 46.38M Environmental Category: C Implementing Agencies: Ministry of Labor, Family and Social Protection; Ministry of Education and Research/Ministry of National Education; Romanian Social Development Fund; National Agency for ROMA Cofinanciers and Other External Partners: N/A

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 11/29/2005 Effectiveness: 03/16/2007 03/16/2007

Appraisal: 03/20/2006 Restructuring(s): 03/28/2011 12/13/2012

Approval: 06/13/2006 Mid-term Review: 04/16/2009 04/13/2010 Closing: 03/01/2011 06/30/2014

C. Ratings Summary

C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

vi

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 22 22 Other social services 68 68 Pre-primary education 10 10

Theme Code (as % of total Bank financing) Education for all 14 14 Other social protection and risk management 29 29 Participation and civic engagement 14 14 Social Protection and Labor Policy & Systems 29 29 Urban services and housing for the poor 14 14

vii

E. Bank Staff

Positions At ICR At Approval Vice President: Laura Tuck Shigeo Katsu Country Director: Mamta Murthi Anand Seth Sector Manager: Andrew Mason Hermann von Gersdorff Project Team Leader: Richard Florescu Ana Maria Sandi/Richard Florescu ICR Team Leader: Ugo Gentilini ICR Primary Author: Suzana Nagele de Campos Abbott

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The SIP’s development objective is to improve the living conditions and social inclusion of the most disadvantaged/vulnerable people in Romanian society by (i) improving the living conditions and social inclusion of Roma living in poor settlements; (ii) increasing the inclusiveness of ECE services in targeted areas; and (iii) improving the quality of services for PWD, youth at risk, and victims of domestic violence. Revised Project Development Objectives (as approved by original approving authority) The project development objective (PDO) was revised through a level-1 project restructuring on March 28, 2011 to include the new technical assistance (TA) activities related to the strengthening of Romania’s social assistance administration under the project. The TA activities were to support implementation of the GORs ongoing social assistance reforms and its then recently approved Social Assistance Reform Strategy. The TA activities also complemented the Bank’s support for those reforms under an ongoing programmatic development policy loan (DPL) series and a Social Assistance System Modernization Project (SASMP) that was under preparation at the time. The revised PDO was to (i) improve the living conditions and social inclusion of the most disadvantaged and vulnerable people in Romania and (ii) strengthen the administration of social assistance benefits.

viii

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at Completion or Target

Years

Indicator 1: % reduction in the gap between targeted settlements and neighboring communities as measured by the living conditions index

Value (Quantitative or Qualitative)

504.46 index points 20 61.77

Date achieved 12/31/2008 06/30/2014 06/09/2014 Comments (incl. % achievement)

Exceeded

Indicator 2: % of Roma targeted poor settlements acknowledge an improvement in basic living conditions


To be taken as communities enter the program

70 73.84%

Date achieved Comments (incl. % achievement)

Achieved. Data is not available for all years, but refers to latest available rate as per RSDFs’ “Monitoring Report: An Independent Study of the Performance Indicators of PIP” (2013)

Indicator 3: Percentage points increase of children from vulnerable groups in targeted communities participating in ECE


70 75 76


Exceeded

Indicator 4: % of services for PWD scoring at least 80% on the standards compliance index for each main category of quality standards.


0 25 0


Not achieved. As per June 2014, 11 Centers for PWD were completed and staffing was in progress. Works in other 4 centers was ongoing and would be completed beyond project closing date. Staff training was canceled.

Indicator 5: % increase of the rate of employment of youth aged 18 years and above benefiting from multifunctional centers.


0 40 39.6

Date achieved 06/30/2007 06/30/2014 06/09/2014

ix

Comments (incl. % achievement)

Substantially Achieved

Indicator 6: The scope of NASB MIS, as measured by the number of large social assistance programs (or their successors) paid through SAFIR, increases from 4 (2010) to all programs by 2013


4 8 8


Exceeded. Included in level-1 restructuring.

Indicator 7: Project beneficiaries (direct and indirect) Value (Quantitative or Qualitative)

n/a 150658 152658


Introduced as corporate strategic indicator

(b) Intermediate Outcome Indicator (s)

Indicator Baseline Value Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: % of targeted Roma settlements having access to improved infrastructure (for example, water and roads)


10% (2008) 60 82.71


Exceeded

Indicator 2: % of Roma from targeted poor settlements agreeing that subprojects reflect community priorities



60 75.66


Exceeded. Data is not available for all years, but refers to latest available rate as per RSDFs’ “Monitoring Report: An Independent Study of the Performance Indicators of PIP” (2013)

x

Indicator 3: Number of annual consultations of Roma in poor settlements with local authorities for addressing community needs (cumulative)


0 360 1,659


Exceeded. Target revised in level-1 restructuring to 360, with baseline set at 120 (2008)

Indicator 4: % increase in number of NAR staff with improved competencies in M&E Value (Quantitative or Qualitative)

0 30 10


Achieved. Target revised to 10 in level-1 restructuring to 30. Was reported as intermediate result (outcome 2) in 2006 PAD

Indicator 5: % of ECE interventions in communities with more than 50% Roma population


0 60


Dropped in level-1 restructuring

Indicator 6: % increase of children from vulnerable groups participating in ECE services through ECE alternative CBS in targeted communities


0 30


Data not available. Included in level-1 restructuring.

Indicator 7: % of PWD services implemented in high-priority areas Value (Quantitative or Qualitative)

0 30


Data not available. Most of the services for PWD became available after the project closing.

Indicator 8: % increase in funding allocation for CBSs by NAPH (for PW) and MER (for ECE)

xi


0 10


Data not available. Modified in level-1 restructuring to read: Percentage increase in funding allocation for CBS by NAPH (for PWD).

Indicator 9: % of community members involved in alternative CBS Value (Quantitative or Qualitative)

0 10



Indicator 10: % increase in ECE staff with skills in inclusive education Value (Quantitative or Qualitative)

0 10 10.56 (3,500 staff trained)


Exceeded.

Indicator 11: % increase in the number of staff with improved competencies in integrated services (domestic violence).


0

Date achieved 12/31/2007 Comments (incl. % achievement)

Data not available. Was reported as intermediate result (outcome 2) in 2006 PAD. Hiring of staff was completed but training has not been carried out due to time constraints.

Indicator 12: % increase in the number of qualified staff in PWS services Value (Quantitative or Qualitative)

0 10 0


Not Achieved. Consultant was selected but activity was not completed as a result of delays in works completion and staff hiring

Indicator 13: % of beneficiaries (including legal tutors) of social inclusion services (CBS for ECE and PWD, LTC institutions, and integrated services) satisfied with the quality of services


0 70 80

Date achieved

xii

Comments (incl. % achievement)

Achieved. Data is not available for all years, but refers to latest available rate as per “Evaluation Study of the Early Childhood Education Program” (2013)

Indicator 14: Increase in number of PWD with access to CBS Value (Quantitative or Qualitative)

0 90


Data not available.

Indicator 15: % decrease of parent from vulnerable groups perceiving discriminatory teaching practices in ECE



20


Data not available (no baseline). However, the “Evaluation Study of the Early Childhood Education Program” (2013) shows that 75% of respondents are satisfied with the way Roma children are treated by educators and 75% of the respondents are satisfied with the way Roma and Romanian children interact.

Indicator 16: % of SIP promoters of mature project proposals eligible for EU financing


0 60 100


Exceeded.

Indicator 17: Evaluations of targeting accuracy of social protection programs, and of administrative and client participation costs of social assistance programs, carried on and disseminated (cumulative number of evaluation reports)


0 4 0


Included in level-1 restructuring. The second selection of consulting services was canceled due to the high cost and time constraints. Will be carried out under SASMP.

Indicator 18: NASB MIS cross-checks information with an increasing number of external databases (Pension House, Public Employment Service, Labor Inspection, Civil Registry, and Tax Registry)

xiii


0 4 5 Data cross-

checks operating

with 5 other databases


Achieved. Included in level-1 restructuring.

xiv

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 05/07/2007 Satisfactory Satisfactory 0.00 2 03/10/2008 Moderately Satisfactory Moderately Satisfactory 0.64 3 09/15/2008 Moderately Satisfactory Moderately Satisfactory 1.33

4 06/03/2009 Moderately Unsatisfactory

Moderately Unsatisfactory 1.87

5 12/16/2009 Moderately Unsatisfactory Moderately Satisfactory 5.91

6 04/30/2010 Moderately Unsatisfactory Moderately Satisfactory 7.79

7 01/04/2011 Moderately Satisfactory Moderately Satisfactory 9.66 8 07/30/2011 Moderately Satisfactory Moderately Satisfactory 13.71 9 03/03/2012 Moderately Satisfactory Moderately Satisfactory 17.84

10 09/22/2012 Moderately Satisfactory Moderately Satisfactory 24.52 11 09/03/2013 Moderately Satisfactory Satisfactory 33.87 12 05/03/2014 Moderately Satisfactory Moderately Satisfactory 41.09 13 06/27/2014 Moderately Satisfactory Moderately Satisfactory 43.65

xv

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

03/28/2011 Yes MS MS 11.26

Reasons for restructuring and key changes made: (i) Revising the Project’s Development Objective to expand its scope; (ii) Reallocating project funds from savings from Component 3 (Social Assistance Programs) and from Component 4 (Capacity Building for Roma Social Inclusion) of the Project to: (a) Component 1 (Priority Interventions Program) to finance additional small infrastructure sub-projects in poor Roma communities; and (b) a new subcomponent of Component 3 to finance technical assistance activities and investments in support of the ongoing reforms and the Government’s Social Assistance Reform Strategy; (iii) Modifying the Results Framework Agreement; and (d) extending the Project’s closing date by 21 months from May 1, 2011 to February 1, 2013.

12/13/2012 Yes MS MS 25.5

Reasons for restructuring and key changes made: (i) Extending the Project’s closing date by 17 months from February 1, 2013 to June 30, 2014; (ii) amending the description of activities from sub-component 3.4 of the Project to allow the upgrade of the management information system for the administration of the social assistance benefits.

xvi

I. Disbursement Profile

xvii

Executive Summary

1. The Social Inclusion Project (SIP) was developed and launched whilst, in the mid-2000s, Romania was preparing to join the European Union (EU). Such process included paying renewed attention to social inclusion matters, particularly among Romania’s large Roma population. At that time, the Bank had contributed to increasing awareness on Roma issues – including helping to launch the Decade of Roma Inclusion 2005-2015 – as well as partnering in project operations. The loan of €47.2 million for the SIP was approved by the Bank’s Board on June 2006 and became effective on March 2007. The original project development objective (PDO) was to improve the living conditions and social inclusion of the most disadvantaged and vulnerable people in Romania. 2. The project was restructured twice. The first restructuring took place in early 2011, including to revising the PDO to expand its scope, reallocating project funds from savings between components and introduce a new subcomponent, modifying the Results Framework Agreement, and extending the Project’s closing date by two years from March 1, 2011 to March 1, 2013. The second restructuring was held in December 2012 to extend the Project’s closing date by 17 months from February 1, 2013 to June 30, 2014, and amend the description of activities from one sub-component. 3. The Project comprised four components, the first of which featured the “Priority Interventions Program” (€11.7 Million). This supported the Romanian Social Development Fund’s (RSDF) program to improve living conditions and social inclusion for Roma through better infrastructure and basic social services. The second component of the SIP, “Inclusive Early Childhood Education Program” (€6.1 Million) in the Ministry of Education and Research (MER), was pursued through various activities. These mainly included construction of Early Childhood Education (ECE) infrastructure in communities with a high percentage of Roma, development of an inclusive ECE curriculum, provision of training to ECE staff, formulation of coherent ECE legislation, communication initiatives, and other activities. 4. The third component, “Social Assistance Programs” (€28.6 Million), originally envisioned three subcomponents housed under the Ministry of Labor, Family and Social Protection (MLFSP). The Disabilities Program subcomponent (€14.4 million) aimed at restructuring of institutionalized care and development of alternative community-based systems of assistance. The subcomponent of Youth at Risk Program (€11.1 million) was designed for youth aged 18 years and above graduating from the institutionalized care system to provide them with increased access to integrated social services. The last subcomponent, Victims of Domestic Violence Program (€3.1 million), was designed to support the development of specialized integrated social services for such victims until their family situation was resolved. A new fourth subcomponent was subsequently introduced in 2011 to support the implementation of the Social Assistance System Modernization Project (SASMP) and included a range of activities in support of MLFSP to increase efficiency in the administration and monitoring of social assistance benefits. Finally, the SIP fourth component, “Capacity Building for Roma Social Inclusion” (€0.8

xviii

Million) was intended to provide assistance to the National Agency for Roma (NAR) and its eight regional branches to coordinate activities such as inclusive planning, monitoring and evaluation. 5. Several factors affected the SIP’s implementation and outcome. During project preparation, two distinct projects were under preparation, one to address the needs of the Roma and another to meet those of ‘other vulnerable groups’. In line with the Government’s vision, however, these were merged into a single project. The preparation phase identified four key risks, which were rated as ‘significant’. While relevant mitigation measures were set out, the project’s risk analysis may have underestimated some key risks, chiefly the complexities around providing institutional coordination, monitoring and evaluation of a comprehensive program implemented at multiple levels and by several ministries. In practice, each of the implementing agencies would use their own administrative, fiduciary, and procurement structures, with no single agency entrusted with overall coordination of the project, its implementation, and reporting requirements (the MLFSP assumed a de facto role in overall reporting coordination). 6. When the project was approved, three of the project’s Operational Manuals (OMs) (i.e., RSDF, MER, and MLFSP) had not been completed and adopted, and the site selection for activities had not been decided. This was mainly due to efforts to ensure participation of local communities and Roma counterparts in identifying locations, priority works, and activities. As a result, baseline data could only be collected once site selection had been decided. Finally, the RF was designed to measure progress achieved through activities that were to be supported by the project, which was adequate as a measure of improvements in ‘living conditions’, but not for defining and tracking ‘social inclusion’. 7. The project’s early implementation was slow. Three years after effectiveness, disbursements totaled only about 10 percent (but commitments totaled slightly over 70 percent). Several factors had a negative impact on the project’s implementation, including the limited capacity of the NAR, lengthy subproject approval procedures, the financial crisis of 2008, low local government capacity, and central government staffing changes. With regards to NAR in particular, due to the protracted lack of progress in implementing activities under Component 4, savings amounting to €0.6 million were reallocated to other components under the restructuring in 2011. Monitoring presented several challenges in the project’s early implementation that resulted in successive ‘Moderately Unsatisfactory’ or ‘Unsatisfactory’ ratings for M&E in the project’s Implementation Status and Results Report (ISR). The RSDF developed its own survey, implemented it, obtained the baseline data (collected in 2008 and 2009), and conducted evaluations. Efforts made by the other implementing agencies to select consultant services to carry out baseline indicator studies were hindered by limited interest of potential bidders. For Component 3, there were four attempts by the MLFSP to contract these services. 8. The project’s overall outcome is rated Moderately Satisfactory. It was, and continues to be, highly relevant and central to the Government’s strategy and the Bank’s

xix

assistance priorities. Its outcomes, as measured by the indicators in the project’s RF, were achieved and surpassed with respect to the first two components, and were partially achieved with respect to the third component. However, the project was not successful in strengthening the NAR to assume a more central role in the coordination, monitoring, and evaluation of programs focused on Roma communities. Based on available data and expectations, the project was implemented efficiently and is expected to either generate positive returns or improve living conditions for disadvantaged persons. In other words, outputs were achieved, although they were compounded by limited evidence on impacts and efficiency. 9. The Bank’s performance in ensuring quality at entry is rated Moderately Satisfactory. Responding to Government’s request for assistance in a very short period, the project addressed most vulnerable groups under one single operation. The project was designed with a clear sustainability objective, not only by using existing government structures, but also by requiring up-front investments commitments from local authorities. While recognizing the risk that it posed and responding to the GOR’s request for rapid assistance and the World Bank’s desire to respond quickly, the preparation team may have underestimated the complexities of the project’s institutional arrangements. Finally, the project’s RF, while relatively concise, presents mostly output indicators and depends on often subjective surveys of beneficiary satisfaction. It did not include important indicators—for example, how many beneficiaries were served by different investments, their gender breakdown, and the number of Roma who benefited—that would have been valuable to monitor progress and retroactively adjust programs to more effectively address the needs of vulnerable groups. However, these data were collected internally by the RSDF and the MER for Components 1 and 2. 10. From the monitoring side in particular, the project did not systematically track the full range RF indicators. For instance, 10 out of the 24 indicators that were to be monitored as per 2011 restructuring were not included in the ensuing ISRs. Data for some of those indicators was, however, availed in commissioned studies, these only provided final data instead of reporting on indicators’ value on an annual basis. Although there were challenges in the establishment of baselines, recruitment of M&E consultants and possible unclear nature of some indicators, more efforts could have been put in place to ensure a systematic collection, monitoring and reporting of RF indicators. 11. The quality of supervision is rated Moderately Satisfactory. Supervision was undertaken in a challenging environment, including, for example, a fluid and frequently changing institutional context. Supervision was regular and focused on obtaining up-to-date information on indicators in the project’s RF. Yet the project’s components were supervised individually and implemented separately by the respective agencies with, as per design, limited coordination among them. Although the 2011 restructuring reallocated funding in favor of well-performing components, at times, supervision efforts could have seized opportunities to adapt, adjust, and reformulate the project in a way that would have enhanced the impact and use of available resources. As a result, €10.2 million that could have been used to finance additional activities by the RSDF and the MER had to be cancelled.

xx

12. The Government performance was Moderately Satisfactory. The Government displayed ownership and commitment to the project throughout implementation despite the frequent changes that affected the political leadership and fiscal constraints. Nevertheless, the Government, despite expressing its objective of addressing the needs of vulnerable groups in a coordinated fashion, did not to put in place a workable mechanism to develop a concerted strategy, prioritize, disseminate, and monitor progress in addressing their needs. 13. The implementing agencies’ performance is rated Moderately Satisfactory. The RSDF’s performance exceeded expectations. Its strong performance benefitted from competent, committed staff and good governance. The MER’s performance was also highly satisfactory because it was able to implement activities under Component 2 effectively and efficiently. The performance of the MLFSP (including the National Agency for Family Protection (NAFP) NationalAuthority for Persons with Handicap (NAPH) until they were made directorates) was weaker than that of either the RSDF or the MER. This was largely the result of several government changes (leadership changed seven times during implementation with consequent changes at lower levels in the Ministry), as well as major restructuring in the aftermath of the 2008 economic crisis. Clearly, local-level implementation was also affected by the Government’s cumbersome procedures. 14. The project’s implementation provides several lessons. For example, project design should set out a clear definition of what constitutes social inclusion, how to measure it, and a theory of change for the project overall; design needs to incorporate a comprehensive pre-assessment of institutional capacities; decentralized and centralized implementation models present potential tensions that need to be considered explicitly during project preparation; the establishment of a coordination mechanism among project components is important, especially when multiple implementing agencies are involved; there is a potential trade-off between rapid preparation of a project and establishing an effective monitoring and evaluation system; committed leadership and champions have a critical role to play in laying the foundations for social inclusion; project design should consider flexible and tailored arrangements that allow for intra-community and spatial-sensitive approaches; the role of and consultation with NGOs and civil society should be embedded within the project cycle on inclusion projects; finally, pilot experiences can provide an effective instrument to test alternative approaches to implementation of assistance delivery models.

xxi

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

15. At the time of appraisal of the Social Inclusion Project (SIP, the project) in early 2006, Romania was focusing all of its efforts on accession to the European Union (EU) and had initiated a significant number of actions to facilitate smooth integration. Among these, social inclusion and poverty reduction measures, targeted at disadvantaged groups, were being designed to contribute to the country’s continued stability, social cohesion, and participation in growth and economic integration. A Joint Inclusion Memorandum (JIM), signed by Romania and the European Commission in June 2005, aimed to prepare the country for full participation in the ‘open method of coordination’ agreed by EU member states on social inclusion. The JIM identified the following vulnerable social groups: the Roma minority, children at high risk, persons with disabilities, youth over 18 years leaving the state child protection system, and victims of domestic violence. 16. Romania’s large Roma population1 was, in particular, in need of social inclusion. An estimated 74 percent of Roma lived in poverty, with almost half of the poor living in extreme poverty. Their infant mortality rate was four times higher than the country’s average, and only 12.9 percent of the active Roma were formally employed. Problems facing the Roma were considered systemic, with interrelated issues with regard to education, housing, employment, and health, and their ability to relate to institutions and influence policies and programs was limited. Also, the poverty risk among children was high, and this had a disproportionate impact among Roma populations2.

1 According to the National Census 2002, only 535,250 persons self-identified as Roma, but sociological surveys reported a much higher Roma population (1.5 million estimated in 1998 by the Institute for Quality of Life). 2 Around 30 percent of children under the age of 15 were poor (compared to 23 percent of the total population); Romania’s infant mortality (18.4 percent) was the highest of all EU member states and candidate countries; most children under five who died at home belonged to Roma families and to families living in extreme poverty; and 12–20 percent of Roma children dropped out of school during primary and secondary education. The availability of early childhood development (ECD) services had declined dramatically during the transition period. A large number of the nurses in crèches were unqualified, and participation of Roma children in preschool education was four times lower than among the rest of the preschool population. Of Romania’s population of 21.7 million at the time, about 430,000 were registered as persons with disabilities (PWD), including about 55,000 children. Among the adults, almost 20,000 PWD lived in 141 long-term care (LTC) institutions that functioned under the authority of local governments. Due to financial constraints, however, those LTC institutions had unacceptable conditions. Furthermore, disability determination criteria were somewhat arbitrary and restrictive, and employment opportunities for persons with less severe disabilities were lacking. Although the number of youth over 18 years in the state residential care system had been halved, slightly under 14,000 children aged 14–17 years still lived in childcare institutions. About 5,000 of them left these institutions annually when they turned 18 years and state assistance ended, and a significant number of these youth were Roma. The number of victims of domestic violence had been increasing—from about 8,000 reported cases in 2004 to slightly over 9,500 in 2005.

1

17. Government strategy. The government of Romania (GOR) had developed, and in some cases, begun implementation of strategies to address the challenges confronting each of these disadvantaged and vulnerable groups. In 2001, the GOR adopted a ten-year National Strategy for Improvement of the Roma Situation (NSIRS). It had also been active in the launch of the Decade of Roma Inclusion (2005–2015), for which Decade Action Plans had been prepared (and were under implementation) for education, employment, housing, and health. The Ministry of Education and Research (MER) had developed an Early Childhood Education (ECE) strategy (2005–2011) as part of an envisaged Convergent Early Childhood Development (ECD) Strategy. The ECE strategy aimed to improve access to and quality of early education for children aged 0–6 years, providing linkages with health and child protection programs. 18. Building on social protection and anti-discrimination legislation, in September 2005, the GOR approved a Strategy for the Social Protection, Integration, and Inclusion of Persons with Disabilities (2006–2103) and related action plans. It initiated activities aimed at prevention of institutionalization, deinstitutionalization, development of alternative community-based services (CBS), and social inclusion for persons with disabilities. A National Strategy for Social Inclusion of Youth over 18 Coming Out of the Residential Care System (2006–2008), which contemplated the concept of multifunctional centers to temporarily assist these young people in preparing for an independent life, was in the process of approval. Finally, the then significant increase in the incidence of domestic violence led to the approval of the National Strategy for the Prevention and Fight Against Domestic Violence in 2005. 19. Rationale for World Bank involvement. The rationale for Bank involvement was strong. The Bank had contributed to increasing awareness on Roma issues by co-organizing a series of regional conferences with the Open Society Foundations and the EU. This led to the creation of the Roma Education Fund and the launch of the Decade of Roma Inclusion (2005–2015). Romania had participated in these events and declared its commitment to reduce the social exclusion of Roma. The GOR’s strategies and plans to modernize the approach to social assistance of PWDs, youth at risk, and victims of domestic violence would benefit from the Bank’s earlier work in these areas, including support to Romania’s highly successful child welfare reform under the Child Welfare Reform Project.3 Further, the project would build on earlier Bank assistance under the two-phase Social Development Fund Program APL (including, in part, implementation arrangements) that also aimed to improve the livelihood of beneficiaries from poor, rural communities, and disadvantaged groups while promoting social capital enhancement and civic engagement among beneficiaries.4

3 Child Welfare Reform Project, P055495, Report No. 17465-RO dated June 9, 1998. 4 First and second phase of SDFP, P049200 and P068808, Report Nos. 17379-RO and 22876-RO, dated December 23, 1998, and November 26, 2001.

2

1.2 Original Project Development Objectives (PDO) and Key Indicators (As Approved)

20. The SIP’s development objective was to improve the living conditions and social inclusion of the most disadvantaged and vulnerable people in the Romanian society by (i) improving the living conditions and social inclusion of Roma living in poor settlements; (ii) increasing the inclusiveness of ECE services in targeted areas; and (iii) improving the quality of services for PWD, youth at risk, and victims of domestic violence. 21. The key indicators identified to measure progress toward achieving project outcomes were the following:

• 20 percent reduction in the gap between targeted poor Roma settlements and neighboring communities as measured by the living conditions index;

• 70 percent of Roma in targeted poor settlements acknowledging an improvement in basic living conditions;

• 5 percentage points increase in children from vulnerable groups in targeted communities participating in ECE;

• Services developed for the PWD score at least 80 percent on the standards compliance index for each category of quality standards; and

• At least 40 percent increase in the rate of employment of youth aged 18 years and above benefiting from multifunctional centers’ services.

22. Intermediate outcome indicators were also defined in the Project Appraisal Document (PAD). These are presented in annex 10.

1.3 Revised PDO (As Approved by Original Approving Authority) and Key Indicators, and Reasons/Justification

23. The project development objective (PDO) was revised through a level-1 project restructuring on March 28, 2011, to include new technical assistance (TA) activities related to the strengthening of Romania’s social assistance administration under the project. The TA activities were to support implementation of the GOR’s ongoing social assistance reforms and its then recently approved Social Assistance Reform strategy. The TA activities also complemented the Bank’s support for those reforms under an ongoing programmatic development policy loan (DPL) series and a Social Assistance System Modernization Project (SASMP) that was under preparation at the time. 24. The revised PDO was to (i) improve the living conditions and social inclusion of the most disadvantaged and vulnerable people in Romania and (ii) strengthen the administration of social assistance benefits. The project’s original outcome indicators remained, but the following new indicator was included to reflect its expanded PDO:5

5 The Restructuring Paper mentioned that two new indicators were included, but only listed the one mentioned here.

3

• The scope of National Agency for Social Benefits (NASB) management

information system (MIS), as measured by the number of large social assistance programs6 (or their successors) paid through SAFIR, increases from 4 (2010) to 8 programs by 2013.

25. The project’s intermediate outcome indicators were also revised to reflect its restructuring as well as to adjust them to implementation progress (annex 10).

1.4 Main Beneficiaries

26. The project’s main direct beneficiaries were to include (i) the most disadvantaged Roma who lived in 100 (later, 116 after the project’s restructuring) poor settlements and would be provided with improved living conditions and efforts aimed at promoting social inclusion; (ii) Roma children aged 0–6 years, who would benefit from the ECE through mainstreaming into the MER’s regular policies and programs; (iii) PWD, with a high dependency on residential services, who would be provided with better quality care and increased access to CBS; (iv) youth over 18 years graduating from the institutionalized child care system, who would no longer be eligible for state support and who would be provided with social services aimed at increasing their readiness for independent living; and (v) victims of domestic violence and their dependents who would benefit from specialized, integrated social services until their family situation was resolved. Project beneficiaries also included the institutions of social protection responsible for implementation of different activities and the new CBS—the Romanian Social Development Fund (RSDF), the MER, the Ministry of Labor, Social Solidarity and Family (MLSSF), the National Authority for Persons with Handicap (NAPH), and the National Agency for Roma (NAR). These institutions would benefit from training, TA, and miscellaneous services to support the strengthening of their service delivery and their monitoring and evaluation (M&E) capabilities. 7 Through project restructuring, the Ministry of Labor, Family, and Social Protection (MLFSP), and the broader government were introduced as beneficiaries, through the institutional strengthening that would be provided to them.

6 The eight largest social assistance programs in 2010 were the State Child Allowance (SCA); Child Raising Benefits (CRB); Complementary Family Allowance (CFA); Single Parent Allowance (SPA); Guaranteed Minimum Income (GMI); Heating Benefits (HB); Indemnity for Disabled Adults (IDA); and Complementary Budget for Disabled Adults (CBDA). 7 The name of the Ministry of Labor, Social Solidarity and Family was formally changed to Ministry of Labor, Family and Social Protection (MLFSP) in 2012 (December 22). The acronyms MLSSF and MLFSP are used in this document to refer to the ministry based on timing, that is, information from the PAD uses MLSSF, and later references use MLFSP.

4

1.5 Original Components (As Approved)

27. The SIP comprised four components: (i) Component 1: Priority Interventions Program (€11.7 Million) 28. This component was designed to support RSDF’s program of poverty alleviation and capacity-building interventions by expanding the fund’s activities to specifically address the needs of the Roma. The project aimed to improve living conditions and social inclusion for Roma living in about 100 poor settlements, bringing those most disadvantaged settlements up to minimum conditions in terms of infrastructure and basic social services. The project was to support (i) financing of demand-driven, integrated Priority Interventions subprojects, on a grant basis awarded by the RSDF following a competitive process from among a list of eligible settlements provided by the NAR; (ii) carrying out of promotion, information, and education campaigns; M&E activities; and the provision of training for local authorities, community groups, and nongovernmental organizations (NGOs); (iii) provision of assistance in capacity building at the community level through preparation, facilitation, evaluation, and supervision of Priority Interventions subprojects and provision of related training, including training to the RSDF; and (iv) provision of TA to poor Roma communities to prepare projects to be financed through EU funds. (ii) Component 2: Inclusive Early Childhood Education Program (€6.1 Million) By helping mainstream Roma into the regular policies and programs of the MER, this component was designed to support the MER’s ECE program (2005–2011) that aimed to ensure every child’s right to education and overall development starting from a very early age. The project would help jump-start the inclusive measures of the MER’s overall ECE program through (i) construction, extensions, rehabilitation, and furnishing of ECE infrastructure in communities with a high percentage of Roma; (ii) development of an inclusive ECE curriculum, provision of training to ECE staff, development and distribution of ECE teaching and learning materials, and provision of TA to entities entitled to submit financing applications for projects to be financed through EU structural funds; (iii) promotion of integrated services and alternative community-based solutions for the ECE, including development of coherent ECE legislation, and provision of training and counseling programs for parents; and (iv) carrying out of M&E, information, education, and communication activities. (iii) Component 3: Social Assistance Programs (€28.6 Million) This component was designed to support programs of the MLSSF aimed at improving the situation of PWD, youth at risk, and victims of domestic violence through three subcomponents: Subcomponent 3.1: Disabilities Program (€14.4 million). This subcomponent aimed to improve the quality of care in residential services and increase the access to CBS for PWD in need, especially those with a high dependency on residential services. It would support the MLSSF/NAPH program for restructuring of institutionalized care, prevention of institutionalization, de-institutionalization, and development of alternative community-

5

based systems of assistance and social inclusion through (i) subprojects—submitted by local authorities and selected in accordance with a competitive grant scheme—aimed at restructuring and rehabilitation of existing institutions and construction of new infrastructure facilities, in line with improved standards developed by the NAPH, and development of alternative CBS to support the de-institutionalization process; (ii) institutional capacity building through provision of training to professional staff in the new standards of care and development of the M&E system for the NAPH; and (iii) provision of occupational counseling services to the beneficiaries of the newly restructured institutions.8 Subcomponent 3.2: Youth at Risk Program (€11.1 million). This subcomponent was designed for youth aged 18 years and above, graduating from the institutionalized care system, to provide them with increased access to integrated social services that would increase their readiness for independent living. It would finance subprojects submitted by local authorities (and awarded on a competitive basis) to support the establishment of a network of multifunctional centers, where eligible beneficiaries would have access to protected housing, complemented by a range of psychological, social, vocational, occupational, and legal counseling services. It would also finance M&E activities, provision of training to counseling providers and MLSSF staff involved in supervising the function of the multifunctional centers established, and implementation of public awareness campaigns. Subcomponent 3.3: Victims of Domestic Violence Program (€3.1 million). This subcomponent was designed to support the development of specialized integrated social services. Victims of domestic violence could have access to these services for a limited period of time until their family situation was resolved, so that they could return to a normal life. It would finance subprojects submitted by local authorities for rehabilitation and/or construction of housing infrastructure to shelter victims (and their dependents) and endowment with furniture and household appliances. It would also finance professional training for staff that would provide counseling services, development of a specific MIS, and implementation of public awareness campaigns. (iv) Component 4: Capacity Building for Roma Social Inclusion (€0.8 Million) This component aimed to provide assistance to the NAR and its eight regional branches to coordinate activities aimed at improving inclusive planning; M&E; and identification, preparation, and implementation of programs to be financed by the EU and other funds, through (i) provision of training to NAR staff, Roma NGOs, local authorities, and Roma community initiative groups; (ii) preparation and implementation of an information, education, and communication strategy; and (iii) strengthening of the M&E system.

8 The provision of occupational counseling services would scale up the positive results obtained in the pilot employment counseling offices for the PWD established under the Social Sector Development Project (SSDP). Under that project, 8 out of 46 district employment agencies established pilot centers for disabled jobseekers, accommodating their infrastructure, office equipment, and staff training to the needs of the PWD.

6

1.6 Revised Components

29. Level-1 project restructuring approved in early 2011 (section 1.3) also included a new subcomponent under the project description, reallocated funding among categories of expenditure, and extended the loan closing date by 21 months from May 1, 2011 to February 1, 2013. The loan’s closing date had previously been extended by two months, from March 1 to May 1, 2011, to provide additional time to process the project restructuring. 30. A new Subcomponent 3.4, social assistance cash transfer program, was added to the project to support the implementation of the SASMP (results-based financing). This subcomponent provided ‘Support to the MLFSP and other relevant governmental institutions in its subordination and/or coordination to increase efficiency in the administration and monitoring of social assistance benefits through, inter alia, (i) upgrading of the MIS for the NASB with modules for family support allowances and guaranteed minimum income (GMI); (ii) harmonization of means-testing procedures for the eligibility of social assistance benefits; (iii) harmonization of disability assessment criteria and the institutional framework for disability benefits and invalidity pensions; (iv) development of institutional capacity and tools to prevent and combat error and fraud; (v) promotion of public information campaigns to present proposed reforms for efficiency and targeting of the social assistance system; and (vi) implementation of M&E activities to assess the incidence of social assistance transfers and their impacts and the savings gained from consolidation of social assistance benefits and simplification of social assistance benefits administration.’ 31. Additional changes in the level-1 restructuring included (i) an additional number of 16 subprojects (a total of 116 instead of 100) aimed at improving access to basic community infrastructure and social services for the most disadvantaged Roma living in poor settlements under Component 1; (ii) reallocating €1.4 million from Component 3 to Component 1 to finance the additional subprojects and €6.0 million to the new Subcomponent 3.4, resulting from cost savings once the subprojects for services to vulnerable groups were approved; and (iii) reallocating €0.6 million in uncommitted funds under Component 4 to Component 1, as the NAR had secured significant grant financing (almost €25.0 million) from the EU structural funds for consulting, training, and public awareness campaign activities. 32. A last level-2 restructuring, approved on December 14, 2012, extended the closing date by an additional 17 months, to June 30, 2014. It also modified the project description of Subcomponent 3.4 to provide for upgrading of the MIS to include modules not only for family support allowance and the GMI but also for any other new social assistance benefits to be agreed with the Bank.

1.7 Other Significant Changes

N/A.

7

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design, and Quality at Entry

33. Project preparation. The project was prepared expeditiously, in slightly over six months from identification to approval by the Bank’s Board of Directors, and included specialists in social protection, operations, fiduciary functions, and other areas. The project was fully consistent with the GOR’s priorities and counted upon its commitment and ownership. During preparation, several project design options were considered. Two projects were under preparation at the time, one to address the needs of the Roma and another to address the needs of ‘other vulnerable groups’.9 However, the government did not want to limit and segregate support within separate projects but rather wanted to address the needs of all vulnerable groups comprehensively. 34. Preparation had classified priorities as ‘short-term interventions’, to include a basic infrastructure package that would be financed by the RSDF; a ‘social inclusion program’ that would finance activities of four ministries (education, health, living conditions and) involved in the preparation of inclusive programs; and ‘long-term priority’ financing activities complementary to those funded by the EU and strengthening the capacity of institutions to tap EU funding for social inclusion.10 Eventually, while the concept note addressed each of these levels of priority, the project focused only on some of those. For example, it did not target the elderly as a vulnerable group nor did it address the totality of needs of the vulnerable groups to achieve ‘social inclusion’ (for example, housing and health). The government had institutional coordination mechanisms in place to ensure that all the needs of all vulnerable groups could be identified, addressed, and monitored in a comprehensive and coordinated manner. 35. The project was in-line with the poverty alleviation and social inclusion pillar of the Country Partnership Strategy 2006-2008 (and with the social and spatial inclusion pillar of the Romania Country Partnership Strategy (CPS) 2009–2013) and the rationale for World Bank involvement was strong (section 1.1).11 Preparation counted on a detailed social assessment that incorporated stakeholder views obtained through a consultative process that included advocacy groups and Roma NGOs as well as NGOs involved in provision of services to children and the disabled. The objective of those consultations was not only to identify issues facing those groups, but more importantly, to design

9 According to World Bank staff, the Bank had embarked on preparing a larger, inclusive preschool education project. When the decision was made to address inclusion in one Bank-funded project for all categories, the Ministry of Finance proposed a larger preschool education project (not Roma-focused) to the Council of the Europe Development Bank. This could be considered somewhat complementary to the SIP. 10 In early project documentation, the possibility of addressing the health needs of vulnerable groups and the needs of the elderly was also considered but later excluded, presumably to provide better focus to project activities. 11 Country Partnership Strategy 2006-2009, Report No. 36147 dated May 16, 2016 and Country Partnership Strategy 2009-2013, Report No. 48665 dated June 12, 2009.

8

arrangements that would promote and facilitate stakeholder participation in implementation and monitoring mechanisms. 36. Earlier, several Bank-financed projects and Institutional Development Fund (IDF) grants (and projects financed by other donors) in Romania had addressed the needs of the most disadvantaged and vulnerable groups, albeit in a less comprehensive manner 12. Lessons of experience from those projects provided important input to guide the project’s preparation. The project built on the highly successful experience of the Social Development Fund APL, implemented by the RSDF, and the Rural Education Project implemented by the MER. An extensive list of challenges in working with Roma communities guided the project’s community-based design by incorporating lessons from pilot activities as well as from the RSDF’s experience in working with these communities. Experience under the Rural Education Project fed into the design of Component 2. Similarly, the successful experience of tapping the CBS as an alternative to institutionalized LTC for PWDs and other vulnerable groups (youth at risk and victims of domestic violence) under the Social Sector Development Project (SSDP) provided important lessons that were incorporated in the design of the project’s Subcomponent 3.1. Finally, experience from other countries in the Europe and Central Asia (ECA) region with a program-based approach (PBA) was taken into account in the project’s design. 37. Preparation revealed four key risks, which were assigned with an overall risk rating of ‘Significant’, mostly since implementation would rely on existing government systems and programs. To address these, the team identified relevant mitigation measures, including capacity-building measures under the SIP and other projects and TA activities in the Bank’s assistance program. The project’s risk analysis underestimated these risks: (i) achieving coordination with what was to be a comprehensive program for addressing the needs of vulnerable groups, (ii) counting on a strong institutional framework for providing that coordination, and (iii) putting in place an effective system for monitoring the needs and the progress made in addressing those needs. It also underestimated the multiple institutional and implementation risks of a project that was to be implemented by several institutions, at the central, local, and community level (and the differences in implementation processes that would entail). 38. Project design. The project was designed as a PBA, whereby the investment project would support several of the GOR’s existing inclusive programs, as opposed to a ‘stand-alone project’ that would support only a predefined investment project. Every year, the GOR and the Bank would agree on specific activities and implementation priorities to be financed by the project in support of the overall inclusive investment programs of each of the executing agencies. This approach to project design was chosen by the preparation team as it would allow Bank financing to assume more of a catalytic role that would help jump-start all of the GOR’s inclusive programs, with fewer resources allocated to each but with greater impact at a broader level. It was also considered relevant as it would support the strengthening of the GOR’s country systems—in terms of investment

12 IDF Grants included a US$350,000 grant for Capacity Building for Roma Inclusion Programs and a US$264,000 grant for Capacity Building for Anti-Poverty Commission.

9

planning, budgeting, and monitoring that was important to align them with EU standards—and build capacity in Romania to absorb EU structural funds in support of its inclusive programs. In fact, project preparation was coordinated closely with the EU delegation in Romania and the project’s results framework (RF) included an intermediate outcome indicator to capture the capacity of local groups to access EU funds for social inclusion programs.13 39. By opting for a catalytic approach that would impact most of the GOR’s inclusive programs, the project’s design, almost by definition, required complex institutional arrangements. These involved implementation by two ministries, the MLFSP—and two agencies reporting to it, the NAPH and the National Agency for Family Protection (NAFP)—and the MER as well as two agencies, the RSDF and NAR. As a mitigating factor, the MLFSP, MER, and RSDF had extensive implementation experience, including with Bank-financed projects. Nevertheless, each of the implementing agencies would use their own administrative, fiduciary, and procurement structures. Individual program coordinators in the agencies would prepare, implement, and monitor their respective agencies’ Annual Development Plans (ADPs) under the overall coordination of a steering committee that included the secretaries of state of the MLFSP and MER in addition to the Ministry of Public Finance (MPF) and the presidents of each of the four agencies. There was no single agency entrusted with overall coordination of the project, its implementation, and reporting requirements, although the MLFSP assumed a de facto role in overall reporting coordination. The project was implemented in accordance with three Operational Manuals (OMs), one each for the RSDF, MER, and MLFSP. 40. The RSDF implemented Component 1 through communities and coordinated activities by relying exclusively on its internal administrative structure. Implementation of activities under Components 2 and 3 were coordinated by project management units (PMUs), of which there were two in the MER (Rural Education PMU and School Rehabilitation PMU), and one in the MLFSP. The PMU in the MLFSP also provided support to fiduciary requirements of activities carried out by the NAR under Component 4—a design feature that at least in perception may have affected the relative standing of the NAR, the agency that was to have a central role in coordination of activities targeted at the Roma population that is, site selection and monitoring of implementation, but not of communicating with the Bank) vis-à-vis other project implementing agencies. 41. Institutional arrangements were additionally complex in that the activities under the project’s four components were, following Romania’s institutional framework, to be implemented by different levels of government. Coordination and implementation of Component 2 was ensured by two PMUs in the MER. County councils were responsible for the processes required for construction of centers for PWD, youth at risk, and victims of domestic violence under Component 3. Nevertheless, during preparation, no capacity assessments were done at the local level. Local levels had varying, generally poor capacity, large distances separating them from the respective PMUs. This led to little

13The relevant intermediate outcome indicator is: ‘Percentage of SIP assisted promoters of mature project proposals eligible for EU financing’.

10

interaction, following of different reporting systems, and unfamiliarity with procurement and financial management procedures for Bank-financed projects, all of which together with cumbersome domestic approval procedures, came to adversely impact the implementation of Component 3. 42. Further, the project was designed to promote the participation of Roma and other disadvantaged persons in the prioritization, identification, and implementation of subprojects that addressed their needs. Due to this, implementation by each of the above implementing agencies involved not only implementation of activities that would build their own capacity but also their working with and through targeted local communities (including representatives of the Roma communities, NGOs, and other stakeholders), dispersed throughout the country, to promote, inform, and build capacity to prepare, implement, and monitor demand-driven inclusive subprojects. It also required inter-agency cooperation among the various implementing agencies, especially with the NAR, which had an important role in terms of providing input for the selection of targeted communities under Components 1 and 2. In an attempt to combine local interests with national strategies, several local communities were consulted during preparation to ensure that the proposed project met their needs. 43. Over and above the project’s institutional complexities, the works were not ready for implementation when the project was approved. The project’s three OMs (RSDF, MER, and MLFSP) had not been completed and adopted, and more importantly, the site selection for activities had not been decided. Mostly, this was the result of the project’s design that aimed to ensure the participation of local communities and Roma counterparts in identifying locations, priority works, and activities. Yet, this was especially important for site selection under Components 2 and 3. These components were to involve large scale works with lengthier and more complex implementation procedures and were to be implemented by the local units of their respective ministries. That would involve not only coordination with the NAR (for Component 2), but more importantly, working with local authorities to mobilize subproject requests. 44. The project’s RF was concise and limited to outcomes and intermediate outcomes that could directly be attributable to its implementation. This was especially important for a project that was to support the implementation of several of the GOR’s inclusive programs. Nevertheless, the project’s outcome indicators relied heavily on composite indices (Living Conditions Index and Standards Compliance Index) and surveys to be carried out in beneficiary communities, which were undefined and could be seen as subjective. As a result, baseline data could only be collected once site selection had been decided. Also, the project’s indicators were inconsistent in their choice of wording, at times referring to Roma and at other times to vulnerable groups. Finally, the RF was designed to measure progress achieved through activities that were to be supported by the project, which was adequate as a measure of improvements in ‘living conditions’ (although any improvement, even building a road in a community, could theoretically be construed as an improvement in basic living conditions) but fell short of measuring ‘social inclusion’. In fact, social inclusion was not explicitly defined in the project’s

11

documents, and its RF did not include specific indicators to track progress in achieving it, relying instead on indicators based on living conditions.

2.2 Implementation

45. The loan of €47.2 million for the SIP was approved by the Bank’s Board on June 13, 2006, and became effective on March 16, 2007 (after two extensions resulting from slow ratification by the Romanian Parliament). The original closing date of the loan was March 1, 2011. The mid-term review was held from March 8 to 12, 2010. 46. The project’s early implementation was slow. Three years after effectiveness, disbursements totaled only about 10 percent (but commitments totaled slightly over 70 percent). The project was considered at risk from mid-2009 until mid-2011. Several factors had a negative impact on the project’s implementation, including the capacity of the NAR, lengthy subproject approval procedures, the financial crisis, local governments’ capacity, and central government staffing changes. Romania’s EU accession process, cost savings in the implementation of planned activities under the project, and a coordinated program of Bank (and other) assistance in support of Romania’s social protection sector had a positive impact on implementation. 47. The NAR. It was established in 2004 with the responsibility of elaborating government policy and strategy on Roma issues, administering funds for Roma programs, representing Roma in specific national and international events, and the overall monitoring of the GOR’s Roma strategy. The agency is led by a President, with the rank of Secretary of State, and functions under the General Secretariat of Government (GSG). During preparation, the NAR’s institutional capacity was known to be weak—the agency had counted on the Bank IDF grant for capacity strengthening—and further support to strengthen its capacity was contemplated under Component 4. The goal was to strengthen the NAR’s capacity to carry out its mandated functions, especially M&E of Roma programs not just under the project but across all GOR programs. 48. In addition, the NAR was also to have a very central role in both site selection and M&E of activities under Components 1 and 2, especially as they affected the Roma population. However, due to its weak institutional capacity, it was not able to do so in a timely manner, hampering the progress in the other components. Indeed, delays in site selection had trickle-down effects on the late launch of infrastructure rehabilitation and construction. It also significantly affected the collection of baseline data for monitoring once site selection had been decided. Weak institutional capacity, changes in management (the NAR had five presidents during the implementation period), high staff turnover, and apparent lack of interest delayed these processes despite repeated agreements with Bank implementation support missions on specific action plans to improve performance. Changes in political leadership in Romania and in the president of the NAR, together with the sensitivity of the issue for which the agency is entrusted, clearly had an impact on the NAR’s irregular performance under the project. The decision to entrust the MLFSP PMU with the fiduciary responsibilities for activities

12

under Component 4 apparently did little to build ownership and commitment to the project within the NAR. Toward the end of the SIP, significant EU financial support (almost €25 million) became available to the NAR for various programs (although not necessarily for the same activities contemplated under the project). With continued lack of progress in implementing activities under Component 4, savings amounting to €0.6 million were reallocated to other components under the restructuring in 2011. The remaining funds were committed for development and maintenance of the NAR’s website. The NAR continued to have a consultative role for the selection of sites in the Roma-related components. The site selection process was smoother for Component 1, since the NAR’s president is on the Board of the RSDF. Nevertheless, the NAR’s monitoring responsibilities were assumed by both the RSDF and MER for their respective components. However, the NAR lacked the capacity to assume the central role that had been envisaged for it with respect to overall coordination of Roma strategies, policies, programs, financing, and monitoring. 49. Lengthy subproject approval procedures. Activities under Component 3 were affected by long delays resulting from the lengthy and cumbersome subproject cycle—totaling in some cases up to 600 days from the launch of the competition for subprojects by the MLFSP until the start of execution of civil works. This was a generic issue for all projects in the Romania portfolio that financed civil works. The site selection process by the MLFSP (and the respective agencies) took over two years to complete and involved consideration by three selection committees. Once the sites were selected, the subproject cycle or process included three main phases: (i) technical design and investment approval; (ii) procurement of civil works; and (iii) execution. Most of these phases required processes (pre-feasibility, feasibility, and bidding) that were implemented by local authorities that had, in general, insufficient capacity to prepare technical documents. In addition, the various processes required approval not only by the MLFSP but also by local authorities, inter-ministerial committees, and even the GOR. These procedures have since been streamlined. Still, their impact on the implementation of Component 3, at several stages of subproject preparation and implementation and among various levels of government, cannot be underestimated. 50. Financial crisis. From the year 2000 up to the global financial crisis of 2008, Romania grew at an average of 6 percent per annum and growth was shared broadly with the bottom two quintiles. The global financial crisis of 2008 triggered a severe recession in Romania, halting income growth and poverty reduction. Social transfers played an important role in mitigating the impact of the crisis on the bottom 40 percent, especially through increases in pension payments. However, the growth in pension costs and other social transfers contributed to a rise in fiscal deficits that led to budget cuts for projects across the portfolio and measures being taken internally to curtail salaries and other expenses. 51. Although less affected than others in the Romania portfolio, the project was still affected by budget cuts and delays in receiving funds for implementation in a timely manner. The GOR had implemented a new public debt management policy whereby the state budget pre-financed Bank-supported (and other) project activities, which are then

13

periodically reimbursed through external financing. However, fiscal constraints presented challenges for the MPF to fully pre-finance project activities in a timely manner, delaying the launch of activities implemented by MLFSP under Component 3 and to a lesser extent, the MER under Component 2. Several administrative and staffing measures, taken in the aftermath of the crisis to curtail expenditures, also impacted the project at the national and local levels. To rationalize and cut costs, the MLFSP was restructured, NAPH and NAFP became first general directorates and later directorates of the ministry, and lost positions and specialized functions. Staffing constraints were imposed. The PMUs were impacted negatively by a government ordinance related to PMU staff salaries (common for other PMUs implementing Bank-financed projects). Staffing ceilings precipitated by budgetary constraints also affected the contracting of specialized staff by local governments that would be responsible for the delivery of services under the subprojects financed under Components 2 and 3. 52. The crisis also appears to have affected the demand for funding of subprojects, especially under Component 3, since local authorities were required to fund not only a share of the original investment but also assume responsibility up front for operations and maintenance for a period of fifteen years upon completion. Faced with the closing of many firms and a decline in local revenues and transfers, local governments became cautious in commitments that would affect future budgets. For example, a total of 37 subprojects (centers for social assistance services) were initially approved for financing by the project selection committees at the level of the MLFSP. However, five subprojects were dropped, mainly due to the scarcity of counterpart funds at the level of some local administrations, which was caused by the economic and financial crisis. 53. Also, as a result of the crisis, curtailing and improving the efficiency of social assistance programs became a priority for the GOR. Preparation of the results-based SASMP that would support a comprehensive reform of Romania’s social assistance programs was underway to support the GOR’s reform. While the reform program would require significant amounts of TA for its implementation, the GOR had opted not to borrow for investment financing.14 With resources that were not being utilized under Component 4, the project was restructured to finance the TA required to support implementation of the GOR’s social assistance reform program. 54. Changes in leadership. In addition to the changes that affected the NAR and other executing agencies, the project’s implementation was affected by several governmental and ministerial changes. From preparation through completion, Romania had four different prime ministers, seven different government coalitions, twelve ministers of the MLFSP, ten ministers of the MER, nine ministers of MoPFb and five presidents of the NAR. These changes affected not only the highest levels but generally the second levels in the executing agencies as well. At the same time, general elections in 2004 and 2008 coincided with local elections, which affected the administrations of local governments

14Although an investment project by definition, the SASMP’s design was to provide disbursement not against incremental project expenditures but on the achievement of activities, outputs, and outcomes as defined by selected ‘disbursement-linked indicators’.

14

that were implementing subprojects at the local level. The RSDF, an autonomous agency and the only executing agency not affected by significant changes in leadership and staffing, worked closely during electoral periods with candidates and then recently elected officials to provide continuity, assume ownership, and maintain trust between communities and local authorities. 55. EU accession. Romania’s accession to the EU took place on January 1, 2007, shortly after the project’s approval by the Bank’s Board. Obligations stemming from the EU membership status of Romania, such as the implementation of the acquiscommunautaire, and the Europe 2020 agenda for smart, sustainable, and inclusive growth have constituted a solid foundation for Romania’s development planning process and provided strategic direction to guide the GOR’s development priorities, including for social inclusion and poverty reduction. Accession to the EU has provided access to considerable resources and obligations and targets in specific areas, including social inclusion (section 3.1).These obligations provided an environment that is supportive of continued commitment to the project’s objectives, and the project aimed to strengthen the capacity of agencies and local communities to prepare proposals to mobilize EU funding.

2.3 Monitoring and Evaluation Design, Implementation, and Utilization

56. M&E design. The PAD specified that M&E of outcomes and results would be done through a combination of top-down and bottom-up approaches, conducted at both the national and local levels using administrative and ad hoc data collection mechanisms. Nevertheless, monitoring of project outcomes presented several challenges. First, the project’s RF included indicators that depended on the availability of evaluation, beneficiary assessment, and site surveys to be carried out by the NAR and MLFSP. Most of the indicators did not have baseline data when implementation began and could only be compiled after sites for project interventions had been decided. Second, outcomes were to be monitored through the systems of each of the implementing agencies and later consolidated into aggregate yearly reports. These would evaluate progress toward the project’s ADPs, frame the following years’ priorities under each of the programs, and feed the system for monitoring the National Action Plan for Implementation of the JIM (JNAP) on social inclusion. The NAR was to have a central role in building a database of Roma living conditions and social exclusion indicators based on data provided by other implementing agencies and its own surveys and in producing regular reports on the Roma’s social exclusion. Finally, the implementing agencies’ capacity for M&E varied widely in general. The RSDF and MER had built some capacity for M&E while both the MLFSP and NAR had very limited capacity. Under the project, funding was allocated to strengthen the M&E systems of the NAPH, MLFSP, NAFP, and NAR. 57. While the project’s RF was concise, it did not contain basic information on project beneficiaries of its several interventions, including descriptive data such as the number of Roma beneficiaries and the beneficiaries’ gender, types of RSDF projects financed (for example, roads, community centers, water, and education services), or indicators that would show institutional progress toward addressing exclusion (for example, number of local governments that incorporated Roma coordinators in their

15

institutional framework). Similarly, monitoring of activities implemented by the MER and MLFSP could have included indicators classified according to vulnerable groups and gender, which would allow for a comprehensive evaluation of the project’s benefits. While some of this information has been compiled ex-post, it would have been desirable to establish and maintain a comprehensive monitoring framework that would inform progress on the governmental programs as a whole while selecting a few key indicators to monitor the project’s progress toward its objectives in parallel. 58. M&E implementation. Monitoring presented several challenges in the project’s early implementation that resulted in successive ‘Moderately Unsatisfactory’ or ‘Unsatisfactory’ ratings for M&E in the project’s Implementation Status and Results Report (ISR). Despite repeated focus by the Bank’s implementation support, none of the agencies had initiated project monitoring or collection of baseline data, in part due to delays in site selection under the various components and in part due to delays on the part of the NAR that was expected to carry out the required surveys once sites were selected according to the original project design. In 2009, an external consultant was contracted by the NAR to help collect the data. When that consultant didn’t comply with contractual obligations, each ministry and agency assumed responsibility for developing methodology and carrying out surveys to compile baseline data and later monitor the project’s outcomes. The RSDF prepared an evaluation tool to be applied in each community to obtain the needed data for project indicators. The tool was applied to 42 Roma communities and 42 control, neighboring non-Roma communities, gradually increasing on a yearly basis (as completed) to all 133 communities. The RSDF developed its own survey, implemented it, and obtained the baseline data. The RSDF’s evaluation tool comprised (i) an impact evaluation questionnaire; (ii) a community ‘fiche’ assessing the conditions of life; (iii) an evaluation of the annual consultations between the local authorities and the community members. The baseline values for key performance indicators were collected in 2008 and 2009. 59. Efforts made by the other implementing agencies to select consultant services to carry out baseline indicator studies (combining methodology design and survey implementation) did not succeed due mostly to the lack of interest on behalf of potential bidders. For Component 3, there were four attempts by the MLFSP to contract these services. 60. During the project’s mid-term review, executing agencies questioned the relevance of some of the monitoring indicators and suggested revisions. The project’s RF was amended in the level-1 project restructuring in 2011 to address the concerns that had been raised as well as to reflect the modifications to the project that had been agreed upon (annex 10). It is not clear if any of the information produced by the project’s monitoring fed into project-related decisions or the GOR’s efforts to monitor and report progress on addressing the needs of vulnerable groups. The project did not succeed in putting in place an integrated framework for monitoring needs, investments, and progress and results of programs targeted at either the Roma or all vulnerable groups as a whole.

16

61. More specifically, as reported in Annex 10, out of the 24 indicators that should have been regularly monitored as per 2011 RF, data for 10 indicators was not included in ISRs following the restructuring. Their systematic tracking would have enhanced the project’s monitoring, evaluation and reporting performance, although for some final data was eventually captured in commissioned studies15. Although there were challenges in the establishment of baselines, recruitment of M&E consultants and possible unclear nature of some indicators16, more efforts could have been put in place to ensure the regular collection, monitoring and reporting of RF indicators.

2.4 Safeguard and Fiduciary Compliance

62. Fiduciary (procurement and financial management) functions were implemented by the RSDF for Component 1, the MER for Component 2, and the MLFSP for Components 3 and 4 in accordance with the project’s three OMs. All these agencies had previous experience in complying with the requirements of Bank-financed projects. Despite the substantial risk to financial management due to complex institutional arrangements that had been identified during appraisal, financial management performance including accounting, budgeting, and reporting was satisfactory throughout implementation, and proper control procedures were maintained. On-site supervision was conducted regularly, twice each year, with spot checks of selected transactions. All audit reports were received, occasionally with slight delays, but opinions were clean with only minor recommendations. The only issue that altered the project’s original financial arrangements related to the closing of the project’s three designated accounts and reimbursement to the Bank of amounts that had been advanced when the government took the decision to pre-finance all externally financed project expenditures with budget resources (section 2.2). 63. Procurement processes were satisfactory but complex nevertheless, mostly because of the different size and scale of the civil works financed under each of the project’s components as well as the different arrangements followed by each of its implementing agencies. For Component 1, the RSDF followed procedures in an Annex to its Operational Manual: Procurement Handbook for RSDF-financed Projects, with small procurement carried out at the local level and site supervision by RSDF supervisors. For Component 2, the MER carried out two infrastructure-related procurement processes, one for school construction and the other for rehabilitation, and then supervised implementation at the county level.

15 See reports (both in Romanian) by Impreuna and ECHOSOC (2013) “Evaluation Study of the Early Childhood Education Program” and RSDF (2013) “Monitoring Report: An Independent Study of the Performance Indicators of PIP”.

16 For example, according to staff the indicator “% of PWD services implemented in high-priority areas” was particularly unclear and difficult to measure.

17

64. In addition, the MER implemented important procurements related to provision of furniture, teaching-learning materials, consulting services for the development of curricula and associated guides and materials, teacher training, parenting programs, and other ‘soft’ activities. Component 3 presented the greatest challenges as the procurement of civil works for the construction and rehabilitation of residential centers was carried out in a decentralized fashion by the local public administrations that owned the centers. The national procurement legislation required approval at several phases of the process by several entities at the local and national level, including the need for inter-ministerial and even government approvals. This led to continuing delays in implementation. The Bank carried out post-procurement reviews on an annual basis, including site visits and physical inspection of the works financed. Only minor issues were identified (for example, limited bids received, minor reporting issues on evaluation reports, and bid opening procedures). One recurring issue was the lack of interest in potential bidders and firms expressing interest in consultancy assignments. 65. The project triggered the Bank’s Environmental Assessment (OP/BP/GP 4.01) safeguard in view of the small infrastructure works and rehabilitation of existing works. It was classified as ‘FI’, with primarily B and some category C subprojects expected. The OMs included a chapter setting out environmental screening criteria and approach to identify and mitigate risks. The involuntary resettlement safeguard policy was not triggered since the project’s OMs stated that subprojects involving land acquisition would not have Bank financing. The implementing agencies’ compliance with environmental screening criteria was supervised periodically, including reviews of subproject feasibility studies, with occasional site visits. Only minor issues were identified, which were promptly addressed. 66. During the mid-term review, a field visit was organized in four counties covering all three of the project’s components that had investment subprojects in different stages of preparation or implementation. Nine subprojects were visited and the review concluded that in most of the cases, the technical and engineering aspects of the environmental protection and mitigation measures included in the subprojects were compatible with the national and international practice and suitable for the works. The performance control and supervision of the works was generally well-organized and coordinated by the PMUs, with direct involvement of site supervisors and local beneficiaries (local authorities or schools representatives). The Construction State Inspectorate participated in all the works contacts for the approval of the works’ quality during the main phases. The mid-term review concluded that all works financed under the project’s three components were implemented in accordance with the health and safety requirements and in compliance with local and Bank environmental safeguards. The review made recommendations regarding site selection for two subprojects implemented by the MER under Component 2: one kindergarten was in close vicinity of an existing church with little consideration for maintaining proper access to the religious facility and the location of another kindergarten would require children to cross a busy, national railroad to reach the facility, raising safety concerns. In both cases, alternative sites were identified and the works rebid.

18

2.5 Post-completion Operation/Next Phase

67. The GOR remains committed to the project’s PDO and to continue making progress in addressing social exclusion. The lessons learned under the project, together with inputs from a reimbursable advisory services (RAS) engagement on poor and disadvantaged communities and another report on ‘Diagnostics and Policy Advice for Supporting Roma Inclusion in Romania’ will feed into the design and implementation of a social inclusion and integrated basic services project that is expected to support interventions in education, health, employment support, and social services coordinated at the local level. The project is at the concept stage. Its design will also be aligned with the draft Social Protection Strategy for Romania, the updated National Roma Integration Strategy, and EU financing for Roma Inclusion. 68. The project financed activities under existing GOR programs that will remain in operation after its completion. Further, the EU has committed an amount of €4.22 million for 2014–2022 to help the GOR address social exclusion. These funds can be used to finance specific programs, such as those under the project, which will support Romania in complying with its targets for social inclusion under the Europe 2020 strategy. In addition, the RSDF has €30 million financing available in EEA Norway Grants for projects in Roma communities, mostly in rural areas. 69. Continuing support to the MLFSP and other governmental institutions aimed at increasing efficiency in the administration and monitoring of social assistance benefits will be provided under the SASMP (still ongoing) as well as through new RAS engagements and a proposed new DPL series. Several TA activities under Component 3.4 supporting the implementation of the ongoing SASMP, including the second phase of the MIS (SAFIR) upgrade for the National Agency for Payments and Social Inspection (formerly NASB) and the implementation of the public awareness campaign will be transferred to the SASMP following a planned forthcoming restructuring that will now (following a GOR decision to adjust its earlier position on borrowing for investment projects) provide financing for TA activities.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design, and Implementation

70. The project’s objective (to improve the living conditions and social inclusion of the most disadvantaged and vulnerable people in the Romanian society) reflected a priority at the time it was approved and one that remains highly relevant today. Almost 2 million of Europe’s 10 million Roma live in Romania. The census reports Romania’s Roma population at 3.3 percent, but unofficial expert estimates reported in the National Roma Integration Strategy range up to 10 percent as many Roma do not self-identify. According to a recent European Commission-United Nations Development Programme-World Bank 2011 report, the vast majority of the Roma population is in the bottom income quintile. Most live with severe material deprivation and are from large

19

households living in sparsely populated rural areas. Poor educational outcomes start early with low preschool enrolment rates, poor quality education, and segregated classrooms and later contribute to poor labor market outcomes. Discrimination continues to be a hurdle the Roma face. Tackling this discrimination with an integrated solution to break inter-generational transmission of poverty and following through in the medium-term is the only way to successfully address the continued social exclusion of low-income populations, including the Roma. 71. The project’s objective is consistent with the GOR’s stated strategic priorities. Romania’s medium-term development strategy is framed within the broader context of the EU’s Europe 2020 strategy, which is designed around the concept of ‘sustainable, smart, and inclusive growth’. Europe 2020 proposes a set of specific targets (including for social inclusion) for the EU as a whole and for each member country.17 Romania’s social inclusion target is to reduce the number of people at risk of poverty or exclusion after social transfers by 580,000. 72. The project is also consistent with the Bank’s FY14–17 CPS, discussed by the Board of Directors on May 22, 201418 which focuses on reducing poverty in Romania and fostering sustainable income growth for the bottom 40 percent of the population. The CPS specifically mentions social protection and inclusion of low-income communities, especially the Roma, as a priority. The CPS has three pillars, of which on is social inclusion, for which it states that Bank assistance will focus on inclusive services for marginalized communities, especially the Roma, and on improving the social protection system. 73. The project’s design was relevant, albeit ambitious, at approval and maintained its relevance throughout implementation. By involving local communities and beneficiaries in the definition, preparation, and implementation of local subprojects, the project ensured that the activities financed were responsive to local needs. At the same time, it helped create capacity at the local level for preparing and proposing additional subprojects that in future could benefit from EU accession funding. Still, its design was undoubtedly complex given the institutional realities of addressing the needs of several vulnerable groups. It was (and continues to be) ambitious in attempting to provide a coordinated approach for addressing the needs of vulnerable groups when the GOR did not and does not have a mechanism in place to oversee and coordinate the needs of these

17 The national targets for Romania under the Europe 2020 strategy included several targets for poverty and social exclusion, including early school-leaving, completion of tertiary education, employment, and of most relevance to the project, reduction of the population at risk of poverty or social exclusion. Romania’s target was to reduce by 580,000 (2.9 percent) the number of people at risk of poverty or social exclusion after social transfers (base year 2008 out of a population of 20 million). In 2008, 23.4 percent of the population was considered at risk of poverty or social exclusion, defined as persons affected by at least one of the three indicators surveyed by EUROSTAT: at risk of poverty, severe material deprivation rate, and those living in households with very low work intensity. By 2012, 22.6 percent of Romania’s population was considered at risk of poverty or social exclusion. 18 World Bank Country Partnership Strategy for Romania for the Period 2014–2017, Report No. 84830-RO, dated April 28, 2014.

20

groups in a comprehensive manner. Further, while it is relatively straightforward to aim to improve living conditions of vulnerable groups, addressing and measuring ‘social inclusion’ is more complex. By incorporating Roma coordinators in local governments, and reforming the curriculum for the ECD and ensuring its inclusive approach, the project design was aimed at addressing this issue. Nevertheless, there are other factors affecting social inclusion that the project did not, and could not, address in such a short time. Overall, strategies to tackle social inclusion have policy and design implications. 74. The project maintained its relevance during implementation through the restructuring approved in 2011. First, realizing that the activities in support of the NAR under Component 4 were not progressing, the restructuring reallocated funding for the NAR (and funding from cost savings) to alternative activities. In retrospect, the decision to reallocate funding from the NAR to other components was pragmatic given the strong implementation performance of other executing agencies, especially the RSDF. However, the decision may have reduced the project’s impact and duplicability in that it was not successful at putting in place a strong mechanism to coordinate and monitor GOR programs targeted at the Roma population. 75. Furthermore, during the final years of implementation, when it became increasingly clear that the demand for funding by local governments for initiatives aimed at vulnerable people by the MLFSP was declining, perhaps the project could have put in place a mechanism to finance an even greater number of investments by the RSDF and MER, which had stronger implementation capacities. The project restructuring also reallocated funding within the MLFSP to address the new objective of strengthening the administration of social benefits. Improved efficiency in the administration of social benefits had become an even higher priority for the GOR in the aftermath of the financial crisis, which resulted in increasing fiscal deficits (that could eventually affect funding for social programs such as those supported under the project) and a halt in previous progress toward reducing poverty. Cost savings and reallocations under the project became instrumental in supporting this new objective in coordination with Bank assistance under the SASMP. 76. The policy documents developed by the MER under Component 2 are fully relevant and compatible with the new Education Law approved in 2011, including the ECE Curriculum 3-6/7, and associated Good Practices guide, the ECE Curriculum for 0-3 and associated Methodological Guide, Quality Standards for ECE programs, Quality Standards for Teaching and Learning Materials, and well as Normative Standards for Minimal Supplies in Kindergartens.

3.2 Achievement of Project Development Objectives

21

77. The achievement of the PDO, as measured by the defined key outcome indicators (annex 10) is considered to be ‘Moderately Satisfactory’. 19 The project was very successful in terms of improving the living conditions and social inclusion of some of the most disadvantaged and vulnerable people in Romanian society. Its outcomes, as measured by the indicators in the Project’s Results Framework, were achieved and surpassed with respect to improving the living conditions and social inclusion of Roma living in poor settlements and increasing the inclusiveness of ECE services in targeted areas (Annex 10). Although delays in implementation precluded it from completing a few of the activities initiated under Component 3, the majority of planned activities were carried out and also completed successfully, and the few that were not are included in the GOR’s programs and are expected to be completed, staffed and become operational in the near future. The Project had an important impact in terms of supporting and strengthening a successful delivery model that incorporated local and beneficiary participation in the preparation, proposal and implementation of sub-projects, and in addition, strengthened the ability to present project proposals for EU funding upon completion. Finally, the Project was successful in strengthening the administration of social benefits through the implementation of several technical assistance activities and information system modernization that contributed to the GOR’s broader social assistance system reform supported under the SASMP. Progress towards the Project’s objectives, as measured by relevant intermediate outcome indicators, is described below. Details of the activities financed that led to their achievement is presented in Annex 2.

Improving the Living Conditions and Social Inclusion of Roma Living in Poor Settlements

78. The Project was successful in improving the living conditions and social inclusion of Roma living in poor settlements, although data is based on information not reported yearly as the Project should have done as per restructuring. Indeed, RSDF’s Community Facilitators compiled assessment sheets of the living conditions index for each of the 133 and neighboring communities at the beginning and end of sub-projects implementation. Based on the data they compiled, the gap in living conditions between the 133 targeted poor Roma settlements financed by the Project and neighboring communities was reduced by 61.8 percent (against a target of 20 percent)20. The Facilitators also compiled responses to annual questionnaires distributed to members of the 133 communities soliciting their views on the improvement in their minimum living conditions. Overall,

19As per the ICR guidelines, the project’s achievement of its PDOs is to be evaluated as a Formally Restructured Project. The required weighting of achievement before and after restructuring is provided in annex 11. In reality, the weighting makes little difference in the overall evaluation as the loan had disbursed very little before restructuring, and the project was rated ‘Moderately Satisfactory’ at the time of and after restructuring. 20 The average value of the living condition index of the 133 Roma communities before sub-project implementation was 10.04 (where the maximum possible value is 28). The value for the neighboring communities was 16.58, yielding a difference of 6.54. Upon completion, the average value of the living conditions index of the 133 Roma communities was 15.8; the index of neighboring communities was 18.30. As a result, the difference between both types of communities decreased from 6.54 before implementation to 2.5 upon completion, representing a decrease of 61.77%.

22

73.8 percent of respondents noted an improvement in the minimum living conditions in their communities, with regards to investments supported by their community sub-projects.21 Also, Facilitators’ questionnaires found that 75.7 percent of members of the 133 Roma communities reported that the sub-projects reflected to a very high extent the communities’ priorities (the remainder found that they reflected priorities to a high extent). 79. Although the Project did not have specific indicators to measure progress towards social inclusion there is evidence that the sub-projects had an impact on that. Roma in poor settlements report a closer link (through annual consultation) with the local authorities, for addressing community needs: 1659 consultations sessions between Roma in poor SIP settlements with local authorities for addressing community needs; the number of annual consultations between members of Roma communities funded under Project and the local authorities, was 4.93, against a target of three consultations. A total of 82.7 percent of Roma communities that benefitted from the Project reported access to improved community infrastructure (roads, water supply systems, etc.), against a target of 60 percent. Also, anecdotal evidence shows that there was an increase in coordination and enhanced dialogue and cooperation between local communities (prevalently Roma) and officials, especially local mayors. Also, there are encouraging examples of role models, for instance Roma teachers, which bolstered confidence in Roma communities and can be expected to generate multiplier effects on the value of and demand for education in Roma communities. These are difficult-to-measure, yet central benefits sparked for longer-term social inclusion, including laying the foundations for building follow-up interventions in the domain. 80. More importantly, the Project had an important impact in terms of institutionalizing a process for addressing concerns of the Roma population in communities. All of the communities that benefited from the Project with the help of RSDF Facilitators, succeeded in establishing community groups and selecting community leaders to prioritize community requirements and many local administrations established the position of Roma Coordinators to work directly with the community groups as an interlocutor that understands and can relay to local officials the realities of the community. The beneficiary participation in the selection/decision making process, and the manner in which communities began communicating with authorities together created a model for using small investments as a catalyst for social inclusion. Intermediate result: Progress is achieved toward addressing the needs identified by disadvantaged Roma in targeted poor settlements.

21 The 73.84% improvement represents a weighted average value resulting from the percentages obtained among different types of sub-projects, as follows: for access roads 81.37% improvement; for community roads improvement, 82.99%; for utilities, 77.92%; for access to education and social services, 63.71%; for access to medical services, 87.57%; for housing, 21.25%; and for access to cultural-educational/leisure activities, 37.93%.

23

81. Despite initial delays in selecting the disadvantaged communities to receive support, progress towards this Intermediate Result was highly satisfactory and the targets established in the Project’s Results Framework were all exceeded. Through a highly participatory process, disseminated and facilitated by RSDF’s Community Facilitators, 133 communities with a majority Roma population (as opposed to the originally contemplated 100), in a pilot and two subsequent rounds, confirmed their eligibility for financing, identified their priorities and submitted sub-project proposals for approval by RSDF. RSDF carried out three rounds of information and communication campaigns and community facilitation campaigns, through which its Community Facilitators verified eligibility, informed community members and local authorities of details and requirements for participation, and assisted them is assessing needs, establishing priorities, organizing initiative groups, setting local contributions, preparing the financing requests, and obtaining the documents required for sub-project processing. This participatory process ensured that the priorities reflected needs identified by the targeted Roma communities themselves. 82. A total of 58,800 persons benefitted from the sub-projects, of which approximately 19,500 were women. In all, 89 km of roads and three bridges were rehabilitated, 17.93 km of potable water distribution network was built, three small hydro-technical works were built, 19 community buildings were rehabilitated, 31 multifunctional centers, 16 day care centers, and 16 cultural centers or clubs were established, 14 information and counseling services and 54 healthcare and health promotion activities were organized and 21 environmental campaigns were carried out. There are, however, no indicators in the Project’s Results Framework to measure the outcomes of these activities (e.g., reduced transport times, improved health indicators), but it can be shown that overall living conditions have improved in the treatment communities.

Increasing the Inclusiveness of ECE Services in Targeted Areas

83. Progress towards this objective was also Highly Satisfactory, especially in relation to the originally established targets in the Project’s Results Framework. The number of children from vulnerable groups in targeted communities participating in ECE increased from 70 percent to 76 percent, exceeding the target of a five percent increase. In all, following 434 meeting with local councils and schools, the MER built and furnished 19 new buildings for preschool units, creating facilities to enroll children in areas where there was a large proportion of Roma children aged 0-6. The MER also rehabilitated and furnished eight existing buildings to improve the quality of education and facilitate the access of more children from disadvantaged communities with a majority Roma population. In all, 69 classrooms were built and/or rehabilitated, benefitting 1,060 children aged 3-6, of which 769 (73 percent) were from disadvantaged groups.

24

Intermediate result: The MER effectively implements inclusive policies and programs.22 84. The MER made substantial progress in grounding new, inclusive and systemic improvements in the ECE curricula. It developed the ECE curriculum for children aged 0-3, and revised the curriculum for children aged 3-6. The new curricula enhance the socio-emotional and physical health of children, and provide continuity between the early education curriculum for 0-3 years and 3-6 years. The curriculum for children aged 3-6 was formally approved and applied in the system. The curriculum for children aged 0-3, together with a Methodological Guide for teaching and other staff have been completed and are awaiting formal approval; the documents have already been sent out to pre-school institutions and they are already used as reference materials. A Good Practice Guide for ECE of children aged 3-6 was developed to provide support for all teaching and non-teaching staff in kindergartens. Quality standards for teaching-learning materials and a Normative for Minimal Supplies in Kindergarten were updated, and Quality Standards for ECE Programs were developed. Both the ECE curriculum for children aged 3-6 and the corresponding Good Practice Guide were printed (6,000 copies each), and have been distributed to (and are in use in) all kindergartens in the country. To disseminate to the public the importance of ECE, the MER developed a communications strategy and action plan for its ECE programs. 85. The MER also put in place a program of in-service training for targeted persons from communities supported by the Project, using a cascade model starting with training the trainers (that were selected from among already qualified or experienced teachers). The main aim of the training was to help the target population, represented by ECE staff (educators, care personnel, nurses, principals and auxiliary personnel), upgrade their teaching and care services and become familiar with and apply inclusive approaches in their activities. Based on the revised curricula, the MER developed, printed and distributed three training modules designed to raise awareness of disadvantaged groups and promote an attitudinal change with respect to the Roma population, and to address issues related to social inclusion and cultural sensitivity. In view of additional needs identified, and benefitting from cost savings of EUR0.95 million under Component 2, and supporting materials were printed and distributed to an additional 1,422 participants. 86. Based on the revised standards for teaching/learning modules, materials and books (including covering inclusion and cultural aspects) were provided to children in 100 kindergartens in targeted communities, based on an assessment of the needs of the ECD units and their priorities. Intermediate result: Vulnerable communities benefiting from alternative community-based solutions in ECE23

22 This intermediate result reads as: MER and MLFSP effectively implement inclusive policies and programs. For purposes of this ICR, it is divided among this and the next section (for MLFSP) to discuss progress in policies implemented in the areas of ECE services, and services for vulnerable persons separately.

25

87. An alternative CBS approach to delivering ECE services to children aged 3-6 was piloted in four isolated and vulnerable communities. The pilot included family kindergartens, specifically targeting Roma and disadvantaged groups, and provided support for training of eight mothers, teaching/learning materials, and incentives for care givers and children. A total of 69 children benefited from the pilot. The MER also implemented a program to train and counsel parents to develop new skills and attitudes and become sensitive to social, psychological, nutrition, health, accident prevention, children’s rights and other issues. Modules for teachers and thematic brochures for parents were printed and developed to support the training delivered by 53 qualified teachers to 1,257 parents in eight MER Resource Centers in each of the country’s regions that were supplied with furniture and equipment for this purpose. 88. The pilot activities represent good practice examples for addressing the needs of hard to reach children in isolated vulnerable groups and communities. The children had access to a totally new and healthier environment; they were introduced to basic things like hands and tooth washing), as well as to typical kindergarten and extra-kindergarten activities. The impact on the community is very important given the exposure of parents to the importance of early childhood education and practices. The challenge now is to disseminate the lessons of the pilot and scale up the apparently highly successful initiatives that have been piloted. 89. The MER also made progress in establishing an ECD monitoring and evaluation system based on clearly identified indicators including a survey instrument to monitor child readiness in domains including: physical health and well-being, social competence, emotional maturity, language and cognitive development, communication skills and general knowledge. The ministry developed a school readiness assessment tool/instrument and a User’s Guide on how to use the Child Development Evaluation Tool in the children’s last year of kindergarten (and by primary teachers in the first year of primary). The MER then piloted the use of these instruments in 60 kindergartens in all eight of the country’s regions; teachers also participated in training program on their use. The MER has compiled the results of the pilot application of these instruments, with a view to improve their quality and ease of use.

Improving the Quality of Services for PWD, Youth at Risk, and Victims of Domestic Violence

Intermediate result: The MLFSP effectively implements inclusive policies and programs. 90. Progress toward this objective was ‘Moderately Satisfactory’, mostly because of the delays suffered due to complex implementation arrangements under Component 3.

23 This intermediate result was revised in the level-1 restructuring on March 28, 2011. It was originally ‘MLFSP and MER identify targeted, disadvantaged groups and adjust programs to better respond to their needs’.

26

Through a competitive process, the MLFSP received proposals from the General Directorates for Social Assistance and Child Protection (GDSACP) at the local country level to raise the quality of care for persons with disabilities in old, overcrowded residential centers. Eight Neuropsychiatric Recovery and Rehabilitation Centers were constructed and/or rehabilitated, benefitting 394 adults with neuropsychiatric disabilities and low skills by providing accommodation in centers with less than 50 adults, nourishment, assistance and personal hygiene, medial and recovery assistance, emergency and therapeutical assistance, and psychological counseling services. 91. For PWD who were considered to have potential for semi-independent life and community integration, 25 protected dwellings, benefitting at most 6 beneficiaries per dwelling, were established to provide accommodation, nourishment, access to leisure and social activities, and support for activities aimed at leading an independent life. Six Assistance and Care Centers capable of serving at most 50 beneficiaries each were established for those persons with severe disabilities who were considered totally dependent on caregivers in view of their reduced potential for independent life. The facilities constructed and/or rehabilitated conform to the GOR’s quality standards for residential centers, day centers, and assisted housing for adults with disabilities that aim to improve the quality of life of PWD. Due to delays in works implementation, professional staff training was not carried out under the project, which constitutes a risk to sustainability. 92. Four centers for victims of domestic violence, with capacity for 12 victims each, were established to provide protection; accommodation; medical care; and social, psychological, legal, and occupational counseling. The period of care in the centers ranges from 60 to 180 days, and to date 209 victims of domestic violence (91 women, 118 minors) have benefitted from the accommodation and services provided. The MLFSP also carried out an extensive public awareness campaign to prevent and fight against domestic violence. The objective of the campaign was to increase the involvement of members of the community and raise awareness of local authorities with regard to the importance of participating in the prevention and response to domestic violence. 93. Several national conferences on the topic were held, in which over 500 persons attended. A survey of public attitudes and opinions with respect to domestic violence was carried out and the existing services for victims evaluated, 35 street events with mobile units and promotional materials were held, a promotional video for television was prepared and aired, all with the objective of increasing public awareness of family violence and how to prevent it. An informal survey of victims of domestic violence who have benefited from services provided by the project reveals that (i) they appreciate the accommodation, the understanding, support and warmth that they receive from shelter staff; (ii) they feel safe in the shelters; (iii) many are concerned with the prospect of leaving the shelter finding employment and housing, and earning an income to support themselves; and (iv) they believe that there is a need for even more publicity and promotion of these social services.

27

94. A total of ten subprojects for constructing Multifunctional Centers for Youth at Risk were approved, but at present three are still under construction (and will be completed with GOR funding). The centers provide temporary accommodation, emotional support, psychological and legal counseling, social counseling, support and guidance for integration, rehabilitation, employment counseling and retraining, and socializing. The centers’ capacity varies from 12 to 30 beds, and victims can remain in the centers for a maximum of 3 years. The 7 centers that are functional are staffed with 52 employees, of which about half have received training to improve their service delivery skills in a variety of areas. Sixty-three percent of the project’s 164 beneficiaries are employed, of which 30.5 percent became employed after entering the centers. An informal survey of beneficiaries’ opinions reveals that (i) most (66.7 percent) of them found the quality of accommodation, meals, and support services provided as very good and 21.7 percent found them good and (ii) 67.5 percent of beneficiaries rated their relationships with center staff as very good and 23.2 percent rated them as good. Intermediate result: Improved capacity of local groups to access EU funds for social inclusive programs 95. Both the RSDF and MER aimed to improve the capacity of local groups to access EU funds for social inclusive programs. Through the information dissemination and facilitation process, RSDF community coordinators worked with communities to assist them in preparing project proposals for EU funding. As of December 2013, the RSDF had received 24 requests for TA, of which six were approved by its Board of Directors (one was subsequently withdrawn and the applicant identified alternative financing). Of the remaining five projects prepared for EU financing, four have been prepared; but in view of the schedules for submission of proposals, only one has been submitted, declared eligible, and approved for financing. The MER also prepared ECE projects for human resources development, eligible under the European Social Fund, thereby leveraging the use of project funding. For this, it designed and implemented a training program (with a training curriculum, course modules, and a practical guide for writing eligible project proposals) that was delivered to 122 participants in five sessions of five days each. The MER also provided follow-up assistance on accessing EU structural funds for projects targeting ECE actions, under the following priority axes of intervention: 1.3 – human resources development in education and training and 2.2 – preventing and correcting early school-leaving. Three months after completion of the training, 34 project proposals were elaborated and four submitted for priority axis 1.3. Ten project proposals were elaborated for priority axis 2.2.

Strengthen the Administration of Social Benefits24

Intermediate result: Improve the administration of the social assistance benefits

24 Additional objective included in project restructuring, dated March 28, 2011.

28

96. Progress toward this sub-component that was incorporated under the project through project restructuring is considered ‘Moderately Satisfactory’ as several of the planned activities were completed, but others were either financed through alternative funding sources or delayed and expected to be financed under a proposed restructuring to the SASMP. The first phase of the SAFIR upgrade (hardware and software has been completed and upgrades were made to the system for it to centralize the beneficiary registry and financing the GMI program25 that had previously been administered in a decentralized setting. This will allow for better control and analysis of the data of about 400 to 250,000 households. The SAFIR was also updated to implement the parametric reform for benefits to mothers with young children whereby the period of maternity leave was reduced from two years to one year, the replacement wage was reduced from 85 to 75 percent and a back-to-work bonus was introduced to make going back to work more attractive. 97. The project mainly financed small upgrades to SAFIR. Three small social assistance programs were eliminated and two (Complementary Family Allowance and Single Parent Allowance) were consolidated to become a Family Allowance, with associated adjustments to the information system. In addition, under the project, the MLFSP completed the national registry (hardware and software) for persons with disabilities—a simple national database that captures and consolidates information from various systems at various government levels, allowing the GOR to track the impacts of disability within families and provides linkages with other GOR systems. The project financed training on the basic processes of a social inspectorate—legislation and preparing a case for the prosecutor to act. The development of a social inspection module in SAFIR (to help document the finding of inspections in electronic format and recalibrate risk models to identify persons with high risk of error) is expected to be financed under the SASMP restructuring. The Public Awareness Strategy for Social Assistance Reforms and Statistical Bulletin were developed, but the implementation of the public awareness campaign is expected to be financed by the SASMP restructuring. Finally, the project supported the NASB with TA in strengthening its ability to detect errors (non-intentional and intentional) through data matching and risk profiling; develop draft legislation for a comprehensive sanction and recovery policy (different benefits had different sanctions, the level of penalties were not correlated with the loss, and there were many ambiguities such as what constitutes fraud and what constitutes error); review the statutes of a social inspector; and initiate a data matching process to eliminate fraud, errors, and duplications in the GOR’s social assistance programs.26

3.3 Efficiency

25 The GMI is Guaranteed Minimum Income is defined by the law regarding social assistance, no. 133-XVI/ 13.06.2008, as the minimum level of computed income that the state guarantees to a family. 26 A Public Financial Management Trust Fund for governance and accountability, in the amount of US$250,000, was also used to finance several activities under this subcomponent.

29

98. The PAD did not include a cost-benefit analysis but an analysis of economic and fiscal rationale for public intervention of its various activities. It remains difficult to carry out economic and financial analysis for the project as a whole since it was demand-driven, multisectoral, multicomponent, and designed by the communities, making economic evaluation techniques for larger projects not applicable. Nevertheless, it is reasonable to assume that the project was implemented in an efficient manner and expected to generate positive economic returns. 99. The project did not compile data to allow a recent comparison of the cost-effectiveness of RSDF’s investments in small infrastructure. Although dated, a study prepared by the RSDF in 2005 found that its costs were systematically lower than those under two other comparable programs.27 As the RSDF continued to exhibit streamlined and strong performance under the project (implementing a greater number of subprojects than originally contemplated), in the absence of reliable, more recent data, it is reasonable to assume that its implementation continues to be an efficient alternative. 100. Investment in the ECE for disadvantaged persons under Component 2 is expected to generate positive economic returns in the medium-term, given the large body of international evidence that highlights the importance of early intervention—from conception to age 8—for child development and later educational and life outcomes. ECD programs are seen as particularly beneficial to children from disadvantaged backgrounds.28 In Romania, only an estimated 37 percent of children aged 3–6 years attend preschool. Although a small step toward increasing enrollment, the project supported the implementation of effective models for delivering ECE programs, including piloting alternative delivery models and more important modifications in the ECE curriculum, associated guides, and standards that address cultural sensitivities and social inclusion. There is evidence that these ECE activities were implemented efficiently, especially since the MER was able to train more teachers and procure additional materials utilizing €0.95 million in cost savings from civil works for construction of ECE centers, mostly due to the efficient, centralized procurement of civil works by MER. 101. Activities implemented by the MLFSP under Component 3 did not have financial and economic efficiency as an objective. By adhering to the GOR’s new quality standards that reduce the average size of institutions for PWD from an average of 126 at the time of appraisal to below 50 residents per center, the GOR’s program supported by the project was actually expected to increase the overall recurrent costs of institutionalized care, at least until the old institutions can be closed. Further, the implementation arrangements for Component 3, although participatory and important in terms of the transfer of knowledge

27 The ICR report for the Second Phase of the Social Development Fund Program (Report No. ICR0000538, dated August 15, 2007) presented the findings of a cost-efficiency comparison of 46 rural roads subprojects implemented by RSDF, the World Bank-financed Rural Development Project, and the European Union’s Special Accession Programme for Agriculture and Rural Development. The study found, that in all cases, the RSDF’s costs were lower than those in the other two programs. 28 World Bank. 2012. Toward an Equal Start: Closing the Early Learning Gap for Roma Children in Eastern Europe.

30

to local authorities, were not as efficient as those implemented by the MER and led to several implementation delays. 102. Finally, adjustments to SAFIR to reduce fraud and corruption in the administration of social assistance benefits (Component 3.4) resulted in savings in overpayment as outlined in table 1.

Table 1. Savings in Overpayment

Number of suspicious cases identified through data matching

Number of cases investigated in 2013

Number of cases found EFC, 2013

Total over payments (debts) 2013 (RON)

Recovered debts by March 31, 2014 (RON)

Costs Dec 31, 2013 (RON)

CRB 36,818 27,720 5,650 11,569,170 5,117,824 115,692

FA 51,664 34,123 15,151 2,643,421 1,690,999 1,129,939

GMI 15,964 9,676 2,959 1,730,223 877,075 SCA 541,502 27,308 2,549 1,189,902 692,476 74,964

Heating 35,851 24,030 4,874 755,368 241,046 490,989

Disabilities 5,457 5,457 2,724 1,301,069 551,914 143,118

Total 128,314 33,907 19,189,153 9,171,334 1,954,701

103. The total debts recovered by end-March reached RON 9,171,334, out of 128,314 cases investigated, resulting in two cost-benefit ratios of 9.58 on overpayments and 4.69 on recovered debts, according to the presentation on reducing error, fraud, and corruption (EFC) in social protection programs, held by the NASB. The final allocation of loan proceeds, by component, is outlined in Table 2.

Table 2. Final Allocation of Costs

Component Implementing Agency

No. of Beneficiaries

Total Allocation of

Costs ( US$,

million )

Cost/Beneficiary (US$)

Priority Intervention Program

RSDF 58797 16.98 284

31

Inclusive Early Childhood Education

Ministry of Education 14523 7.56 520

Social Assistance Programs

Ministry of Labor 1158 33.71*

Capacity Building for Roma Social Inclusion CANCELED

National Agency of Roma

– 0.25

Total 74478 58.5

Note: * €10.255 Million (US$11.417 Million) were cancelled.

3.4 Justification of Overall Outcome Rating

Rating: Moderately Satisfactory 104. The project’s overall outcome is rated Moderately Satisfactory. It was and continues to be highly relevant and central to the GOR’s strategy and the Bank’s assistance priorities. Its outcomes, as measured by the indicators in the project’s RF, were achieved and surpassed with respect to improving the living conditions and social inclusion of Roma living in poor settlements and increasing the inclusiveness of ECE services in targeted areas, and were partially achieved with respect to improving the quality of services for PWD, Youth at Risk and Victims of Domestic Violence. The project’s institutional objectives, especially with respect to the MER’s development of ECE curriculum and associated standards, were met. However, the project was not successful in strengthening the NAR to assume a more central role in the coordination, monitoring, and evaluation of programs focused on Roma communities. Based on available data and expectations, the project was implemented efficiently and is expected to either generate positive returns or improve living conditions for disadvantaged persons. Most of the project’s outputs were achieved, but there is little information on either its impact or efficiency. In short, outputs were achieved although they were compounded by limited evidence on impacts and efficiency.

3.5 Overarching Themes, Other Outcomes, and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development. 105. Poverty impacts and social development have been discussed in section 3.2, which provides the actual results achieved by the project toward improving the standards of living of its beneficiaries.

32

(b) Institutional Change/Strengthening. 106. Although the project did not have a specific institutional strengthening objective, it did support several activities in this respect. The project’s main institutional development impact was on the MER for revisions to the ECE curricula for children aged 0–3 and 3–6 years. The curriculum for children aged 3–6 years was formally approved and is already being used throughout the country and that for children aged 0–3 years is awaiting formal approval (while they are already in use by practitioners as reference materials). Both are supported by the new Education Law approved in February 9, 2011. Support to the MLFSP under Component 3.4, involving improvements in the system for social assistance, although not completed under the project, also had an important institutional aspect. Finally, the institutionalization of the position of Roma coordinators in several local councils to act as liaisons between the communities and local governments has shown promising results in terms of addressing social inclusion. (c) Other Unintended Outcomes and Impacts (Positive or Negative) N/A.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

N/A.

4. Assessment of Risk to Development Outcome

Rating: Negligible 107. Assessment of risk to development outcome is rated Negligible. In analyzing the risk to development outcome, two separate risks were considered: (i) the risk that the strengthening of social inclusion programs at the national level can be sustained and can continue to make progress in reducing social exclusion and (ii) the risk that the investments financed by the project at the local level will be sustained and maintained. At the national level, the risk to development outcome is considered negligible based on the GOR’s commitments on various fronts, including EU commitments for Europe 2020, its National Roma Integration strategy, and the ongoing preparation of a National Strategy on Social Inclusion and Poverty Reduction for the 2014–2020 period (under preparation by the MLFSP, with support from the Bank RAS, and under a condition for accessing EU structural funds in the programming period). Both the RSDF and the MER have strong implementation capacity and commitment and have their mandate with respect to vulnerable groups grounded in legislation. Several project-supported activities have routine allocations for these programs supported in regular, albeit constrained, GOR budget financing. Finally, continued Bank and EU financing, among others, should maintain the project’s objectives high on the government’s agenda.

33

108. At the local levels, the risk to sustainability of project-supported investments is considered Moderate. Project design had contemplated issues of sustainability, requiring that local councils assume ex-ante the obligations for operations and maintenance of investments for a period of fifteen years (or five years in the case of investments financed by the RSDF) as a pre-condition for being awarded financing. In fact, this was one factor that affected demand for financing at the local level in the aftermath of the economic crisis, when local governments were reluctant to assume additional financial commitments given uncertainties with respect to their future revenues. The RSDF is considering the continued use of community facilitation of the post-implementation period to avoid the possibility of facilities being closed on completion. In addition, community members, through their facilitators, can contact the RSDF to complain if the local authorities do not comply with their obligations during this period. 109. The development of the ECE curriculum for children aged 3–6 years has been completed, the document was formally approved and together with the Good Practices Guide for ECE for children aged 3–6 years, has been printed, distributed, and is in use in all kindergartens in the country. For alternative ECE solutions, a successful pilot activity, a sustainability challenge remains since the project was actually implemented over a short period and resources are needed to continue this initiative at the local level. There is a good example provided by one county (Iaşi) that issued a decision allocating resources for continuing the activities initiated under the project. The Bank has recommended that the MER identify resources and/or facilitate access to other funds to support communities that cannot otherwise continue these activities. In addition, it has recommended that the MER should facilitate the promotion of these activities through communication and awareness events at county and national levels involving relevant local authorities, representatives of school inspectorates, and other Roma-related organizations. 110. Finally, representatives of the NAR expressed to the ICR team that the NAR has begun a process to establish an M&E system that would be instrumental in carrying out its mandated function.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry - Rating: Moderately Satisfactory 111. Bank performance in ensuring quality at entry is rated Moderately Satisfactory. On the positive side, the project responded to a strategic priority of the GOR tied to its obligations toward EU accession. Responding to a GOR request for assistance in a very short period, the project addressed most vulnerable groups under one single operation. The project’s design was built on the lessons learnt from earlier Bank-financed operations. The project was designed with a clear sustainability objective, not only by using existing government structures and staffing (as opposed to implementation by a stand-alone

34

implementation unit) but also by requiring up-front commitments from local authorities to operate and maintain investments for a predetermined period upon completion. 112. However, there were some important shortfalls in preparation. First, while recognizing the risk that it posed and responding to the GOR’s request for rapid assistance and the World Bank’s desire to respond quickly, the preparation team severely underestimated the complexities of the project’s institutional arrangements, especially the differences in implementation (procurement and works supervision) procedures of each of the implementing agencies for the four components, and the time that would be required to implement the project as designed While fully participatory, the MLFSP’s approach to prioritizing the project’s somewhat larger investments at the local level and later implementing them through local councils (but with cumbersome critical approvals required at the national level) pre-supposed a much more complex and lengthy critical path than, for example, for investments financed by the RSDF. 113. The difficulties that this created became all the more evident in the absence of a unit to coordinate the GOR’s approach to address the needs of all vulnerable persons. Somewhat related, the preparation team underestimated the weak institutional capacity of the NAR, which was made all the more difficult by the ‘political’ nature of the agency and the negative impact that NAR’s weak performance would have on the project’s implementation and ability to put in place a strong institution to help prioritize, monitor, and evaluate the effectiveness of GOR programs targeted at the Roma population. The project was not fully prepared when the Bank’s financing was approved—sites had not been selected, OMs were yet to be finalized, baseline data that was dependent on site selection had not been collected as a result—and the difficulties in working with a weak NAR, which was to have a central role in this process, further delayed the start of implementation. To be fair, sites were not supposed to be pre-selected exactly in order to ensure the participation of communities, especially in the case of the RSDF-implemented component. In this case, the project’s planned implementation period could have taken this into account. 114. Finally, the project’s RF, while relatively concise, presents mostly output indicators and depends on often subjective surveys of beneficiary satisfaction (what community would respond that their living conditions have not improved if a road has been paved?). It did not include important indicators—for example, how many beneficiaries were served by different investments, their gender breakdown, and the number of Roma who benefited—that would have been valuable to monitor progress and retroactively adjust programs to more effectively address the needs of vulnerable groups. However, these data were collected internally by the RSDF and the MER for Components 1 and 2. (b) Quality of Supervision -- Rating: Moderately Satisfactory 115. The quality of supervision is rated Moderately Satisfactory. One country office-based task manager (who had participated in its preparation) was responsible for supervision, with the support of other specialists, throughout implementation until loan

35

closing. Supervision was regular and focused on obtaining up-to-date information on indicators in the project’s RF. 116. Although the 2011 restructuring reallocated funding in favor of well-performing components, at times, supervision efforts could have seized opportunities to adapt, adjust, and reformulate the project in a way that enhanced the impact and use of available resources. As a result, €10 255 259, 17 that could have been used to finance additional activities by the RSDF and the MER had to be cancelled. For instance, early issues with NAR’s performance were addressed repeatedly through action plans. However, in light of NAR’s diminishing capacity to absorb funds, these were eventually reallocated to other components. Supervision of the project tried repeatedly to support the NAR in improving effectiveness, especially since a decision to reduce financing for NAR was considered difficult. Despite the efforts, NAR’s performance lagged behind, with implications for overall coordination of Roma-related issues, for which NAR’s role was central. In the 2011 restructuring, issues with delays (chiefly due to cumbersome procedures and limited local demand) had become evident. At this point, supervision could have considered refocusing Component 3 toward those investments that could be realistically implemented, including reallocating more funding to other components that were progressing adequately. 117. Supervision was undertaken in a challenging environment, including, for example, a fluid and frequently changing institutional context. Yet the project’s components were supervised individually and implemented separately by the respective agencies with, as per design, limited coordination among them. From the monitoring side in particular, the project did not track its RF indicators systematically. As mentioned in section 2.3, 10 out of the 24 indicators that were supposed to be regularly monitored as per 2011 restructuring were not included in the ensuing ISRs. Final values for some of those indicators were, however, availed in commissioned studies, these only provided final data instead of reporting on indicators’ value on an annual basis. 118. The project restructuring responded in a timely manner to the emerging priority of TA for the social assistance reform. However, this helped address a challenge emerging in another initiative and provided limited contribution in enhancing coordination among the project’s components. The restructuring did not amend the project’s RF to address an issue with the definition of its indicators (that is, percentage increase when the base was 0) or increase the target for Component 1 when additional funds had been allocated. The RF did not include indicators enabling a focused monitoring of the project’s benefits on beneficiary populations, and it may have been useful to request implementing agencies to provide them regularly. During a visit to the country in preparation for this ICR, representatives from local NGOs cited the consultations during preparation but expressed the concern that these tended to diminish as projects were approved. (c) Justification of Rating for Overall Bank Performance -- Rating: Moderately Satisfactory

36

119. Overall Bank performance is rated Moderately Satisfactory based on similar ratings for both Bank performance in ensuring quality at entry and quality of supervision.

5.2 Borrower Performance

(a) Government Performance --Rating: Moderately Satisfactory 120. The Government performance was Moderately Satisfactory. For purposes of this section of the ICR, government is considered to be the broader GOR, especially the MPF, MoE, MLFSP, and RSDF. The government displayed ownership and commitment to the project throughout implementation despite the frequent changes that affected the political leadership. This was witnessed through continued funding at the national level for the project’s investments despite the fiscal constraints resulting from the economic crisis starting in 2008. The government also sought the Bank’s and EU’s funding in support of the project’s objectives through complementary vehicles such as the RAS. Nevertheless, issues relating to government processes, such as requirements for effectiveness, cumbersome implementation, and approval requirements for investments carried out at local levels, delayed the start of implementation and required approvals throughout. However, more importantly, the GOR, despite expressing its objective of addressing the needs of vulnerable groups in a coordinated fashion, was not able to put in place a workable mechanism to develop a concerted strategy, prioritize, disseminate, and monitor its progress in addressing their needs. (b) Implementing Agency or Agencies Performance -- Rating: Moderately Satisfactory 121. The implementing agencies’ performance is rated Moderately Satisfactory. The project was implemented by two ministries, the MLFSP (including NAPH and NAFP) and MER, and two agencies, the RSDF and NAR. The Ministry of Finance was an implementing agency insofar as it was responsible for allocating the required budgetary allocations to each of the implementing agencies. 122. The RSDF’s performance exceeded expectations. Its strong performance, based in part on its previous highly satisfactory experience in implementing Bank-financed projects, benefitted from competent, committed staff and good governance. Its procedures pre-suppose outreach by RSDF facilitators and beneficiary involvement throughout, both of which bode well for future sustainability and inclusion. The RSDF’s implementation model and procedures have become an example of good practice and have recently succeeded in attracting additional grant financing from European donors. 123. The MER previously had a successful experience in implementing Bank-financed projects and was therefore fully familiar with the requirements that financing entails. Its performance was also highly satisfactory because it was able to implement activities under Component 2 effectively and efficiently. The MER was able to circumvent the GOR’s investment and procurement approval procedures by centralizing implementation

37

and packaging works, thereby limiting the many approvals that would be required for individual subprojects. Further, the MER pursued and obtained approval for systemic changes in the legislation, curricula, and teaching practices for the ECD that will have a lasting impact on vulnerable groups, especially Roma, well beyond closing. 124. While assuming a de facto coordinating role for the project, the performance of the MLFSP (including the NAPH and NAFP until they were made directorates) was weaker than that of either the RSDF or the MER. This was largely the result of three factors. First, the successive government changes had an inordinate impact on the MLFSP, whose leadership changed seven times during implementation (along with consequent changes at lower levels in the ministry). Second, the MLFSP went through a major restructuring resulting from GOR budget constraints in the aftermath of the 2008 economic crisis. Finally, of all the project’s components and subcomponents, those implemented by MLFSP through local governments were the most affected by the GOR’s cumbersome investment and procurement approval procedures. The long processes required for approval of investments, coupled with the staffing and other changes in the MLFSP, presented a difficult environment for maintaining implementation on track. 125. The NAR had the weakest performance of the implementing agencies, demonstrating intermittent commitment to the project, and, because of the central role it was expected to play, delaying the implementation start of other components. Local governments and city and county councils were also responsible for implementing investments, especially under Component 3 where they were responsible for tendering, contracting, and supervising civil work for centers of social assistance services. The poor institutional capacity of these local governments, coupled with a weakened demand for financing (section 2.2) at the local level, required intensive supervision from the MLFSP and impacted the outcomes under Component 3. The Ministry of Public Finance allocated funding to implementing agencies in a timely fashion, albeit with delays on occasion. (c) Justification of Rating for Overall Borrower Performance -- Rating: Moderately Satisfactory 126. Overall borrower performance is rated Moderately Satisfactory based on similar ratings for both government performance and implementing agencies performance.

6. Lessons Learned

127. The project’s implementation provides several lessons with respect to project design, implementation, and especially the challenges of addressing the needs of Roma and disadvantaged groups. 128. Social inclusion is a broad-encompassing concept. Project design should set out a clear definition of ‘social inclusion’ and how to measure it. The concept of social inclusion can be interpreted at different levels, through the lens of different

38

disciplines and from a wide range of perspectives, some of which may not necessarily be fully compatible. Therefore, it is important that the design of projects that have social inclusion as an objective provide a concrete, working definition of social inclusion that is relevant to different actors and that can be clearly attained and measured. Such definition and metrics should possibly be part of a broader ‘theory of change’ for a project, including laying out pathways and mechanisms to achieve the intended objectives. 129. Project design needs to incorporate a comprehensive pre-assessment of institutional capacities at all levels of government. This would be important in order to calibrate project objectives and activities, establish realistic implementation schedules and make possible course corrections. Under the Project, the various implementing agencies had very different operating and implementation procedures that affected the pace with which activities were implemented. These differences were exacerbated since, over time, the capacity of implementing agencies was severely affected by a number of factors, such as the reduction in government staffing levels in Romania as a result of the financial crisis that impacted staff motivations due to institutional reform (i.e., NAPH downgraded from Agency to Directorate); and also the pressure on limited local capacities for attaining and managing funding from the EU and other sources. In such a rapidly changing environment (that most projects experience), a full ex-ante assessment of implementing agencies’ capacities, processes and implementation models at all levels assumes an even greater importance to ensure that objectives are realistic, program design is commensurate to existing arrangements, gaps are identified, and flexible mechanisms are prepared to adjust should circumstances require so. 130. Decentralized and centralized implementation models present potential tradeoffs that need to be considered explicitly during project preparation. Decentralized implementation, such as that by MLFSP, builds buy-in, local capacity, sustainability, and responsiveness to local needs. However, it requires additional time, and may not be compatible with the short-time schedules of other, more centralized, implementation models, such as that of the MER. If the decentralized approach is taken, local authorities require strong implementation support, especially when required to follow guidelines of externally financed projects with which they may not be familiar. 131. The establishment of a coordination mechanism among project components is important, especially when multiple agencies are involved in project implementation. Social inclusion programs tend to operate in a context of fragmented approaches, across sectors, and including a range of different interventions, diverse actors and differing institutional arrangements. The existence of an overall coordinating institutional umbrella would serve to ensure coordination and connection among project components and activities. This type of coordination should not rely only on the requirements of a World Bank-financed project, but should be matched by coordination mechanisms established within the government and that cut across the relevant dimensions envisioned by social inclusion projects. For example, a number of countries have introduced inter-ministerial working groups for facilitating coordination and the exchange of information on a given theme, and Romania may consider doing so for social inclusion.

39

132. A potential tension might emerge between rapid preparation of a project and establishing an effective monitoring and evaluation system. Baseline data should be available before implementation begins, with metrics identified and protocols for data collection and monitoring agreed upon with partners and implementing agencies. Specific operational activities, such as project site selection, should be undertaken in advance of project implementation and as part of the preparation phase. Yet many of these activities could be compromised by rapid project preparation and design. 133. Project sustainability should be framed in ways that recognize the importance of securing commitments while at the same time not hampering the demand and capacity of local authorities. While it is important to ensure up-front commitments to sustain interventions over time (as the Project did), this may present a trade-off between short-term demand of local authorities to participate in projects and the longer-run sustainability through up-front commitments. In the case of the Project, local demand was somewhat limited precisely because of the sustainability requirements in contexts of often limited financial and other capacities, and uncertainties. While devising an appropriate set of incentives between central and local governments is a recurrent challenge in most countries, trade-offs should be more fully recognized and weighed and project expectations set accordingly. 134. Committed leadership and champions, including, in the case of the Project, local mayors and Roma community representatives, have a critical role to play in laying the foundations for social inclusion. The ability of interventions to foster social inclusion would, ultimately, heavily hinge upon the sustained engagement, commitment, mutual trust and partnerships between Roma communities and government officials (central and local). Social inclusion projects should include as an indicator to monitor progress specific metrics to measure progress in incorporating Roma representatives in local governments or other community organizations. 135. Project design should consider flexible and tailored arrangements that allow for intra-community and spatial-sensitive approaches. Under the Project, there were important cultural and socio-economic differences among Roma communities that should be more explicitly recognized and considered in project interventions (e.g., differentiating approaches to mobile and sedentary communities). Similarly, programs in urban areas should more deliberately account for the different socio-institutional setting compared to rural areas (e.g., definition of ‘community’, their level of cohesiveness, etc.). In particular, such differences may have implications for the complexity of undertaking consultation with Roma communities and their level of participation in decision-making (i.e., more challenging in urban areas with more fragmented communities), as well as the degree of integration with different sectors (and requirements for technical support). For example, road-construction programs in urban areas may entail closer integration with hard-core urban development interventions (e.g., in water, sanitation and drainage systems), with implications for technical standards required in the planning, execution and supervision of works.

40

136. Design of social inclusion projects should include measures that capture how activities ultimately contribute to social inclusion. The physical construction and rehabilitation of social centers, community infrastructure and trainings of teachers – the core of the SIP activities – are key to provide marginal areas with kindergartens, paved roads, or community baths that previously lacked these. Yet, these interventions, per se, may not represent a full measure of social inclusion and empowerment. It is important that projects’ results frameworks not only capture and measure what was built or maintained, but how those assets contributed and translated into enhanced dialogue, mutual understanding, trust and partnership between Roma and non-Roma Romanians as a people. An indicator, for example, might be the number of communities where local councils include Roma representatives. Also, it would be important to try capturing whether elements of Roma traditions and culture (e.g., materials, decorations, posters, etc.) are included in kindergartens, somewhat mirroring the learning materials for teachers produced by the MoE. 137. The role of and consultation with an organization responsible for addressing the needs of minorities and civil society should be embedded within the project cycle on inclusion projects. It is important to ensure that government agencies that have a legal mandate to work with and represent Roma communities are engaged as partners alongside other institutions. In the case of NAR, the agency proved to have limited direct relationship with the World Bank as it was the only project implementing agency that was financially and programmatically subordinated to a line Ministry. As an agency that was expected to have a coordinating role (if not for the Project, for the GOR’s progress in addressing Roma inclusion), it may well have been more appropriate for NAR to not be subordinated. 138. While the involvement of Roma institutions and NGOs cannot guarantee success, per se, civil society should be involved throughout the project cycle through engagement and consultation. An active participation of NGOs will help ensure that the priorities, perspectives and voice of communities is considered as an integral part of the project preparation, implementation and evaluation stage, as well as a core tenet of the broader dialogue on social inclusion beyond a specific project. The community facilitator in the activities financed by RSDF played a key role in bridging institutions and communities, including providing assistance to ensure that projects reflect community priorities as identified from the bottom-up. Yet it is important to ensure that such facilitation role does not translate into the ‘clearance’ of predefined lists, but rather identifies needs and interventions as expressed and demanded by communities. 139. Projects should incorporate mechanisms to promote sharing of innovations and lessons, and outreach. There was a wealth of experience that communities, kindergartens, and other institutions accumulated in implementing activities under the Project that would be highly relevant for other eventual stakeholders. For example, initiatives such as a mayors yearly local gathering – e.g., knowledge fairs, ‘community of practice’, or similar – would help share challenges and solutions among peers and other actors. For programs involving Roma communities, NAR could play an important role in identifying, documenting and disseminating emerging models and best practices. Such

41

forums would also provide a vehicle for encouraging innovations (e.g., alternative education model), discuss how they could be scaled-up, and ensure support of local and central authorities. Similarly, lessons-sharing platforms could also help bolster outreach, communication and awareness about social inclusion projects, issues and experiences, both positive and less-positive ones. 140. Pilot experiences can provide an effective instrument to test alternative approaches to implementation of assistance delivery models. The alternative CBS pilot produced some encouraging, positive approaches to scaling up the provision of ECE services in remote areas. Yet, for pilot experiences to be productive, there is a need for platforms in which to disseminate, communicate and raise awareness to benefits, possibilities and lessons.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agencies 141. The Borrower provided a few editorial comments that have been incorporated in the text. The Borrower subsequently provided a second round of comments in a letter dated December 5, 2014, in which it recommended that the ratings for Borrower, implementing agency and World Bank performance be satisfactory (as opposed to moderately satisfactory). This recommendation was justified by the Borrower as with the exception of Component 3, all components achieved their objectives, and RSDF was able to absorb more funds than originally allocated.

(b) Co-financiers

N/A.

(c) Other partners and stakeholders

N/A.

42

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in US$, Million Equivalent)

Components Appraisal Estimate (US$, millions)

Actual/Latest Estimate

(US$, millions)

Percentage of Appraisal

Component 1: Priority Interventions Program 13.25 16.98 128

Component 2: Inclusive Early Childhood Education (ECE) Program

6.9 7.5 108

Component 3: Social Assistance Programs 32.4 33.71 104

Component 4: Capacity Building for Roma Social Inclusion

0.9 0.25 27

Total Baseline Cost 53.45 58.44 109

Physical Contingencies 0.00 0.00 0.00

Price Contingencies 0.00 0.00 0.00

Total Project Costs 53.45 58.44 109

Front-end fee PPF 0.00 0.00 0.00

Front-end fee IBRD 0.00 0.00 0.00

Total Financing Required 53.45 58.44 109

43

(b) Financing

Source of Funds Type of Co-financing

Appraisal Estimate

(US$, millions)

Actual/Latest Estimate

(US$, millions)

Percentage of Appraisal

Borrower 12.00 15.00 125

Local Communities 0.90 0.90 100

International Bank for Reconstruction and Development

47.14 58.50 124

44

Annex 2. Outputs by Component

Component 1: Priority Interventions Program (€11.7 Million)

This component was designed to support RSDF’s program of poverty alleviation and capacity-building interventions, by expanding the Fund’s activities to specifically address the needs of the Roma. The project aimed to improve living conditions and social inclusions for Roma living in about 100 poor settlements, bringing those most disadvantaged settlements up to minimum conditions in terms of infrastructure and basic social services. The project was to support: (i) financing of demand-driven, integrated Priority Interventions subprojects, on a grant basis awarded by RSDF following a competitive process from among a list of eligible settlements provided by the NAR; (ii) carrying out of promotion, information, and education campaigns, M&E activities, and the provision of training for local authorities, community groups, and NGOs; (iii) provision of assistance in capacity-building at the community level through preparation, facilitation, evaluation and supervision of Priority Interventions subprojects, and provision of related training, including training to RSDF; and (iv) provision of TA to poor Roma communities to prepare projects to be financed through EU funds. RSDF financed 133 integrated projects—that benefited 58,800 people, of which approximately 19,500 were women—as follows: 90 had a community road rehabilitation component; 30 had a utility access component (water supply, sewage network, including street

sewage, public baths and washrooms); 62 had a social services component (day care and educational centers, vocational

training and professional counseling services, information and health campaigns); 17 had a medical services component (establishment of medical office or cabinet); 9 had a housing component (rehabilitation of houses, apartment buildings, flood

protection works, public toilets); 21 had a community center component (cultural and educational centers, art and

sports clubs, music and dance clubs/bands); and 21 had an environmental protection component.

In addition, the following activities were produced and/or delivered under Component 1: 4 public information campaigns, conducted in the target communities Information dissemination to 214 communities 18 regional information seminars, attended by 647 people 23 training sessions for the members of the project coordination committees,

benefiting 991 participants 3 experience-sharing networks between beneficiaries, 12 network meetings

organized 171 eligible Roma communities received community facilitation services

45

€17.86 million was allocated to component 1, of which €1.37 million (76.71 percent) was external funding. Of the remaining €16.94 million (94.85 percent), €13.0 million was external funding and was used to finance activities under the component. Of the €16.94 million cost of this component, €14.15 million (83.56 percent) was allocated to subprojects, and €1.05 million (6.31 percent) for activities aimed at increasing community capacity to contribute to local development (TA—information, community facilitation, project evaluation and supervision, consultancy for development of other projects—and training for the representatives of the beneficiary communities). In addition, the operational costs, goods, and training for RSDF staff totalled €1.72 million (10.22 percent) of the total expenditure for Component 1. Of the amounts allocated to subprojects, €11.55 million (representing approximately 81.6 percent) financed construction works for improving the livelihood in Roma communities (rehabilitation of roads, water supplies, and building education and community centers). The total costs of Component 1 were allocated as follows:

Construction Works

Goods and Materials

Training and

Consulting

Operational Costs (Related to Social

Services)

Project Management Costs

81.6% 6.6% 1.1% 5.7% 5.1%

46

Component 2: Inclusive Early Childhood Education Program (€6.1 Million)

By helping mainstream Roma into the regular policies and programs of the MER, this component was designed to support the MER’s ECE program (2005–2011) that aimed to ensure every child’s right to education and overall development starting from a very early age. The project would help jump-start the inclusive measures of MER’s overall ECE program through: (i) construction, extensions, rehabilitation, and furnishing of ECE infrastructure in communities with a high percentage of Roma; (ii) development of an inclusive ECE curriculum, the provision of training to ECE staff, the development and distribution of ECE teaching and learning materials, and the provision of TA to entities entitled to submit financing applications for projects to be financed through EU structural funds; (iii) promotion of integrated services and alternative community-based solutions for ECE, including the development of coherent ECE legislation, and the provision of training and counseling programs for parents; and (iv) carrying out of M&E, information, education, and communication activities. Rehabilitation of ECE infrastructure Component 2 financed the building and furnishing of 19 new buildings for preschool units, creating facilities to enroll more children in the areas where there was a large percentage of Roma children aged 0–6 years. It also financed the rehabilitation and furnishing of approximately eight existing buildings, to improve the quality of education and facilitate access to more students coming from disadvantaged communities with a majority Roma population.

No. County School unit Type of Works

Number of classrooms

Number of

enrolled children

Number of disadvantaged

children

1 Arad Kindergarten half time sat Vanatori,

comunaMisca D + R 4 93 84

2 Arges Kindergarten half time sat Pojorata,

comunaLeresti CN 2 30 30

3 Botosani Kindergarten half

time sat Buda, comunaCosula

D + R 2 39 29

4 Brasov Kindergarten half

time sat Soars, comuna Soars

D + R 3 22 0

5 Buzau

Kindergarten half time nr. 6 Cartier

Zidari, municipiulRamnicu

Sarat

CN 2 29 29

47



Number of

enrolled children


children

6 Calarasi

Kindergarten half time sat

SarulestiGara, comunaSarulesti

CN 4 54 27

7 Dambovita Kindergarten half time sat Iedera de

Sus, comunaIedera CN 2 36 33

8 Dolj Kindergarten half time sat Salcuta, comunaSalcuta

CN 2 10 4

9 Galati

Kindergarten half time nr. 4 sat

Podoleni, comunaBarcea

CN 3 55 42

10 Iasi Kindergarten half time sat Crucea, comunaLungani

CN 3 60 53

11 Maramures Kindergarten half time sat Chelinta, comunaUlmeni

CN 3 45 45

12 Mehedinti

Kindergarten half time sat Balta

Verde, comunaGogosu

CN 2 40 35

13 Olt Kindergarten half

time sat Chilii, comunaFalcoiu

D + R 4 64 48

14 Prahova

Kindergarten half time sat

LuncaPrahovei, comunaMagureni

D + R 2 34 34

15 Salaj Kindergarten half time sat Dragu, comunaDragu

CN 2 28 20

16 Satu Mare Kindergarten half time sat Tataresti,

comunaViile D+R 2 40 22

17 Suceava Kindergarten half time sat Bogata,

comunaBaia CN 3 44 32

48



Number of

enrolled children


children

18 Valcea Kindergarten half time sat Balota, comunaRacovita

D + R 2 23 23

19 Vaslui Kindergarten half time sat Puscasi, comunaPuscasi

D + R 4 75 51

20 Bihor Kindergarten half time Marghita CN 2 55 15

21 Covasna

Kindergarten half time

ValeaSeacaSanzieni

CN 2 47 23

22 Ilfov Kindergarten half time Ganeasa CN 1 25 0

23 Cluj Kindergarten half time FizeşuGherlii

C + R + E 3 20 20

24 Alba Kindergarten half

time SilivasHopirta

C + R + E 3 18 16

25 Mures Kindergarten half time Craciunesti

C + R + E 2 16 11

26 Sibiu Kindergarten half time Dumbraveni C + R 3 36 30

27 Gorj Kindergarten half time Albeni

C + R + E 2 22 13

Total 1060 769

Note: CN: new construction (building) D+R: demolition & reconstruction RK: minor works concerning finishing jobs C+R: consolidation & rehabilitation E: extension for the existing building

49

Development of ECE curriculum, human resources and teaching/learning materials The development of the ECE curriculum

National individual consultants developed the ECE curriculum for children aged 0–3 years and revised the curriculum for children aged 3–6 years. They worked with inspectors from the Ministry of National Education (MNE), and the Department for Primary and Secondary Education and the Department for Minority to develop the curriculum to be integrated into the MNE policy and practice and be accepted by education decision-makers and beneficiaries. The new ECE curriculum enhances the socio-emotional and physical health of children, and the continuity between early education curriculum for 0–3 years and 3–6 years.

Since the educational curriculum for children aged 0–3 years had not previously

existed, a methodological guide for teaching and other staff was prepared to orient the specific activities and organize the learning environment in crèches. Both the ECE curriculum for children aged 0–3 years and the methodological guide are still awaiting formal approval, which is necessary for their legal application.

A good practice guide for ECE for children aged 3–6 years was also developed by

individual consultants. It provided support for all teaching and non-teaching staff in kindergartens.

The quality standards for teaching and learning materials and the normative for

minimal supplies in kindergarten were updated. Quality standards for ECE programs were developed through consultations with representatives from the Romanian Quality Assurance Agency for Pre-university Education, to be used as a grounding document for the development of other standards.

The development of human resources (teaching and care services)

An international consulting company provided TA for training the trainers, coordinated and monitored the in-service training for upgrading skills of staff primarily working in project supported ECE units, and developed modules for ECE and a preparatory guide for all staff. To organize the training activities, the consulting company selected an appropriate number of persons, from the already qualified or experienced teachers and trainers, to ensure training for the targeted persons from the communities with building and rehabilitation interventions and other similar communities in proximity, lacking qualification and competence in ECE. They provided in-service training (by gathering people in clusters), and presented and discussed the basic module for all the trainees and the specific ones for distinct groups including follow-up sessions. The clusters have been placed where resources allowed quality teaching environment and availability of education staff. The main aim of the training was to help the target population, represented by the ECE staff—educators, care personnel, nurses, principals from kindergartens, and auxiliary personnel—to upgrade their skills (teaching and care

50

services), and also become familiar and apply inclusive approaches in all their activities.

Training modules were designed to raise awareness about disadvantaged groups

and bring about an attitudinal change toward Roma, address issues related to social inclusion and cultural sensitivity:

o Module 1: General module on concepts for early childhood education, focused on inter-cultural principles

o Module 2: Interactive methods module on practices in ECE services o Module 3: Educational management module on management of diversity

in the ECE system The modules were developed based on the revised curriculum for children aged

3–6 years and the early childhood education guidelines for children aged 0–3 years, using the methodological guide developed under the project and other relevant documents for ECE that have been recently developed and implemented in the ECE system.

2,600 modules were printed and distributed to trainees, and the number of

participants was 2,769. Based on the training needs of the new-entry ECE staff and budget savings, a

second round of training was organized in 2014 by another consulting company, selected as a result of QBS.

During the second round of training, 1,500 modules were printed and distributed to trainees, and the number of participants was 1,422.

On the whole, more than 4,000 people were trained, as targeted in the beginning of the project.

The development of standards of ECE teaching and learning materials, and provision of such materials primarily in project ECE units

6,000 copies of the ECE curriculum for children aged 3–6/7 years and the good practice guide for ECE for children aged 3–6/7 years were printed in two tranches, and distributed to all kindergartens, through the county inspectorates.

Teaching and learning materials and books were provided for the children from targeted communities (100 kindergartens), as follows:

o set of balls o tunnels for crawling o magnetic boards with letters and numerals o magnetic boards for teachers o sets of geometric shapes in various colors, sizes, and thickness o construction sets - small pieces, set of wooden cubes for construction, and

big pieces o sets of musical instruments o models of human body and internal organs o models of various animals, insects, and fishes

51

12 book titles (575 copies of each title) were distributed based on the assessment of the needs of ECD units and their priorities. The selected books also covered the inclusion and cultural aspects.

Access to the EU Structural Funds

The use of SIP funds was maximized by utilizing them for the preparation of ECE projects for human resources development eligible under the European Social Fund (ESF), further contributing to the implementation of the overall MNE ECE program. Consultants were contracted to design and implement training and provide follow-

up assistance on accessing EU structural funds for projects targeting ECE actions, under the following priority axes of intervention:

o 1.3 - Human resources development in education and training o 2.2 - Preventing and correcting early school leaving

The training was provided to 122 participants, in 5 sessions of 5 days each, as follows:

o Priority axis 1.3: 3 sessions with 77 participants (from Teachers’ House, School Inspectorates, County Centers for Psycho-pedagogical Assistance, schools, and kindergartens) in 36 counties

o Priority axis 2.2: 2 sessions with 44 participants (from kindergartens) in 13 counties

The training involved experts from the Intermediary Body for Human Resource Development Operational Program, the MNE unit responsible for managing the funding policies and instruments available for human resources development through the European Social Fund.

All trainees received certificates for participation. The following training materials were provided:

o training curriculum o training course modules o practical guide for writing eligible proposals under the European Social

Fund Three months after the completion of the training, 34 eligible project proposals had been prepared of which 4 had been submitted for priority axis 1.3, and 10 had been prepared for priority axis 2.2. Promotion of integrated services and CBSs for ECE Alternative community-based solution

The project financed a consultant to design a plan for alternative community-based solutions.

Based on this plan, an international consulting company implemented an alternative CBS in four pilot-isolated and vulnerable communities with no access to education services for children aged between 3 and 7 years.

52

The intervention included family kindergartens, specifically targeting Roma and disadvantaged groups, support for training, teaching and learning materials, and incentives for care mothers and children.

The pilot communities were:

o Poiana cu cetate, Grajduri parish, Iaşi county o Baneasa town in Carpinis district, Constanta county o Maciova, Caras-Severin county o Racos village, Racos parish, Brasov county

In the four communities, eight care mothers were trained based on the adapted alternative curriculum for children aged 3–6 years and the methodological guide for care mothers. During project implementation, 69 children benefited from CBS, most of them Roma.

Training and counseling programs for parents

A consultant was contracted to train and counsel parents to become sensitive to social, psychological, nutrition, health, accident prevention, children’s rights, and other specific issues when raising children and to develop new skills and attitudes.

A needs analysis for teaching staff and parents from the target group was developed—in the field of particularities of education, development, and care of the young child (0–7 years old).

A set of modules for trainers and thematic brochures for parents were developed and printed, to support the training and counseling sessions.

The training program included 53 qualified teachers who trained and counseled 1,257 parents.

A final study was developed, including 3 questionnaires: one for teachers, one for parents, and one for the trainers.

The services were implemented in eight resource centers: o ValeaLunga Kindergarten, Alba county o Bunesti Kindergarten, Brasov county o Modelu Kindergarten, Calarasi county o Iveşti Kindergarten, Galati county o Lunca Cetatuii Kindergarten, Iaşi county o Glina Kindergarten, Ilfov county o Odorheu Kindergarten, Satu Mare county o Pătrăuţi Kindergarten, Suceava county

All eight resource centers received furniture and equipment: tables for equipment, tables for discussions, chairs, laptops, multifunctional printers, video projectors, projection screens, and filing machines.

Monitoring & Evaluation and IEC activities National and international consultants supported MNE in setting up an ECD

M&E system based on clearly identified indicators, including a survey instrument

53

to monitor child readiness in domains including: physical health and well-being, social competence, emotional maturity, language and cognitive development, communication skills, and general knowledge.

The consultants developed a child’s school readiness assessment tool and a user’s guide on how to use the Child Development Evaluation Tool (CDET) in the child’s last year of kindergarten and also, by primary teachers, in the child’s first year of schooling. The tool’s major aim is to define a child’s profile at the end of kindergarten, which will serve as reference data in supporting the child’s development and learning in the next years (child’s level of development, domains in which the child needs more support, child’s strengths, and child’s style of learning).

Later, different consultants were contracted to pilot the two instruments. The piloting process was carried out in 42 units, two kindergartens and two schools in each of the eight regions (South, South-West, West, North-West, North-East, Center, and Bucharest-Ilfov).

The piloting process included teachers and parents from 60 kindergartens and schools. All the participants received copies of the instrument to apply it to each child they supervise.

Upon conclusion of the pilots, a study was carried out incorporating the results with a view of improving the quality and ease of using the CDET and user’s guide by teachers and parents.

Consultants also designed and prepared a communication strategy and action plan for the ECE program. They developed a strategic communication appraisal document, a communication strategy for the IECE program, and an action plan for implementing the communication strategy.

Project Management Units

Within the MNE, two PMUs were responsible for managing the project’s component 2: the Externally Financed Projects Management Unit (EFPMU) and the School and University Network Rehabilitation Projects Management Unit (SUNRPMU). The PMUs had general coordination responsibilities, and were responsible and accountable for procurement, financial management, and reporting, as well as M&E activities.

Component 3: Social Assistance Programs (€28.6 Million)

This component was designed to support programs of the MLSSF aimed at improving the situation of PWD, youth at risk, and domestic violence victims through three subcomponents, as follows: Subcomponent 3.1: Disabilities program (€14.4 million). This subcomponent aimed to improve the quality of care in residential services and increase the access to CBS for PWD in need, especially those with a high dependency on residential services. It would support the MLSSF/NAPH program for institutionalized care restructuring, prevention of institutionalization, de-institutionalization, and development of alternative community-based systems of assistance and social inclusion through: (i) subprojects—submitted by

54

local authorities and selected in accordance with a competitive grant scheme—aimed at restructuring and rehabilitation of existing institutions and construction of new infrastructure facilities, in-line with improved standards developed by NAPH, and development of alternative CBS to support the de-institutionalization process; (ii) institutional capacity-building through both the provision of training to professional staff in the new standards of care and the development of the M&E system for NAPH; and (iii) provision of occupational counseling services to the beneficiaries of the newly restructured institutions. Under Component 3.1, the project financed the restructuring or closing of existing residential care services, and the construction and establishment of new residential care centers with a maximum capacity of 50 residents and protected dwellings with a maximum capacity of six persons. In all, six neuropsychiatric recovery and rehabilitation centers, seven assistance and care centers, and 25 protected dwellings were established (and services provided), as follows:

Type of Center Beneficiary Category

Number of Beneficiaries Services Provided

Neuropsychiatric Recovery and Rehabilitation Center

Adult with neuropsychiatric disabilities and low skills

394 - accommodation - nourishment - assistance and personal hygiene - medical and recovery assistace, - emergency and therapeutical assistance - psychological counseling services - ergo therapy services and medical, psychological, and social recovery (ocupational therapy, art therapy, and music therapy)

Protected Dwellings

Persons with disabilities that have potential for semi-independent life and community integration

150 - accommodation - nourishment - leisure and social activities - support activities for independent life

55

Type of Center Beneficiary Category

Number of Beneficiaries Services Provided

Assistance and Care Center

Persons with severe disabilities that have reduced potential for independent life, being totally dependent on a care-giver

296 - accommodation - nourishment - care and personal hygiene îngrijire şi igienă personală - medical and recovery assistace - psychological counseling services - leisure and social activities

The new services conform to specific quality standards for residential centers, day centers, and assisted housing for adults with disabilities, approved by presidential order on National Authority for Disabled Persons no. 559/2008. Subcomponent 3.2: Youth at risk program (€11.1 million). This subcomponent was designed for youth aged 18 and above, graduating from the institutionalized care system, to provide them with increased access to integrated social services that would increase their readiness for independent living. It would finance subprojects submitted by local authorities (and awarded on a competitive basis) to support the establishment of the network of multifunctional centers, where eligible beneficiaries would have access to protected housing, complemented by a range of psychological, social, vocational, occupational, and legal counseling services. It would also finance M&E activities, provision of training to the counseling providers and MLSSF staff involved in supervising the function of the multifunctional centers established, and implementation of public awareness campaigns. Ten subprojects were approved for the establishment of multifunctional centers for youth at risk. Of these, seven are already in operation. The centers provide temporary accommodation, emotional support, psychological and legal counseling, support and guidance for integration, rehabilitation and retraining, and socializing. They are staffed by administrative (chief, night watchman - if applicable, a cleaning lady, or janitor) and specialized staff (one full-time social worker for 20 people), and other professionals who comprise a multi-disciplinary team required to deliver services (a psychologist, a psycho/social educator, a socio-educational animator, advisor, and a vocational/training counselor). The maximum period of stay for residents is three years. Details of each of the ten centers are provided below:

56

Multi-functional Center for Youth

at Risk Opening Date Capacity

Number of

Beneficiaries

Number of Employed

Beneficiaries

Number of Beneficiaries

Employed after they Entered the

Center

Employed Beneficiaries

within the Total of

Beneficiaries (%)

Number of Center

Employees

DGASPC HARGHITA Centrul social cu destinaţiemultifuncţională "Un micpopasînviaţa mea", Sanmartin (CN) judetulHarghita

05/31/13 12 beds 12 9

6 50% 5, of which 2 are employees of the center and 3 are the social service provider’s employees

MUNICIPIUL GALAŢI Centrul social cu destinaţie multifuncţionalădinmun. Galaţi (CN) judetul Galati

06/10/13 30 beds 34 19 0 0% 12

Centrul social cu destinaţiemultifuncţională, GuraHumoruluijudetulSuceava

09/01/12 40 beds 10 10 10 100% 3

57



Number of

Beneficiaries

Number of Employed

Beneficiaries



Center


within the Total of

Beneficiaries (%)

Number of Center

Employees

ASC - ORADEA Centru social cudestinatiemultifunctionala CANDEO judetul Bihor

12/20/13 40 beds 30 16 8 26,66% 6, of which 2 are employees of the center and 4 are the social service provider’s employees

ASC ORADEA - DIGNITAS Centrul social cudestinaţiemultifuncţională DIGNITAS (CN) judetul Bihor

12/20/13 32 beds 16 11 2 12,5% 7, of which 2 are employees of the center and 5 are the social service provider’s employees

58



Number of

Beneficiaries

Number of Employed

Beneficiaries



Center


within the Total of

Beneficiaries (%)

Number of Center

Employees

Centrul social cudestinaţiemultifuncţionalăpentrutinerii care părăsescsistemul de protecţie, dinjudetul Iasi (3 X CN)

10/16/12 30 beds (10 per home)

49 29 21 42,85% 14 social service provider’s employees

DGASPC VRANCEA Centrul social cudestinaţiemultifuncţională " Sf. Teodor " (CN), judetul Vrancea

11/01/13 40 beds 13 9 3 23,07% 5 social service provider’s employees

DGASPC SĂLAJ Centrul social cudestinaţiemultifuncţională, Jibou (CN) judetulSalaj

The center is not yet functional

15 beds

59



Number of

Beneficiaries

Number of Employed

Beneficiaries



Center


within the Total of

Beneficiaries (%)

Number of Center

Employees

DGASPC SĂLAJ Centrul social cudestinaţiemultifuncţionalăCehuSilvaniei (CN) judetulSalaj


15 beds

DGASPC SĂLAJ Centrul social cudestinaţiemultifuncţionalăŞimleulSilvaniei (CN) judetulSalaj


10 beds

264 164 103 50 30.48% 52

60

Subcomponent 3.3: Victims of domestic violence program (€3.1 million). This subcomponent was designed to support the development of specialized integrated social services. Victims of domestic violence could have access to these services for a limited period of time, until their family situation was solved so that they could return to normal life. It would finance subprojects submitted by local authorities for rehabilitation and/or construction of housing infrastructure to shelter victims (and their dependents) and endowment with furniture and household appliances. It would also finance professional training for staff that would provide counseling services, the development of a specific MIS, and the implementation of public awareness campaigns. Component 3.3 of the project financed the construction of four shelters for victims of domestic violence, and social welfare units that provide protection, accommodation, care, and counseling for victims of domestic violence. The centers are in Bucharest and in three counties. The maximum stay at the shelters is between 60 and 180 days, with the possibility of extension. The centers are staffed to provide social, psychological, legal, and occupational counseling, as well as medical care and counseling. Details are provided below:

61

Shelters for victims of domestic

violence

Opening Date

Center Capacity

Number of Beneficiaries Service Types

Social Counseling

Psychological Counseling

Legal Counseling

Medical Care

Occupational Counseling

Centrul de servicii integrate pentruvictimileviolenţei in familie, Iaşi, judetulIaşi

08/01/13 12 places

18 female victims and

32 minors (4-16 years)

50 32 18

- 20 medical counseling

- 18 medical evaluation

- 4; - 1 victim has been employed

Centruldestinatvictimelorviolenţeiînfamilie, Bucureşti, sect. 3

01/20/14 12 places

19 female victims and 25 minors

(15 girls and 10 boys)

44 16 19



- 5; - 1 person was employed

DAS Targoviste - Centrul social pt. prevenireaşicombatereaviolenţeidomestice " Împreunăvomreuşi "- judetulDambovita

12/20/12 12 places 42 female

victims and 38 minors

80 80 42



- 9; - 4 victims have been employed

DGASPC Suceava - CentrumultifuncţionaldestinatvictimelorviolenţeiînfamilieGuraHumorului, judetulSuceava

2012 12 places 12 female

victims and 23 minors

35 35 12

-1 protection order



-3

209 victims 209 163 91 64 21

62

In addition, under Component 3.3, the project financed: An awareness raising campaign to prevent and fight against domestic violence.

The aim of the campaign was to increase the involvement of the members of the community, as well as to raise awareness among local authorities with regard to the importance of their participation in the prevention and fight against this phenomenon, mobilizing all the existing resources, and raising awareness among the public regarding the prevention and fight against domestic violence, in order to change the way in which ‘the community sees this phenomenon’. Activities carried out included:

o Preparing and organizing the launch conference in Bucharest in November 2012

o Preparing and organizing seven regional conferences, which were attended by 50 participants each:

- regional conference Constanţa on November 9, 2012 - regional conference Iaşi on November 15, 2012 - regional conference Târgovişte on November 19, 2012 - regional conference Timişoara on November 22, 2012 - regional conference Craiova on November 26, 2012 - regional conference Sibiu on November 29, 2012 - regional conference ClujNapoca on December 6, 2012

o Preparing and organizing the closing conference in Bucharest on January 28, 2013, which was attended by 80 participants

A survey of public attitudes and public awareness regarding domestic violence, an evaluation of the existing information at the level of institutional actors, and an evaluation of the social services and existing protection mechanisms for domestic violence victims and aggressors—between November 2012 and January 2013

35 street events—caravan type, from November to December 2012, including preparation of the caravan (preparing mobile units and promotional materials for each route, preparing the request and obtaining authorizations for all 35 street events), and deployment of the caravan (sending information to the media and the main stakeholders regarding the deployment of the caravan, organizing training sessions for the personnel taking part in the event, and deployment of the 35 street events in three different routes).

An opinion poll to find out people’s attitudes toward and awareness of the family violence phenomenon, broadcasting of TV and radio spots ‘Good night, Mommy!’, distribution of flyers, posters (A1 - 10 pieces, A2 - 1,410 pieces, A3 - 17,390 pieces, A4 - 16,470 pieces), stickers - 87,600 pieces, and other promotional materials (balloons - 2,000 pieces, push button pens - 4,100 pieces, caps - 200 pieces, paper cases - 4,100 pieces, A5 notebooks - 4,100 pieces, 1 banner, 2 canvas roll-ups, and one flag). The poll was targeted at children and adolescents, parents, professionals working with children, decision makers, public authorities and civil servants, civil society organizations, public at large, victims of domestic violence, and family offenders.

An additional six street events in Bucharest, where the public was informed about the family violence phenomenon. There was active participation from the media participation for all street events (video and radio spots regarding the campaign).

63

Subcomponent 3.4, social assistance cash transfer program supported the implementation of the SASMP (results-based financing). This subcomponent provided: ‘Support to MLFSP and other relevant governmental institutions in its subordination and/or coordination to increase efficiency in the administration and monitoring of social assistance benefits through, inter alia,: (i) upgrading of the MIS for the NASB with modules for family support allowances and GMI; (ii) harmonization of means-testing procedures for the eligibility of social assistance benefits; (iii) harmonization of disability assessment criteria and the institutional framework for disability benefits and invalidity pensions; (iv) development of institutional capacity and tools to prevent and combat error and fraud; (v) promotion of public information campaigns to present the proposed reforms for the efficiency and targeting of the social assistance system, and (vi) implementation of M&E activities to assess: (a) the incidence of social assistance transfers and their impacts; and (b) the savings gained from the consolidation of social assistance benefits and simplification of social assistance benefits administration”. Component 3.4 of the Project financed: (i) TA for fulfillment of the Disbursement Linked Indicators (DLI) agreed in the Social Assistance System Modernization Project, and (ii) the upgrading the Management Information System (SAFIR, MIS) for the National Agency for Payments and Social Inspection (NAPSI former NASB) with modules for the family support allowance and the Guaranteed Minimum Income (GMI), and with any other new modules for the social assistance benefits paid through this system, as well as improvement of the analysis and reporting capacity and performances of this system according to the specific needs of the sector. The assistance provided under the Project includes those activities presented below:

ACTION IBRD (Euro) GOR(Euro)

Support for the development of the action plan of the social assistance strategy (DLI 1) 4.335,00 765,00

Development of a statistical bulletin to track the distributional impact of social protection spending, including supplementary data collection from Family Budget Survey (DLI 7) 47.948,10 8.461,43

Upgrade SAFIR (3): (SW and HW) to incorporate (i) new rules for GMI, Family Support benefits, Child Raising Benefits; (ii) Supplementary services to include EO 124/2011; (iii) Improvements in functioning of the operational and decisional modules, and ”on site” support 4.914.365,88 867.240,80

Assess the capacity of SAFIR to cross-check information with other public databases; (DLI 14) 8.415,00 1.485,00

Legal TA for the development of a new sanction policy and investigative powers for Social 12.719,40 2.244,60

64

Inspections (DLI 17)

Capacity building (IT and office equipment, training, study tours) for:

(i) MoLFPS and ANPIS staff training in ORACLE 14.922,13 2.633,32

(ii) MoLFPS staff training – WB Courses on Safety Nets 14.607,78 2.577,84

(iii) ANPIS Social Inspection staff training (two training sessions in the UK) 13.975,37 2.466,25

(iv) Social Inspection staff training on combating fraud and errors 37.121,45 6.550,84 (v) Translation services for delivery of Social Inspection staff training on combating fraud and errors 4.588,61 808,75

(VI) Office and IT equipment for Social Assistance General Directorate 11.501,52 2.029,68

Consulting services for the development of the Communication Strategy. Preparation of Terms of Reference for the preparation of key components of the Strategy, and Monitoring and Evaluation of the results of the Strategy for the Social Assistance System Modernization Project in Romania. 101.287,29 17.874,23

TOTAL 5.185.787,54 915.137,74

Component 4: Capacity Building for Roma Social Inclusion (€0.8 Million)

This component aimed to provide assistance to the NAR and its eight regional branches to coordinate activities aimed at improving inclusive planning, M&E and identification, preparation and implementation of programs to be financed by EU and other funds, through: (i) provision of training to NAR staff, Roma NGOs, local authorities, and Roma community initiative groups; (ii) the preparation and implementation of an information, education, and communication strategy; and (iii) strengthening of the M&E system. Very few activities were implemented by NAR under Component 4, and most of the resources were reallocated to other components in the project’s restructuring. The activities financed included the following:

65

Consultants’ Services Amount (€, including VAT)

Consultancy services to assess the capacity of Roma National Agency (RNA) in coordinating its own activities, including management of EU funds

24,791.32

Pre-intervention evaluation study of the ‘inclusive early education’ 20,000.00

IT consultancy services for development of the institutional capacity of NAR 15,000.00

Information technology equipment 26,164.00

Furniture 1,361.00

Office equipment 9,972.00

66

Annex 3. Economic and Financial Analysis

See Section 3.3

67

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Lending Raluca Marina Banioti Program Assistant ECCRO

ECCU5 GGODR GSPDR GHNDR GGODR GEDDR EASPR - HIS GSPDR ECSHD - HIS ECSHD - HIS GSURR

Christian Bodewig Program Leader Bogdan Constantin Constantinescu Sr Financial Management Specialist

Richard Florescu Senior Operations Officer Kari L. Hurt Senior Operations Officer Vladislav Krasikov Senior Procurement Specialist Mariana Doina Moarcas Senior Operations Officer Daniel M. Mont Senior Poverty Specialist Lucian Bucur Pop Senior Economist Silviu Calin Radulescu Sr Health Spec. Ana Maria Sandi Lead Education Specialist Mark C. Woodward Lead Social Development Specialist

Supervision/ICR Suzana Abbott Consultant GHNDR

ECSO2 - HIS ECCRO ECCRO GGODR GSPDR ECCRO GGODR GEDDR GENDR GSPDR GSPDR ECSHD - HIS ECSHD - HIS

Nurul Alam Senior Procurement Specialist Corina Alexandrescu Senior Program Assistant Alexandra Colin ET Temporary Bogdan Constantin Constantinescu Sr Financial Management Specialist Ugo Gentillini Sr Social Protection Specialist Camelia Iulia Gusescu Program Assistant Vladislav Krasikov Senior Procurement Specialist Mariana Doina Moarcas Senior Operations Officer Cesar Niculescu Environmental Specialist Isadora Nouel Program Assistant Lucian Bucur Pop Senior Economist Ana Maria Sandi Lead Education Specialist

Dan Ioan Sava E T Consultant

68

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands

(including travel and consultant costs)

Lending

FY05 37.81 61.34 FY06 87.83 176.10 FY07 0 0

Total: 237.44 Supervision/ICR

FY07 16.82 35.45 FY08 21.07 50.28 FY09 19.66 49.11 FY10 19.90 48.42 FY11 23.26 61.11 FY12 25.36 80.84 FY13 14.62 48.15 FY14 8.62 31.08 FY15 9.54 52.41

Total: 133.49 456.85

69

Annex 5. Beneficiary Survey Results

(if any) N/A.

70

Annex 6. Stakeholder Workshop Report and Results

N/A.

71

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

SOCIAL INCLUSION PROJECT

Loan number 4825

BORROWER’S CONTRIBUTION TO THE IMPLEMENTATION COMPLETION REPORT

The Project

The objective of the Project was to improve the living conditions and the social inclusion of the most disadvantaged and vulnerable people in Romania.

The Project’s main direct beneficiaries were to include: (i) the most disadvantaged Roma that lived in 100 (later, 16 after the Project’s restructuring) poor settlements who would be provided with improved living conditions and efforts aimed at promoting social inclusion; (ii) (ii) Roma children aged 0-6, who would benefit from ECE through mainstreaming into the MER’s regular policies and programs; (iii) (iii) PWD with a high dependency on residential services who would be provided with better quality care and increased access to community-based services; (iv) (iv) Youth aged 18 years and above graduating from the institutionalized child care system (who would no longer be eligible for state support) and who would be provided with social services aimed at increasing their readiness for independent living; and (v) (v) Victims of domestic violence (and their dependents) who would benefit from specialized, integrated social services until their family situation was resolved. Project beneficiaries also included the institutions of social protection responsible for implementation of different activities, including the Romanian Social Development Fund (RSDF), the MER, the Ministry of Labor, Social Solidarity and Family (MLSSF), the National Authority for Persons with Handicap (NAPH) and the National Agency for Roma (NAR), and of new Community-Based Services (CBS), that would benefit from training, technical assistance and miscellaneous services to support the strengthening of their service delivery and of their monitoring and evaluation capabilities.

The responsibilities of implementation of project parts were distributed as follows:

Part I - Romanian Social Development Fund

Part II – Ministry of Education

72

Part III – Ministry of Labor, Social Solidarity and Family

Part IV – National Agency for Roma The original Parts of the project and the supported activities were:

Part I: Priority Interventions Program

Support to Romanian Social Development Fund (RSDF) to improve living conditions and social inclusion for Roma living in about one hundred (100) poor settlements through: (i) the financing of Priority Interventions Sub-projects awarded on a competitive basis; (ii) the carrying out of promotion, information and education campaigns and monitoring evaluation activities, and the provision of training for local authorities, community groups and NGOs; (iii) the provision of assistance in capacity building at community level through preparation, facilitation, evaluation and supervision of Priority Intervention Sub-projects and provision of training in relation thereto, including training to RSDF; and (iv) the provision of technical assistance to poor Roma communities to prepare projects to be financed through EU funds.

Part II: Inclusive Early Childhood Education Program

Support to MER to mainstream Roma into the regular policies ano1 programs of MER through: (i) the construction, extension, rehabilitation and furnishing of ECE infrastructure in communities with a high percentage of Roma; (ii) the development of an inclusive ECE curriculum, the provision of training to ECE staff, the development and distribution of ECE teaching and learning materials, and the provision of technical assistance to entities entitled to submit financing applications for projects to be financed through EU structural funds; (iii) the promotion of integrated services and alternative community-based Solutions for ECE, including the development of coherent ECE legislation, and the provision of training and counseling programs for parents; and (iv) the carrying out of monitoring and evaluation and information, education and communication activities.

Part III: Social Assistance Programs

(47) Disabilities Program

Support to MLSSF and NAPH to improve the quality of care in residential services and to increase the access to community-based services for persons with disabilities in need through: (i) the financing of Disabilities Sub-projects awarded on a competitive basis; (iii) the provision of training to professional staff that will implement new standards of care; (iii) the development of the monitoring and evaluation system for NAPH; and (iv) the provision of occupational counseling services.

2. Youth at Risk Program

Support to MLSSF to enhance the social integration of youth aged eighteen (18) and more graduating from the institutionalized child care system, through: (i) the financing of Youth at Risk Sub-projects awarded on a competitive basis; (ii) the carrying out of monitoring and evaluation activities, and the development of specific methodologies; (iii) the provision of training to the staff of the multifunctional centers to

73

be established under the Youth at Risk Sub-projects, and to the MLSSF staff involved in supervising the functioning of such multifunctional centers; and (iv) the carrying out of public awareness activities.

3. Victims of Domestic Violence Program

Support to MLSSF and NAFP to develop specialized integrated social services for victims of domestic violence, through: (i) the financing of Domestic Violence •Sub-projects awarded on a competitive basis; (ii) the provision of training to the staff of centers for victims of domestic violence and to NAFP staff; (iii) the carrying out of monitoring and evaluation activities, the development of specific methodologies and the development of a specific management information system regarding victims and aggressors to be maintained by NAFP; and (iv) the carrying out of public awareness activities.

Part IV: Capacity Building for Roma Social inclusion

Support to NAR and its eight (8) regional branches to coordinate activities aimed at improving inclusive planning, monitoring and evaluation and identification, preparation and implementation of programs to be financed through EU and other funds, through: (i) the provision of training to NAR staff, Roma NGOs, local authorities and Roma community initiative groups; (ii) the preparation and implementation of an information, education and communication strategy; and (iii) the strengthening of the monitoring and evaluation system.

The initial allocation of the amounts of the Loan to each Category was, according to the Loan Agreement ratified by Law no. 40/2007:

Category Amount of the

Loan Allocated (Expressed in EUR)

Goods, Works, Consultants’ services (including audits), Training and grants for Part I of the Project

11,700,000

Goods, Works, Consultants’ services (including audits), and Training for Part II of the Project

6,100,000

Goods, Works, Consultants’ services (including audits), Training, Incremental Operating Costs, Disabilities Sub-projects, Youth at Risk Sub-projects and Domestic Violence Sub-projects for Part III

28,600,000

Goods, Works, Consultants’ services, Training and Incremental Operating Costs for Part IV the Project

800,000

OTOTAL AMOUNT 47.200.000

After project restructuring to reflect the evolutions in the implementation and to better match the needs of the modernization of the social assistance system the following modifications were included in the Loan Agreement through two sets of amendments:

74

Amendment 1 ratified by Governmental Decision no.12/2011

• A new objective was added: (ii) increase the efficiency of the administration of social assistance benefits.” • Part I was amended in order to increase the number of poor settlements from one hundred (100) to one hundred and twenty five (125). • In Part III, a new sub-paragraph 4 - Social Assistance Cash Transfer Program was added “Support to MLFSP and other relevant governmental institutions in its subordination and/or coordination to increase efficiency in the administration and monitoring of the social assistance benefits through: (i) upgrading of the Management Information System (MIS) for the National Agency for Social Benefits (NASB) with modules for the family support allowances and the Guaranteed Minimum Income (GMI); (ii) harmonization of the means-testing procedures for the eligibility of social assistance benefits; (iii) harmonization of disability assessment criteria and of the institutional framework for disability benefits and invalidity pensions; (iv) developing capacity to prevent and combat error and fraud; and (v) promoting public information campaigns to present the proposed reforms for the efficiency and targeting of the social assistance system.” • The allocation of the amounts of the Loan to each Category was modified as follows:

Category Amount of the Loan

Allocated (expressed in

EUR) (1) Goods, works, consultants’ services, Training, Incremental Operating Costs and grants for Part I of the Project as identified in the Annual Development Plans for such Part I of the Project

14,700,000

(2) Goods, works, consultants’ services, Training and Incremental Operating Costs for Part II of the Project as identified in the Annual Development Plans for such Part II of the Project

6,100,000

(3) Goods, works, consultants’ services, Training, Incremental Operating Costs and grants for Part III of the Project as identified in the Annual Development Plans for such Part III of the Project

26,200,000

(4) Goods, works, consultants’ services, Training and Incremental Operating Costs for Part IV the Project as identified in the Annual Development Plans for such Part IV of the Project

200,000

TOTAL AMOUNT 47,200,000

Amendment 2 ratified by Governmental Decision no. 30/2013

75

• The letter (i) of new sub-paragraph 4 - Social Assistance Cash Transfer

Program modified to read as follows :

“ (i) upgrading the Management Information System (MIS) for the National Agency for Payments and Social Inspection (NAPSI former NASB) with modules for the family support allowance and the Guaranteed Minimum Income (GMI), and with any other new modules for the social assistance benefits paid through this system, as well as improvement of the analysis and reporting capacity and performances of this system according to the specific needs of the sector”;

• The closing date was extended till July 1, 2014. During Project implementation, RSDF, MER, MLSSF maintained adequate management structures – Project Management Units (PMU), MLSSF PMU providing the project management services for Part III and Part IV Operational Manuals were developed and agreed with the Bank for implementation of each part of the Project

Project implementation

The responsibilities of implementation of project parts were distributed as follows:

Part I - Romanian Social Development Fund

Part II – Ministry of Education

Part III – Ministry of Labor, Social Solidarity and Family

Part IV – National Agency for Roma

The main achievements in implementation of the project parts are presented below as they were notified by the implementation entities

Part I: Priority Interventions Program

Priority Intervention Programme aimed at improving the minimum living conditions and increasing the level of participation to the local decision making process of people living in poor Roma settlements, through promoting social inclusion. The programme provided to Roma people living in poor settlements from rural and urban areas, the opportunity to improve their abilities to solve problems, through a „learning by doing”

76

process, by identifying their needs and articulating priorities, by participating to the design, implementation and monitoring of subprojects. PIP consisted of a grant scheme designed to finance projects initiated by local communities and to provide support (technical assistance and training) for the beneficiaries, in order to build the Roma communities’ capacity to contribute at local development. The main expected outcomes of the programme was to improve living conditions and social inclusion for Roma living in 133 poor settlements, to upgrade these disadvantaged Roma settlements up to minimum standards in terms of infrastructure and basic services. Under the SIP, the following activities were supported: (i) financing of grants for demand-driven, integrated community development subprojects, targeting poor Roma settlements from a list of priorities provided by the National Agency for Roma; (ii) carrying out of information and education campaigns, and the provision of training for local authorities, community groups and NGOs; (iii) provision of assistance in capacity building at community level through preparation, facilitation, evaluation and supervision of Priority Intervention subprojects and provision of related training; (iv) provision of technical assistance to poor Roma communities to prepare projects to be financed through EU structural funds and (v) carrying out of monitoring and evaluation, promotion and communication, networking and training activities for the project implementation. The stage of achieving the outcomes at the end of the programme implementation is the following: Indicator Target Achieved The percentage by which the difference between the Roma communities included in the program and the neighboring communities has reduced, with regard to the living conditions, further to carrying out the program

20% 61,77%

The percentage by which the difference between the Roma communities included in the program and the neighboring communities has reduced, with regard to the minimum living conditions, further to carrying out the program

70% 73,84%

The percentage of Roma communities included in the program that have access to improved infrastructure (ex: roads, water supply systems etc.)

60% 82,71%

The percentage of projects for whose elaboration RSDF offered specialized assistance and which proved eligible for EU funding 60% 100%

Percentage of members of the Roma communities included in the program who declare that the RSDF financing projects reflect the community’s priorities

60% 75,66%

Number of annual consultations between the members of the Roma communities included in the program and the local authorities aiming at meeting the community’s needs

3 4,93

Some anecdotal evidence supporting the quantitative accomplishments, e.g. beneficiaries’ perspectives as gathered during field visits, are present6ed below:

77

“I could say that Bora is still a success story as regard the measures which we, as an organization and people dedicated to a cause, take, in respect of the inclusion of people belonging to disadvantaged ethnic groups and social categories. What has been achieved here is a step forward towards the measures we will take next”, School Inspector for Roma Education Matters, Sandu Ion. “If so far we only addressed the material part and invested in the water, gas and road infrastructure, it is now time to also address the spiritual part. We hope that this is a small and first step towards emancipation, because many Roma children attend school, kindergarten, they have a medical office and this way we are trying to attract the largest possible number.” Mayor of Slobozia, Alexandru Stoica, at the inauguration of the center “We could show our appreciation for RSDF’s support, by keeping the area clean, by taking care of the asphalt, not destroying it, as we have to take care of them because they are for us and for our children.” Moldovan Călin Filip, volunteer, Petelea, Mureş County “UMTF Roma community has 169 members and a special program was developed for them, financed with RSDF funds. We invested in the improvement of their health conditions and in their integration in the Romanian community. As I was saying, the problems that traditional copper pot makers who live here encounter are many and very diversified... Therefore, we conceived this integrated project which did not imply only the construction of a public bath, but also the education of Roma on matters of health and social counseling. These are approaches addressed to children, but also to adults, and they are carried out here, at the public bath, as well as in the Roma homes.” Elena Coşofreţ, project coordinator Vânători Neamt, Neamţ County “I couldn’t say that we don’t know how to wash, to be clean, but we don’t have the means. Now we have a public bath in our community, with a washroom and a drier. We are thankful for the understanding. We are not educated enough to understand all problems. But thanks to the understanding and patience of the project facilitators, a good thing was done for the Roma of this community.” Codrea Lăcătuş, Leader of the Roma community of Vânători Neamt, Neamţ County “All persons present agreed that the project has achieved its objectives. The two matters proposed for settlement were entirely resolved. The access road to the community is now practicable on its entire length and the day care center is and will be a permanent support for the families in the community and their children who face problems at school. The students have improved their school records and they have widened their knowledge horizon”. (focus group at the end of the implementation, Brateiu, Sibiu) “The members of the community are satisfied with the results of the project. Both with the road rehabilitation and with the after school services. As dissatisfaction – they said all children should benefit from after school services.” (focus group at the end of the implementation, FNC Livadă community, Călăraşi)

78

“The community can become involved in activities that improve their life. They became more confident and they understood that if they become involved and they collaborate, they can achieve their objectives. Roma people can collaborate with significant persons from the city hall management and their behavior during the implementation of the project (voluntary work, the LMU’s activity- local monitoring unit) has reassured APL that they can rely on Roma people. The leaders appointed by the Roma community have proved that they are reliable and have done their duty, rewarding the community’s trust. Roma may also learn to carry out monitoring activities, even prepare documents if they are trained and supported”. (focus group at the end of the implementation, Brateiu, Sibiu) “Further to the involvement in the project, we learned to be confident that good things can be done for the community, we learned to try and do something for the community, to request assistance from the city hall, to access other funds”. (focus group at the end of the implementation, FNC Livadă community, Călăraşi) “As dissatisfaction, we could mention that the bitumen layer poured on the road does not seem durable”. (focus group at the end of the implementation, Brateiu, Sibiu) “To a great extent, the community members are satisfied with how the project was implemented by the Coordination and APL Group. As reasons of satisfaction, we could mention: the pouring of asphalt on the access road to the community and the construction of the social group. As reasons of dissatisfaction, we could mention: the fact that asphalt was not poured on the road and alleys inside the community and the fact that, although the social group is constructed and equipped, it is still not functional”. (focus group at the end of the implementation, Brad, Hunedoara) The evaluation performed by the Local Monitoring Units of the project outcomes, upon their conclusion, has revealed the satisfaction of most members of the Roma community, both with regard to achieving the projects’ objectives, and with regard to how they were implemented, although they noted that they only solved part of the problems faced by these communities and they expressed the desire for their other needs, deemed equally urgent, to be resolved as well in the future. The impossibility to perform more extensive or better quality (infrastructure) investment works, due to the limited financial resources supplied by the project was also raised by the representatives of the Roma communities.

• Outputs financed by the Loan: o Number and small description of Sub-Projects financed (broken down by

type, focus, purpose, capacity) In total, within the PIP 133 integrated projects were financed, of which: - 90 had a community road rehabilitation component - 30 had a utility access component (water supply, sewage network, including street sewage, public baths/washrooms)

79

- 62 had a social services component (day care/educational centers, vocational training/professional counseling services, information and counseling services, health campaigns) - 17 had a medical services component (establishment of medical office/cabinet) - 9 had a housing component (rehabilitation of houses, apartment buildings, flood protection works, public toilets) - 21 had a community center component (cultural and educational centers, art and sports clubs, music and dance clubs/bands) - 21 had an environmental protection component

o Number of beneficiaries that have benefitted from Projects/Sub-Projects, broken down by gender, and by disadvantaged groups.

The total number of beneficiaries was approximately 58,800, of which approximately 19,500 women.

o Other activities financed (e.g., MIS, information campaigns, training, studies, consultations etc.). For example, 2 trainings on social counseling for victims of domestic violence for X staff of a given center X; 500 experts participating in a conference, etc.

- 4 public information campaigns, conducted in the target communities, one for

each call: 214 communities informed - 18 regional information seminars, attended by 647 people - 23 training sessions for the members of the Project Coordination Committees

(inclosing representatives of the initiative groups): 991 participants - 3 experience sharing networks between beneficiaries – 12 network meetings

organized

o Other activities financed, with description of activities and beneficiaries

- 171 eligible Roma communities received community facilitation services

• Prospects for Sustainability:

Responsibility for ensuring the maintenance of the objectives resulting from the implementation of projects funded by PIP and for continuing to provide social services created by the project lies entirely with the Management Agent who implemented the project. In this regard, since the submission of projects, Management Agents were required to submit a Plan for maintenance of integrated project after PIP funding ends, and a Local Council decision which are assumed to continue financing the maintenance

80

of investments financed by project (for infrastructure works) and the services provided to beneficiaries in the project (in the case of social services created by the project).

• Prospects for Continued Funding of activities and new Sub-Projects FRDS promoted participatory approach, integrated projects for poor communities and best practices drawn from the Priority Intervention Program to other donors. Thus, Financial Mechanism of the European Economic Area (EEA) is funding now a similar component which focuses on increasing access to education in the Roma communities (targeting about 37 communities).

• Major factors that affected implementation of Component/Sub-Component - the lack of preparation of the communities included in the pilot stage for the intervention provided under PIP. The conditions stipulated in the PIP Operation Manual, based on which ANR followed to select the pilot communities have not been fulfilled but very little. Therefore, on the date of RSDF intervention, with just one exception, the beneficiaries in the 10 eligible communities for the pilot stage have not had established their priorities, had not identified at least one project idea, respectively there were no conditions for the mobilization of the resources (contribution, people). At the same time, no community had, at least under preparation, the technical documentation required for the submission of a project under PIP; - lack of ownership documents for the lands and houses of the Roma community members, which made it impossible to satisfy within PIP the needs designated as priority by the majority of the eligible community members (rehabilitation of the houses or their connection to the power supply network), which lead to the necessity of extending the period for establishing the community priorities, long after the originally estimated term; - the location of some eligible Roma communities in floodable areas, with no access to road infrastructure or drinking water resources, on private or claimed lands, which made it impossible to design investment works for the community infrastructure. That is why, in such cases, only integrated projects with exclusively social services components could be submitted; - lack of investment funds at the local budgets level, which entailed the allocation of more important time resources in the community facilitation process, in order to identify alternative financing sources to cover the costs for the technical documentation required for submitting the projects; - reduced capacity for target communities mobilization (including the demand of those involved in the initiative groups to be remunerated for the work performed for the project), the process of identification of priority problems by the communities requiring a longer period of time than the one estimated by RSDF / allocated for this process in other programmes previously developed by RSDF;

81

- reduced cohesion within the target communities, which often generated difficulties in gathering a unitary group to get involved in the PIP projects (communities consisting of several different Roma groups / varieties); - mistrust of the community members in the successful development of a project to improve their living conditions due to the multitude of previous promises (in particular during election periods) which were not kept; - desire of the local actors (local authorities, political parties, NGOs, etc.) to exploit the PIP projects for electoral and own image purposes, which may generate mistrust and confusion among the target communities during the projects preparation and implementation period; - lack of perceptiveness of some local authorities regarding the accomplishment of some investments in the Roma communities; - lack of resources at local level (financial, human, etc.), which makes it difficult to assure the preparation of viable and sustainable projects; - reduced expertise level of some of the beneficiaries / Management Agents in the field of project management, which lead to delays in the projects progress and procedures infringement tendency; - the insufficient funds made available for RSDF for PIP implementation during the last 3 years, which entailed the slowing down of the projects implementation rhythm and accumulation, in time, of delays in the payment of instalments to the projects. This fact made RSDF to move the focus of its activities from supporting the progress at high rhythm of the projects under financing to their controlled slowing down, to the mediation of the relationships between the Management Agents responsible for the implementation of the projects and the companies employed to accomplish the projects objectives (for which the delayed payments for the accomplished works represented a reason for tensions and dissatisfaction), as well as to maintaining a certain level of trust among the beneficiaries, dissatisfied, in their turn, by not finalizing in time the activities proposed by the project, by the delayed payment regarding the services, delivery of goods and, in particular, the payment for the construction works execution; - the existence of legal provisions which restricted the personnel employment by the local authorities during the last years, which imposed the identification, together with the Management Agents, of other legal alternatives for contracting members in the financed projects implementation teams, respectively for taking over the costs for the salaries of the experts employed in PIP projects by the local authorities after finalizing the projects; - the reduced interest of the beneficiaries/ Management Agents in accessing the technical assistance services supplied by RSDF in view of helping them to also attract other financings intended for supporting the local/ community development process (in particular EU funds), reduced interest due to inclusively the limited financing opportunities existing during this period, in particular for poor rural communities, respectively the restrictive financing conditions and the reduced capacity of the Management Agents to fulfil them (in the absence of human resources with expertise in the implementation of EU projects and of the financial resources to assure the required cash flow, in the conditions of massive delays in assuring the financing/ reimbursements of the expenses). The lack of expertise in this field of the Management Agents could also be noticed from the low quality of many of the technical assistance applications

82

submitted to RSDF. The low information level of the local actors regarding the existing financing opportunities and the conditions requested by the financers made the majority of the technical assistance applications received by RSDF to be either non-eligible or insufficiently substantiated. The quick exhaustion of PNDR funds as well as the absence/ permanent postponement of opening new financing rounds under POSDRU for grant projects – the most attractive and accessible financing sources for local communities eligible under PIP – represented other reasons that explain the low number of technical assistance applications received by RSDF and their low quality. - the reduced involvement of the representatives of some of the financed Roma communities in the projects monitoring and participating evaluation activities, due to the lack of expertise, to the low level of education and of social capital, to the lack of time or long periods of absence of the Local Monitoring Units members from the community; - the limited financial resources supplied by the project (the maximum grant value under PIP) and the local financial capacity for maintaining the objectives did not allow the accomplishment of major investment works (in infrastructure) or of a higher quality, which lead to the discontent of some Roma communities representatives (several of the problems they are confronting were not solved, impossibility of adopting more complex or more expensive technical solutions, with consequences on the level of satisfaction of the community members regarding the quality of the performed works, etc.).

• Any views on the preparation/implementation support provided by the World Bank

- consultation meetings with the involved actors have been organized; - the World Bank team had an important contribution in establishing the project design

and implementation conditions, by continuous consultation with RSDF; - during the implementation, RSDF received support for monitoring the process, for

identifying the problems and finding the solutions to solve them.

• Aspects (design, procedural, strategic, etc.) worth replicating (positive lessons learned from implementation)

- supply of community facilitation services, including during the projects implementation period, in particular for supporting the local monitoring and evaluation activities, and during post-implementation for supporting the steps to assure the sustainability of the project outputs; - provision of support to the potential beneficiaries for preparing the projects, and in particular for preparing the budgets and the social component design; - allowing a sufficiently long period of time for the projects preparation (minimum 3 months) and for the projects implementation (minimum 2-3 years); - provision of technical support / verification during the implementation; - involvement, with priority, of the local Roma NGOs in the projects, as they are aware of the situation and as this does not imply high travel / personnel costs or just good practice transfer from other NGOs situated far away – cost effectiveness; - involvement of Roma community members in the management / implementation teams, if they have the required training and expertise;

83

- maintaining a relative autonomy of the implementing entity, respectively as concerns the financing and payment approvals – the same financial mechanism applied until now within the programmes financed by the World Bank, based on RSDF’s responsibility, without additional reporting and approval levels; - utilization of the World Bank procedures as concerns the procurement for RSDF, respectively the community procurement for the implementing entities; - involvement of the community members in performing the activities and their step-by-step information regarding the development of PIP projects, action which generated benefits, such as: awareness of the importance to participate, even if volunteer, in activities of common interest and, in particular, awareness of the necessity to contribute to maintaining the projects outputs; - training of the beneficiaries and of the specialists involved in their support, in view of implementing the projects in proper conditions; - facilitation of collaboration relations development, of experience exchange and good practice transfer between/ for the beneficiaries. The experience exchange and partnership establishment activities, grouped under the general name of “networking” activities were intended for the information and experience exchange between the actors involved in the financed projects implementation and having similar mission and interests, with the purpose of increasing their capacity to assure the good functioning and sustainability of the projects; - sustaining a participating monitoring process, aimed at involving the community members in the close monitoring of the implementation process and in the project outputs evaluation. The evaluation of the participating monitoring and evaluation activities – performed by RSDF facilitators together with ULM members and members of the Projects Coordination Groups – in the middle of the implementation period, represented a good opportunity to identify, together with the Roma communities representatives, all the problems occurred during the projects implementation and participating monitoring and evaluation activities performance, respectively to find the best solutions for overcoming these problems and finalizing the projects in good conditions.

• Aspects (design, procedural, strategic, etc.) that should be adjusted/refocused before replicating or continuing with activities

- the necessity of revising certain targets of the Programme. For instance, the

performance indicator regarding “the percentage of Roma communities members (who benefited from financing under PIP) who declare that their minimum living conditions have improved as a result of the Programme implementation”, for which the target of 70% was established, proved to be difficult to document / achieve, as the general perception of the Roma communities members on the general living conditions in the community is difficult to change on a short term, and only by means of individual interventions aiming a small part of the (complex) needs of a poor Roma community;

- the analysis of the possibility to give up the lists of eligible Roma communities supplied by ANR and allowing the access of all eligible communities to the resources provided by the Programme, in conditions of open competition;

84

- refocusing the intervention on poor multiethnic communities, with a high/ relevant percentage of Roma ethnics, in order to avoid possible tensions/ conflicts at local level, to raise the interest of local authorities in developing such projects and to better facilitate the organization of intercultural activities under the projects, with the majority population/ other ethnicities in order to avoid the possible interpretations concerning the segregation of the Roma community members;

- involvement of the initiative groups both in training activities and in networking activities with the financing beneficiaries.

- increase the maximum level of grant / funding for a project, to be able to cover/ solve more of the community needs and at a higher level of quality

At its closing date, of the 17.86 million euro allocated to the program (of which, 13,7 million euro, representing 76.71%, were external funds) were actually spent approximately 16.94 million euro (94.85%), of which approx.13 million euro (76.75%) representing external funds. Out of 16.94 million euro, 14.15 million euro(83.56%) has been allocated for projects, and 1.05 million euro(6.31%) for activities aimed at increasing community capacity to contribute to local development(technical assistance – information, community facilitation, project evaluation and supervision, consultancy for other projects development - and training for the representatives of the beneficiary communities). Additionally, the operational costs, goods and training for RSDF accounted 1.72 million euro (10,22% of total expenditure). Regarding project implementationcosts,11.55 million euro out of total14.15 million euro (representingapprox.81.6%) were allocated for construction works for improving the livelihood in Roma communities (rehabilitation of roads, water supplies, building educational/community canters etc.). Overall, proportionally, the utilization of the funds allocated to project implementation has the following structure:

Construction works

Goods and materials

Training /Consulting

Operational costs(related to social services)

Project management costs

81.6% 6.6% 1.1% 5.7% 5.1% Of activities aimed at increasing the capacity of Roma communities to contribute to local development, the community facilitation-focusing mainly on consulting community members in various stages of preparation and implementation of the project-assumed total expenditure of approx.166,000 euro, of whichapprox.75,800 euro has been allocated for community facilitation specific to the implementation phase of projects, including participatory monitoring at the level of the community.

Part II: Inclusive Early Childhood Education Program

85

Support to MER to mainstream Roma into the regular policies ano1 programs of MER through: (i) the construction, extension, rehabilitation and furnishing of ECE infrastructure in communities with a high percentage of Roma; (ii) the development of an inclusive ECE curriculum, the provision of training to ECE staff, the development and distribution of ECE teaching and learning materials, and the provision of technical assistance to entities entitled to submit financing applications for projects to be financed through EU structural funds; (iii) the promotion of integrated services and alternative community-based Solutions for ECE, including the development of coherent ECE legislation, and the provision of training and counseling programs for parents; and (iv) the carrying out of monitoring and evaluation and information, education and communication activities.

Rehabilitation of ECE infrastructure The program supported building and furnishing of nineteen (19) new buildings for preschool units from the list, creating facilities to enroll more children in the areas where there was a large percentage of Roma children with age between 0-6. At the same time, the project supported the rehabilitation and furnishing of approximately eight (8) existing buildings, to improve the quality of education and facilitate access of more students coming from disadvantaged communities with a majority of Roma population.

No. County School unit

Rehabilitation solution


No of enrolled children

No of disadvantaged children

1 Arad Kindergarten half time sat Vanatori, comuna Misca D + R 4 93 84

2 Arges Kindergarten half time sat Pojorata, comuna Leresti CN 2 30 30

3 Botosani Kindergarten half time sat Buda, comuna Cosula D + R 2 39 29

4 Brasov Kindergarten half time sat Soars, comuna Soars D + R 3 22 0

5 Buzau Kindergarten half time nr. 6 Cartier Zidari, municipiul Ramnicu Sarat CN 2 29 29

6 Calarasi Kindergarten half time sat Sarulesti Gara, comuna Sarulesti CN 4 54 27

7 Dambovita

Kindergarten half time sat Iedera de Sus, comuna Iedera CN 2 36 33

8 Dolj Kindergarten half time sat Salcuta, comunaSalcuta CN 2 10 4

9 Galati Kindergarten half time nr. 4 sat Podoleni, comuna Barcea CN 3 55 42

10 Iasi Kindergarten half time sat Crucea, comuna Lungani CN 3 60 53

11 Maramu Kindergarten half time sat Chelinta, CN 3 45 45

86

res comuna Ulmeni

12 Mehedinti

Kindergarten half time sat Balta Verde, comuna Gogosu CN 2 40 35

13 Olt Kindergarten half time sat Chilii, comun Falcoiu D + R 4 64 48

14 Prahova Kindergarten half time sat Lunca Prahovei, comuna Magureni D + R 2 34 34

15 Salaj Kindergarten half time sat Dragu, comunaDragu CN 2 28 20

16 Satu Mare

Kindergarten half time sat Tataresti, comunaViile D+R 2 40 22

17 Suceava Kindergarten half time sat Bogata, comuna Baia CN 3 44 32

18 Valcea Kindergarten half time sat Balota, comuna Racovita D + R 2 23 23

19 Vaslui Kindergarten half time sat Puscasi, comuna Puscasi D + R 4 75 51

20 Bihor Kindergarten half time Marghita CN 2 55 15

21 Covasna Kindergarten half time Valea Seaca Sanzieni CN 2 47 23

22 Ilfov Kindergarten half time Ganeasa CN 1 25 0 23 Cluj Kindergarten half time Fizeşu Gherlii C + R + E 3 20 20 24 Alba Kindergarten half time Silivas Hopirta C + R + E 3 18 16 25 Mures Kindergarten half time Craciunesti C + R + E 2 16 11 26 Sibiu Kindergarten half time Dumbraveni C + R 3 36 30 27 Gorj Kindergarten half time Albeni C + R + E 2 22 13

Total 1060 769

NOTE:

CN : new construction (building) D+R : demolition & reconstruction RK : minor works concerning finishing jobs C+R : consolidation & rehabilitation E : extension for the existing building 1.Development of ECE curriculum, human resources and teaching/learning materials The development of the ECE curriculum National individual consultants have elaborated the ECE curriculum for children 0-3 years and revised the curriculum for children 3-6 years. They have worked with inspectors from the Ministry of National Education (MNE) and Department for Primary and Secondary Education, Department for Minority to develop the curriculum to be

87

integrated into the MNE policy and practice and to be accepted by education decision-makers and beneficiaries. The new ECE curriculum enhances the socio-emotional and physical health of children, and the continuity between early education curriculum for 0-3 years and 3-6 years. Since educational curriculum for children 0-3 has not been developed before, a Methodological guide for teaching and other staff has been provided in order to orient the specific activity and to organize the learning environment in crèches. Both ECE curriculum for 0-3 years and methodological guide are still awaiting formal approval, necessary for their legal application. A good practice guide for ECE of children from 3 to 6 years of age has also been developed by individual consultants and provided support for all teaching and non-teaching staff in kindergartens. Also, under this sub-component, the quality standards for teaching-learning materials and the Normative for Minimal Supplies in Kindergarten were updated and quality standards for ECE programs were developed through consultations with representatives from the Romanian Quality Assurance Agency for Pre-university Education, to be used as a grounding document for the development of other standards. The development of human resources (teaching and care services) An international consulting company, selected as a result of QCBS in 2009, ensured the technical assistance for training the trainers, coordinate and monitor the in-service training for upgrading skills of staff primarily working in project supported ECE units, develop the modules for ECE and the preparatory guide for all staff. In order to organize the training activities, the consulting company selected an appropriate number of persons, from the already qualified or experienced teachers/trainers to ensure the training for the targeted persons from the communities with building/rehabilitation interventions and also from other similar communities in proximity, lacking qualification and competence in ECE. They provided in-service training gathering people in clusters, presented and discussed the basic module for all the trainees and the specific ones for distinct groups including follow-up sessions. The clusters have been placed where resources allowed quality teaching environment and availability of education staff. The main aim of the training was to help the target population, represented by the ECE staff: educators, care personnel, nurses, principals from kindergartens and auxiliary personnel to upgrade their skills (teaching and care services) and also become familiar and apply inclusive approaches in all their activities. The training modules have been designed to raise awareness on disadvantaged groups and bring about an attitudinal change about Roma, to address issues related to social inclusion, and cultural sensitivity: Module 1: General Module – Concepts for early childhood education, focused on intercultural principles Module 2: Interactive Methods Module – Practices in ECE services Module 3: Educational Management Module – Management of diversity in ECE system. The modules have been developed based on the revised curriculum for 3-6 and the early childhood education guidelines for 0-3, using the methodological guide developed under

88

the project and other relevant documents for ECE that have been recently developed and implemented in the ECE system. During the training, 2,600 modules were printed and distributed to trainees, and the number of participants was 2,769. Based on the training needs of the new-entry ECE staff and budget savings, a second round of training has been organized in 2014 by another consulting company, selected as a result of QBS. During the second round of training, 1,500 modules were printed and distributed to trainees, and the number of participants was 1,422. On the whole, more than 4,000 people were trained, as targeted in the beginning of the project.

The development of standards of ECE teaching and learning materials and provision of such materials primarily in project ECE units EFPMU developed two selection procedures (shopping) for printing the ECE Curriculum for Children from 3 to 6/7 Years of Age and the Good Practice Guide for ECE of Children from 3 to 6/7 Years of Age, previously developed under sub-component The development of the ECE curriculum. The two materials have been printed in 6,000 copies each, in two tranches. They have been distributed to all kindergartens, through the county inspectorates. Within the project, teaching-learning materials and books were provided for the children from targeted communities (100 kindergartens):

• set of balls, tunnel for crawling, magnetic board with letters and numerals, magnetic board for teacher, set of geometric shapes in various colors, sizes and thickness, set for construction - small pieces, set of wooden cubes for construction - big pieces, set of musical instruments, human body / internal organs models, models of various animals, insects, fishes

• 12 book titles – 575 pcs for each title They were distributed based on assessment of the needs of ECD units and their priorities. The books selected covered also the inclusion and cultural aspects. The access to the EU Structural Funds The use of SIP funds was maximized by utilizing them for the preparation of ECE projects for human resources development eligible under the European Social Fund (ESF), further contributing to the implementation of the overall MNE ECE program. For this purpose, a consultancy firm was selected by LCS to conduct the design and implementation of training and follow-up assistance programs on accessing EU structural funds for projects targeting ECE actions, under the following Priority axes of intervention:1.3. - Human resources development in education and training and 2.2. - Preventing and correcting early school leaving. The training was provided to a number of 122 participants, in 5 sessions of 5 days each as follows:

89

• Priority axis 1.3: 3 sessions with 77 participants (from Teachers’ House, School Inspectorates, County Centers for Psycho-pedagogical Assistance, schools, kindergartens) in 36 counties,

• Priority axis 2.2: 2 sessions with 44 participants (from kindergartens), in 13 counties. All trainees received certificates for participation. The training materials provided were: the training curriculum, the training course modules and the practical guide for writing eligible proposals under the European Social Fund. 3 months after the training completion, the number of eligible project proposals elaborated/submitted was: 34 elaborated and 4 submitted for Priority axis 1.3, and 10 elaborated for Priority axis 2.2. The training involved also experts from the Intermediary Body for Human Resource Development Operational Program, the unit from the level of MNE responsible for managing the funding policies and instruments available for human resources development through the European Social Fund.

2. Promotion of integrated services and CBSs for ECE Alternative community-based solution A national individual consultant was selected to design a plan for alternative community-based solutions. Based on this plan, an international consulting company, selected by QCBS, implemented an alternative CBS in 4 pilot isolated and vulnerable communities with no access to education services for children aged between 3 and 7 years. The intervention included family kindergartens, specifically targeting Roma and disadvantaged groups, support for training, teaching/learning materials, incentives for care mothers and children. The pilot communities were: 1. Poiana cu cetate, Grajduri parish, Iasi County 2. Carpinis district from Baneasa town, Constanta County 3. Maciova, Caras-Severincounty 4. Racos village, Racos parish, Brasov County In the 4 communities, 8 care mothers have been trained based on the adapted alternative Curriculum for children aged 3 to 6 years and the Methodological guide for care mothers. During project implementation, 69 children benefited from CBS, most of them being roma. Training and counseling programs for parents A national consultant has been selected by QCBS and to train and counsel parents to become sensitive to social, psychological, nutrition, health, accident prevention, children’ rights and other specific issues when raising children and to develop new skills and attitudes. A needs analysis for the teaching staff and parents from the target group has been developed – in the field of particularities of education, development and care of the young child (0-7 years old). A set of modules for trainers and thematic brochures for parents were developed and printed, to support the training and counseling sessions.

90

The training program included 53 qualified teachers who trained and counseled 1,257 parents. A final study was developed, including 3 questionnaires: one for teachers, one for parents and one for trainers. The services were implemented in 8 Resources Centers: - ValeaLunga Kindergarten, Alba county - Bunesti Kindergarten, Brasov county - Modelu Kindergarten, Calarasi county - Iveşti Kindergarten, Galati county - Lunca Cetatuii Kindergarten, Iasi county - Glina Kindergarten, Ilfov county - OdorheuKindergarten,Satu Mare county - Pătrăuţi Kindergarten, Suceava county All 8 Resources Centers received furniture and equipment: tables for equipment, tables for discussions, chairs, laptops, multifunctional printers, video projectors, projection screens, filing machines.

3. Monitoring & Evaluation and IEC activities An international individual consultant and 2 national individual consultants were selected to support MNE in setting up an ECD monitoring and evaluation system based on clearly identified indicators including a survey instrument to monitor child readiness in domains including: physical health and well-being, social competence, emotional maturity, language and cognitive development, communication skills and general knowledge. They developed a child’s school readiness assessment tool/instrument and a User’s Guide on how to use the Child Development Evaluation Tool (CDET) in the child’s last year of kindergarten and also by primary teachers in the child’s first year of schooling. The tool’s major aim is to define a child’s profile at the end of kindergarten which will serve as reference data in supporting child’s development and learning in the next years (child’s level of development, domains in which children needs more support, child’s strengths, child’s style of learning, etc.). Further on, in order to pilot the 2 instruments, a national consulting company was selected by LCS. The piloting process was carried out in 42 units, 2 kindergartens and 2 schools in each of the 8 regions (South, South-West, West, North-West, North-East, Center and Bucharest-Ilfov). The number of participants in the training session was represented by 50% of the kindergarten teachers in 8 regions. The piloting process included teachers and parents from 60 kindergartens and schools. All the participants received copies of the instrument in order to apply it to each child they supervise. In the end on the piloting process, a study was elaborated with the results of these activities, in view of improving the quality and easiness in using the CDET and User’s Guide by teachers and parents. A local consulting company was selected by CQ to conduct the design of the communication strategy and action plan for the Early Childhood Education Program. It developed a strategic communication appraisal document, a Communication Strategy for IECE Program and an Action Plan for implementing the Communication Strategy.

91

Project Management Units Within the Ministry of National Education, the overall responsibility for managing the project was placed with two national executing bodies (PMUs): the Externally Financed Projects Management Unit (EFPMU) and the School and University Network Rehabilitation Projects Management Unit (SUNRPMU). The PMUs have had general coordination responsibilities, have been accountable for procurement, financial management and reporting, as well as monitoring and evaluation activities. The PMUs have created an interface for the coordination of the project with MNE and its local correspondents and ensures that all Government inputs committed to the project are made available. Sustainability Under the project sub-component The development of the ECE curriculum, the Curriculum for children aged 3-6 years has been revised and, together with the Good practice guide for ECE of children from 3 to 6 years of age, have been printed and distributed to kindergartens in all the country, to be used in the education process. The ECE curriculum for children 0-3 years and its Methodological guide have been elaborated and are waiting formal approval to be used in kindergartens. Both PMUs due to their experience over project management are still implementing other projects with different foreign donors, World Bank, Development Bank of European Council, European Union etc., each unit having in implementation various projects with resources that have been created in this project. FINANCIAL STATEMENT Disbursement Throughout the project life, the Borrower has maintained a satisfactory financial management system. During the period 2007-2009, for the purposes of the project, the Borrower opened and maintained in EUR a special deposit account opened with a commercial bank, used exclusively to cover the project eligible expenditures financed from the Loan. Starting from the second quarter of 2009, based on Government’s Decision no. 64/2007 regarding public debt, both the Externally Financed Projects Management Unit and the School and University Network Rehabilitation Projects Management Unit have been using the Treasury Accounts for making payments, including from WB counterpart, as the funds have been received from MPF as pre-financing in RON with a Treasury Account. The Borrower has respected the relevant IBRD loan financial covenants, by submitting to the WB quarterly financial monitoring reports and annual audit reports in a timely manner and in a format and content acceptable to the Bank. Audit opinions have been unqualified (clean) and no significant internal control issues have been mentioned. Counterpart financing received from the Romanian Government has, in general, been satisfactory during the project life. Disbursement till the project end is: 7.1 mil € out of 7.7 mil €, 92.27% of the project loan funds.

92

B. Inclusive Early Childhood Education Program Budget/Planned Expenditure

% Spent

B.1. Rehabilitation of ECE Infrastructure 5,289,109.00 4,746,469.92 89.74% B.2. Development of ECE Curriculum, Human Resources and Teaching Materials 1,246,510.00 1,026407.56 82.34% B.3. Promotion of Integrated Services and CBSs for ECE 375,527.00 397,183.29 105.77% B.4. Monitoring & Evaluation and IEC activities 838,854.00 934,659.79 111.42%

7,700,000.00 7,104,720.56 92.27%

Procurement activity The procurement activity was subject to prior-review according to the Loan Agreement’s provisions and also subject to the post-review supervision missions that analyzed goods, works and consultants contracts. The procurement was done respecting the principle of transparency, equal treatment, eligibility and all contracts where Bank procedures applied are in agreement with the Bank procurement guidelines.

Part III: Social Assistance Programs

1. Disabilities Program The Social Inclusion Project (SIP) - program for people with disabilities (Part 1) - funding on a competitive basis to the Sub-projects for people with disabilities

Government of Romania, through the Government Program decided to develop and diversify social services for vulnerable groups not covered by existing services. The crucial role of social services in a constantly evolving society is given by the social and economic changes that may have seriously/ badly effects, because they often generate situations of vulnerability or increase vulnerability to certain categories of people. Analyses revealed the precarious situation of social services infrastructure and the need to investments in buildings for social sites, rehabilitation, modernization and equipping of the buildings in which those services are created/ developed. However, the need for investments arouse following of the establishment of minimum quality standards for social services which has to be met by the existing social sites, as well as those who has to be built, depending on the type of beneficiaries.

93

This program aims to support Ministry of Labor, Family and Social Protection for the Elderly (MMFPSPV) and the National Authority for Disabled People (NADP), to raise the quality of care in residential settings: - (i) financing on a competitive basis to sub-projects for people with disabilities; - (ii) providing training professional staff who will apply new standards of care; - (iii) developing a monitoring and evaluation system for NADP; - (iiii) providing occupational counseling. Program for peoples with disabilities supports the implementation of the National Strategy for protection, integration and social inclusion of people with disabilities in the period 2006-2013, approved by Governmental Decision no. 1175/2005 The specific objectives of the Program for people with disabilities are: - Reducing the number of peoples with disabilities in old and overcrowded centers/ residential complex isolated by restructuring or closure of these centers / complexes according to county strategy and county restructuring/ closure plan; - Increasing the institutional capacity of the local authorities through the development of social services according to the beneficiaries profile and their individual needs, social services to promoting social inclusion for persons with disabilities; - Improving quality of life of adults with disabilities from the target group, who are benefiting from hosting, supporting and caring in residential care centers and protected shelters, newly created and adequately equipped. Project implementation was carried out by the General Directorates for Social Assistance and Child Protection at the local, county level, to whom Ministry of Labour, Family and Social Protection exercised the role of Financier on behalf of the Romanian Government. Assessments and selection of projects under this subcomponent were developed during 2008 and 2009. Submission of the project was done in accordance with the provisions of the Operations Manual and the requirements specified in the notice. Subsequently, we obtained advices from the World Bank, Technical Economical Council of the Ministry of Labor, Family Social Protection and Elderly, and Interministerial Council for Advising the National Interest Public Works and Housing After obtaining these approvals, MMFPSPV promoted projects for Government Decision for the approving the technical-economic indicators of the investment objective. Financing contracts were concluded after the publication of the Government Decision for approving to the technical-economic indicators on investment objective. Subsequently, recipients of funding, namely the General Directorates for Social Assistance and Child Protection at local county level, proceeded to start implementation. After completion of the contract, the applicant started procurement necessary to achieve the investment objective.

94

Documentation preparation and submission of tenders for the procurement of services, works and goods were developed by the applicant, based on the model submitted by the Project Management Unit. The projects aimed the following activities - Closing / restructuring the centers/ old building complexes with a capacity of 80 peoples; - Creating of residential centers with capacity of max. 50 places that offers services to European quality standards - Creating of protected houses with capacity of 6 seats; The results of implementing the projects are:

1. Protected dwellings for adults with disabilities - DGASPC Constanţa Investment objectives: 6 Protected Dwelling – in NegruVodă for 36 beneficiaries, adults with disabilities (6 beneficiaries/protected dwelling) 4 Protected Dwelling - in Topraisar for 24 beneficiaries, adults with disabilities (6 beneficiaries/protected dwelling) 2. Restructuring the Hîrlău assistance and care center from Iasi county implemented by DGASPC Iaşi Investment objectives: 1 Assistance and Care Center Budăi, for 48 beneficiaries, adults with disabilities 4 Protected Dwellings – Iaşi, for 24 de beneficiaries, adults with disabilities (6 beneficiaries/protected dwelling) 3. Restructuring the Răcăciuni Neuropsychiatric Recovery and Rehabilitation Center - judeţul Bacău, implemented by DGASPC Bacău Investment objectives: 1 Assistance and Care Center– Filipeşti, for 48 beneficiaries, adults with disabilities 4 Protected dwellings – Filipeşti, for 24 beneficiaries, adults with disabilities (6 beneficiaries/protected dwelling) 4. Restructuring Tigveni Center of Integration Through Occupational Therapy, implemented by DGASPC Argeş Investment objectives: 1 Neuropsychiatric Recovery and Rehabilitation Center– Călineşti for 48 beneficiaries, adults with disabilities 2 Protected dwellings – Tigveni, for 12 beneficiari, beneficiaries, adults with disabilities (6 beneficiaries/protected dwelling) 5. Neuropsychiatric Recovery and Rehabilitation Center and Protected Dwellings Carei, implemented by DGASPC Satu Mare

95

Investment objectives: 1 Neuropsychiatric Recovery and Rehabilitation Center– Carei, for 48 beneficiaries, adults with disabilities 6. Neuropsychiatric Recovery and Rehabilitation Center no. 2 Lugoj, implemented by DGASPC Timiş Investment objectives: 1 Neuropsychiatric Recovery and Rehabilitation Center, for 48 beneficiaries, adults with disabilities 7.Restructuring the Brâncoveneşti Neuropsychiatric Recovery and Rehabilitation Center by creating a the new ,,Sf. Maria" Brâncoveneşti Neuropsychiatric Recovery and Rehabilitation Center, implementated by DGASPC Mureş Investment objectives: 1 Brâncoveneşti Neuropsychiatric Recovery and Rehabilitation Center, for 50 beneficiaries, adults with disabilities 8. Closing the Vultureşti Recovery and Rehabilitation Center, implemented by DGASPC Argeş Investment objectives: 1 Vultureşti Neuropsychiatric Recovery and Rehabilitation Center, for 50 beneficiaries, adults with disabilities 5 Protected Dwellings – 1 in Vultureşti and 4 in Buzoieşti, for 30 beneficiari, beneficiaries, adults with disabilities (6 beneficiaries/protected dwelling) 9. Restructuring the Social Services granted to persons with disabilities in Alba county, implemented by DGASPC Alba Investment objectives: 1 Assistance and Care Center - Baia de Arieş, for 50 beneficiari, beneficiaries, adults with disabilities 10. Restructuring the Bădăcin Center of Integration Through Occupational Therapy, implemented by DGASPC Sălaj Investment objectives: 1 Assistance and Care Center -Nuşfalău, for 50 beneficiari, beneficiaries, adults with disabilities 11. Restructuring the Urlaţi Center of Integration Through Occupational Therapy, implemented by DGASPC Prahova Investment objectives: 1 Assistance and Care Center - Urlaţi for 50 beneficiari, beneficiaries, adults with disabilities 12. Rehabilitation and expansion of Câţcău Assistance and Care Center, implemented by DGASPC Cluj Investment objectives: 1 Assistance and Care Center - Câţcău for 50 beneficiari, beneficiaries, adults with disabilities 13. Restructuring the Brâncoveneşti Neuropsychiatric Recovery and Rehabilitation Center by creating a the new "Primula" Brâncoveneşti Neuropsychiatric Recovery and Rehabilitation Center, implemented by DGASPC Mureş Investment objectives: 1 "Primula "Neuropsychiatric Recovery and Rehabilitation Center, for 50 beneficiaries, adults with disabilities

96

14. Restructuring Costîna Neuropsychiatric Recovery and rehabilitation center by establishing the "O noua şansă" Neuropsychiatric Recovery and Rehabilitation Center, implemented by DGASPC Suceava Investment objectives: 1 Neuropsychiatric Recovery and Rehabilitation Center– Todireşti, for 50 beneficiaries, adults with disabilities 15. Residential center for adults with disabilities from Ionăşeni and Botoşani towns, implemented by DGASPC Botoşani Investment objectives: 1 Recovery and Rehabilitation Center - Stăuceni, Victoria Village, for 50 beneficiaries, adults with disabilities

Type of center Beneficiar category Number of beneficiaries

Services provided

Neuropsychiatric Recovery and Rehabilitation Center

Adult with neuropsychiatric disabilities with low skills

394 - accommodation - nourishment - assistance and personal hygiene - medical and recovery assistace, - emergency and therapeutical assistance - psychological counseling services - ergo therapy services and medical, psychological and social recovery (ocupational therapy, arttherapy and music therapy)

Protected Dwellings Persons with disabilities that have potential for semi-independent life and community integration

150 - accommodation - nourishment - leisure and social activities - support activities for independent life

Assistance and Care Center

Persons with severe disabilities that have reduces potential for independent life,

296 - accommodation - nourishment - care and personal hygiene îngrijire şi igienă personală

97

being total depended of the caregiver

- medical and recovery assistace, - psychological counseling services - leisure and social activities

The new services were realised in accordance with the Specific quality standards for residential centers, day centers and assisted housing for adults with disabilities, approved by order of the president of the National Authority for Disabled Persons no. 559/2008. The specific objectives of the Program for people with disabilities were: - reducing the number of people with disabilities in the old, isolated and overcrowded residential centers, by restructuring these centers, according to the district strategy and county plan of restructuring / closing; - increasing the institutional capacity of the local authorities on the development of social services according to the profile of the beneficiaries and their individual needs, the social services to promote social inclusion of persons with disabilities; - increasing the quality of life of adults with disabilities in the target group which benefit by hosting, support and attention in the residential centers and the assisted housing, newly created and properly equipped. So, the resulting profitability by implementing of the project activities is the increased quality of life of beneficiaries, respective the persons with disabilities in the residential centers undergoing restructuring through the development and diversification of specialized social services. Prospects for Sustainability: The projects sustainability is ensured by the grant beneficiaries, respective the General Directorates for Social Assistance and Child Protection for a period of 15 years, with obligation to keep the destination and operating costs of newly created services for adults with disabilities. Following the restructuring of the old residential centers with large capacity is necessary to identify new sources of funding to establish new residential centers, with small capacity for the people with disabilities.

2. Youth at Risk Program

98

The purpose of the Program for Youth at Risk component of the Social Inclusion Project (SIP), financed by the loan offered to the Romanian Government by the International Bank for Reconstruction and Development (IBRD) is to promote social inclusion of young people leaving the child protection system, representing a type of population whose vulnerability is caused, in particular, by their limited capacity to adapt to an independent living. The program aimed to create multifunctional centres with the capacity to provide accommodation, psychological and legal counselling, social work and employment provision in order to develop skills for independent living, to recover and social reintegrate the youth at risk. The overall goal of the program was to prevent the risk of social exclusion of young people who leave the child protection system through the development and diversification of social services for this vulnerable population. The specific objectives have been:

• diminishing the number of young people in the situation of not having a safe shelter by creating multifunctional centers that provide accommodation and socio-professional reintegration services.

• increasing the institutional capacity of the local public authorities on the development of social services, according to the profile of the beneficiaries and their real needs;

• improving the quality of services by using trained personnel in a specific area of intervention;

• increasing the quality of life of young people from the target group benefiting both accommodation in adequately equipped multifunctional centres and social assistance and support in order to achieve their socio-professional integration.

• progress towards expected Outcomes (based on Outcome and Intermediate Outcome Indicators for the Component/Sub-Component) in the Project Appraisal Document

o Include anecdotal evidence supporting the quantitative accomplishments, e.g. beneficiaries’ perspectives as gathered during field visits, etc.

Beneficiaries’ opinion regarding the services of shelter, asked after your request, reveals the following:

• concerning the quality of accommodation, meals and support services provided in the center, 66,66%of the center’s beneficiaries appreciate the quality of these services as very good, 21,66% as good, 10,83% as satisfactory and 0,85% as unsatisfactory (figure no.1).

• concerning relationships with center staff, 67,5% of the center’s beneficiaries appreciate these relationships as very good, 23,33% as good, 8,3% as satisfactory and 0,83% as unsatisfactory (figure no.2).

• regarding the number of employed beneficiaries, through the total number of 164 beneficiaries of the multifunctional centres, 103 are employed representing a percentage of 62,8% within the total number of beneficiaries and 50 are employed after they entered the center representing a percentage of 62,8% within the total number of beneficiaries 30,48%.

99

• concerning the number of staff, we have 7 functional centers (within the total number of 10) with 52 employees, within more than a half is staff with improved competences in integrated services.

• most beneficiaries are concerned by the perspective of leaving the multifunctional centre, of withdrawal/integration on labour market, getting an adequate income needed for independent living.

Program for Youth at Risk, component of the Social Inclusion Project (SIP) was implemented in 2008 - 2013 in order to create multifunctional centres for youth at risk. The results of the 2 organized selections were: 10 projects are financed by the Romania Government and IBRD within 7 centers are functional. Main types of social services provided by the centers are: temporary accommodation, emotional support, psychological and legal counselling, social counseling, support and guidance for integration, rehabilitation and retraining, socializing. Minimum staffing structure includes: administrative staff (chief, night watchman - if applicable, a cleaning lady or janitor) and specialized staff: one full-time social worker to 20 people and other professionals who

67.50%

23.33%

8.30%

0.83%

Figure no.2- Relationship with center's staff

very good

good

satisfactory

unsatisfactory

100

constitute the multidisciplinary team required to implement the measures provided by the intervention and individualized plan: psychologist, psycho/ social educator, socio-educational animator, advisor, vocational/training counselor etc.

No. Multifunctional centres for youth at risk

Center’s opening

date

Reporting

Period

Period of stay in shelter

Center’s capacity

Number of

beneficiaries

Number of employed beneficiaries

Number of beneficiari

es employed after they entered

the center

% employed beneficiaries within the total

of beneficiar

ies

Number of

center’s employe

es

1 DGASPC HARGHITA Centrul social cu destinaţiemultifuncţională "Un micpopasînviaţa mea", Sanmartin (CN) judetulHarghita

31.05.2013 31.05.2013 – 09.2014

time span of maximum 3 years

12 beds

12 9

6 50% 5 of which 2 are employees of the center and 3 are social service provider’s employees

2. MUNICIPIUL GALAŢI Centrul social cu destinaţie multifuncţionalădinmun. Galaţi (CN) judetul Galati

10.06.2013 10.06.2013 - 09.2014


30 beds

34 19 0 0% 12

3 Centrul social cu destinaţiemultifuncţională, GuraHumoruluijudetulSuceava

01.09.2012 01.09.2012- 09.2014


40 beds

10 10 10 100% 3

4 ASC - ORADEA Centru social cudestinatiemultifunctionala CANDEO judetul Bihor

20.12.2013 20.12.2013-09.2014


40 beds

30 16 8 26,66% 6 of which 2 are employees of the center and 4 are social service provider’s employees

5 ASC 20.12.2013 20.12.2 time span 32 16 11 2 12,5% 7 of

101

ORADEA - DIGNITAS Centrul social cudestinaţiemultifuncţională DIGNITAS (CN) judetul Bihor

013-09.2014

of maximum 3 years

beds which 2 are employees of the center and 5 are social service provider’s employees

6 Centrul social cudestinaţiemultifuncţionalăpentrutinerii care părăsescsistemul de protecţie, dinjudetul Iasi (3 X CN)

16.10.2012 16.10.2012 -09.2014


30 beds (10/home)


7 DGASPC VRANCEA Centrul social cudestinaţiemultifuncţională " Sf. Teodor " (CN), judetul Vrancea

01.11.2013 01.11.2013 -09.2014


40 beds


8 DGASPC SĂLAJ Centrul social cudestinaţiemultifuncţională, Jibou (CN) judetulSalaj

The center has not become functional


15 beds

9 DGASPC SĂLAJ Centrul social cudestinaţiemultifuncţionalăCehuSilvaniei (CN) judetulSalaj



15 beds

10 DGASPC SĂLAJ Centrul social cudestinaţiemultifuncţionalăŞimleulSilvaniei (CN) judetulSalaj



10 beds

Total 264 164 103 50 30,48% 52

102

Local public authorities have committed to maintain the activity of multifunctional centers and to ensure its functionality for at least five years from the date of the final receipt of the work funded by the SIP and, after this time span, to maintain its activity in social services’ domain another 5 years. No applications were submitted for other projects regarding this issue but the need still exists. Regarding the possible other sources of financing, it could be used structural funds allocated to Romania for the next programming period, 2014-2020. Major factors that affected implementation of Component/Sub-Component The main problems that affected implementation of Component3/Sub-Component 3.2 - youth at risk:

• the lack of interest from eligible applicants; • the lack of financial funds from the General Directorates Social Assistance and

Child Protection. The contribution required from local public authorities - 10% of the total amount of funding requested.

• Any views on the preparation/implementation support provided by the World Bank

Very good collaboration with the Project Management Unit of the Ministry of Labor, Family, Social Protection and Elderly and World Bank staff.


Legislation should be amended in order to have a faster procedure for approving investment objectives (approvals were obtained from the County Council, Technical and Economic Council of the Ministry of Labor, Family, Social Protection and Elderly, the Interministerial Council and Government.

It’s recommended to give a pre-financing in order to carry out the projects more easily.


Because the local authorities could not allocated enough money to co-finance projects (It was required a contribution of 10% of the total amount of funding requested), General Directorates of Social Assistance and Child Protection applied for a few projects.

3. Victims of Domestic Violence Program

Program for family violence victims, component of the Social Inclusion Project (SIP), financed by the loan offered to the Romanian Government by the International Bank for Reconstruction and Development (IBRD). The purpose of the program is to

103

enhance the living conditions and the social inclusion of the disadvantaged group of family violence victims. The objectives have been: Creating new social services for family violence victims by developing a network of centers, training the specialists, raising awareness regarding the phenomenon and its consequences and the development of a toolkit for the specialists within the field. Therefore, four projects ware financed by the Romanian Government and IBRD (four shelters for the victims of domestic violence in Bucharest, District 3 and in the counties of Suceava, Dâmboviţa and Iaşi). • Progress towards expected Outcomes (based on Outcome and Intermediate

Outcome Indicators for the Component/Sub-Component) in the Project Appraisal Document

o Include anecdotal evidence supporting the quantitative accomplishments, e.g. beneficiaries’ perspectives as gathered during field visits, etc.

Beneficiaries’ opinion regarding the services of shelter, asked after your request, reveals the following: All beneficiaries appreciate the accommodation conditions, the understanding, support and warmth that they receive from the staff of the shelter. Victims feel safe in the shelter. They are concerned, most of them are worried by the perspective of leaving the social assistance-employment, housing, getting an income necessary maintenance on their own. The victims believe that isn’t enough publicity and promotion of the social services; their proposals are related to visual media in the neighborhoods in the blocks scale in the buildings, in the stores in the public institutions network of family doctors and audio media-radio stations stores.

• Outputs financed by the Loan: A. Program for family violence victims, component of the Social Inclusion Project (SIP), implemented in 2008 - 2013 in order to establish new shelters for victims. The results of the 2 organized selections were: 4 projects are financed by the Romania Government and IBRD (4 shelters the victims of the domestic violence, Bucharest: District 3 and in the counties Suceava, Dâmboviţa and Iasi). The centers for sheltering victims of domestic violence are social welfare units and provide protection, accommodation, care taking and counseling to the victims of domestic violence forced to resort to this social welfare service.

104

29Shelter = Center for sheltering the victims of domestic violence

Number

The shelters29 for

victims of domestic violence

Start date shelter

operation

Reporting Period

Period of stay in shelter

Places for victims

Number of domestic violence

cases

Service Types Social

counseling Psycholog

ical counselin

g

Legal counseling

Medical care Occupational

counseling

1 Centrul de servicii integrate pentruvictimileviolenţei in familie, Iaşi, judetulIaşi

August 01th 2013

August 01th 2013 - September 2014

60 days – 180 days, with the possibility of extension

12 places 18 female victims and 32 minors (4-16 years)

50 32 18 - 20 medical counseling; - 18 medical evaluation


2. Centruldestinatvictimelorviolenţeiînfamilie, Bucureşti, sect. 3

January 20th 2014

January 20th 2014 - September 2014


12 places 19 female victims and 25 minors (15 girls and 10 boys)

44 16 19 - 10 medical counseling; - 5 medical evaluation

- 5; - 1 person was employed

3 DAS Targoviste - Centrul social pt. prevenireaşicombatereaviolenţeidomestice " Împreunăvomreuşi "- judetulDambovita

December 20th 2012

December 20th 2012 – September 2014

60 days - 180 days, with the possibility of extension

12 places 42 female victims and 38 minors

80 80 42 - 25 medical counseling; - 17


105

B. In the period September 2012 - January 2013, The Awareness Raising campaign to prevent and fight against the domestic violence” was implemented, financed through the Loan Agreement No. 4825 RO signed by the Romania Government and The International Bank for Reconstruction and Development (IBRD) and ratified by Law no. 40/2007. The aim of campaign was to increase the involvement of the member of the community, as well as to raise of awareness at the level of local authorities with regard to the importance of their participation in the prevention and fight against the domestic phenomenon, mobilizing all the existing resources and raising awareness of the public opinion regarding the prevention and fight against the domestic violence, in order to change the way in which “the community see this phenomenon”.

medical evaluation

4 DGASPC Suceava - CentrumultifuncţionaldestinatvictimelorviolenţeiînfamilieGuraHumorului, judetulSuceava

2012 2012 - September 2014


12 places 12 female victims and 23 minors

35 35 -12 -1 protection order

- 9 medical counseling; - 6 medical evaluation

- 3

Total cases 209 victims 209 163 91 - 64 medical counseling; - 46 medical evaluation

- 21 victims were counseled occupational; - 6 victims have been employed

106

Activities aimed through this campaign were: • Preparing and organizing the launching conference, Bucharest, 1 November 2012 • Preparing and organizing 7 regional conferences attended by 50 participants each

- regional conference Constanţa, 09 November 2012 - regional conference Iaşi, 15 November 2012 - regional conference Târgovişte, 19 November 2012 - regional conference Timişoara, 22 November 2012 - regional conference Craiova, 26 November 2012 - regional conference Sibiu, 29 November 2012 - regional conference ClujNapoca, 06 December 2012

• Preparing and organizing the closing conference attended by 80 participants, Bucharest, 28 January 2013

• Accomplishment of poll for searching the public attitudes and the public awareness regarding the domestic violence phenomenon, the evaluation of the existing information at the level of institutional actors concerned by the domain, as well as the evaluation of the social services and of the existing protection mechanisms for the domestic victims and aggressors, November 2012 – January 2013

• Organization of 35 street events – caravan type, November – December 2012 • Preparation of the caravan (preparing the mobile units and promotional materials

for each route, preparing the request and obtaining the authorizations for all of the 35 street events)

• Deployment of the caravan (sending information to the media and main stakeholders regarding the deployment of the caravan, organizing training sessions for the personnel taking part in the event, and deployment of the 35 street events in three different routes).

Type of activities: Organization of conferences, opinion poll, drafting reports, organizing numerous street events. Some of those events have been implemented simultaneously. Results and outputs: An opinion poll was conducted to find out people’s attitudes towards and public awareness of the family violence phenomenon, broadcasting of TV and radio spots “Good night, Mommy!”, distribution of flyers, posters (A1-10 pieces, A2–1410 pieces, A3-17390 pieces, A4-16470 pieces), stickers-87600 pieces and other promotional materials (balloons 2000 pieces, push button pens - 4100 pieces, caps – 200 pieces, paper cases – 4100 pieces, notebooks A5 – 4100 pieces, 1 banner, 2 canvas roll-ups and one flag). Target group(s): Children and adolescents, parents, professionals working with children, decision makers, public authorities and civil servants, civil society organizations, public at large, the victims of domestic violence, family offenders. Success factor(s): 500 participants (2 national conferences and 7 regional conferences), working meetings, local events specifically organized for the three routes of the caravan, namely:

107

• Route 1: Slobozia - Tulcea-Braila-Galati-Vaslui-Botosani-Suceava-Piatra Neamt-Bacau-Focsani-Buzau-Ploiesti ; • Route 2: Alexandria-Slatina-Rm Valcea-TgMures-Bistrita-Sf. Gheorghe-Brasov-Pitesti; • Route 3: Dr. TurnuSeverin - Resita- Arad-Oradea-Zalau-Satu Mare- Alba Iulia - Hunedoara - Tg. Jiu . In Bucharest (District 1, District 2, District 3, District 4, District 5, District 6), six street events were deployed. The public was informed about the family violence phenomenon. There was an active media participation in all of the street events (video and radio spots of the campaign).

• Any possible indicators to substantiate that the Project’s activities were cost-

effective, and expected to increase efficiency and generate positive economic returns:

o Construction/training etc. costs compared to other similar construction or other costs

o Measures of increased efficiency in providing services/activities to target population, e.g., reaching a greater number of beneficiaries with existing staff, improved targeting of social assistance

o Possibly, citing average economic returns to ECD

• Prospects for Sustainability: o Actions taken to ensure continued funding for operations and maintenance

of activities financed o By which agency, what source of funding o Internal government plans for replicating/disseminating, etc. positive

experiences Public shelters are financed from local budgets for at least 10 years.

• Prospects for Continued Funding of activities and new Sub-Projects o Projects submitted for EU Contingency Financing o Financing secured, by source of funding and number/type of Sub-Project o Financing applied for by source of funding and number/type of Sub-

Project o Possible other sources of financing

No applications were submitted for other projects regarding this issue but the need still exists.

• Major factors that affected implementation of Component/Sub-Component

It was found the lack of financial funds during this projects especially from the General Directorates Social Assistance and Child Protection.

108

• Any views on the preparation/implementation support provided by the World

Bank It was a very good collaboration with the Project Management Unit of the Ministry of Labor, Family, Social Protection and Elderly and World Bank staff.


Legislation should be amended in order to have a faster procedure for approving investment objectives (approvals were obtained from the County Council, Technical and Economic Council of the Ministry of Labor, Family, Social Protection and Elderly, the Interministerial Council and Government. It’s recommended to give a pre-financing in order to carry out the projects more easily.


Because the local authorities could not allocated enough money to co-finance projects, General Directorates Social Assistance and Child Protection applied for a few projects, so this is a reason for which these institutions should be financially supported. 4 - Social Assistance Cash Transfer Program This component of the Part III was mainly devoted to: (i) offer TA support for fulfillment of the Disbursement Linked Indicators (DLI) agreed in the Social Assistance System Modernization Project, and (ii) upgrading the Management Information System (MIS) for the National Agency for Payments and Social Inspection (NAPSI former NASB) with modules for the family support allowance and the Guaranteed Minimum Income (GMI), and with any other new modules for the social assistance benefits paid through this system, as well as improvement of the analysis and reporting capacity and performances of this system according to the specific needs of the sector. The actions completed till the closing date of the project are presented below:

ACTION BIRD (Euro) GOR(Euro)

Support for the development of the action plan of the social assistance strategy (DLI 1) 4.335,00 765,00 Development of a statistical bulletin to track the distributional impact of social protection spending, including supplementary data collection from Family Budget Survey (DLI 7) 47.948,10 8.461,43

109

Upgrade SAFIR (3): (SW and HW) (i) with new rules for GMI, Family Support benefits, Child Raising Benefits; (ii) supplementary services to include EO 124/2011; (iii) introducing improvements in functioning of the operational and decisional modules, and ”on site” support (many DLIs) 4.914.365,88 867.240,80 Asses the capacity of SAFIR to cross-check information with other public databases; (DLI 14) 8.415,00 1.485,00 Legal TA for the development of a new sanction policy and investigative powers for Social Inspections (DLI 17) 12.719,40 2.244,60

Capacity building (IT and office equipment, training, study tours). (i) MoLFPS and ANPIS staff training in ORACLE 14.922,13 2.633,32 (ii) MoLFPS staff training – WB Courses on Safety Net 14.607,78 2.577,84 (iii) ANPIS Social Inspection staff traning (two traning sesions in UK) 13.975,37 2.466,25 (iv) Social Inspection staff training on combating fraud & errors 37.121,45 6.550,84 (v) Translation services for delivery of Social Inspection staff training on combating fraud & errors 4.588,61 808,75

(VI) Office and IT equipment for Social Assistance General Directorate 11.501,52 2.029,68 Consulting services for the development of the Communication Strategy. Preparation of Terms of Reference for the preparation of key components of the Strategy, and Monitoring and Evaluation of the results of the Strategy for the Social Assistance System Modernization Project in Romania. 101.287,29 17.874,23 TOTAL 5.185.787,54 915.137,74

Part IV: Capacity Building for Roma Social inclusion

Support to NAR and its eight (8) regional branches to coordinate activities aimed at improving inclusive planning, monitoring and evaluation and identification,

110

preparation and implementation of programs to be financed through EU and other funds, through: (i) the provision of training to NAR staff, Roma NGOs, local authorities and Roma community initiative groups; (ii) the preparation and implementation of an information, education and communication strategy; and (iii) the strengthening of the monitoring and evaluation system. NAR found little interest in implementation of the project objectives which is way, with the occasion of project restructuring the funds allocated to this part of the project were reduced from 200.000 to 80.000 Euro just to support the acquisition of some consulting services and some equipments as follows: Consultancy services Amount/Euro

including VAT Consultancy services to assess the capacity of Roma National Agency (RNA) in coordinating its own activities, including management of EU funds

24,791.32

Pre-intervention evaluation study of the "Inclusive early education"

20,000.00

IT Consultancy services for development the institutional capacity of National Agency for Roma

15,000.00

Goods IT 26,164.00 Furniture 1,361.00 Office equipments 9,972.00 Financial statement of Part III and Part IV at October 31, 2014

Component

EUR Sursa Total

BIRD GoR Disabilities Program 9.377.386,28 1.655.718,73 11.033.105,01 Youth at Risk Program 2.893.066,23 511.133,15 3.404.199,38 Victims of Domestic Violence Program 702.022,26 126.667,30 828.689,56

111

Social Assistance Cash Transfer Program 4.702.282,56 834.065,14 5.536.347,70 Project Monitoring - UMP 70.869,03 651.311,54 722.180,57 Partea III 17.745.626,36 3.778.895,86 21.524.522,22 Estimat Nov. 1.248.648,06 220.349,66 1.468.997,72 18.994.274,42 3.999.245,52 22.993.519,94 Partea IV 81.980,69 17.213,80 99.194,49 TOTAL III+IV 19.076.255,11 4.016.459,32 23.092.714,43

112

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders

No Comments

113

Annex 9. List of Supporting Documents

Government of Romania. Borrower’s Contribution to the Implementation Completion Report. Social Inclusion Project. Government of Romania, World Bank. Mission Aide Memoires. Social Inclusion Project. Impreuna and ECHOSOC (2013) “Evaluation Study of the Early Childhood Education Program”. Report produced for Government of Romania (in Romanian). Bucharest. RSDF (2013) “Monitoring Report: An Independent Study of the Perfromance Indicators of PIP”. (In Romanian). Bucharest. World Bank. 2006. Project Appraisal Document. Social Inclusion Project. Report No. 36181-RO. ———. 2011. Restructuring Paper. Social Inclusion Project. Report No. 59725-RO. ———. 2012. Restructuring Paper. Social Inclusion Project. Report No: 73767-RO. ———. 2014. Country Partnership Strategy for Romania for the Period 2014–2017. Report No. 84830-RO. ———. Implementation Status and Results Reports. Social Inclusion Project P093096. May 2007–June 2014.

114

Annex 10. Results Framework

Baseline30 Target Actual at Closing Comment

Key Performance (Outcome) Indicators

% reduction in the gap between targeted settlements and neighboring communities as measured by the living conditions index

504.46 index points 31

(2008)

403.7 (20% reduction) 61.77% reduction Exceeded.

30 At approval, or at the time of the level-1 restructuring for new indicators (March 2011) 31 Baseline data was not available at Appraisal. The baseline data of targeted and neighboring communities mentioned in this and several other indicators in the Results Framework could only be collected once sites had been selected, i.e. after the first year.

115


% of Roma in targeted poor settlements acknowledge an improvement in basic living conditions

0 70 73.84

Achieved. Data is not available for all years, but

Actual at Closing data

refers to latest available rate as

per RSDFs’ “Monitoring Report: An Independent Study of the Performance Indicators of PIP” (2013)

Percentage points increase of children from vulnerable groups in targeted communities participating in ECE

0

5 percentage points 6 percentage points Exceeded

116


% of services for PWD scoring at least 80% on the standards compliance index for each main category of quality standards

0 25 0

Not achieved. As per June 2014, 11 Centers for PWD were completed and staffing was in progress. Works in another 4 centers was ongoing and would be completed beyond project closing date. Staff training was canceled.

% increase of the rate of employment of youth aged 18 years and above benefiting from multifunctional centers

0 40 39.6 Substantially Achieved

117


The scope of NASB MIS, as measured by the number of large social assistance programs (or their successors) paid through SAFIR, increases from 4 (2010) to all programs by 2013

4 8 832

Exceeded. Introduced in level-1 restructuring

Project beneficiaries (direct and indirect) n/a 150,658 152,658

Core sector indicator introduced during project implementation

32 Universal Child Allowances, Child Raise Benefits, Family Support Allowance, GMI, Heating Allowances, Disabled Chile Raise Benefits, Benefits for PWDs, Placement Monthly Allowance.

118

Intermediate Results Indicators

Intermediate Result (Outcome 1): Progress is achieved toward addressing the needs identified by disadvantaged Roma in targeted poor settlements

% of targeted Roma settlements having access to improved infrastructure (for example, water and roads)

10 (2008) 60 82.71 Exceeded

% of Roma from targeted poor settlements agreeing that subprojects reflect community priorities

Originally no baseline31

62 (2008)

60 (2006)

74 (2011)

75.66

Achieved. Target revised

in level-1 restructuring to 74. Data is not available for all

years, but “Actual at

Closing” data refers to latest

available rate as per RSDFs’ “Monitoring Report: An Independent Study of the Performance Indicators of PIP” (2013)

119

Number of annual consultations of Roma in poor settlements with local authorities for addressing community needs (cumulative)

Originally no baseline31

120 (2008)

3 (2006)

360 (2011)

1,659

Exceeded Target revised

in level-1 restructuring to

360

% increase in number of NAR staff with improved competencies in M&E

0

70 (2006)

30 (2011)

30

Achieved. Target revised to 30 in level-1 restructuring.

Was reported as intermediate

result (outcome 2) in 2006 PAD

Intermediate Result (Outcome 2): MLFSP and MER identify targeted, disadvantaged groups and adjust programs to better respond to their needs33

% of ECE interventions in communities with more than 50% Roma population

0 60 Dropped in level-1 restructuring

% increase of children from vulnerable groups participating in ECE services through ECE alternative CBS in targeted communities

0 30


33 Intermediate Result (Outcome 2) was revised in the level-1 restructuring on March 28, 2011 to read: Vulnerable communities benefiting from alternative community-based solutions in ECE.

120

% of PWD services implemented in high-priority areas 0 30

Data not available. Most of the services for PWD became available after the project closing.

% increase in funding allocation for CBSs by NAPH (for PWD) and MER (for ECE)

0 10

Data not available. Modified in level-1 restructuring to read: Percentage increase in funding allocation for CBS by NAPH (for PWD).

% of community members involved in alternative CBS 0 10

Data not available. Introduced in level-1 restructuring.

Intermediate Result (Outcome 3): MER and MLFSP effectively implement inclusive policies and programs

% increase in ECE staff with skills in inclusive education 0 10

10.56

(3,500 staff trained)

Exceeded.

121

% increase in the number of staff with improved competencies in integrated services

0 70

Data not available. Was reported as intermediate result (outcome 2) in 2006 PAD. Hiring of staff was completed but training has not been carried out due to time constraints

% increase in number of qualified staff in PWD services 0 10 0

Not Achieved. Consultant was selected but activity was not completed as a result of delays in works completion and staff hiring

122

% of beneficiaries (including legal tutors) of social inclusion services (CBS for ECE and PWD, LTC institutions, and integrated services) satisfied with the quality of services

0 70 80

Achieved. Data is not available for all years, but refers to latest available rate as per “Evaluation Study of the Early Childhood Education Program” (2013)

Increase in number of PWD with access to CBS

0

9034

Data not available

34 The PAD had a target of 900, which was a typo.

123

% decrease of parents from vulnerable groups perceiving discriminatory teaching practices in ECE

0 20

Data not available (no baseline). However, the “Evaluation Study of the Early Childhood Education Program” (2013) shows that 75% of respondents are satisfied with the way Roma children are treated by educators and 75% of the respondents are satisfied with the way Roma and Romanian children interact.

124

Intermediate Result (Outcome 4): Improved capacity of local groups to access EU funds for social inclusive programs

% of SIP assisted promoters of mature project proposals eligible for EU financing

0 60 100 Exceeded.

Intermediate Result (Outcome 5): Improve the administration of social assistance benefits35

Evaluations of targeting accuracy of social protection programs, and of administrative and client participation costs of social assistance programs, carried on and disseminated (cumulative number of evaluation reports)

0 4 0

Introduced in level-1 restructuring. The second selection of consulting services was canceled due to the high cost and time constraints. Will be carried out under the SASMP.

35 Intermediate Result (Outcome 5) was included in the level-1 restructuring on March 28, 2011

125

NASB MIS cross-checks information with an increasing number of external databases (Pension House, Public Employment Service, Labor Inspection, Civil Registry, and Tax Registry)

0 4

5 Data cross-checks

operating with 5 other databases36

Achieved. Introduced in level-1 restructuring

36 Pension House, Employment Agency, Ministry of Finance, Ministry of Interior, and MER

126

Annex 11. Outcome Rating for Project with Formally Revised Project Objectives

Against Original PDO

Against Revised PDO Overall Comments

Rating Moderately Satisfactory

Moderately Satisfactory

-

No major change in rating, as the level-1 project restructuring was to utilize project savings to finance TA in support of additional activities (as opposed to fundamentally changing the project and its PDO)

Rating Value 4 4

Weight (% disbursed before/after PDO change)

17% 83%

Weighted Value (2X3) 0.68 3.32 4

Final Rating (rounded) Moderately

Satisfactory

127

Annex 12. Project photos

Priority Intervention Program

128

Inclusive Early Childhood Education

Social Assistance Programs

129

MAP

130

Document of The World Bankdocuments.worldbank.org/curated/en/316581468058523732/... ·...

Documents

Transcript of Document of The World Bankdocuments.worldbank.org/curated/en/316581468058523732/... ·...