(RFP Number: LRFP-2017-9132946) Mid-term Evaluation Final ... · Services as Part of Child Care...

“Toward Social Inclusion of Vulnerable Children: Expanding

Alternative Care, Family Support and Inclusive Education

Services as Part of Child Care Reform” Project

(RFP Number: LRFP-2017-9132946)

Mid-term Evaluation

Final Report

Submitted by

UNICEF country office in Armenia

Timeframe of the evaluation September 2014 - June 2017

Geographical scope of the evaluation Yerevan, Lori, Syunik

Management of the evaluation UNICEF country office in Armenia

Independent evaluation team VISTAA Plus LLC (Armenia)

This evaluation is made possible by the support of the American people through the United

State Agency for the International Development (USAID). The contents of this evaluation are

the sole responsibility of VISTAA Plus LLC and do not necessarily reflect the views of USAID,

the United States Government or UNICEF.

December 2017

Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support

and Inclusive Education Services as part of Child Care Reform” project

1

CONTENTS

CONTENTS ................................................................................................. 1

I. List of TABLES AND CHARTS ...................................................................... 2

Ii. List of Acronyms ....................................................................................... 3

Introductory Note by VISTAA Evaluation Team ........................................................................................... 5

IIi. Executive summary .................................................................................. 7

CHAPTER I. INTRODUCTION TO EVALUATION..................................................... 18

- Object of the Evaluation, Purpose, Objectives and Scope .......................................... 18

- Background of the Intervention and Context ....................................................... 19

- Logic of Interventions ................................................................................. 23

CHAPTER II. EVALUATION METHODOLOGY........................................................ 26

- Qualitative Data Collection: Approach and Implementation ....................................... 28

- Quantitative Data Collection: Approach and Implementation ..................................... 31

- Major Limitations ..................................................................................... 34

- Ethical Considerations, Human Rights and Gender ................................................. 35

CHAPTER III. FINDINGS ................................................................................. 37

CHAPTER IV: CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS ................. 82

CONCLUSIONS ............................................................................................ 82

LESSONS LEARNT ......................................................................................... 84

RECOMMENDATIONS ................................................................................... 84



2

I. LIST OF TABLES AND CHARTS

Table 1: Key Expected Results under the Project ......................................................................................... 23

Table 2: Stakeholder Mapping ....................................................................................................................... 27

Table 3: Distribution of In-Depth Interviews by Marz and Informant Groups .......................................... 28

Table 4: Focus Group Distribution by Marz and Type of Key Informant ................................................... 30

Table 5: Proportion of Schools by RPPSC Service Area ............................................................................... 31

Table 6: Number of Schools Included in the Sample Based on Availability of Assessed Children from

Rural and Urban Schools ................................................................................................................................ 32

Table 7: List of Schools in the Final Sample .................................................................................................. 32

Table 8: Readiness to Implement Inclusive Education (average value) ...................................................... 47

Table 9: Roles and Competencies of Project Partners .................................................................................. 50

Table 10: Transformation Status of Institutions ........................................................................................... 54

Table 11: Utility of Training Session ............................................................................................................. 61

Table 12: General Knowledge Index by Sub-region ..................................................................................... 63

Table 13: Application of knowledge (Data source-240) ............................................................................... 64

Table 14: Socializing Patterns of Children with SEN (Data source-175) .................................................... 65

Table 15: Project Budget and Spending ......................................................................................................... 68

Table 16: Assessed Children by Gender ........................................................................................................ 80

Chart 1: Rationale for Attitude ...................................................................................................................... 48

Chart 2: Average Value of Training as Assessed by Respondents (5 being highest and 1 lowest) ............. 62

Chart 3: Change in Absolute Knowledge ...................................................................................................... 63

Chart 4: Gender Distribution across Foster Families .................................................................................... 81



3

II. LIST OF ACRONYMS

ADP Area Development Programs

AMD Armenian Dram

BoH Bridge of Hope

CFSC Child and Family Support Center

CPD Country Program Documents

CRC Child Rights Convention

CRPD Convention on the Rights of Persons with Disabilities

CSCF Child Support Center Foundation

CwD Children with Disabilities

DFID Department for International Development

DI Deinstitutionalization

EU European Union

FAR Fund for Armenian Relief

FG Focus Group

FM Frequency Modulation

FWCPU Family Women and Child Protection Units of Regional Administrations

G2G Government to Government

GoA Government of Armenia

GTC Guardianship and Trusteeship Committees

IE Inclusive Education

IP Implementing Partner

ISS Integrated Social Services

ITT Indicator Tracking Table

LSP/TSP Local Social Planning/Territorial Social Planning

M&E Monitoring and Evaluation

MoES Ministry of Education and Science

MoLSA Ministry of Labour and Social Affairs of Armenia

MoTAD Ministry of Territorial Administration and Development

MoU Memorandum of Understanding

NGO Non-Governmental Organization



4

NIE National Institute of Education

NORK Social Services Technology and Awareness Center

OSI Open Society Institute

PIO Public International Organizations

PPC Psychological-Pedagogical Center

PPSC Psychological-Pedagogical-Psychological Support Centre

PSA Public Service Announcement

Q&A Question and Answer

RA Republic of Armenia

RFP Request for Proposal

RoA Republic of Armenia

RPPSC Republican Psychological-Pedagogical-Psychological Support Centre

SC Save the Children

SDG Sustainable Development Goal

SEN Special Education Need

SOAR Society for Orphaned Armenian Relief

SPSS Statistical Package for the Social Sciences

ToR Terms of Reference

ToT Training of Trainers

TSP Territorial Social Plan

UK United Kingdom

UN United Nations

UNCRC United Nations Child Rights Convention

UNDP United Nations Development Programme

UNICEF United Nations Children's Fund

USAID United States Agency for International Development

USD United States Dollar

WB World Bank

WV World Vision



5

Introductory Note by VISTAA Evaluation Team

To the Final Report Commissioned by UNICEF in Armenia for

Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative

Care, Family Support and Inclusive Education Services as Part of Child Care Reform” Project (RFP

Number: LRFP-2017-9132946)

August 14 – November 30, 2017

In August 2017, VISTAA was awarded the contract for conducting the Mid-term Evaluation of the

‘‘Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and

Inclusive Education Services as part of Child Care Reform’’ project (LRFP-2017-9132946).

The Project itself has started in September 2014 with an end date of September 2019. In compliance with

the Request for Proposals and discussions with UNICEF, the evaluation should have focused on the first

2.5 years of the Project, more specifically on the period from the September 2014 to June 2017.

Since the contract was awarded in mid-August, 2017, the field work for quantitative and qualitative data

collection started on the 14th of September and lasted till the end of October (following finalization of the

methodology, sampling, questionnaires, etc.). Meanwhile, a number of significant developments occurred

during this period both with respect to Project management and implementation as well as Project-

induced achievements, including:

- On November 16, 2017 the Government session approved the proposed amendments to the Family

Code and related legislation and submitted those to the National Assembly of Armenia. Amendments

were adopted during the last session of the Parliament in December 2017. According to the proposed

amendments, starting from 2018, children in difficult life circumstances will be eligible to become the

beneficiaries of the foster care system. This development followed the approval of the Concept for

Foster Care in March 2016 allowing for different types of foster care, for which UNICEF and Project

partners actively advocated. The 2018 Child Protection Program has also included strong stipulations

for foster care and the state budget AMD 166,343.5 thousand is committed to fund 94 foster families,

which is around 4 time increase compared to the previous years (reaching a total of 119 foster families

starting 2018). These significant achievements are attributable both to the political will of the

Government, to Project interventions, consistent and forceful advocacy efforts of UNICEF Armenia

and Save the Children.

- With respect to the deinstitutionalization efforts as well there is considerable progress following the

Government decrees on transformation of three institutions (two special schools in Lori and one night

care institution in Syunik) and closure of one residential institution. The released State funds were

reallocated to family strengthening and alternative care, ultimately resulting in the return of more

children to families.

Specifically, the official decision on Kapan institution was passed in September 2017. It was signed in

November 2017 and became effective as of 16/11/2017, following the RA Decision N 1398-N on Renaming



6

the Kapan Child Care and Protection Boarding Institution SNCO into Syunik Marz Child and Family

Support Center Foundation, Approving of the Carter and Providing Property.

Draft Government decree regarding transformation of a night care institution and closure of an orphanage

in Lori region has been re-submitted to the Government for approval.

• Since September 2017, all schools in Lori and Tavush launched the full inclusion concept supported

by the teacher trainings, coaching, establishment of seven Psychological-pedagogical support centers,

four in Tavush and three in Lori.

However, in September 2017 USAID Armenia made a decision to terminate its agreement with UNICEF

(111-IO-14-00004 AID-111-IO-14-00004) thus suspending some of the Project activities. Despite the

termination, both UNICEF and USAID agreed to proceed with the evaluation based on the assumption

that its findings and lessons learnt could be useful for both organizations in their future efforts in support

of Government of Armenia’s reforms agenda in child welfare and inclusive education.

As a consequence of this situation, the assignment that was initially designed as a mid-term evaluation

evolved into more of a final review of the Project activities. However, there are certain factors and realities

to be kept in mind while reading this document:

• Some of the Project interventions were still on-going during the data collection in September- October

(depending on the contract/agreement expiry dates of specific implementing partners). As such, for

several key informants it was difficult to delineate the before June-September activities from the

ongoing ones, especially since the ground work for many of them has been done through Project

efforts. As noted above, the significant progress made with respect to the Family Code, Foster Care

and transformation of the institutions and return of significant number of children to the families was

to a great extent possible due to the Project efforts. While this report tries to maintain its focus on the

defined timeline for the evaluation, sometimes there are references to activities/events that happened

later on, since they constitute an integral part of the overall process and not mentioning those would

skew the actual achievements of the Project.

• The evaluation’s mid-term nature was somewhat affected by the overall situation since informants

were aware of the recent developments and oftentimes expressed opinions and ideas that are more

relevant to a final evaluation than to a mid-term one where the possible improvements and shifts

should be the focus.

• This said, since the Project was designed to support the Government’s recent reforms which were

taking shape and gaining momentum as the evaluation was being conducted, the evaluation team

believes that it still maintains its mid-term focus if not with respect to the Project itself, then with

respect to the reforms. All the key stakeholders involved in the Project (including UNICEF in

Armenia, USAID, implementing partners such as Save the Children, World Vision, Bridge of Hope,

FAR CSCF) are important players in advocating for these ongoing reforms and as such the evaluation

team hopes that the findings and recommendations from the assignment will be considered and

utilized to further this work.



7

III. EXECUTIVE SUMMARY

Object and Purpose of Evaluation: UNICEF in Armenia’s mandate and operations in the country have

consistently emphasized child protection and inclusive education as priority areas. Realizing children’s

right to education alongside with inclusion of children with disabilities in mainstream education, as well

as support to ongoing child welfare reforms through deinstitutionalization and transformation of

residential care institutions, has been the focus of UNICEF Armenia’s 2012-2015 and 2016-2020 Country

Program Documents (CPD).

In early 2014, UNICEF Armenia in cooperation with World Vision (WV), Save the Children (SC) and

Bridge of Hope (BoH) submitted a project proposal to USAID to expand alternative care, family support,

and inclusive education services as a part of the child-care reforms. Following an extensive review, USAID

approved the Proposal in September 2014 as a 5-year agreement (09/09/2014 - 09/08/2019). This evaluation

focuses on the project “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,

Family Support and Inclusive Education Services as Part of Child Care Reform” (hereinafter the Project)

with its key components.

The main goal of the Mid-Term Evaluation of the “Toward Social Inclusion of Vulnerable Children:

Expanding Alternative Care, Family Support and Inclusive Education Services as part of Child Care

Reform’’ project (the Project) is to assess the relevance, effectiveness, efficiency as well as impact and

sustainability of interventions with specific objectives defined in the ToR. The time frame covered by

evaluation is September 2014-June 2017 with geographical coverage focusing on Lori, Syunik and

Yerevan. As per the ToR, the primary users of the evaluation are expected to be UNICEF, USAID and

Government counterparts, implementing partners as well as other national and international partners.

Methodology: Based on the stakeholder mapping and the RFP requirements, a sampling and data

collection methodology was developed by the evaluation team and approved by UNICEF in Armenia.

Consequently, mixed (quantitative and qualitative) data collection methods were used by the evaluation

team for data collection from relevant groups:

• Expert interviews were used for data collection from national level key stakeholders (ministries, NIE,

etc.) as well as the Office of Human Rights Defender, legal and child protection experts, etc. In total,

25 experts were interviewed as part of this stakeholder group.

• In depth interviews (including group interviews) were used to collect data from Project

beneficiary/stakeholder groups identified through stakeholder mapping such as parents/family

members, school principals, staff of RPPSC, CFSC, regional authorities, service providers and others.

In total, in-depth interviews (including 3 small group interviews) were conducted with over 40 key

informants.

• Focus groups were conducted with teachers and parents trained under the inclusive education

component, trained foster parents, staff of institutions and services. 14 Focus groups were conducted

with a total of 104 participants.

• Desk review and observations were used as additional qualitative data collection methods.

• In compliance with the ToR for this assignment, trained school teachers under the Inclusive Education

component (implemented by the BoH) were the target respondents of the quantitative data collection.

As agreed with UNICEF during initial meetings, the evaluation prioritized Syunik for the quantitative

data collection given the lengthier experience (2 years) of the schools in this region of involvement in



8

inclusive education. Quantitative interviews were implemented with 312 teachers of Syunik Marz (see

Chapter II for details of the sampling approach and overall methodology and Annex 4 for survey

instrument and interview guides).

The evaluation identified some data collection limitations including non-involvement of primary

beneficiaries, i.e. children, extended timeline of the evaluation since some key results occurred beyond

June 2017, different intensity and sequencing of interventions in Lori and Syunik, and others.

The evaluation findings are presented using the evaluation criteria, i.e. relevance, effectiveness, efficiency,

sustainability and impact. As prescribed by the ToR, “Given that this is a mid-term evaluation, two of the

evaluation criteria this project will be evaluated against, namely sustainability and impact, are to be

considered to the extent possible”. This issue was discussed later with UNICEF partners as well to

reconfirm the position that sustainability and impact will be addressed only to the extent possible.

RELEVANCE

General Findings

Finding 1: Due to UNICEF’s and partners’ continuous efforts and advocacy, policy makers and decision

makers in Armenia have realized the importance and urgency to act upon their commitments on

CRC/CRPD and initiate reforms that emphasize family environment and inclusiveness in education, as

well as minimize reliance on institutions. As a result, the number of institutionalized children has

decreased significantly. However, overall acceptance and understanding of the reforms varies from high

to limited among the different stakeholder groups. In the past 25-years UNICEF in Armenia’s efforts were

heavily focused on advocating for and supporting child protection and inclusive education reforms and

policies introduced and implemented by the Government of Armenia. UNICEF also supported the

establishment of the Integrated Social Services system as an institutional mechanism to address child

protection through the social system. UNICEF’s consistent engagement with the MoLSA, MoES and other

partners resulted in increased acceptance of the need to transform institutions and set-up alternative

services, building up a momentum for the reforms in child welfare and inclusive education. The Project

itself almost identically mirrors the reforms agenda and provides support to its key directions to sustain

these achievements. However, the challenges that persist with respect to both reorganization of the

institutions, introduction of inclusive education, setting up new social services, at times unpreparedness

of biological families to deal with returning children, create some uncertainties about the timing and

sequencing of the reforms both among stakeholders and beneficiaries.

Child Welfare Reforms

Finding 2. Different Project stakeholders (including policy makers, institution staff and families) confirm

that due to UNICEF’s leadership and consistent efforts, there is overwhelming agreement about the family

being the most favorable environment to ensure a child’s best interest and wellbeing. However, since the

launch of relevant reforms (deinstitutionalization) new and flexible solutions were required from UNICEF

and partners to address the situation with the lack of a functioning system (in terms of alternative social

services, acceptable quality of life at home, social conditions, etc.). As discussed throughout this report,

the momentum for the deinstitutionalization reforms has been built over many years due to UNICEF’s



9

and partners’ advocacy resulting in the Government declaring deinstitutionalization a priority in 2011,

and the consequent developments. The MoLSA’s commitment to the RA Prime Minister to close all night

boarding institutions by the end of 2017 is an important evidence as to how these efforts have paid off. As

a result of these efforts the number of institutionalized children has decreased drastically in the country.

However, despite the perceived need for deinstitutionalization reforms, there is also agreement that it is

not an overnight process and needs serious preparation. Reform stakeholders believe that

deinstitutionalization will not succeed without establishing the community based alternative services in

advance, as well as providing individual solutions for each family reunited with the children returning

home. Still, this has been the case with the ongoing reforms since Child and Family support centers and

other alternative care services are either not established or not fully functional. To address the situation,

UNICEF has brought on board other partners involved in service provision to vulnerable children and

their families (Society for Orphaned Armenian Relief (SOAR) and SOS Children’s villages).

Foster Care

Finding 3: The institute of foster family is new to Armenia and is somewhat divergent with its traditional

culture. However, given the ongoing deinstitutionalization reforms, it has been considered as one of the

most appropriate alternatives for children placement and promoted as such. The Project interventions,

conducted mainly by Save the Children Armenia and FAR CSCF, have been timely and relevant with

respect to advancing foster families as a type of an alternative care in support of the deinstitutionalization

reforms. They focused on legal changes needed for introducing this form of alternative care alongside to

capacity building of foster families. However, some stereotypes about placement with foster families still

persist since despite efforts to ensure acceptance of foster care on the local level, yet there was no national

level public awareness campaign to advocate for this form of care with the larger public.

Inclusive Education

Finding 4: Introduction of the concept of inclusive education (IE) through different pilot projects and

work of both civil society organizations and policy makers has prepared the Armenian society for

embracing inclusive education in mainstream public schools. Perceptions and assessment of the surveyed

teachers on the country’s general readiness for inclusive education is rather favorable for rollout.

However, there are also those who believe that the pace of the reforms is too fast and there is more work

needed before comprehensive rollout of the system. Over the years critical steps were taken toward

integration of inclusive education into mainstream schools both on policy and practical implementation

levels. These resulted in a gradual shift of perceptions and opinions as well as general acceptance of the

notion of inclusive education. However, persisting challenges relating to the capacity of specialists,

adequacy of school infrastructure and other concerns give way to certain questions about readiness to fully

embrace IE at different levels and by different groups. In this respect, the work done under the Project is

undoubtedly contributing to changing the attitude of educators, parents, children and others toward

children with special educational needs and their place in mainstream education.

EFECTIVENESS



10

General Findings

Finding 5: The child welfare /inclusive education reforms are complex multifaceted undertakings. As such,

the Project structure reflects the reforms complexity through its multiple components, implementing

partners and coordination with line ministries and agencies. While the Project is intended to support the

Government’s reforms agenda, the reforms pace and project pace are at times in conflict with each other

resulting in delayed Project activities. The Project donor and implementers, the Armenian officials, the

child care institutions’ and public-school staff, and other stakeholders, all agree that the reforms, while

necessary, are complex and sensitive in addition to being resource intensive and requiring long-term

efforts. With the understanding of these complexities and to effectively implement the Project, UNICEF

in Armenia has partnered with the most competent organizations that have long-standing experience of

supporting interventions in these areas. UNICEF, as the lead organization undertook coordination efforts,

communication with government partners and USAID, policy and advocacy work, as well as some direct

interventions in such areas of its expertise as mobilization and training of ISS/case managers, local social

planning and others. While generally several results were achieved in the past 2, 5 years, there have been

certain bottlenecks that either slowed down or diverted the Project activities, including the change in the

Government resulting in delays in approving critical legislation. USAID as a donor as well as implementing

partners, while acknowledging UNICEF’s leading and mobilizing role in the Project, point out to certain

coordination challenges, both with the Government and the IPs. Other stakeholders emphasize UNICEF’s

critical guidance both for the reforms and the project, despite the existing challenges.


Finding 6. While UNICEF and partners have developed and proposed to MoLSA a model for establishing

Child and Family Support Centers and supported reorganization of institutions and their transformation

into CFSCs, several factors contributed to certain inconsistencies and drawbacks in the process, mostly

due to delays and shifts in the reform priorities led by the Government. Noubarashen, Kapan and

Vanadzor have followed different timelines and patterns of transformation that affected institution/center

work and beneficiary experiences with placement/follow up and service provision. According to UNICEF

and World Vision reports, four (4) institutions have been targeted for reorganization under the Project,

including Yerevan Child Care and Protections Institute #1 (Korea Dzor), Yerevan Child Care and

Protections Institute #2 (Noubarashen), Kapan Child Care and Protection Boarding Institution and

Vanadzor Child Care and Protection Boarding Institution. However, the situation with each of these

institutions is different in terms of their legal status, Project interventions, their transformation into a

Child and Family Support Center (CFSC), provision of service, etc. As much as possible, the Project has

attempted to smooth the process by maintaining constant engagement with the Government through

supporting the legal foundations needed for moving forward (decrees, charters, etc), proposing a model

for the CFSCs, training case managers and other social service providers as well as mobilizing partner

NGOs to address the needs of children reunified with their families. By June 2017/ there has been mixed

success, including the establishment of the first CFSC and reunification of children from Noubarashen

with their families (over 50), assessment of hundreds of children and families, provision of support to

reunited families in form of material assistance and specific services. In terms of institutional mechanisms

for the functioning of the social service system, capacity building (on-the-job trainings/coaching and

mentoring) of case managers and other service providers, in-depth assessment of children (more than 400



11

children) from institutions and development of individual care plans (more than 300 children), as well as

development inter-agency protocols of cooperation in the area of child protection have been important

steps forward.

As to the drawbacks, absence of community based family support services in the communities of origin

where children return, has remained the paramount issue to be addressed. While a model for CFSCs has

taken shape, its full application is still to be ensured.

Foster Care

Finding 7. The Project succeeded in conducting a two-phase training for over 74 foster parents. While

improved parenting skills is noted as a training benefit by several of them, delayed placement of children

is creating disappointment among the potential foster families. 74 potential and experienced foster parents

(instead of initially planned 120) were trained by FAR’s Child Support Center Foundation (CSCF) in 5

regions, i.e. Lori, Syunik, Armavir and Ararat Marzes and Yerevan. CSCF also developed 12 training

modules and methodologies such as Guidelines for Monitoring Foster Families and Foster Care Program,

Principles and Standards for Foster Care, Public Awareness Manual. Most training participants (both FG

participants and in-depth interview informants) speak very highly about the qualifications of trainers as

well as usefulness of the trainings resulting in acquiring some critical knowledge and skills. However, for

over a year after the trainings were conducted none of the trained parents had an opportunity to practice

their foster care skills. Many of them expressed concerns that they might soon forget the knowledge and

skills acquired.

Inclusive Education

Finding 8. Implementation wise IE-related interventions have been one of the most effective components

of the Project with trainings conducted among 1670 teachers in Syunik Marz (102 schools) and 3760 in

Lori (162 schools) respectively, and successful involvement of key players and stakeholders (NIE,

coordination with the Republican Centre of Pedagogical Psychological Centre, and other involved

stakeholders including parents). As a result, several key results were fully achieved, including the

establishment of 4 Pedagogical-Psychological Support Centres (PPSC), capacity building of around 200

specialists in Syunik and Lori RPPSCs, training of over 5000 teaching staff with 70% of surveyed teachers

indicating the usefulness of trainings.

Finding 9. Knowledge of the basics of inclusive education has increased significantly as a result of teaching

staff trainings resulting in some level of practical application of learnt concepts and approaches as well as

overall acceptance of the new system (as perceived by surveyed teachers). Assessing the level of knowledge

increase among school teachers from target regions was first done by BoH and partners through pre-post

knowledge tests, according to which overall growth of knowledge among 1670 participants in Syunik is

53%. A knowledge index developed based on the current survey results (+1 to -1 interval) ranks the Syunik

teachers’ knowledge at 0.87. 84, 6 % of respondents also answered positively to the question about

knowledge application. Thus, Project interventions have increased the knowledge on IE among the

teachers in target regions resulting in certain improvements in application as well as improved attitudes



12

toward children with SEN. However, there are still several issues and challenges to be overcome to reach

full-fledged integration of IE in mainstream education. They include but are not limited to teachers’ ability

to assess educational needs of children with SEN, as well as develop and implement individual educational

plans and others.

EFFICIENCY

General Findings

Finding 10: From the Project perspective, the combination of USAID’s G2G support coupled with the

implementation of the soft interventions under the Project (with each partner contributing to within the

overall budget) was initially perceived as a good tool to increase Project efficiency and resource use, but

it did not work as planned. As of June 2017, around 50% of the overall Project budget has been expended

with mixed outcomes that vary from Project component to component. From state budget perspective,

unwillingness of the two line ministries to join certain aspects of the reforms agenda under one umbrella,

has raised questions among different stakeholders about efficient use of funds and resources. Given the

close interconnections between the Project interventions and the two G2G agreements (between USAID

and MoLSA, USAID and MoES), the latter two include strong conditionality language in support of the

Project implementation. However, while this approach should have resulted in favorable consequences

for advancing the reforms supported by the Project, the real progress did not fit the initial plans due to

different reasons. Lengthy negotiations between the two Governments on the different aspects of G2G

funding had certain impact on the Project progress, especially since the Project partners were not involved

in these discussions despite the fact that their respective work was affected by those.

With respect to UNICEF and IP use of Project funding and resources, as of mid-July, over 50% of the

overall Project budget has been spent, with most of UNICEF and IP contributions committed. However,

most of the tangible accomplishments as of June 30, 2017 were attributable to BoH interventions that has

completed its work in two regions (Lori and Syunik) and has spent its budget of over 500,000 USD. Some

other significant accomplishments included training of over 250 case managers and social service providers

that exceeded the initial target of 145 as well as adoption of secondary legislation resulting from UNICEF

in Armenia’s advocacy efforts and laying foundations for future progress. In agreement with all partners,

the unspent portion of SC’s budget was frozen pending the submission of the Family Code to the

Parliament and subsequent activities, while WV also faced an unexpended budget balance (31% spent)

due to delays in its planned work (mainly related to renovation and purchase of equipment and furniture

for the newly established CFSCs).


Finding 11. UNICEF and MoLSA have successfully joined efforts to mobilize all ongoing interventions

related to a child’s rights to protection and social systems reforms to support the Project activities and

interventions. UNICEF and MoLSA have consistently and successfully capitalized on other critical reforms

initiatives, namely introduction of ISS and case management to support the ongoing deinstitutionalization

and IE reforms. This approach has been especially effective in the current situation when the communities

of origin face a lack of services and social workers, and case managers have been closing that gap to the



13

extent possible. Thus, over 100 case managers and 192 social service providers were trained under the

Project, both through in classroom and on job modalities. CSCF worked with 29 case managers in Lori and

Syunik (11 and 18 respectively, as well as additional 16 case managers from Kotayk and Yerevan) providing

on-the-job guidance and mentorship to conduct needs assessment and case analysis both for special schools

and child care institutions.

Finding 12. To a certain extent, both UNICEF, WV and CSCF are all involved in the deinstitutionalization

component of the Project. As a result, there is more expertise and targeted efforts invested on the one

hand, but somewhat confusing role distribution and coordination on the other. UNICEF’s role as the lead

agency responsible for the coordination of the Project implementation was defined in the Project Proposal

and further implementation agreements. However, in addition, UNICEF also undertook several other

responsibilities with respect to assisting MoLSA in developing the strategy on deinstitutionalization and

establishment of alternative services. Though according to the initial Project setup World Vision was

responsible for establishing alternative community based services (including the CFSC model, costing,

staff training, etc.), it eventually became involved in the assessments of children, support of families

reunited with children, and other activities. CSCF also has been involved in child assessments, and

supporting case managers’ work with families. Consequently, a number of functions were replicated by

these different organizations. However, the positive aspect of this situation is that all involved

organizations have vested interest in promoting child welfare reforms and supporting families and

alternative services as part of their mission. Despite the occasional ambiguity of roles and responsibilities,

when needed, they all mobilize available resources to serve the child’s best interest.

Foster Care

Finding 13: The progress and the future of the foster care component of the Project was challenged by the

delays of the approval of the Family Code that would enable children in difficult life circumstances to

benefit from foster care services and the fact that in 2017 no funds from the state budget were committed

to increase the number of foster families, thus challenging foster family motivation. Since 2008, the

Introduction of Foster Family concept has been supported by the RA State Budget, which annually

committed funding for foster care of 25 children deprived of parental care. Currently, 25 children are

placed in 21 families. For 2017 as well, no additional funds were committed to increasing the number of

foster families, which affected Project efficiency since the foster families trainings happened in 2017

without any immediate placement of children in families. However, some positive developments are

expected in this regard in 2018. The 2018 Child Rights Protection Program approved by the Government

on September 29, 2017, plans to increase the number of children in foster families to 120. The State Budget

will fund these items of the Program by committing 166,343.5 thousand drams. It should be noted that

the approval of the Family Code, as well as the allocation of increased State funds for foster care for 2018,

are essential progress in this respect since the amendments will also allow qualifying children in difficult

life circumstances for foster care, thus increasing the potential number of children to be placed.

Inclusive Education

Finding 14: Despite significant knowledge about inclusive education, somewhat favorable perception by

schools and the entire community, lack of special conditions and resources needed has been voiced as a



14

critical challenge to implementing inclusive education. Additionally, the changes in the financing of the

system also affect its efficiency and effectiveness. Ensuring full efficiency of the IE component is

dependent on a good balance between soft interventions and infrastructure improvements, donor

assistance and state commitment. All these factors are also critical for further sustainability of IE

integration into the mainstream education system. While the soft interventions under the Project have

proceeded intensively and effectively, the physical improvements of the schools involved in the IE system

envisioned by the G2G agreement have not followed. In addition to physical resources, lack of specialized

professionals has also been noted as one of the impediments to implementing inclusive education. And

finally, throughout interviews and discussions concerns were raised with respect to financing mechanism

of the inclusive education system, which has affected both the motivation for its implementation and the

involvement of the specialists.

SUSTAINABIILTY

Finding 15: Sustainability of the Project interventions in support of reforms efforts depends on a number

of overarching and interlinked factors ranging from availability of alternative care mechanisms to capacity

of the professionals involved in delivery of services to public awareness and others. While the Project has

successfully established preconditions for some, others still need to be initiated and implemented.

Sustainability guarantees of these multiple and complex interventions are equally diverse and manifold.

They include laying down the foundations of the prerequisite legal framework, availability of financial

resources, availability of professional human resources to implement the reforms at local levels, such as

social services, schools, PPSCs, others, availability of alternative care services, specific guides and

regulations about service operations, public awareness and acceptance of reforms, etc. While some of these

factors are addressed by the Project, there are two aspects of the reforms that, as agreed by interviewed

experts, need significant improvement if sustainability of reforms is to be ensured. These are 1) national

level public awareness building about the reforms and 2) working with parents (both of children to be

reunited with their families and larger parents groups affected by the introduction of inclusive education

into mainstream public schools).

CROSS CUTTING ISSUES

Finding 16. While the Project has emphasized the gender aspect of its interventions requesting data

disaggregation by gender and general sensitivity to gender issues, there seems to be no visible differences

in the way deinstitutionalization or inclusive education is affecting boys and girls. However, with respect

to the specialists involved in both reforms, female professionals have a prevailing presence.

Conclusions: Over the last two decades UNICEF and development partners have successfully advocated

with the Government of Armenia for the child’s right to live in family environment and have access to

quality inclusive education as being in the child’s best interest. By 2014, when the Project was being

designed, the momentum for the reforms in both directions was irreversibly established and the Project

intended to support and sustain that momentum. The Project almost identically mirrored the key aspects

of these two reforms directions by focusing on improving the pre-requisite legal framework, supporting

transformation of existing institutions and establishment of alternative services, engaging into capacity

building of professionals and specialists required for implementation of the reforms and other relevant



15

efforts. Each of the Project components was tasked to a competent organization with reputation and

experience of working in the relevant areas.

By June 2017, the Project has reached some key results, including reorganization of a number of

institutions (both child care and right protection and special schools, establishment of one CFSC and 4

RPPSCs, needs assessment and placement of hundreds of children), training of thousands of teaching staff

under the IE component, as well as other professionals, development and adoption/approval of legal acts,

etc. However, there have also been pushbacks, such as the absence of a unified institution transformation

model, no visible progress with setting up of alternative community based services, delays with the

amendments to the Family Code and subsequent challenges with the placement of the children in foster

care, and others. While some of these interruptions are attributable to the Government’s uneven actions

with respect to their reforms agenda, others have been due to internal Project dynamics.

Generally, given the complex and sensitive nature of the reforms supported by the Project, more time is

needed for effective implementation of some of its aspects. However, even at this point there is confidence

about the impact of the Project interventions, especially with respect to legal foundations and strategic

priorities of the reforms. As discussed throughout the report, adoption of several decrees, concept papers,

State programs coupled with informal statements have cemented the commitment of the RA Government

to the ongoing reforms. Specific steps have been taken to close or downsize institutions (both child care

and special schools) and reunite thousands of children with their families. As such, there is confidence

among stakeholders that deinstitutionalization and inclusive education will be further implemented in

Armenia, in the best interest of the children affected. As overwhelmingly agreed by the evaluation

informants, UNICEF Armenia’s role in this process has been instrumental.

Recommendations included in this part of the report build up on the Findings and Conclusions and are

also grouped by relevant evaluation criteria.

Relevance

1. USAID as the Project donor and UNICEF as the lead Project partner should continue their efforts to

bring to fruition the Government’s reforms with respect to child welfare and inclusive education.

While currently the Project serves as the main locomotive in support of the reforms agenda, and as

such is important, UNICEF in Armenia should also prioritize its policy expertise in terms of child

rights protection and use all available channels to build on the current momentum for these reforms,

which was to a large degree achieved due to UNICEF in Armenia’s continuous efforts over the years.

Effectiveness and Efficiency

2. If possible, as a tactics moving forward, USAID should consider separating the two reform directions

(child welfare and inclusive education) into two standalone projects while UNICEF’s relevant sections

could take over responsibility for each direction (i.e. Education section taking over Inclusive

Education given its longstanding experience with work in that area). While the current structure has

been designed to ensure harmonization of the two reforms and their key directions, coordination

across Government partners, implementing partners, and different stakeholders has been challenging.

If the separation is still under the auspices of UNICEF, the harmonization could be furthered.



16

3. In the future, while undertaking funding of a Project in support of ongoing reforms, USAID should

consider a more structured approach, especially when tying soft interventions to G2G support. One

alterative is starting small and laying the foundations of a bigger project, then expanding it and adding

budget support components once confident about the progress. In case UNICEF undertakes the role

of a Project implementer, it should prioritize pilot projects that showcase the benefits of the specific

approaches of reforms implementation and serve as a model to be replicated by the Government. To

illustrate, with the current child welfare and deinstitutionalization reforms, there seems to be a need

for a reorganization and transformation model and focusing on establishing and making functional

one CFSC could have served as such a pilot. Another alternative is piloting all possible child and family

support services in one region, including CFSCs, RPPSs, strengthening case managers, social workers

and other social services to demonstrate the interconnections between these key institutions and

services and demonstrate how effectively their cooperation could be shaped in support of children in

difficult life circumstances.

4. All Project stakeholders, including USAID as donor, MoLSA and MoES as well as UNICEF should be

aware of any duplications between the roles and responsibilities of different implementing partners

and should define the tasks of partners based on their core competencies. More involvement from the

implementing partners in policy related communication with their respective Government

stakeholders could also result in improved Project coordination.

Sustainability

5. As noted throughout the report, sustainability of the Project efforts is very much dependent on

availability of resources and such factors as availability of community based family support services,

professionally trained specialists, guidelines and manuals in support to service provision and others.

This said, all key players in the reforms, including the Government of Armenia, USAID, UNICEF and

other implementing partners should use available resources and capabilities to ensure these critical

gaps are closed. While each player may utilize different models, methodologies and mechanisms to

ensure further sustainability of the reforms, coordination of efforts will remain a critical priority.

6. In the future, USAID, UNICEF and IPs should consider including a national level public awareness

campaign in similar interventions that support large scale and sensitive reforms. Given UNICEF’s

competencies with respect to designing and implementing such campaigns, it can become one of the

key intervention areas led and managed by UNICEF.

7. Both the Government of Armenia and UNICEF should continue efforts directed to capacity building

of professionals needed for the successful implementation of this reforms both through the Project

and any other channels/initiatives available. In this respect, it is important to identify the key partner

organizations that should be strengthened to support child welfare reforms. While the IE and BoH

experience provides a good example of how the NIE and the Republican PPC became key Project

partners involved in both teacher trainings and capacity building of RPPSC specialists, the role should

be taken over by a State agency and not fall to an NGO active in that area. With respect to the child

welfare reforms, MoLSA should not take over this function given its role as a policy setter but the

National Institute of Labour and Social Research could be considered as a possible candidate for this

role.

8. MoLSA and UNICEF should continue using case managers in support of the reforms implementation

and enhancing their professional experience and resources to work further with children reunified

with their families and other children in need of support. Community social workers should also be



17

brought in to support both child welfare and inclusive education reforms implementation. While a

case manager’s involvement in the ongoing Project and reforms is commendable and necessary, they

are not present in the communities on daily basis and are mostly involved in referrals. Similarly,

regional/Marz administration level departments and their staff are focused on decisions in regards to

placement, documentation processing and other related activities. This creates a gap of specialists on

the community level, and social workers (as evidenced by WV model) could become that important

link in the reforms implementation.

9. Working with parents and children should be an important part of reforms that are touching their

lives and their future. As such, the Project interventions would have benefited from more intensive

engagement with both groups. While this can be partially achieved through a public awareness

campaign, targeted activities are also needed.

And finally, the Government of Armenia (both MoLSA and MoES) should undertake a more robust

leading and coordinating role with respect to all aspects of the reforms, ranging from legislative changes

to capacity building to coordination between different stakeholders including government partners,

donors and implementers. Timely and relevant delivery of all the inputs required for moving forward the

reforms process should be ensured by relevant stakeholders, including adoption of decrees and

commitment of state budget resources (e.g. for foster care or IE) and be responsible and accountable of the

overall reforms outcomes, both positive and negative.



18

CHAPTER I. INTRODUCTION TO EVALUATION

- Object of the Evaluation, Purpose, Objectives and Scope

UNICEF in Armenia’s mandate and operations in the country have consistently emphasized child

protection and inclusive education as priority areas1. Realizing a child’s right to education alongside with

inclusion of children with disabilities in mainstream education, as well as support to ongoing child welfare

reforms through deinstitutionalization and transformation of residential care institutions has been the

focus of UNICEF in Armenia’s 2012-2015 and 2016-2020 Country Programme Documents (CPD). Thus,

throughout the last decade UNICEF has worked with the Government of Armenia to promote policy

reforms and their implementation in these areas. UNICEF has also advocated for increased cooperation

with key development partners to build up consensus around primary strategic reforms in the social sector

that either directly or indirectly support reforms and improved policies in these two areas. Consequently,

several partnerships have emerged, including:

- with the World Bank Around Integrated Social Services (ISS) reforms and capacity building of case

managers that are at the forefront of the deinstitutionalization reforms as the support base for the

children leaving the institutions;

- with the EU in promoting the social protection of families affected by migration, including children,

and finally

- with USAID in supporting deinstitutionalization and inclusive education.

In early 2014, UNICEF Armenia in cooperation with World Vision (WV), Save the Children (SC) and

Bridge of Hope (BoH) submitted a project proposal to USAID to expand alternative care, family support,

and inclusive education services as a part of child-care reforms. It should be noted that while this proposal

was not in response to USAID Armenia’s targeted solicitation, its Country Development Cooperation

Strategy FY 2013-2017 includes a focus on supporting policies that promote social inclusion and provide

services for vulnerable children and the disabled. With the heads of the two agencies (UNICEF in Armenia

and USAID Armenia) understanding the importance of joint efforts in support of Government of

Armenia’s child welfare reforms and inclusive education, a special USAID contracting instrument for

Public International Organizations (PIO) was utilized for approving the project. In compliance with PIO

procedure, UNICEF submitted a concept paper, which then underwent a rigorous review for over a year

before the project was approved. Consequently, in September 2014 the 5-year agreement (09/09/2014 -

09/08/2019) was signed between USAID Armenia and UNICEF in Armenia for implementing the “Toward

Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive

Education Services as Part of Child Care Reform” Project. UNICEF was selected as the lead project partner,

while other implementing partners were involved in the areas of their strong expertise.

Thus, the object of this evaluation is the Project with its key components including child welfare reforms

focused on deinstitutionalization/transformation of institutions and introduction of alternative care

services including foster family system, rollout of inclusive education, as well as any relevant cross-cutting

interventions such as improved case management, setting of legal frameworks and other.

1 http://www.unicef.am/en/articles/aboutunicef

http://www.unicef.am/en/articles/aboutunicef



19

The main goal of the Mid-Term Evaluation of the Toward Social Inclusion of Vulnerable Children:

Expanding Alternative Care, Family Support and Inclusive Education Services as part of Child Care

Reform’’ project (the Project) is to assess the relevance, effectiveness, efficiency as well as impact and

sustainability of interventions with specific objectives defined in the ToR2:

- Family and community based alternative services and their role in prevention of institutionalization;

- Institutional mechanisms for the functioning of social service system;

- The process of strengthening the family substitution service system with the diversification of types

of foster care;

- The process of strengthening the inclusive education system to provide quality learning for all

children;

- Legal and regulatory framework developed to support child-care reforms;

- Institutional cooperation and coordination between implementing partners/key actors (both

government and donor community) in responding to vulnerable families and children in the frames

of the project (including financial);

- Coherence and coordination between the objectives/components of the project to meet the

overarching goal.

Gender and human rights implications of the Project and the reforms supported through it are also an

important aspect of the evaluation.

As to the scope of the assignment, the ToR defined its timeline, geographic coverage and the data

collection methods to be used the following should be noted:

• This is a mid-term evaluation, which covers the period from September 2014 to June 2017. Since the

project duration is 5 years and the closing date was initially set as September 2019, the mid-term

review of the Project intended to “assess the progress in meeting the Project goals and

outcomes….provide early lessons learnt and recommendations as well as identify discrepancies

between expected results and actual achievements” As noted above, the evaluation is intended for the

use of UNICEF, USAID, the Government of Armenia, Implementing Partners and other international

and national partners to inform “further action in the areas of child protection within the context of

deinstitutionalization”;

• Yerevan, Syunik and Lori as the target regions benefiting from Project activities, and;

• Use of mixed quantitative and qualitative methods to collect data from primary informants,

stakeholders and beneficiaries.

- Background of the Intervention and Context

In 1992, the Republic of Armenia joined the United Nations Child Rights Convention (UNCRC), which

entered into force in the RA on 22nd July 1993. Following the ratification of the Convention, Armenia

2 UNICEF Armenia’s Request for Proposals to conduct Mid-term Evaluation of the ‘‘Toward Social Inclusion of

Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as part of Child

Care Reform’’ project (LRFP-2017-9132946) (ToR).



20

initiated legislative changes (including the passage of the Law on the Rights of the Child in 1996) that set

the foundation for furthering a child rights protection system and the child care/welfare reforms.

Another important milestone was the ratification of CRPD by the GoA in 2010 that accelerated child care

system reforms and movement towards inclusive education systems. Several CRPD articles (Article 8,

Article 23) emphasize such requirements as human rights of children with disabilities, respect for the

Home and the Family, preventing concealment, abandonment, neglect and segregation of children with

disabilities, provision of early and comprehensive information, services and support to children with

disabilities and their families. Article 24 is specifically focused on education and the importance of IE.

UNICEF has consistently been at the forefront of these developments, advocating for the rights of children

with disabilities and their inclusion through guiding the evolution of IE practices and policies in Armenia,

followed by technical support for improvement of IE legal framework and strengthening the IE system.

In addition, UNICEF played a critical role in modelling 31 inclusive schools serving over 500 children

with special educational needs and supporting the Government-established center for early identification

of special educational needs and referral of CwD to mainstream education. Such a center was established

by the Government as part of the current action plan for implementation of a fully inclusive education

system by 2025. The plan is to establish 23 Psychological-pedagogical support centres through

transformation of special schools.

From there on the country took several steps toward fostering international best practice with respect to

a child’s right to protection and upholding the child’s best interest, including children with disabilities.

Several legal acts and programs were adopted such as the Amendments to the Law on General Education

in 2014, the 2004-2015 National Programme for the Protection of the Rights of the Child, the subsequent

2013-2016 and 2017-2021 programs. However, despite these important steps, there still exist critical gaps

in terms of enforcing the legislation and implementing an effective child rights protection system in

Armenia. Evidence to such state of affairs is provided by UNICEF commissioned 2008 analysis3 to assess

compliance of national legislation with the Convention, a 2015 Save the Children publication on Child

Rights Situation Analysis: Armenia4, statements and reports by child protection NGOs, and other sources.

While the problems are related to different aspects of children’s rights, overreliance on institutions and a

weak child right protection system has been among identified priorities.

In the past 25 years that UNICEF has been operating in Armenia, it has advocated consistently and

unwaveringly with the Government of Armenia in support of child rights protection and the child’s best

interest. Children’s rights to survival, growth, education and development, family life, protection and

participation, with focus on greater social inclusion, especially of the most vulnerable and excluded

children have been and remain the overarching goal of UNICEF Armenia’s Country Programmes both in

2012-2015 and 2016-2020. UNICEF’s focus on ensuring the child’s best interests are adhered to throughout

different policy reforms and is guided by concerns about child poverty, number of children placed in

different institutions, the weak child rights’ protection and monitoring systems and other important

considerations. More specifically:

3 Legislative reform related to the Convention on the rights of the Child. National case studies: Armenia, Barbados

and Ghana, 2008

4 https://armenia.savethechildren.net/sites/armenia.savethechildren.net/files/library/book-eng.pdf

https://armenia.savethechildren.net/sites/armenia.savethechildren.net/files/library/book-eng.pdf



21

• The child poverty rate in Armenia is higher than the general poverty (29.4) according to Social

Snapshot and Poverty in Armenia, 2017 report5, ranging from 37, 2 % in the age group 15-17 (highest)

to 32.8% in the age group 10-14 (lowest). Generally, larger households with more children are exposed

to a higher poverty risk oftentimes resulting in children’s institutionalization due to social issues.

• Back in 2014, when the Project interventions were being designed, 14 child care and protection

institutions were functioning under the Ministry of Labour and Social Affairs (MoLSA), including 8

boarding schools (for children aged 6-18), 3 specialized care institutions and 3 general care institutions

serving a total of 1431 children, of which 54% were male and 46% female. With respect to

communities of origin the child distribution was the following: 25. 2 % are from Yerevan, 17.7 % from

Lori and 16.4 % from Shirak. While Lori and Shirak had higher poverty rates, there are other regions

with high poverty such as Gegharkunik or Aragatsotn from where the number of institutionalized

children was not that high. As noted in the Project Proposal document, “only approximately 10% of

children in orphanages are orphans. The majority of children in state and non-state institutions are

children from socially vulnerable families who have at least one parent alive…Yet after the placement

in an institution the majority of children spend their childhood there due to the lack of support

provided to families and children by social protection services”.

• Before the amendment of the Law on General Education in 2014, over 2,400 children with disabilities

had been accommodated by 23 special schools that were segregated educational institutions operating

under the Ministry of Education and Science (MoES). However, as noted in the above cited document,

“what brings many of the children to these schools is not only their special education needs but the

hard-socio-economic conditions or various dysfunctions of their families, since schools provide food,

care and shelter.” By 2015 Armenia had 130 inclusive schools providing education to more than 2500

children with special needs. While this has been a significant accomplishment for the country that

has a tradition of reliance on special schools and institutions, it covered less than 10% of Armenia’s

schools and was mostly implemented in urban areas. Children with disabilities are among the most

marginalized groups facing multiple deprivations. As the 2011-2012 national statistics indicate 71 per

cent do not attend preschool, 18 per cent do not attend school, 12 per cent attend special schools, 13

per cent live in institutions (special schools and orphanages), 34 per cent do not attend any community

events and 73 per cent do not use rehabilitation services.

• The current three-tier Child Protection System functioning in Armenia consists of the National

Committee on Protection of Children - a policy setting body at the highest level of the system. The

second tier is comprised of the Family, Women and Child Protection Units (FWCPU) within Marz

administrations and Yerevan municipality that are tasked with implementation and monitoring of the

policies adopted by the government of Armenia with respect to protection of children, including the

protection of children in difficult life circumstances. These units are also responsible for designing and

implementing targeted interventions for the vulnerable beneficiaries. The third tier of this system is

at the community level, where the Guardianship and Trusteeship Committees (GTC) are vested with

authority to make decisions about the care and placement of the children that have found themselves

in difficult life circumstances. There is consensus among several stakeholders of the system that the

only more or less functioning part of the system is its second tier.

5 http://armstat.am/file/article/poverty_2016a_2.pdf

http://armstat.am/file/article/poverty_2016a_2.pdf



22

• From 2010 onwards, UNICEF in Armenia has provided technical assistance to the Government of

Armenia for the identification, introduction and implementation of a model of social services that

would effectively address the needs of vulnerable families in Armenia, and ensure the best interest of

children. The Integrated Social Services model (based on Council of Europe’s Guidelines on

Transformative Social Protection) was both recommended and supported by UNICEF Armenia as one

that has proved to have a positive impact on vulnerable families as an effective system for tackling

poverty, preventing children from appearing in difficult circumstances, being separated from their

families or placed in institutions.

To summarize, while Armenia has made significant progress on child protection in recent years, the

country still faces numerous challenges in meeting its obligations under the Convention on the Rights of

the Child. These include establishing a comprehensive and coherent child care and protection system,

decreasing the number of children in institutions, expanding community based services (including

alternative family-like solutions) to preventing the inflow of children to these institutions

To address the situation, most of UNICEF Armenia’s program components and related interventions are

aimed at ensuring that different child rights are adhered to and fully exercised through its support to

relevant policies and legislation, data collection, specific projects targeting child health, education,

disability, and social protection. With respect to social protection of children UNICEF Armenia has

undertaken several efforts, including:

• Provision of technical assistance for the revised Family Code addressing foster care (2005-2010 CPD);

• In partnership with the European Union, UNICEF Armenia contributed to technical assistance to the

development of policies, plans and normative frameworks for the child protection implementation

and oversight bodies (2005-2010 CPD);

• UNICEF Armenia launched the "Every Child needs a Family" campaign in late 2011. While it is

primarily focused on advocacy for children with disabilities (who are more likely to appear in

institutions), the campaign highlighted at the same time, the issue of deinstitutionalization and the

need build up alternative child care services (2010-2015 CPD).

UNICEF Armenia supports the deinstitutionalization reforms to close residential care institutions for

children and promote community-based professional services as an alternative. In this respect, UNICEF

Armenia’s efforts focused on contributing to review and develop a number of key documents regulating

child protection issues including the National Child Protection Plan of Action 2012-2016; draft Law on

Domestic Violence (December 2017); Government’s decree on Adoption; a policy document on

deinstitutionalization and others.

UNICEF Armenia’s most significant accomplishment with respect to promoting child protection and child

care reforms have been the reduction of institutionalized children from 12,000 to 3,700 in the last two

decades as a result of its consistent efforts to promote and support relevant reforms by the government. It

has been part of these efforts that UNICEF successfully lobbied to secure USAID’s funding to support

transitional costs of the deinstitutionalization reforms of the Government of Armenia and promote

inclusive education in Armenia through the Toward Social Inclusion of Vulnerable Children: Expanding

Alternative Care, Family Support, and Inclusive Education Services as part of Child-Care Reform” Project



23

(the Project). It is expected that such comprehensive assistance could play an essential role in reducing

the number of children in residential care institutions even more and will establish alternative family and

community based services for vulnerable children and their families.

- Logic of Interventions

The Project’s logic of intervention is fully described in the Monitoring and Evaluation Plan including the

Indicator Tracking tables. Five (5) expected outcomes with their relevant output results are reflecting the

key Project intervention areas and contributing to the Project goal, i.e. “Children Realize their right to

live in a family environment and access community based social and educational services”. Table 1 below

summarizes key expected results under the Project.

Table 1: Key Expected Results under the Project

Level Expected Result

Goal Children Realize their right to live in a family environment and access community based social and

educational services

Outcome 1 Institutional mechanisms for the functioning of social service system are set up

Outcome 2 Alternative community-based family support services are accessible to vulnerable children and

families

Outcome 3 Family substitution service system is strengthened with the diversification of types of foster care,

establishment of monitoring and evaluation system

Outcome 4 Inclusive Education System is strengthened and expanded to provide quality learning for ALL

children in targeted regions

Outcome 5 An enabling legal and regulatory framework is established to support Childcare Reforms

While Outcomes 2-4 reflect specific targeted intervention areas under the Project that are implemented

by UNICEF and its partners, Outcomes 1 and 5 cut across the Project and are relevant for all its aspects.

As noted in the M&E Plan, “Lower level indicators (outputs) and their targets will mainly be drawn from

program joint work plans developed with Programme partners. Higher level indicators (outcome) and

their targets will be directly linked to the analysis to be conducted during the Programme implementation

to estimate the impacts of the interventions”.6 In addition to the Project-specific indicators that are

designed to track Project progress, the M&E plan includes a set of USAID’s standard indicators for unified

reporting purposes.

According to the Project Theory of Change the inputs and outputs of the interventions that contribute to

its outcome and impact level results are to be achieved through both mid-term and long-term system

changes. More specifically, it is expected that the strengthening of inclusive education system, family

6 Monitoring and Evaluation Plan: Revision1, June 2016



24

substitution services alongside with accessible alternative community based family services will contribute

to realizing children’s rights to live in a family environment and access community-based social and

educational services. And important aspect of this results chain is an enabling legal and regulatory

framework to support child-care reforms and improved institutional mechanisms for the functioning of

social service system.

While the Government is one of the key stakeholders for achieving such system changes through

legislative reforms, introduction of policies, setting up institutional frameworks and services, the local

communities, civil society and the public at large have an important role to play in ensuring all children’s

rights to live in a family environment, to access health, social and educational services and expanded basic

and alternative care services for most vulnerable children.

To achieve its goals and objectives the Project works through a number of inter-connected components

including:

1. Institutional strengthening of the Social System that targets empowerment of case managers in the

target regions to perform social needs assessments and follow-up the cases of children and their

families with the involvement of appropriate agents. In addition, establishing institutional cooperation

between services, formalizing information exchange and referral procedures related to vulnerable

children is emphasized;

2. Establishment of Family Support Services through region-based approaches to ensure necessary

alternative services are established in the target communities for children, in parallel to the process of

deinstitutionalization from residential care facilities;

3. A comprehensive foster care mechanism with diverse foster care types and robust monitoring

mechanism is available for vulnerable children;

4. Strengthening of the Inclusive Education System through establishing PPSCs in target regions,

training teachers and principals of mainstream school. Adjusting mainstream schools, etc.;

5. The effective implementation of all these interventions entails establishing an enabling legal and

regulatory framework in support of child-care reforms and inclusive education, which cuts across

these four components.

Geographically, the Project interventions are concentrated in Syunik and Lori regions of Armenia as well

as capital Yerevan. Key groups to directly benefit from the Project include:

- Children living in different types of residential care institutions and their families (6,000 persons

including 1,800 children (50% of children living in institutions)) who will be reunited with their

families;

- Schools’ professional personnel to be trained on inclusive education (625 schools, approx. 25,600

professional personnel);

- Children in target communities that will have access to newly established community services

(estimated 20% of total child population in the target community with access to services);

- Foster families (in all of Armenia, but preferably from the community or region) that will host 80

children from transformed residential institutions;

- 45 trained case managers and 100 trained social workers.



25

Indirect beneficiaries of the Project include children and families from nearby communities who can

benefit from the new/expanded services (30,000 persons), schools that will receive support from the newly

established Pedagogical support centers (350 schools, approx. 14,000 professional personnel),

communities, regional and local authorities.

It should also be noted that according to Project design and timeline, interventions in the three target

areas have not been occurring simultaneously, across all components. To illustrate, formally Noubarashen

reorganization happened first, while in Kapan the works proceeded prior to Government decree (passed

in November 2017). With respect to Inclusive Education, trainings happened in Syunik almost two years

prior to Lori. Some of this sequencing was preplanned, while in some cases several factors as well as the

context contributed to the situation (see discussion across the report).

Throughout this report, achievement of planned results under the Project is discussed with respect to its

individual components and the Project generally. However, it should be noted, that given the timeframe

of the evaluation (2.5 years from the start of the Project) and the fact that over 1.5 years was committed

to preparatory work, there is not much progress on the outcome level results to report on at this stage. As

the discussion of the individual Project components indicates, some of the output level indicators have

not yet been achieved as well due to a number of reasons that are discussed throughout the Findings part

of the report.



26

CHAPTER II. EVALUATION METHODOLOGY

In response to the goals and objectives of the assignment as well as the scope defined by the ToR, the

evaluation team proposed and implemented an evaluation methodology comprising of the following

components:

• Stakeholder Mapping and Approach to Designing the Evaluation Sample

• Data Collection Methods, including:

- Desk Review

- Qualitative Methods

- Quantitative Methods

• Data Analysis and Reporting

In compliance with UNICEF in Armenia’s requirements included in the ToR, mixed methods were used

for data collection purposes to ensure “complementary strengths and non-overlapping weaknesses… The

analysis will be built on triangulating information collected from different stakeholders and through

different methods…”7 As detailed below, in agreement with UNICEF, quantitative data collection targeted

teachers involved in IE training in Syunik region under the Project. As such, the IE related findings of the

report triangulate data from the survey, in-depth interviews, FGs as well as a desk review. For all other

components, the analyses build on the in-depth interviews, FGs as well as any quantitative or qualitative

information derived from the desk review.

Since understanding project and evaluation stakeholders is a critical part of developing an evaluation

methodology for elaborating an evaluation sample and choosing the most effective data collection methods

with respect to each group, identification and mapping of key stakeholders was conducted. The exercise

derived information from the evaluation ToR, desk review of materials provided by UNICEF, as well as

initial meeting with UNICEF and Implementing Partners. Stakeholder mapping was conducted with the

following considerations in mind:

• While the overarching goal of the Programme is to ensure that children “realize their rights to live in

a family environment and access community based social and educational services”8, it consists of a

number of components some of which are cross-cutting, while others can be also viewed

independently;

• Those aspects of the Project that relate to advancing the Child Care reforms agenda, establishing an

enabling legal and regulatory framework, institutional mechanisms and local social planning cut across

all the Project components and require commitment from the Armenian Government and major

partners including the Ministry of Labour and Social Affairs, Ministry of Education and Science,

National Institute of Education, Marz administration as well as USAID, UNICEF and other

international players;

7 ToR

8 Project Indicator Definition Table



27

• Each of the key focus areas of the Project such as the alternative community-based family support

services, family substitution service system and inclusive education have a specific target group as

beneficiaries and stakeholders.

With these considerations in mind key stakeholder groups and sub-groups were identified and are

summarized in Table 2 below.

Table 2: Stakeholder Mapping

Project beneficiaries

• Children in difficult life circumstances living in different types of

residential care and educational institutions

• Children in target regions/communities benefiting from newly

established services

• Families/parents of these children including foster families

• Case managers and social service providers (police, health, education,

regional and local self-governmental bodies, etc.) whose capacity is

enhanced to work with the above groups

• School professional personnel whose capacity is enhanced to work with

the above groups

National level key stakeholders • National Assembly

• Government (Prime Minister’s Office) of Armenia

• Ministry of Labour and Social Affairs (MoLSA)

• Ministry of Education and Science (MoES)

• Ministry of Territorial Administration and Development (MoTAD)

• National Institute of Education (NIE) and trainers involved

• National Institute of Labour and Social Research

• NORK Informational Center

Regional/community level key

stakeholders

• Regional administration officials involved in child care reforms (i.e.

heads and staff of relevant administration units such as Marz

departments on Family, Women and Child Protection (FWCPU), Social

Security and Health, Education, Science and Sport departments),

Guardianship and Trusteeship Commissions, local self-governmental

bodies, i.e. community leaders

• NIE regional structures in Lori and Syunik

• Institutions and newly established RPPSCs, FCSCs, residential care

facilities, special schools

• Mainstream schools involved in inclusive education reforms

Other Stakeholders • USAID, UNICEF and other international organizations such as EU,

UNDP or World Bank

• Project Implementing partners

• Office of Human Rights Defender, NGOs and experts (including legal)

involved in child protection in Armenia

Based on the above stakeholder mapping and the RFP requirements, a sampling and data collection

methodology was developed. Extensive discussions between the evaluation team and UNICEF in

Armenia’s team resulted in the following key agreements about data collection from each specific group:



28

• Children that are the primary beneficiaries of the child welfare and inclusive education reforms

and Project initiatives were not targeted for the purposes of this particular assignment, since it

focuses primarily on the processes and mid-term accomplishments rather than the impact of the

Project.

• Quantitative data collection followed the requirements set in the ToR and targeted teachers

involved in Bridge of Hope trainings on inclusive education with a focus on Syunik region where

the intervention has taken place over the last two years.

• Different qualitative data collection methods (detailed below) were used for collecting data from

all the other stakeholder groups, including teachers.

- Qualitative Data Collection: Approach and Implementation

Qualitative data collection: the following methods and related sampling approach were utilized for the

purposes of this evaluation:

• Expert interviews: Since expert interviews are used to gain insights from a person's special knowledge

and experiences which result from the actions, responsibilities, obligations of the specific functional

status, this method was used for data collection from national level key stakeholders (ministries, NIE,

etc.), as well as some of the stakeholders identified under “Other Stakeholders” category, i.e. Office of

Human Rights Defender, legal and child protection experts, etc. Information saturation approach was

used by the evaluation team, i.e. interviews were conducted as long as new information was being

received. In addition to the initially identified list of experts, snowballing approach was also used for

engaging additional experts referred to by those already interviewed (which resulted in dropping off

some experts from the initial list). In total, 25 experts were interviewed from this stakeholder group,

which also includes 5 UNICEF staff members (interviewed as a group). Annex 3. List of Respondents

by Data Collection Method includes the list of the experts involved in the evaluation (expert interview

is specified as the method of data collection).

In depth interviews (including group interviews) with key informants: For the purpose of this evaluation

in-depth interviews were used to collect data from Project beneficiary/stakeholder groups identified

through a stakeholder mapping (see Table 1). They were parents/family members, school principals, staff

of RPPSC, CFSC, regional authorities, service providers and others. The sampling took into account such

factors as geography (Lori, Syunik and Yerevan), and types of informant groups related to each aspect of

the project as well as cross cutting groups (e.g. parents, staff of institutions from which children were

released (care institution, special school, residential, etc.), different service providers etc.). Gender was a

key consideration, and as much as possible, a balance was ensured among the key informants. To ensure

validity of the information received from each sub-category under this stakeholder group, a minimum of

3 interviews was conducted with the representatives of each particular sub-group, including

families/parents, case managers and social service providers, school principals, staff on institutions and

newly formed services. Table 3 below summarizes the interviews conducted for this particular data

collection method.

Table 3: Distribution of In-Depth Interviews by Marz and Informant Groups



29

Parent/Family member

(possible gender balance

ensured)

School Principals Former/current

Institutions and

alternative

services

Marz level authorities and

social service providers

Yerevan • 2 interviews with

biological family

members/parents of a

former Noubarashen

child resident

2 school principals 1 staff members

of Yerevan night

care institution

N2 in

Noubarashen and

Day care center

1 interview with a social

service provider

2 interviews with case

managers

Lori • 1 interview with

experienced foster

family members

• 4 trained foster parents

2 school principals

(urban/rural)

2 interviews with

Vanadzor Child

Care and

Protection

Boarding

Institution staff

2 interviews with

Vanadzor, Spitak,

RPPSC

representatives

1 interview with heads of

Marz administration

departments

4 interviews with case

managers and social

service providers (e.g.

police)

Syunik • 3 interviews with

biological family

members/parents Kapan

Child Care and

Protection Boarding

Institution

• 1 interview with a

family member/mother

of a former special

school child

2 school principals

(urban/rural)

1 interview with

Kapan Child Care

and Protection

Boarding

Institution staff

3 interviews with

Kapan, Sisian and

Goris RPPSC

representatives

2 interviews with

representatives of

special schools for

Children with

Intellectual

Disabilities in

Kapan, Goris

3 interviews with heads of

Marz administration

departments

4interviews with a case

manager and a social

service provider

1 SG interview with

Kapan FWCPU

Thus, in total, in-depth interviews (including more than 4 small group interviews) were conducted with

over 40 key informants.

• Focus groups: For the purposes of this evaluation FGs were conducted with trained teachers,

parents (trained under the inclusive education component), trained foster parents, staff of

institutions and services. The participants of Focus Group discussions were selected from the



30

databases received from the implementing partner (BoH and CSCF) and finalized at the place

depending on availability of requested participants. However, an effort was made to ensure a

balanced representation of different subject teachers as much as possible. Homogeneity was

ensured by limiting the FGs to teachers only without principals or other management staff present.

The same approach was taken in case of RPPSC staff focus groups. Most of FG participants for IE

were female reflecting the general statistics in Armenia with over 90% of teachers being women.

FGs were conducted by a moderator and an assistant, the former moderating the discussion and

the latter taking notes and providing technical support.

Table 4 summarizes Focus Group distribution by Marz and type of key informant

Table 4: Focus Group Distribution by Marz and Type of Key Informant

Foster families Teachers Parents Institution staff

Yerevan 1 FG (6 participants

from Yerevan,

Armavir and Ararat)

1 FG with

Noubarashen staff (6

participants)

Lori 3 FG (in Vanadzor

school # 16, Lernapat

school, Spitak # 5,

total 26 participants)

2 FG (in Vanadzor

school #16, Lernapat

school, total 13

parents)

1 FG with Spitak

RPPSC (6

participants)

Syunik 2 FG (Goris school #

3, Angeghakot

school, total 20

participants)

2 FG (Kapan school #

5, Goris school # 3,

total 14 parents)

1 FG with Kapan

RPPSC (6

participants)

1 FG Kapan Child

Care and Protection

Boarding Institution

(7 participants)

In total, 14 Focus groups were conducted reaching out to 104 participants.

Some additional qualitative data collection methods that were employed by the evaluation team are

described below:

• Observations were used for better understanding, as well as comparing and contrasting the work

of institutions and newly provided services focusing on the following two aspects: physical

infrastructure and its relevance, as well as service provided/methods used. Observations were

conducted in the following locations:

1. Noubarashen Family and Child Support Center

2. Goris RPPSC and Vanadzor former special school

3. 2 public schools in Lori and 2 in Syunik Marzes



31

And finally, a Desk Review of all the documents provided by UNICEF and IPs (such as quarterly reports,

ITTs, legal documents, assessments and research, training modules, pre-and post-knowledge tests) was

conducted as an integral part of the overall data collection (see Annex 2. List of Desk Review and

Background Materials).

Prior to the fieldwork the qualitative data collection guides were developed, pre-tested and shared with

UNICEF in Armenia (see Annex 4. Interview Guides and Survey Instrument).

- Quantitative Data Collection: Approach and Implementation

In compliance with the ToR (Annex 1) for this assignment, trained school teachers under the Inclusive

Education component (implemented by the BoH) were the target respondents of quantitative data

collection. As agreed with UNICEF during initial meetings, the evaluation prioritized Syunik for the

quantitative data collection given the lengthier experience (2 years) of the schools in this region of

involvement in inclusive education.

Quantitative interviews were implemented with 312 teachers of Syunik Marz.

The initial sample size was determined with 5% error margin and 95% confidence interval. The following

sample size formula for infinite population is used to arrive at a representative number of respondents

when population estimate is known (Godden, 2004):

𝑛 =𝑍2×𝑝(1 − 𝑝)

𝑀2

Where:

n = Sample Size for infinite population

Z = Z value (1.96 for 95% confidence level)

P = population proportion (expressed as decimal) (0.5 (50%)

M = Margin of Error at 5% (0.05)

Since the general sample (i.e. trained teachers in Syunik) is 1670, the calculated sample size is 312.

A multi-stage randomized sample was then constructed on the following levels:

1. The number of the urban and rural schools to be involved in the sampling was decided based on the

rural-urban distribution of teachers in Syunik (54% and 46% respectively), 168 of surveyed teachers

represented rural areas, while 144 came from urban areas.

2. Taking into consideration that the number of rural teachers is at least 8, and the number of the urban

teachers is at least twice more, respectively 8 and 16 interviews were conducted in each rural and

urban school selected into the sample, i.e. 21 rural and 9 urban schools that were selected. These 21

rural and 9 urban schools were allocated according to the service areas of RPPSCs in Sisian, Goris and

Kapan (which also serves the Meghri region) based on the proportion of schools in each area. The

rural/urban distribution in each area is presented in Table 5.

Table 5: Proportion of Schools by RPPSC Service Area



32

Proportion in percentage Interviews conducted Total

Urban Rural Urban Rural

Sisian 19% 31% 32 64 96

Goris 22% 22% 32 36 68

Kapan 59% 47% 80 68 148

Total 100% 100% 144 168 312

3. Site visits indicated that while in all urban schools of these areas there exist assessed children, the

situation is different in some rural communities with some schools still not having assessed the

children. Thus, the sample construction took into consideration this factor as well.

Table 6: Number of Schools Included in the Sample Based on Availability of Assessed Children from

Rural and Urban Schools

4. In the next stage of sampling, the schools were randomly selected from the alphabetical list and

the sample was constructed based on the above discussed factors. Table 7 below provides the list

of sampled schools.

Table 7: List of Schools in the Final Sample

Area Urban Rural

Inclusive Non-inclusive

Kapan Kapan #3 Davit Bek Artsvanik

9 21 schools were planned initially. However, 22 were selected into the sample to have substitution options in case

if the foreseen number of interviews would not be possible to conduct in the initial 21 schools.

Distribution of Schools per Assessment Factor

Urban Rural

There are assessed

children

There are no

assessed children

Sisian 2 4 4

Goris 2 2 2

Kapan 5 5 5

Total 9 11 11

229



33

Kapan #5 Dzagedzor Achanan

Kapan #6 Geghanush Norashenik

Kapan #10 Lernadzor Okhtar

Kapan #13 Syunik Verin Khotanan

Goris Goris #2 Karashen Khndzoresk

Goris #5 Tegh #1 Kornidzor

Sisian Sisian #1 Angeghakot Aghitu

Sisian #3 Ishkhanasar Bnunis

Shaghat Uyts

Shaki Tolors

Prior to fielding the quantitative survey, the questionnaire was pre-tested among 15 respondents (5% of

the planned interviews). For the purpose of effective resource management, the interviews were

conducted at 2 schools of Yerevan. The basis for the selection of schools for the pilot was the level of

successful implementation of inclusive education at schools assessed by study experts. Per expert estimates

three types of schools were classified: very successful (e.g. Mainstream School N 100), medium successful

(e.g. Mainstream School N 160) and less successful (e.g. Mainstream school N 125). Interviews were

conducted in school # 100 and # 160 with 8 and 7 interviews respectively.

The instrument for standardized face-to-face interviews included questions under the following main

domains:

1. Inclusive Education

• General awareness/knowledge

• Perceptions

• Practice

2. Assessment of training relevance and effectiveness

3. Availability of Supportive Resources/RPPSCs

4. Respondent profile (socio-demographic and other information)

Majority of questions included in the questionnaire are close-ended with a number of open-ended

questions that were coded based on verbatim quotations once the database was created and processed.

The questionnaire is attached to this document as Annex 4.

With respect to data analysis and reporting, VISTAA strictly followed the requirements of the RFP for

using the evaluation criteria and evaluation questions as an analysis framework, guidance for providing

good recommendations in compliance with findings, disaggregation of data, ethics and gender

considerations, and others.

Data analysis: Transcribing, tabulating and systemizing data gathered through both quantitative and

qualitative methods and cross checking with findings of the desk review and observation records was used



34

as a basis for analyzing and presenting the data. All data received through desk review, FGs, interviews

and surveys were analyzed and systemized to be used for the findings and recommendations of the report.

VISTAA used the SPSS statistical package/software for quantitative data entry and analyses.

Quality control of the collected data was performed by the Field Supervisor & Quality Coordinator under

the overall supervision and guidance of the Quantitative Expert responsible for all the stages and aspects

of the quantitative survey. All standardized questionnaires were checked by Field Supervisor & Quality

coordinator, by reviewing the questionnaires, checking and cleaning the database.

As to the control of qualitative data, data collection followed evaluation questions per evaluation criteria

to ensure consistency of the information received. IE related findings of the report triangulate data from

survey, in-depth interviews, FGs as well as desk review. For all other components, the analyses builds on

the in-depth interviews, FGs as well as any quantitative or qualitative information derived from the desk

review.

- Major Limitations

Some data limitations were identified both in the evaluation design and implementation phase. One of the

primary limitations was related to the challenges of reaching out to the children that are the primary

beneficiaries of project interventions. While this was not a requirement of the ToR, the evaluation team

approached the issue from different aspects, trying to assess to what extent this gap will affect the

evaluation rigor. However, as noted above, consultations with UNICEF in Armenia resulted in the mutual

agreement that at this stage the focus of the assignment was mostly on the implementation processes and

mid-term results, rather than the impact of the interventions, which should be visible primarily on

children. In addition, interviews and discussions (as well as the survey instrument) with parents/families,

institution and service staff, school teachers and principals always included inquiries about children’s

opinions and perceptions on how the reforms in question affect their wellbeing (while this was giving

secondary data on children’s opinions and perceptions, it was still considered to be valid information).

Some other limitations included:

- The timeline of the evaluation: several of the key results expected to have been accomplished as part

of the Project progress occurred beyond the timeline following the groundwork laid by UNICEF in

Armenia and its partners. However, since the field work happened in the fall of 2017, it was at times

difficult keeping the informants focused on the evaluation timeframe. To illustrate, some institutions

reorganized after June (such as special schools into RPPSCs), but since interviews were happening in

October they touched upon issues relevant to the reorganization.

- Data Collection Limitations: In very limited cases (11 of 312), surveys were not done through face to

face interviews, but distributed to teachers, explaining the instrument logic and asking them to fill in.

This was due to time limitations and short working days of teachers, who wanted to leave earlier or

in a few cases when the interviewer team was delayed due to road conditions or other unexpected

issues. A lower number than expected participants were present at parent groups FGs, than initially

planned because of teachers’ involvement as parents. While the teachers could also be interviewed in

their roles as parents, VISTAA decided removing them not to influence the expression of opinions by

other parents.



35

- The different intensity and sequencing of interventions in Lori and Syunik also caused some

divergence in data, given the different regional experiences and cycles with respect to the Project

implementation. This was also one of the reasons why Lori was not included in the teacher survey,

assuming that the BoH trainings happened there just recently, leaving no time for the schools to

implement inclusive education integration into mainstream public schools and reflect on perceptions,

applications and other aspects included in the survey instrument.

- Generally, given the sensitivity of the reforms directions supported by the Project

(deinstitutionalization and inclusive education), their implication not only by the directly affected

groups but the larger society and public, there was significant diversity of opinions and perceptions

on many of the evaluation questions making generalizations somewhat challenging.

- Ethical Considerations, Human Rights and Gender

Ethical dimensions were taken into consideration by the evaluation team, particularly when working with

beneficiary families (vulnerable families with children). Prior to conducting interviews and focus group

discussions, the evaluators consulted them and ensured their informed consent. The evaluation team

protected the anonymity and confidentiality of the individuals and observed ethical guidelines as set out

by United Nations (UN) Evaluation Group in its standards and norms10.These standards are based on the

following principles taken into due consideration:

• Hold in confidence all information given to the evaluation team by the Client;

• Explain the rationale, objectives and the methodology used to institutions and /or individuals

taking part in the evaluation and receive informed consent from all participants;

• Listen and value all participants’ views during interviews and focus groups;

• Respect the values of the Client for whom the evaluation is undertaken;

• Reflect on own bias due to previous experience, education background, gender, ethnicity and

other factors;

• Respect the anonymity of individuals when analyzing data;

• Cross-check and triangulate all results, wherever feasible;

• Feedback on the research results to the participating institutions or individuals, where the Client

approves;

• Prohibit use of the research data for other studies without Client and participants’ permission.

Human rights based approaches to development and gender mainstreaming are considered as foundational

normative principles for UNICEF. Therefore, the extent to which these principles have been adhered to

by UNICEF Armenia in implementing the current Project, they were integrated across the evaluation

10 http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4

http://www.uneval.org/normsandstandards/index.jsp

http://www.unEvaluation.org/ethicalguidelines

http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4

http://www.uneval.org/normsandstandards/index.jsp

http://www.unevaluation.org/ethicalguidelines



36

focusing on human rights based approach and gender considerations that produce equitable impacts on

children’s lives and realization of their rights. A special finding is included in this report on the gender

aspects of Project interventions as required by the ToR. The team has made an effort to review the impact

of the on-going child welfare and IE reforms on both girls and boys (to the extent available that data

allowed for) as well as assessing gender balance of different stakeholders involved in the reforms and

affecting it, such as teachers, foster parents, etc.



37

CHAPTER III. FINDINGS

Evaluation findings are presented by evaluation criteria i.e. relevance, effectiveness, efficiency,

sustainability and impact. As prescribed by the ToR “Given that this is a mid-term evaluation, two of the

evaluation criteria this project will be evaluated against, namely sustainability and impact, are to be

considered to the extent possible”. This issue was discussed later with UNICEF partners as well to

reconfirm the position that sustainability and impact will be addressed only to the extent possible. Some

of the findings discussed under each evaluation criteria are general in their nature, pertaining to the overall

Project and the reforms agenda supported through the Project. Others are specific to each individual

component of the Project, such as Inclusive Education, deinstitutionalization and transformation of

institutions, foster care, institutional mechanisms for functioning of social service systems, etc. One cross-

cutting finding discusses gender aspects and the implications on the Project.

RELEVANCE

General Findings

Finding 1: Due to UNICEF’s and partners’ continuous efforts and advocacy, policy makers and decision

makers in Armenia have realized the importance and urgency to act upon their commitments on

CRC/CRPD and initiate reforms that emphasize family environment and inclusiveness in education, as

well as minimize reliance on institutions. As a result, the number of institutionalized children has

decreased significantly. However, overall acceptance and understanding of the reforms varies from high

to limited among the different stakeholder groups.

As detailed in the background section of this report, in the past 25 years UNICEF in Armenia, with

coordinated efforts of all partners involved in this programme, including World Vision, Save the Children,

Bridge of Hope and FAR CSCF, were heavily focused on advocating for and supporting child protection

and inclusive education reforms and policies introduced and implemented by the Government of

Armenia. This consistent engagement with the MoLSA, MoES and other partners resulted in “increased

acceptance of the need to transform institutions and set-up of alternative services”11, including12:

• In 2005 the amendments to the Family Code were approved incorporating provisions on foster care;

• September 2010 ratification of the Convention on the Rights of Persons with Disabilities (CRPD) by

the Government of Armenia;

• In 2011 the Government declared de-institutionalization a priority;

• In 2012 the Minister of Education and Science, and the Minister of Labour and Social Issues signed a

joint decree to start the process of transformation of the institutions under the MoES. International

Classification of Functioning (ICF), Disability and Health was introduced as a tool for assessing and

11 Project Proposal: Prepared by UNICEF Armenia in cooperation with WV, SC and BoH

11 UNICEF In Armenia’s CPDs, annual reports and Project quarterly reports are used as sources



38

certifying disability and special education needs thus shifting the model for understanding disability

and planning interventions from medical-based to social-based. In 2017 3 ICF codes were

contextualized and piloted in Armenia, then adopted by the Government;

• In 2013 both ministers requested support for the development of transformation plans and the revising

of funding plans for residential care institutions. The Prime Minister approved the transformation

plans for two residential care institutions, which were developed with technical support from UNICEF

-.

• In 2014 amendment to the Law on General Education introduced elements that are critical for the

further development of an IE system in Armenia. The Project supported drafting of a number of bi-

laws that contribute to the operationalization of those amendments;

• The new State Programme for Education Development 2030 also includes provisions on ensuring full

inclusiveness of the education system by 2025.

• From 2010 onward introduction of the Integrated Social Services system emphasizing reduction of

social needs and professional case management has been supported by UNICEF as an alternative to the

much criticized three-tier child protection system;

• The Government decision of 10.03.2016 approved the Concept for Foster Care of Children in Difficult

Life Circumstances allowing for different types of foster care. As part of the process, 12 legal acts and

proposed amendments to existing acts have been drafted and once adopted, will ensure that children

in difficult life circumstances benefit from foster care services alongside with children deprived of

parental care. It is a major result in the Government’s commitment to fund 120 more foster families

from the state budget as of 2018;

• The National Strategy for Child Rights Protection 2013- 2016 and 2017-2021 contain strong provisions

further consolidating commitments to transforming the child welfare system. In addition, the

Government Programme for 2017-2022 and Human Rights Action Plan for 2017-2022 prioritize de-

institutionalization reforms.

The 2013-2016 National Strategy for Child Rights Protection declared the child’s wellbeing in a family

environment and in the society a priority through reducing the number of children in child care and

protection institutions and preventing placement of more children into these institutions. The goal and

the priorities of the current (2017-2021) National Strategy include protection of interests of children in

difficult life circumstances, improving the system of child’s right protection and ensuring equitable,

inclusive and quality education. The actions envisioned by the Strategy include reorganization of large

institutions into social centers, improving the system of trusteeship and guardianship, enhanced foster

family care, establishment of daycare centers, i.e. steps that will allow keeping the child in the biological

family. Institutions are considered only as a last resort with an emphasis on the idea of small group homes

instead of large facilities when institutionalization, in exceptional cases, cannot be avoided.

All these developments allow believing that the momentum for the reforms were established by late 2014,

when the Project interventions started. The Project itself almost identically mirrors the reforms directions

and provides support to its key interventions to sustain these achievements.



39

The Government of Armenia’s official position states the child’s right to live in a family environment

under parental care, and targets return of children currently placed in state care to their biological family

or to alternative care. Similarly, inclusive education has evolved into a mainstream policy direction

implemented across the country after the 2014 Amendment of the Law on General Education. In 2016,

the Government of RA approved the Action Plan and Timetable for Provision of Total Inclusive Education

System. Finally, the 2017 draft State Programme for Education Development 2030 has been developed

taking into consideration the United Nations’ Transforming our world: the 2030 Agenda for Sustainable

Development and Incheon Declaration on Education 2030. Ensuring inclusive and equitable quality

education and promote lifelong learning opportunities for all.

In addition to these formal undertakings, officials at both MoLSA and MoES have articulated their

commitment to the ongoing reforms and have consistently upheld their position both in the public and

during discussions with UNICEF and other donors. All these efforts have resulted in several initiatives

that allowed decreasing the number of institutionalized children in Armenia through different pilot

projects and roll out of the reforms, which are supported by this Project.

The Government’s commitments are supported, to a certain extent, by those stakeholders that are directly

involved in the implementation of the reforms, i.e. staff of reorganized institutions and public schools,

regional authorities, case managers and social workers. However, the challenges that persist with respect

to both reorganization of institutions, introduction of inclusive education, setting up new social services,

at times unpreparedness of biological families to deal with returning children, create uncertainties about

the timing and sequencing of the ongoing reforms. Opinions like “it is too early”, “communities are not

ready”, “services are lacking” have been voiced by different key informants. With respect to families who

are directly impacted by the reforms, reactions are twofold. While emotionally and morally their majority

seems happy to have the child back, social and financial implications of the move make them question

whether the child is benefitting from reunification with the family. Similarly, with respect to inclusive

education, families voice questions about the readiness of the mainstream school system and the larger

public to accept the integration of children with disabilities.

All institutions should close down, children should be sent back to families. Expert Interview, National level

official

The idea is very good, but the families are not ready for this, yet. They should be prepared both financially and

psychologically for this change- to be able to reunite the family. In-depth interview, Regional administration

I have been lucky to have those (services) several years when the state helped me out with the boarding

kindergarten and institution (for my child). What should I have done, if not for that opportunity? Who would

have taken care of my child while I was earning the living for us? In-depth interview, Parent

It is too early to speak about the results of the introduction of the inclusive education. We are reforming and

learning in the process. Expert Interview, Independent Expert.


Finding 2. Different Project stakeholders (including policy makers, institution staff and families) confirm

that due to UNICEF’s leadership and consistent efforts, there is overwhelming agreement about the family

being the most favorable environment to ensure a child’s best interest and wellbeing. However, since the

launch of relevant reforms (deinstitutionalization) new and flexible solutions were required from UNICEF



40

and partners to address the situation with the lack of a functioning system (in terms of alternative social

services, acceptable quality of life at home, social conditions, etc.).

The common opinion voiced by different Project stakeholders (such as staff of the transformed and to-be

transformed institutions, the heads of the Family, Women and Children’s Right Protection Departments

of the regional administration, WV as an implementer, government officials and the Human Rights

Defender’s Office, and others) confirmed that the child welfare reforms were needed and timely. Over the

years, a consensus has been reached about the family being the best natural environment for the child to

grow and develop, thus serving the child’s best interest and wellbeing. As discussed throughout this report,

the momentum for deinstitutionalization reforms has been built over many years due to UNICEF’s and

the partners’ advocacy resulting in the Government declaring deinstitutionalization a priority in 2011,

and the consequent developments. The MoLSA’s commitment made to the RA Prime Minister to close all

night boarding institutions by the end of 2017 is another important evidence as to how these efforts have

paid off.

Overall, UNICEF and the partner’s efforts resulted in the reduction of institutionalized children from

12,000 to 3,700 in the last two decades due to consistent efforts to promote and support relevant reforms

by the Government. With respect to specific Project results, by June 2017, 30413 children from child care

institutions and special schools were reunited with their families, including 140 children from Lori (3

special schools and one night care institution), 103 in Syunik (3 special schools) and 61 from both

reorganized institutions in Yerevan (Noubarashen night care and 3 1 residential care) as a direct result of

the Project interventions. It should be noted that in the period between 2014-2017 when the Project was

operational, the Government of Armenia reported reunification of around 7,000 children from the

institutions downsized across the country, which was due to direct and/or indirect advocacy by UNICEF

and partners within the Project. Some quotes below provide evidence as to how positively some of the

key informants perceived family reunification.

Of course, it is not even questioned that the children from even the poorest families would prefer going to bed at

home, and waking up with their parents (and siblings), rather than in an institution… and we are talking about

the emotional bonds within the family, which clearly does not exist in any, even the best, institution. In-depth

interview, Regional administration

It is a joy to know that my kids are at home…it was a torture to have them sleeping not at home (crying)…I will do any job to

keep them alive. In-depth interview, Parent

However, despite the perceived need for deinstitutionalization reforms and several specific steps taken to

ensure reunification of children with their families, there is also agreement that it is not an overnight

process and it needs serious preparation. Reforms stakeholders (social workers, Marz level officials, staff

of institutions and CFSC) believe that it should not be done prior to providing the community based

alternative services, as well as the individual solutions for each family whose children return home, which

is said to oftentimes be the case with the ongoing reforms. Many informants also pointed to the reasons

which caused families to place their children in institutions in the first place (supported also by UNICEF

13 The numbers have been provided by UNICEF to USAID based on the information received from each relevant IPs

and Government of Armenia.



41

reports and the in-depth assessment implemented in 2015 by the “Center for Educational Research and

Consulting” funded by World Vision). While social and poverty related considerations prevailed in many

cases, there are others that are equally valid, including disability of a parent, alienated parents, and other

factors that at times make the child’s return to the family traumatic and in conflict with the child’s best

interest.

I myself oppose the fact of drastic change in terms of immediately sending the children home without having all

the services established and fully operating, especially with the current state of communities. In depth interview,

Stakeholder/Institution

We deal with young human beings in the process of development and self-establishment. This returning back to

the family is going to be as hard as the bringing the children to the institution. Long-time work is needed with

the children and parents to help them to reunite. The children very often see their teachers as a bigger authority

than the parents, which creates problems at home both for the children and parents. FGD, Stakeholder/Institution

It should be noted that in compliance with the internationally recognized norms and requirements with

regards to children’s rights, the country level implementation of the reforms emphases two key pre-

conditions: 1) strengthening the family’s ability to nurture their children and provide appropriate care,

and 2) establish community based social centers (Child and Family Support Centers (CFSC)) to provide

the needed support and services to children leaving institutions to return to their families. The CFSCs and

other social services are expected to support the family in a number of child-related activities, such as

“organization and provision of services of socio-psychological, socio-pedagogical and socio-legal nature”14;

social skills consultations to the children and their families; “provision of vocational orientation, art-based

therapy and occupational therapy” 15; organization and provision of primary medical aid and healthcare

services, as well as other types of services. With respect to family enhancement, while financial and in-

kind support (25,000ADM during 6 months and food/other necessity packages) is considered an important

part of preparing them to receive the children back home, improving parenting skills is a viewed as an

essential aspect of CFSC functions alongside with other relevant interventions described above. In addition

to the social services, it is expected that several institutions and professionals should extend support to

these families ranging from case managers, to Marz administration, to schools, and others.

While certain elements of this approach are implemented in Armenia, numerous challenges and concerns

still exist. Key informants prioritized some of those, including:

• Transformation of institutions and creation of CFSCs is a complex process, including aspects related

to assessment of children and families, capacity building of staff working in the centers, physical

adaptation of the centers, and others that require significant time to accomplish and are still

underdeveloped.

14 Description of Provided Services within the framework of “Provision of Social Services to Children by Social Care

Day Centers” Program, RA MoLSA, October, 2016

19\ ibid



42

• Readiness of the community of origin is another issue. While the institutions are transformed into

CFSCs, the children return to their own communities where the services are not available, their

integration both into family, school and the general community is an issue. Foster care as an alternative

is not fully functional and is sometimes not well perceived by the biological parents.

• Preparing and monitoring families of children reunified with their families is another key issue.

Despite provision of a meagre allowance by the state and some in-kind contributions by the Project

and other NGOs, there remain several unaddressed challenges such as housing, the physical and

mental health of the parents, even the safety of the child retuning home, which could in some cases

be questionable.

• Lack of parenting skills among many of the parents in the families that are taking children back home.

As one of the informants noticed, “there grew up a generation of parents, who are lacking parenting

skills and responsibilities and (they) think … that institution replaces the parent… this is wrong

(thinking) from the child and the parent and the state”.

With this understanding of the situation, especially at the communities of the origin, UNICEF and partners

have undertaken certain steps to support children and their families after their reunification. Given the

lack of community based alternative services, UNICEF has reached out to other partners involved in

service provision to vulnerable children and their families, such as SOAR and SOS. Thus, by late 2015

USAID, UNICEF, and MOLSA had negotiated with SOAR, the biggest diaspora organization supporting

Armenia’s orphanages, to redirect the financial and in-kind assistance from residential care institutions to

children reunited with their families. In 2016, the Project reached out to SOS Children’s villages to engage

in the process as well through providing social assistance to vulnerable families, and sharing their model

of outreach services. Case managers trained under the Project also became instrumental in the follow up

with these children and their families. Thus, NGOs alongside with the case managers trained under the

Project have been critical in filling in the gaps in services that families faced following reunification.

In addition, UNICEF in Armenia has introduced the idea of Small Group Homes to the Government of

Armenia as a temporary solution in those few cases where placement with either biological or a foster

family is impossible. The Government seems inclined to go ahead with this model as a more efficient

community based alternative service for special groups of children whose placement in families is not an

option. This includes adolescents or children with behavioral issues, children with disabilities who either

have no parents and/or whose parents are unwilling to get the child back, as well as those at risk of

violence/abuse in the family settings). While there have been some concerns raised about how

introduction of this new model will affect the existing alternatives for care (e.g. foster care) or how

appropriate it is to trade one type of institution for the other, UNICEF in Armenia emphasizes the

transitional nature of the model and its use as a temporary solution and only in those cases when placement

with a family is out of the question.

… UNICEF does not view the idea of small homes as a permanent solution… the fact that the decision on

immediately turning Dilijan and Byureghavan into family type small homes was not sent to the GoA is the result

of UNICEF’s engagement and influence. Expert Interview, International organization

Notwithstanding the Government’s public explicit commitment to the reforms, there have also been

situations when steps were taken even with respect to the legal framework which did not move the process



43

forward. For example, the situation with the decree # 1112-N on the Order of Provision of Care and

Conditions to Children, Elderly People, and People with Disabilities (September 2015) provides a good

illustration as to how caution and vigilance should be constantly exercised to promote the Project goals.

The decree in question did not prioritize the right of the child to live in a family environment as a first

choice, generally the sequencing of alternative care options was not in compliance with the international

best practices. Under USAID’s leadership the Project partners (UNICEF and WV) provided a legal analysis,

helping the decision makers to see that the decree was headed in the wrong direction.

To summarize, at the current stage of the reforms, several critical steps have been undertaken by the

Government of Armenia to downsize institutions and reunify children with their families. All these efforts

have been due to UNICEF and development partners’ consistent efforts and advocacy to promote the

family environment as being in child’s best interest. However, there are still concerns about community

based family support services not being established at the same pace as the closing down of institutions.

Children returning to their families from the boarding institutions do not have a support base locally if

the community of origin is different from the currently reorganized boarding institution (which happened

in Yerevan, with Noubarashen CFSC) and if there is no other relevant NGO operating in the community.

In most of the cases children face problems re-integrating back into the family, school and the community

given the circumstances discussed above, while in the worst cases they are sent to another institution.

While the solutions proposed by the Project are addressing the short-term needs of children and families,

long-term institutional mechanisms are yet to be made operational.

A few illustrations that follow provide very specific examples of how these different factors influence the

placement and the future of a child both positively and not so.

Case 1. A single mother with a boy (from Noubarashen) works double shifts at the hospital to make a

decent living for both of them. Since the family is under extreme financial challenges, SOS Kids Villages

has renovated the roof of the one room apartment (in the dormitory).

Case 2. A family of 7 (grandmother, father, mother and 4 children) “residing” in Noubarashen community,

Yerevan, with literally no housing conditions, directly dangerous for children to stay in – both in terms

of health, hygiene and psychological health. While the family is happy to have the children back and is

working hard to provide for them, their problem is the housing-one room, utilities outside, no privacy,

etc.

Case 3. In Kapan, there is a family where the mother works at the hospital as a sanitary help with night

shifts. The mother used to have a second job at a restaurant. Though she has quit the job she is officially

employed and thus she receives no welfare. Her younger child is attending the institution and stays

overnight when the mother is working. The Project provided the girl with a bed, desk, chairs and

wardrobe to facilitate her return to the family.

Case 4. Two brothers from Noubarashen: the mother and the grandmother refused having the children

back home and also didn’t agree to placement in a foster family, hence the two boys were moved from

Noubarashen to Byureghavan boarding institution “till at least one of them grows and is capable for taking

care of both”.

Foster Care



44

Finding 3: The institute of foster family is new to Armenia and is somewhat divergent with its traditional

culture. However, given the ongoing deinstitutionalization reforms, it has been considered as one of the

most appropriate alternatives for children placement and promoted as such.

One of the primary guarantees of the ongoing deinstitutionalization reforms is further placement of

children from institutions in families as part of meeting the best interest of children and their right to live

in a family. Continuous efforts (discussed throughout this report) by UNICEF and development partners

have ensured that policy makers and child protection actors in Armenia agree that biological families are

prioritized on the continuum of care for children. However, in those instances when the placement with

the biological family is impossible, care alternatives include guardianship care (by the kin of the child) or

foster care “where children are placed by a competent authority for the purpose of alternative care in the

domestic environment of a family other than the children’s own family, and has been selected, qualified,

approved and supervised for providing such care”16. In the past decade, several key efforts were undertaken

to introduce foster care as an alternative for care/placement of children in difficult life circumstances or

children without parental care, including:

• In 2005 UNICEF contracted CSCF to pilot foster care project in Lori and Gegharkunik regions. As a

result, 25 foster families received 30 children. Based on the results/lessons learnt from this project

technical assistance was provided to MoLSA in revising the Family Code to address foster care and the

amendments were approved in 2005.

• Starting from 2008 the Government of Armenia took the responsibility for funding and expanding

foster care in the country with 25 families enrolled in the system hosting 60 children over the past

several years and with 17 children in foster care currently.

• As a result of UNICEF’s and Save the Children’s technical support and advocacy efforts, the Concept

for Foster Care of Children in Difficult Life Circumstances was approved by the Government decision

of 10.03. 2016, allowing for different types of foster care. As part of the process, 12 legal acts and

proposed amendments to existing acts have been drafted by SC, but have not been passed yet pending

the adoption of the amendments to the Family Code (adopted in December 2017), which currently

allows foster care for only those children that are without parental care.17 Passage of these documents

will allow children in difficult life circumstances to benefit from foster care services alongside with

children deprived of parental care.

• The protocol decision N18 of the RA Government’s May 12, 2016 session on approving the Concept

of Alternative Service to Children in Difficult Life Circumstances, which makes it possible for this

group to become beneficiaries of foster care as well.

• Both 2013-2016 and 2017-2021 National Strategies for Child Rights Protection and annual

Government Programmes on Child Rights Protection include language about Foster care as an

alternative for child placement. To illustrate, the 2018 draft Government Programme defines the need

for “preventing institutionalization of children through expanding the network of alternative services,

including foster care”.18

16 Project Proposal

17 As the Introductory Note to this report specifies, the package of legal documents related to the amendments of the

Family code was submitted to the National Assembly in November 2017 and adopted in December 2017.

18 Draft Government decision on approving 2018 Program and Action Plan on Child Rights Protection



45

Despite these positive developments establishing a legal framework and piloting out foster care, there still

exist several challenges with respect to attitudes toward foster care and support to its implementation both

by relevant regional authorities (Guardianship and Trusteeship Committees (GTCs, Regional Social

Services, etc.) and the society at large. During the FGs and in-depth interviews several participants (around

60%) pointed to instances when the authorities did not support their enrolment in foster care.

Who told you to apply? Why to hurry? The State is not funding this program, so why are you applying? FGD,

Foster Parent

Regional Marz administration departments showed some resistance to the program. The State was playing a double

role- explicitly speaking about being for it, but showing some resistance through Marz authorities. Expert

interview, Implementing Agency

In other cases, they face negative attitude inside the community,

You have a child, a good house, so what is the use of becoming a foster family? FGD, Foster Parent

In one of the rural communities they complained the foster parents are abusing the child. Our several monitoring

visits indicated that the child was merely helping the parents in agricultural works. The attitude toward foster

care was so bad that they wanted to see it fail. In-depth interview, Implementing Agency

However, there were also cases when FG participants and informants noted (around 40%) that they faced

no issues:

Honestly speaking, we had no problems. We went to marzpetaran (regional administration), they came to assess

our house. Someone came from municipality as well. We submitted our documents and in two months got the

registration. FGD, Foster Parent

So, it is pure luck… whoever gets a good official, the process goes much easier. FGD, Foster Parent

According to FDG participants and other key informants, some of these stereotypes persist despite efforts

to increase awareness and understanding of foster care at community level and among the professional

community, such as:

• informational meetings in 38 communities (instead of planned 18);

• 496 persons reached as a result of professional forums instead of planned 300;

• 1783 persons reached as a result of community meetings instead of planned 300;

• 132 persons reached through different other meetings organized by CSCF.

In addition to these Project activities, UNICEF and CSCF undertook some extra efforts in this direction

including developing and dissemination (at the above forums) Foster Care Q&A, posters, one-pager

promoting the concept of foster care and other materials. UNICEF also separately organized a series of TV

programs and awareness through social media reaching up to 4000 persons. However, it should be noted

that the national level public awareness campaign was not fully implemented conditioned by the delay of



46

the adoption of the new Family Code. There were some publications in media, but the film and the Public

Service Announcements that could capture a much larger audience was delayed due to prolonged

finalization of the developed scripts for one of the PSAs (by all partners involved) in accordance with the

advocacy standards to better reflect the messaging to potential foster families, as well as due to the

agreement among all partners to broadcast the visual materials after the adoption of the Family Code

(provisions on foster care).

To summarize, the Project interventions have been timely and relevant with respect to advancing foster

care as an alternative care in support of the deinstitutionalization reforms. However, alongside with legal

changes and capacity building of foster families (discussed below), working with the key stakeholders and

larger public to ensure acceptance of this form of care is an important aspect of the Project that should be

further enhanced.

Inclusive Education

Finding 4: Introduction of the concept of inclusive education (IE) through different pilot projects and

work of both civil society organizations and policy makers has prepared the Armenian society for

embracing inclusive education in mainstream public schools. Perceptions and assessment of the surveyed

teachers on the country’s general readiness for inclusive education is rather favorable for rollout.

However, there are also those who believe that the pace of the reforms is too fast and there is more work

needed before comprehensive rollout of the system.

In 2005, the Government of Armenia adopted the concept of Inclusive Education and the Law on

Education of Persons with Special Education Needs was passed, with Inclusive Education included in the

draft Education Development National Programme for 2008-2015. However, a 2009 external evaluation

of Inclusive Education Policies and Practices in Armenia, administered by UNICEF Armenia office,

pointed out that this framework allowed for a parallel system of education (regular and special). Based on

this evaluation recommendations and as a follow up to CRPD ratification by the Government of Armenia,

the Law on Education was amended in 2014, and a new State Programme for Education Development

2030 was drafted. The Law and the new state program aim to align the education system development

with the newly adopted Sustainable Development Goals (SDGs), the Incheon Declaration, and the

adoption of the Bologna process, all of which occurred in 2015. Thus, both the Law and the Project

consider “the formation of an educational system which enables every citizen in all stages of their life to

receive quality education corresponding to their requirements, intellectual competencies and aspirations,

meets the demands of the economy and society, and fully serves the national interests”, and “The Republic

of Armenia shall declare the universal inclusive education as a guarantee of ensuring the right of each

child to education. The policy of inclusive education shall be aimed at ensuring the accessibility,

opportunity of equal participation in and quality of education of each child"19.

These achievements in the area of inclusive education have been possible due to efforts of UNICEF,

Mission East OSI, USAID and other development partners that consistently advocated for the introduction

of inclusive education in the country by exposing decision makers to international best practice,

emphasizing the human right based approach, mobilizing NGOs around the issue, organizing study tours

19 Article 4, RA Law on Education, supplemented by HO-200-N of 1 December 2014



47

and trainings. The first school (Yerevan School No. 27) opened its doors to children with disabilities in

1998, followed by modeling inclusive education in eight schools of Yerevan and Tavush region by UNICEF

and BoH. Since then the number of inclusive schools has been growing steadily through application of the

same model, reaching 100 by 2014 and providing education to more than 2500 children with special

educational needs. A major contribution to this development was the launch of joint the BoH and MoES

pilot project (funded by the Mission East and the Danish Government) in Tavush in 2010, aiming to create

conditions in all mainstream schools of Tavush for educating children with special educational needs and

preparing the system to transition to overall inclusiveness by 2022. As one of the interviewed NIE regional

staff members noted, “a mere hundred years ago it was difficult to imagine the existence of mixed gender

schools, which is currently the rule rather than the exception. The same can very well happen with

inclusive education.”

The feedback received through the teacher survey conducted for the purposes of this evaluation provides

additional evidence that the Armenian society has overcome its initial reservations about integrating

children with special educational needs into mainstream schools and is ready to embrace the concept of

inclusiveness.

Thus, over 80% of respondents have indicated their personal acceptance of the concept of inclusive

education, while 70% of the surveyed teachers also believe that Armenia is somewhat ready to implement

the inclusive education reforms. The teachers also have high confidence in their colleagues’ readiness to

implement inclusive education reforms. Thus, when asked to rank the readiness of different players to

introduction of the IE on a scale from 1 to 5 (1 being lowest and 5 highest), they have assessed their

colleagues’ readiness at 4.1. Interestingly, parents’ readiness is ranked as average at 3.1, while the schools

are perceived to be the least (at 2.8) ready for implementing the new system mostly due to unfit physical

infrastructure and lack of needed resources.

Table 8: Readiness to Implement Inclusive Education (average value) 20

(On a scale of 1-5 (1 being lowest and 5 highest), please assess to what extent the following groups (community

members/parents, the school, including its facilities and your colleagues with respect to their perceptions,

knowledge and skills) are ready for introduction of IE):

Community members, parents (attitude) 3.0

Their Schools (building and facilities) 2.8

Their colleges (attitude, knowledge, skills) 4.1

The respondents were asked “what role (positive or negative) the introduction of IE plays for different

stakeholder groups. The response “mostly positive” was highest (63.8%) for the official bodies (MoES, NIE)

followed by the parents of children in the inclusive education system (54.8%) and school administration

(48.7%). The response “mostly positive” was lowest for children in mainstream education and their parents

- 19.6% and 11.5% respectively. When asked to what extent they personally accept the concept of

inclusive education, 81.1% of respondents answered they accept it either fully or to some extent (see

20 Full list of output tables is included in Annex 7



48

Annex 7, Table 9). The respondents that provided positive and negative answers to this question, have

given justifications for their attitude. Over 54, 2 % of respondents point out to such positive implications

from the introduction of the inclusive education system as respect towards all children and an integrating

function for the overall society, while over 10% believe that it is disruptive for the entire learning/teaching

process. Chart 1 below summarizes positive and negative arguments provided by respondents as pros and

cons for inclusive education

Chart 1: Rationale for Attitude

It should be noted that while generally qualitative data also supported the need for changes to introduce

IE, more negative arguments were voiced throughout in-depth interviews and focus group discussions

with different informant groups. Both teacher and parents expressed opinions about the society not being

ready to embrace inclusive education, more time needed, parallels were drawn with how overtime there

was acceptance of people of color. Parents were more non-receptive of the idea speaking about their

reluctance to have their children attend school alongside with children from “special schools.”

Receptiveness toward inclusive education depended also on such factors as geography, lack of conditions

and resources, severity of condition of the children with special educational needs. Thus, the concept of

inclusive education is more acceptable in Syunik than in Lori, which is understandable given that more

time and effort has been invested in Syunik.

We can do nothing. If we are required to work like that, we will work like that. FGD, School /Lori

Before, I could not imagine what should be done with those children. With time, seeing their progress, I realized

that inclusion was the right thing to do. FGD, School /Syunik

54%

5%

4%

11%

11%

6%

9%Respect toward the rights of all children

IE system has an integrating function forthe society

Children feel better, psychological aspect

Lack of adequate conditions (physical,sufficient attention during the class, etc.)

Negative impact on the class, disruption ofthe classwork.

Need for individual work with SENchildren

Other



49

Lack of ramps, adapted restrooms, most importantly lack of special teachers, speech therapists and other

specialists, i.e. all the pre-conditions and resources needed for effective inclusive education are other areas

where unresolved issues raise skepticism about the system.

Another important factor that was voiced by informants is related to the specific condition that the

children with special educational needs have. They mostly agree that if the issue is a complicated mental

health one or mental disability, inclusion is difficult and at times, dangerous. Former special school staff

also were concerned about children with severe conditions who received special care in institutions and

were deprived of it in mainstream schools.

Most informants believed that inclusion could be more smoothly achieved in case of physical disability

and not for complicated mental disabilities.

I knew that a child in wheelchair will join our class from December. Months in advance I prepared my students,

showed a film, told stories and you should have seen the care and attention they showed this kid. FGD, School

/Syunik

To summarize, over the years, critical steps were taken toward integration of inclusive education

principles into mainstream schools both on policy and practical levels. These resulted in gradual shift of

perceptions and opinions and general acceptance of the notion of inclusive education. However, persisting

challenges related to capacity of specialists, adequacy of school infrastructure, resistance of parents and

others give way to certain questions about how ready different groups in the society are to fully embrace

IE. In this respect, the work done under the Project is undoubtedly contributing to changing the attitude

of educators, parents, children and others toward children with special educational needs and their place

in mainstream education.

EFECTIVENESS

General Findings

Finding 5: The child welfare /inclusive education reforms are complex multifaceted undertakings. As such,

the Project structure reflects the reforms complexity through its multiple components, implementing

partners and coordination with line ministries and agencies. While the Project is intended to support the

Government’s reforms agenda, the reforms pace and project pace are at times in conflict with each other

resulting in delayed Project activities.

Project donor and implementers, Armenian officials, child care institution and public school staff and

other stakeholders agree that both reform directions (child welfare/de-institutionalization and inclusive

education) while necessary, are complex, sensitive as well as resource intensive and requiring long-term

efforts. Deinstitutionalization requires reorganization of existing institutions (including legal aspects,

assessment of staff and related decisions, infrastructure improvements, etc.), assessment of children and

their return to biological families and support thereof, establishment of alternative services including

foster care and provision of relevant services, involvement of multiple players ranging from social service

providers to regional authorities to families themselves. Introduction of inclusive education into

mainstream schools is no less complex. It starts with closing down special schools and establishing

Psychological-pedagogical Support Centers, placing children with disabilities in mainstream education



50

and training teachers to understand the concept of inclusiveness and address the needs of the children,

working with parents (both those that have children involved in inclusive education and others), working

with children themselves. To summarize, both reforms have aspects related to reorganization, setting up

new services, infrastructure, capacity building, social support, public awareness and buy-in and others.

Consequently, they require mobilizing several stakeholders such as relevant Ministries, regional and local

authorities, social, pedagogical, psychological and health service providers, children and their families,

and the larger public.

With the understanding of these complexities and to effectively implement the Project, UNICEF in

Armenia has partnered with the most competent organizations that have long-standing experience of

supporting intervention in these areas. UNICEF’s original partners included World Vision, Save the

Children and Bridge of Hope, each responsible for a Project component matching its competencies and

experience. Later on, UNICEF also brought on board FAR’s Children’s Support Centre Foundation (CSCF),

a local NGO that initially was not included for USAID funding. Table 8 below summarizes the role and

competencies of each Project partner.

Table 9: Roles and Competencies of Project Partners

Organization and Respective

Project Component

Competence and Prior Experience

World Vision (WV): Support to

Transformation of Institutions

and set up of new services

Child protection is a cross-cutting theme throughout WVA’s 13 Area

Development Programs (ADP) in 6 Marzes and Yerevan. WV has

implemented a series of child protection projects funded by USAID and the

UK Government’s Department for International Development (DFID).

Through “Strengthened Child Protection in Armenia” $2.5-million project

funded by USAID WV has acted in support of GoA Deinstitutionalization (DI)

Strategy by enabling reunification of 172 children with their biological

families (as of June 2013).

Save the Children (SC): Foster

Care

Save the Children has been implementing child protection and child rights

governance initiatives with the focus on strengthening state systems and

mechanisms for implementation and monitoring of child’s rights. Through its

programs, Save the Children ensured training of parents and caregivers on

positive parenting approaches that need to be applied for children’s

upbringing. In 2012-2013, SC Armenia conducted research on perspectives of

foster care in Armenia.

Bridge of Hope (BoH): Inclusive

Education

Since its establishment, BoH has advocated on disability as a human rights

issue in accordance with the social model of disability. Bridge of Hope has got

branches in 4 cities of Tavush region providing services to 1200 children with

disabilities and special education needs. Since 2001 “Bridge of Hope”, as the

key partner of MoES for inclusive education, has implemented large

education programs in cooperation with the Ministry, civil society and

international agencies.

Children’s Support Centre

Foundation (CSCF): Case

CSCF is the only crisis intervention service for children and their families in

Armenia. It provides crisis intervention for children aged 3 to 18 who are in

difficult situations, including abandoned or homeless children, street



51

management, Foster care and

Transformation of Institutions

children, children with behavioral problems, children in conflict with family

and the law, etc. The center’s goal is to prevent these children from becoming

institutionalized.

UNICEF, as the lead organization undertook coordination efforts, communication with government

partners and USAID, policy and advocacy work, as well as some direct implementation activities in areas

of its expertise, including involvement of ISS/case managers, service providers in child protection field,

local social planning and others. Another important structure involved in Project implementation is the

operational level steering committee comprised of key government partners, USIAD and Project

implementation partners that has regular meetings around key Project developments.

It should also be added, that while the overall term of the Project is 5 years, each partner had an agreement

with UNICEF according to which the intensity and length of their involvement in the Project differed,

including:

• Initially the BoH’s contract term was until February 2017, but they requested a one-year extension to

continue their work in Armavir Marz. However, given the uncertainty of the Government’s plans for

Armavir Marz, the contract was extended only until June 2017 to wrap up activities in Lori Marz.

• The contract with Save the Children expired in May 2017 and was not extended, due to the mutual

agreement that their further work depended on the adoption of the Family Code.

• WV had a 36-months contract ending in December 2017, however as noted in UNICEF’s January-

March 2017 report, “due to slow progress in decision making regarding the transformation of

Noubarashen CFCS and the residential care institutions in Lori and Syunik regions, WV is facing

considerable underspending since significant part of their budget is dedicated to renovation and

equipment purposes”.

While overall several results were achieved in the past 2.5 years, the situation with each implementing

partner indicates that there have been a number of bottlenecks (beyond the control of the Project) which

either slowed down or diverted the Project activities.21

Project Area Key Results Challenges

Reorganization and

transformation of

institutions

• One institution closed (Yerevan child

protection and care institution #1) and one

transformed into a CFSC (Yerevan child

protection and care institution #2)

• Assessment of 150 children in 2 residential

institutions in Lori and Syunik

• Assessment of Staff in Noubarashen, Kapan

and Vanadzor.

• Slowing down of the process by

MoLSA, Noubarashen

reorganization delayed,

approach unclear

• G2G support to MoLSA still

under discussion

• Lori assessment by WV

outdated

• CFSC staff training delayed until

autumn 2017

21 UNICEF, BoH, WV quarterly reports are used as a source



52

Institutionalization of

Social Service system (case

management, assessment

of children, care plans and

family assistance packages

to reunified children,

territorial social planning)

• Two training packages for case managers and

service providers developed by CFSC

• Over 100 case managers trained through two

5-day training sessions

• Over 192 social service providers trained

• Different capacity level of case

managers

• Lack of community-based

alternative services for case

managers to refer cases

Foster Care • Amendments to the Family Code with

corresponding 12 bylaws regulating the

diversified foster care model were drafted

and accepted by MoLSA

• Over 74 potential foster parents trained on

foster care regulations, registered and 55

accredited

• 132 experts and stakeholders involved in

public hearings on legislative reforms

• Over 2500 persons involved in community

level awareness meetings and professional

forums

• Family Code amendments still

pending22

• Lack of state budget funding for

new foster care services23

• Broadcast of PSA and

documentary for improved

awareness not released

• Foster family database not

created

• Full system application on hold

Inclusive education • 4 special schools transformed into RPPSCs

(Syunik and Lori)

• Over 650 children assessed in Lori and

Syunik by RPPSs

• Over 90 specialists of special schools and

RPPSCs involved in a series of trainings

• Over 5000 teaching staff trained in 264

schools in Syunik and Lori Marzes, including

264 principles

• 4506 parents attended one-day seminars

• Number of children served by RPPSCs

• MOES request to postpone the

trainings of the teaching staff of

162 schools in Lori region from

August-October to November-

February resulting in slower

pace of expenditure

• Infrastructure improvements

under G2G still pending

Legal framework • Action Plan and timetable for provision of

total inclusive Education system approved

• The Concept for Foster Care of Children in

Difficult Life Circumstances approved

• Several decrees and decisions passed to

implement institutional reorganization, to

approve methodologies, etc.

• Family Code amendments still

pending24

As the above summary analyses and review of indicator tracking tables indicates, the key outputs so far

are attributed to interventions by primarily BoH, (which has completed most of its undertakings under

the Project) and CSCF with respect to exceeded number of case managers and social service providers

22 See the introductory note above.





53

trained (over 250 instead of planned 145) as well as foster parent training and community level advocacy.

While SC (through CSCF) has conducted the trainings for foster parents and did significant work with

respect to legal amendments, the massive broadcasting of visual materials on foster care model is still

pending. Due to slow pace of transformation in Noubarashen, as well as child care and protection

institutions in Lori and Syunik Marzes, WV has accumulated significant underspending. Throughout the

evaluation, it was difficult to delineate the roles of WV and CSCF with respect of assessments, and SC and

CSCF with respect to trainings under foster care. The CSCF role is more complicated, since it was

contracted out by UNICEF to conduct trainings for case managers, but eventually became involved in

foster care training and assessments of children from child care and protection institutions. As noted by

UNICEF, once CSCF was on board, it was decided to combine child- focused case management trainings

with foster care trainings under direct contract with UNICEF, for efficiency purposes. As for in-depth

assessment of children, it seems that the initial assessment for Yerevan # 2 was conducted by WV, while

in Syunik both trained case managers supported by CSCF and WV social workers were involved in the

process. In Lori, CSCF and case managers conducted social assessment of families of the children from

special schools to complement the educational needs assessment by RPPSCs. Assessment of children from

Vanadzor Child Care and Protection Boarding Institution was initially conducted by WV, but was

considered outdated given the delay in the reorganization of institutions alongside concerns about its

quality. It was later tasked to CSCF, and was completed in March 2017.

Overall, the uneven curve of reforms progress is also a challenge - as some key informants noted “things

slow down for months, then there is more intense activity. The changes in the government also created

delays in the approval of critical legislation. Another evidence on divergence of reforms and Project

priorities, is MoLSA’s commitment made to the Prime Minister to close all night boarding institutions by

the end of 2017, resulting in actions toward that direction, while the Project is focused on a limited

number of institutions. Consequently, the Project interventions and expected results being directly linked

to how energetically the Government is implementing its reforms agenda, there have been several

challenges, delays and short-comings in almost all Project directions.

However, it should be noted that despite these challenges and complications, UNICEF has put significant

efforts in pushing the reforms forward through negotiations with the Government and in finding ways to

resolve implementing partner related issues (like bringing on board CSCF). While both the Government,

USAID as the donor, as well as implementing partners, acknowledge UNICEF’s leading and mobilizing

role in the Project, they also point out to certain coordination challenges, which are summarized below.

Each IP is implementing their specific component, but UNICEF is the lead partner involved in policy and advocacy

matters together with USAID. They should all sit together around negotiation table, but it is always UNICEF that

is in charge for communication with the Government. Expert Interview, Implementing Entity

Having all these components under one Project was in a way a good coordination strategy since all key players

were involved and informed, being on the same page about the overall reforms directions. Expert Interview, State

Official

Almost 1.5 years for preparation-which was acceptable given the complexity of the project and the associated

reforms-however after that USAID expected better progress. Lack of quantitative achievements is one thing, but

the qualitative aspect of UNICEF’s work is what bothered USAID, mostly lack of evidence based justifications that

could help the Government move in the correct direction. Expert Interview, Donor

The Project was badly planned and badly implemented. Expert Interview, Implementing Agency



54

Coordination meetings between WV-FAR-SC, also ad hoc meetings with UNICEF, they are open, we can call

them, email, but there is a lack of a systematized approach. Expert Interview, Implementing Agency

Need for better coordination of activities in Lori Marz. Expert Interview, Implementing Agency

An issue may be discussed through a chain of emails for months with no response from UNICEF, then suddenly

we learn that it was resolved. Expert interview, Implementing Agency


Finding 6. While UNICEF and partners have developed and proposed to MoLSA a model for establishing

Child and Family Support Centers and supported reorganization of institutions and their transformation

into CFSCs, several factors contributed to certain inconsistencies and drawbacks in the process, mostly

due to delays and shifts in the reforms priorities led by the Government. Noubarashen, Kapan and

Vanadzor have followed different timelines and patterns of transformation that affected institution/center

work and beneficiary experiences with placement/follow up service provision.

According to UNICEF and World Vision reports, four (4) institutions have been targeted for

reorganization under the Project, including Yerevan Child Care and Protections Institute #1 (Korea Dzor),

Yerevan Child Care and Protections Institute #2 (Noubarashen), Kapan Child Care and Protection

Boarding Institution, and Vanadzor Child Care and Protection Boarding Institution. In addition, there has

been considerable ground work for transformation of Vanadzor orphanage through advocacy efforts and

planning. However, the situation with each of these institutions is different in terms of their legal status,

Project interventions, their transformation into a Child and Family Support Center (CFSC), provision of

service, etc. (see Table 10 below)

Table 10: Transformation Status of Institutions

Institution Current Status Project interventions

Yerevan Child Care

and Protections

Institute #1 (Koreai

Dzor)

Closed MoLSA closed without prior assessment and

a clear model; some children returned to

families, but most of them went to other

institutions. WV and other NGOs supported

those children that remained in Kanaker-

Zeytoun (WV ADP) and the children

became their beneficiaries.

Yerevan Child Care

and Protections

Institute #2

(Noubarashen)

Transformed into Child and

Family Support Center by GOA

decree in August 2016

Targeted by the Project, WV involved in

assessing the staff and later training,

proposed initial model that was used to

certain extent by MOLSA (65 children

leaving institutions, of which only 6 were

originally from Noubarashen district, others

returning to communities of origin),

support to children and families by ISS case

managers, employment services, SOAR,



55

CSCF, SOS villages (for more than 6 months

to children reunified with their families

from this institution)

Kapan Child Care and

Protection Boarding

Institution

Decree on reorganization

planned to be passed in October

2017

Staff assessment; child and family

assessment for 45 children, support to

children and families including material

support such as food, medication, firewood,

etc. (for 25,000 drams per month per family)

Vanadzor Child Care

and Protection

Boarding Institution

Decree on reorganization

planned to be effective as of

November 16, 2017

Staff assessment, child assessment and

support to children and families, including

material support such as food, medication,

fire wood, etc. (for 25,000 drams per month

per family), advocacy with regional

administration, trainings of case managers

in the region, as well as trainings on positive

parenting

Building up on its experience from the USAID-funded “Strengthened Child Protection in Armenia”

project (implemented in 2010-2013, through which over 170 children were unified with their biological

families), and the best international practice, WV developed a model for creating family support services,

which was submitted to MoLSA in April 2016. The model detailed the services to be provided by each

professional of the center, the qualification of the professionals and the set-up of the rooms, including the

description of items that should be in each room of the service center.

..we also showed that these centers don’t need to be big buildings or excessively overstaffed institutions, and can

service 3 times more children on much less budget than spent today. Expert Interview, Implementing Agency

It was expected that WV should start working with the staff of the institutions before they would be

reorganized, immediately after the staff needs assessments were implemented– to support the institutions

to hire new professional staff (psychologists, social workers, social educators, speech therapists), and to

train the staff who transitioned from the closed institutions, to develop relevant methodologies and

interventions. However, due to the changes in the state priority, decisions were made to close the Yerevan

Child Care and Protection Boarding Institute No. 1 and to reorganize the Yerevan Child Care and

Protection Boarding Institute No. 2 (Noubarashen)25, before approving the model for CFS Centers

submitted to the MoLSA by WV. Interestingly, months after the model was shared and the Noubarashen

decree passed, the Ministry requested the Project to support with Noubarashen CFSC model.

It became clear in the very beginning that the State was not ready for the reforms in terms of not having a well-

defined, clear implementation schedule and action plan. There seemed to be lack of trust in the reforms among

the public administration agencies. On the other hand – this was a new area for them, so there was most probably

25 GoA Decision N 831-N, dated as of August 11, 2016 on Reorganization of the “Yerevan Child Care and Protection

Boarding Institution N 2” into “Child and Family Support Center” SNCO.



56

some fear, that the reforms might not be successful, which is also understandable. Expert Interview, Implementing

Agency

It took almost another year for the Government to come up with a structured process for reorganization

and transformation of the other institutions, which is formalized in the decrees for each specific one

(Kapan and Vanadzor, decrees passed in October 2017, transformation to begin in January 2018). The

overall reorganization process was structured to start with renaming the institution, drafting its charter,

legal succession and solving property rights issues, the right for implementation of aid services within the

scope of the foundation, etc. With respect to transforming the institution into a CFSC, World Vision’s

model for the Child and Family Support Center was reflected in the charters. While Noubarashen’s

transformation became a pilot for other institutions, pending issues with Noubarashen are indicative about

the differences as to how MoLSA, USAID and the Project partners perceive the model for the newly

established CFSCs. There is an ongoing controversy between the Donor (USAID), the Project partners and

MoLSA with regards to the scope and the budget for renovation activities in Noubarashen. While MoLSA

would like to establish a large-scale demo CFSC, USAID and UNICEF believe that smaller and more

flexible centers close to the communities of origin are a better solution which will serve the children and

their families better, as well as being more accessible.

... According to the model, there is no need to renovate the entire building (2 parts of it), though seismic

enhancement was done since it is important... The two floors of one of Noubarashen buildings is sufficient for the

Center we propose…but we cannot move forward before there is an agreement between the Donor and the

Ministry. Expert Interview, Implementing Agency

Some other consequences of this situation are related to delays with the training of the institution/CFSC

staff. Budget wise, it was considered inefficient to provide training to the Noubarashen staff separately but

rather train the staff of all the three institutions targeted by the Project, once Kapan and Vanadzor status

is more or less clear. As the process of reorganization moved forward (the MoLSA at least committed to

the reorganization of the three institutions within the scope of the Project26), staff assessment and

professional staff recruitment in both Kapan and Vanadzor Boarding Institutions began and a series of

joint trainings were conducted for all the three institutions (Noubarashen, Kapan and Vandazor Boarding

Institutions) in September-October 2017.

While these are positive developments that occurred beyond June 2017, the discussed issues had their

consequences since Noubarashen CFSC staff was not trained for almost a year after its establishment and

had to operate without professional guidance. Several other implications of these shifts and delays in terms

of both support to the children and their families, and the setup of the new service, i.e. the Noubarashen

CFSC, are discussed below.

26 RA Decision N 1398-N on Renaming the Kapan Child Care and Protection Boarding Institution SNCO into Syunik

Marz Child and Family Support Center Foundation, approving of the Carter and Providing Property ; draft decision

on Vanadzor Boarding Institution and Orphanage was part of the desk study, but the official decision is not made

yet.



57

• Children from No. 1 Boarding Institution returned home in the summer and never came back to the

institution, disturbing Project plans to start the work with those children and their families in mid-

autumn, and causing “loss of some precious time to work with the children and families”.

• The mismatched schedules, delay on approving the model (by the Government), as well as the

engagement of World Vision in other activities under the Project (i.e. evaluation of seismic stability

of Noubarashen Child and Family Support Center and the related construction activities) delayed the

preparation and implementation of the training packages for the staff of the already re-organized

institution.

• The staff of the reorganized institution that stopped “accepting” (i.e. enrolling) new children in

autumn 2016, had officially been renamed into Child and Family Support Center as of January 2017

(aka Noubarashen CFSC). For an entire year it operated based on the general Job Descriptions of the

positions, their skills and intuition, as well as “finding and reading things on internet, mostly in

Russian”.

• While the CFSC is established, only 6 children originally from Noubarashen continue making use of

its services (as stated in the Charter of the Center, the children, who need the services of the Center

will continue making use of those from 6 to 12 months, with a following re-assessment), while the

Center does not track the destiny of the children that returned to other communities of origin,

especially rural areas where there are no NGOs providing services. As noted above, this issue is closely

interlinked with the ongoing conflict between MoLSA and the Project donor and partners about the

G2G investment into the center infrastructure. While MoLSA is willing to commit over 1 million USD

to creating a model first, CFSC, USAID and Project partners believe at least three such centers have to

evolve (in Erebouni and Shengavit as well) to support more children in their communities of origin.

• The destiny of those children who cannot return home remains unresolved. In Noubarashen there

were 5 children that could not be sent back home due to unacceptable situations in or of the family.

These children were placed in other institutions, since even if there are registered foster families, it is

still an issue to “place” the child in foster care, due to limiting legislation.

Despite these concerns and persisting issues, Noubarashen CFS Center is currently considered as the

pioneer and as an “experienced” center by now and it provides a basis for a few lessons learnt for the next

to-be-reorganized institutions.

• While only 5-6 former Noubarashen resident children are served by the Center, the total number of

children currently benefiting from its services is more than 100 since children in difficult life

circumstances from this community are also being served by the Center. More than 300 children have

been assessed as potential recipients of the services of the Noubarashen community as well as

Shengavit and Erebuni. This is a positive example of effective utilization of the Center and in line with

the Project proposal.

• As several informants agreed (institution staff, implementers and experts) staff training should follow

the needs assessment and recommendations for changes, happening in advance, long before the

reorganization, so that the staff is prepared to work with children and parents alike in their new roles.

In addition, placement of children should be pre-arranged, monitoring provided, and all relevant

structures mobilized in the communities of origin.

The situation of Kapan Night Boarding Institution to be reorganized into Syunik Marz Child and Family

Support Center seems to be much better. While the transformation is expected to happen following the



58

adoption of the RA Decision N 1398-N in November 2017, assessment of the children in the institution

and their families was conducted by the case managers (with coaching by CSCF supervisors) and further

support in terms of providing family assistance packages by World Vision. Most children (35) are staying

home at night, but spending their days at the institution, including having their meals there. Since many

of them are from Kapan and its outskirts, only a handful of children that are from the villages stay

overnight. Besides, the professional team has already been enriched, two psychologists joint the team in

2017, and professional training is planned for the team for autumn 2017. The future staff of the new service

center has enough time to study the experience of the CFSC in Yerevan, visit it, discuss the “do’s and

don’ts”, which is quite helpful for the Director to start organizing its activities. Simultaneously, CSCF-

trained case managers and WV’s local social workers are involved with the families of the children and

the boarding institution on nearly daily basis. Assistance packages including food and furniture, have been

provided to families in need of those by UNICEF and WV. All the ground work that has been done by the

Project in Kapan allows for the transformation of the institution and the operations of the future service

center to be much smoother. With Vanadzor Child Care and Protection Boarding Institution, WV made

the child assessments over two years ago. However, due to delays in the Government decision to start the

reorganization and some concerns over the quality of the assessments implemented by the company hired

by WV, assessment results are outdated. The assessments were updated in January 2017 by the case

managers under the professional coaching and mentoring of CSCF supervisors. Two stage in-depth

discussions for each case were organized by CSCF and UNICEF to validate the assessments, mobilize the

partners and stakeholders, and to find common solutions to individual child’s case. During the interview

with the institution Directors (the former one and the currently acting one) it was quite clear that they

are well aware of the reorganization process, have started the process of hiring new staff for the future

service center and have also studied the lessons learnt of the Yerevan CFSC. It should be noted that with

respect to alternative services, Vanadzor (and Lori Marz in general) seem to be in an even more favorable

condition, given the presence of several NGOs in the area. World Vision itself does not have a

representation in Vanadzor, but it is present in Stepanavan and Alaverdi, which makes it possible to cover

the children from the adjacent rural communities that will return to these communities from the boarding

institution.

To summarize, the reorganization and transformation of child care institutions have understandably taken

an uneven path given its different complex aspects and involvement of different stakeholders and interests.

As much as possible, the Project has attempted to smooth the process by maintaining constant engagement

with the Government to push the legal foundations needed for moving forward (decrees, charters, etc),

proposing a model for the CFSCs, training case managers and other social service providers as well as

mobilizing partner NGOs to address the needs of children reunified with their families, to the extent

possible. By June 2017 there has been mixed success, including the establishment of the first CFSC and

reunification of children from Noubarashen with their families (over 50), assessment of hundreds of

children and families, provision of support to reunited families in form of material assistance and specific

services. As to the drawbacks, absence of community based family support services in the communities of

origin where children return, has remained the paramount issue to be addressed. As noticed above,

UNICEF and partners have found ways to overcome most of this hurdle, but not through long-term

solutions. While a model for CFSCs has taken shape, its full application is still to be ensured. To illustrate,

while the Charter of the service centers is clearly defining the general scope of activities of the staff of the

new institution, a major concern brought up during all the interviews and FG discussions, was about the



59

lack of training and education on how to work in this new type of institution. The Job Descriptions

attached to the Order on the establishment of the service centers do not give much information on the

specific professional performance of the professional staff. While the mandate of the service center is well

understood, in general - to provide services to the family – psychological, social, legal, medical, parental

skills and household economy, the “how to do that” part is still missing. Although trainings are planned

for the general professional staff of the service centers and specific, discipline-based trainings will follow,

there is concern that one-time short-term training is not enough to build long lasting capacity of the team.

All these concerns expressed by the institution staff about effective execution of their functions are

additional evidence to the need of a clear and consistent process and model for reorganization of

institutions into CFSCs with specific guidelines, methodologies and manuals in place, as well as a quality

control mechanism/oversight to ensure that minimum standards are adhered to.

Foster Care

Finding 7. The Project succeeded in conducting a two-phase training for over 74 foster parents. While

improved parenting skills is noted as a training benefit by several of them, delayed placement of children

is creating disappointment among the potential foster families.

Trainings for 74 potential and experienced foster parents (instead of initially planned 120) was

implemented by FAR’s Child Support Center Foundation (CSCF) in 5 regions, i.e. Lori, Syunik, Armavir

and Ararat Marzes and Yerevan. CSCF specialists were also responsible for developing 12 training modules

and methodologies such as Guidelines for Monitoring Foster Families and Foster Care Program, Principles

and Standards for Foster Care, Public Awareness Manual. In addition, they supported

registration/accreditation of foster families and training of other social service providers such as the case

managers.

Trainings of potential foster parents were implemented in 2 phases through three-day sessions in each

phase. Training modules for the first phase included such topics as reforms implementation, the role of

foster families, procedures for registering as foster families, while the second phase focused on issues

related to types of families, child psychology and other related topics. As part of the training experienced

foster families shared their experiences with the potential candidates.

Some disagreements occurred between Save the Children and CSCF throughout the implementation of

this component. To illustrate, since recruitment of foster families required awareness building about this

type of care among the potential candidates, intensive ground work was required to accomplish this task

in preparation for trainings and registration. According to the CSCF head, this function was not included

in their assignment but eventually they had to visit over 200 villages to meet with people and inform them

about foster care as an alternative to biological family and institutional care.

Initially, the recruitment of foster families almost failed. I went to 268 communities using established connections

with heads of communities and schools. Public awareness was at zero level before that, no families were applying.

We also worked with media and succeeded in inspiring interest toward foster care. Expert Interview, Implementing

Agency



60

Focus groups and in-depth interviews with potential foster families also indicated that they learnt about

the foster care from CSCF, local TVs, and experienced foster families. As a result of these efforts 93 families

were enrolled in the program. Based on feedback of interviewed potential foster parents, the following

types of families interested in foster care were identified:

• Families with no children;

• Families with one child who want their children to have siblings;

• Families with grown up children, who wish to continue caring for younger children.

A large group of foster parents (involved in the FG discussion) are those who wanted, but did not succeed

in adopting children and they perceive foster care as a solution to this issue.

Most training participants (both FG participants and in-depth interview informants) speak very highly

about the qualifications of trainers as well as usefulness of the trainings resulting in acquiring some critical

knowledge and skills, including:

• Giving consideration to the child's point of view in different situations;

• Tolerance to unacceptable behavior displayed by children sometimes;

• Patience.

I finally made my husband attend the training with me, at least for an hour. He became so engrossed that he

stayed until the end. FGD, Parent

However, it should be noted that the majority of FG participants voiced concerns that for over a year after

the trainings none of them had an opportunity to practice their foster care skills. Many of them expressed

concerns that they might soon forget the knowledge and skills acquired. Below are a few quotations

summarizing trained foster parents’ insights about the uses of training and their concerns about delayed

applications of skills and knowledge.

If in the next few months we are not given a child, I will not be interested in this program any more. I am waiting

for that call every day. FGD, Parent

Inclusive Education

Finding 8. Implementation wise IE-related interventions have been one of the most effective components

of the Project with trainings conducted among 1670 teachers in Syunik Marz (102 schools) and 3760 in

Lori (162 schools) respectively, and successful involvement of key players and stakeholders (NIE,

coordination with the Republican Centre of Pedagogical Psychological Centre, and other involved

stakeholders including parents). Thus, several key results were fully achieved with some areas still

requiring improvements.

Bridge of Hope NGO is implementing the Inclusive Education (IE) aspect of the Project, and has worked

in several directions to ensure all pre-conditions are set for implementing the system in the Project target

regions, Syunik and Lori Marzes. This included:

• Four Pedagogical-Psychological Support Centres (PPSC) were established/supported in 2 target

regions which provide professional support to mainstream schools. With technical assistance and

policy advice from UNICEF, MoES and Bridge of Hope has already defined the functions and



61

operational modality of these centres. A situation analysis/resource mapping was conducted and a

transformation plan was developed for special schools. A special experience exchange between Syunik

and Lori PPSC professional staff was organized to share the lessons learned during the first phase of

the transformation.

• Based on “Amendments and additions in the RoA Law on Mainstream Education” December 01, 2014

Decree ՀՕ-200-Ն a list of documents was developed to support the implementation of the Law.

Together with MoES, BoH established 6 expert working groups to develop relevant documents and

normative acts for the Government’s approval including “Action plan and timetable for the provision

of a total Inclusive Education System”, PPSC Service Definition, Mainstream school Statute and others.

On February 18, 2016 the Government of RoA approved it.

• Around 200 specialists involved in Syunik and Lori RPPSCs and former special school staff underwent

trainings on such topics as ICF-based assessment of special educational needs of children, formative

assessment; development of individual learning plan, curricula adaptation and others.

• Inclusive education training for around 266 mainstream/regular schools in Syunik (102) and Lori (164)

Marzes reached out to over 5000 teaching staff engaging teachers, school principals, and local experts

in knowledge building and capacity enhancement. The first ToT was conducted for the principals of

these 266 schools by NIE, who then trained their staff locally.

Several other activities and interventions were implemented as part of this Project component including

children’s assessment, assessment of facility conditions (including mainstream schools) and

recommendations for improvements under G2G, and others. In designing and implementing all these

activities, BoH has identified the partners best qualified to deliver the work, including National Institute

of Education and Republican PPSC.

The intensity and rigor of all these interventions is not lost on one of the key target groups, i.e. teachers.

76.9% of surveyed teachers responded that they have attended trainings on inclusive education in the past

2 years with over 95% saying they attended the course in full. 70% of respondents stated confidently that

the trainings were useful to them (see Table 12 below).

Table 11: Utility of Training Session

Were the trainings useful for you? %

Definitely useful 70.0

Somewhat useful 27.1

Not that useful 1.7

Not useful at all 1.3

Total 100.0

With respect to how the teachers assess the different aspects off the trainings, such as effectiveness,

organization, content, etc., their overall average assessment is rather high on the scale of 1-5 (5 being the

highest). See the chart 2 below.



62

Chart 2: Average Value of Training as Assessed by Respondents (5 being highest and 1 lowest)

Generally, during the FGs the teachers also expressed satisfaction with the trainings, they recalled the

BoH and NIE trainers and pointed out that involvement of professional organizations like NIE and RPPSCs

is a good approach. RPPSC trainings are also very highly valued and the evaluation team was able to

observe teacher assistant trainings in progress during the field work. However, despite these positive

findings both from quantitative and qualitative research, there still remain a number of issues that should

be addressed to effectively introduce inclusive education to mainstream schools. These include:

• One time and limited length of trainings with several informants indicating that a 5-day training is

not sufficient for this complex issue and requesting more;

• In some of the schools, the principals just held one general meeting to introduce the concept of IE

without taking the time to transfer the full content of the material.

• The work with parents is uneven across different schools. In some of them principals and teachers

have worked with parents (where IE is functioning longer), in others there have been zero efforts in

that direction.

And finally, monitoring of how IE is being implemented across the schools is an important aspect of the

Project that should be enhanced to achieve full and meaningful integration of IE into mainstream schools

and insure its acceptance by teachers, parents and children alike.

Finding 9. Knowledge of the basics of inclusive education has increased significantly as a result of teaching

staff trainings resulting in some level of practical application of learnt concepts and approaches as well as

overall acceptance of the new system (as perceived by surveyed teachers).

Assessing the level of knowledge increase among school teachers from target regions was first done by

BoH and partners through pre-post knowledge tests. According to BoH final report, for Goris and Sisian

area knowledge growth is 54% and for Kapan and Meghri area it is 50%, and overall growth of knowledge

4.6

4.5

4.3

4.2

4.2

4 4.1 4.2 4.3 4.4 4.5 4.6 4.7

Delivery of material (clear explanation, interactivemethods, etc.)

Content (addressing all issues, introducing newmaterials, etc.)

Logistics (time, place, refreshments, etc.)

Training effectiveness

Utility for teacher



63

measured among 1670 participants is 53%. With respect to Lori, the percentage of the teachers who

answered correctly to all the questions increased from 0 to 82 percent (see chart 3 below).

Chart 3: Change in Absolute Knowledge

Source: BoH Final Report

With respect to the teacher survey implemented as part of this evaluation, teacher knowledge on basic

concepts of IE was also tested, including the definition of IE, some statements about the IE as well as a

question on which children are considered as having SEN (special education need). 93.9% of respondents

were able to choose the correct IE definition. A knowledge index has been calculated on this question (+1

to -1 interval) with the general results comprising 0.87. See Table 12 below that summarizes the index per

sub-region in Syunik.

Table 12: General Knowledge Index by Sub-region

Index of knowledge

Average Kapan Goris Sisian

Definition of IE 0.87 0.91 0.85 0.85

The medical model of disability is at the basis of inclusive education -0.56 -0.69 -0.59 -0.4

Every teacher should have the skills to identify the observation and learning

abilities of the student

0.96 1.00 1.00 0.88

Subject teachers do not participate in the development and implementation

of the individual education plan. Only the parents, the class teacher, the

special pedagogue and the psychologist are involved in the process

0.39 0.41 0.47 0.29

The individual education plan is developed for one school year or based on

the short-term issue.

0.83 0.85 0.88 0.77

Children that are considered SEN 0.78 0.83 0.76 0.75

99.81%

18.22%

0.19%

81.78%

Pre Post

Answered right all thequestions

Answered wrong at leastone question



64

Aggregated value of Knowledge index 0.65 0.66 0.67 0.63

With respect to other statements (on medical model of disability versus social, teacher participation in

developing and implementing individual educational plans and others) as well as SEN definition, a

generalized knowledge index was also calculated and the results are summarized in Table13 below.

Table 13: Application of knowledge (Data source-240)

Do you apply knowledge gained through training Frequency %

Yes, regularly 96 40.0%

Mostly yes 107 44.6%

Mostly no 14 5.8%

No 23 9.6%

Total 240 100.0%

Qualitative research indicated that there were knowledge related differences across the two regions, i.e.

Syunik and Lori. In the first one the trainings happened back in 2016-2017, while in Lori mostly in 2017.

Thus, in Lori Marz the principals and teachers had better knowledge of theory, while in Syunik the

practical understanding of the IE system was better.

Inquiries made both through quantitative and qualitative data collection allowed assessing to what extent

teachers are applying their knowledge on IE. As table 14 below shows, 84, 6 % of respondents answered

positively to the question about knowledge application. However, it should be noted that out of 40%

responding that they regularly apply the knowledge gained, the majority were teachers of elementary

schools.

Some other questions related to application of knowledge addressed the teachers’ involvement in

assessments of children’s learning abilities. While 60.6% of respondents answered this question positively,

around 40% of trained teachers in Syunik Marz are still not involved in the process.

In addition, qualitative research allowed understanding how individual educational plans are developed

and implemented. It appears that in those schools where there is a multi-disciplinary team, the team

specialists are the ones primarily involved in the process, and the required collaboration between the

teachers (including homeroom teacher), parents, psychologist and other specialists is not widely practiced.

Some of the FG participants referred to the plans that are needed for working with children with SEN,

but did not associate those with individual education plans.

The evaluation also attempted to look at IE application through the perspective of children’s inclusion and

acceptance of the new system. Only 14.3 % of respondents said they faced no problems with children with

SEN, with 48.6% indicating some problems, and over 21% citing frequent problems. Some of the teachers

participating in FGs noted progress as a result of their work, while others claimed there is no use working

with these children. As one of the informants noted, “I work with him for two months already to teach

him the colors, no results, he does not memorize…”



65

One hundred seventy five (175) teachers mentioned that they have students with special educational

needs in their classrooms. 56.5 % of these teachers believed that children with SEN are socializing with

all other children in class/school, while 26.7% thinks that they mostly deal with those children that treat

them well (see Table 15).

Table 14: Socializing Patterns of Children with SEN (Data source-175)27

With whom do Children with SEN mostly socialize in

class/school

Frequency %

With all children 131 56.5%

Only with each other 5 2.2%

With neighbors’ children in the same class 16 6.9%

With children of keen in the same class 15 6.5%

With children who treat them well 62 26.7%

With no one 3 1.3%

Total 232 100.0%

FGs and in-depth interviews gave additional insights into children behaviors and attitudes, such as:

• Integration of children from special schools is easier in rural areas where the community knows the

child and the family and accepts them;

• Children with SEN have more problem in middle school when they are teased and isolated. It becomes

difficult for children with physical disabilities to access the school or the classroom;

• The children with SEN are taken out of the class for different reasons (e.g. for food or separate

classwork), which highlights their differences;

• Some of the children with more complicated issues who were dismissed from special schools become

involved in home based education. Consequently, they become more isolated, while in the past there

were integrated in at special school.

And finally, a number of different issues were identified during FGs and in-depth interviews that

challenge effective implementation of the IE in mainstream schools, including:

• In bigger classes, teachers have problems with distributing their time evenly and a lot of time and

focus goes to children with SEN. There is one teacher assistant per 100 children, which is not enough,

especially when there are a number of children with SEN in the school. It is sometimes difficult to

find individual approaches to these children.

• Knowledge assessment/testing of these children is still an issue. There are no special tests/exams for

them, just additional time allocation during exams.

27 Teachers had the option to provide more than one answer to this question



66

• At times children with SEN display atypical behavior, while at other time they use their limitations

as an excuse saying “I cannot do this, I do not want to do this, have mercy”.

To summarize, Project interventions have increased the knowledge on IE among the teachers in target

regions resulting in certain improvements in application and attitudes toward children with SEN.

However, there are still several issues and challenges to be overcome to reach full-fledged integration of

IE in mainstream education.



67

EFFICIENCY

General findings

Finding 10: From the Project perspective, the combination of USAID’s G2G support coupled with the

implementation of the soft interventions under the Project (with each partner contributing to within the

overall budget) was initially perceived as a good tool to increase Project efficiency and resource use, but

it did not work as planned. As of June 2017, around 50% of the overall Project budget has been expended

with mixed outcomes that vary from Project component to component. From state budget perspective,

unwillingness of the two line ministries to join certain aspects of the reforms agenda under one umbrella,

has raised questions among different stakeholders about efficient use of funds and resources.

Under its 2013-2017 Strategy USAID approach to providing aid is based on triangular partnerships with

the host government (through Government to Government (G2G) mechanism), a technical assistance

vehicle through an international organization and a monitoring scheme through local civil society. The

approach was not different in the case of the “Toward Social Inclusion of Vulnerable Children: Expanding

Alternative Care, Family Support and Inclusive Education Services as Part of Child Care Reform” Project.

While the technical assistance guided the reforms implementation through developing alternative service

models, making assessments and providing recommendations for transformation plans and advocating for

legal changes, it was envisioned that the two G2G agreements between USAID and MoLSA (2015-2018),

as well as USAID and MoE (2016-2019) would support the process through infrastructure improvements,

including:

• Infrastructure costs for the set-up of new community based alternative services guided by World

Vision Armenia should be covered by the Government of Armenia with the G2G funding from

USAID.

• While technical assistance and policy advice from UNICEF, MoES and Bridge of Hope targets defining

the functions and operational modality of RPPSCs, the infrastructure costs for service set-up should

be covered by the Government of Armenia with the G2G funding from USAID.

• School plans on providing an inclusive learning environment should include infrastructure

adjustments (wheelchair ramps, accessible bathrooms, FM systems, resource rooms to provide after-

school services and other) that should be covered by the Government of Armenia with the G2G

funding from USAID.

Given these close interconnections between the Project and the two G2G

agreements, the latter two include strong conditionality language with

respect to supporting the Project implementation. However, while this

approach should have resulted in favorable outcomes for advancing the

reforms supported by the Project, the real progress did not fit the initial

plans due to different reasons:

• Disagreement between USAID, Project partners and MoLSA about

Noubarashen reorganization model;

• Delayed reorganization of child care and protection institutions in Lori and Syunik Marzes;

• The Government request that the transitional support to families be provided from the G2G agreement

funds. This was not part of the G2G agreement and eventually state budget resources were committed

once USAID stood firm by its position;

GtG agreements have not been shared

with partners, they do not have an idea of

what is included. But pre-conditions are

good, otherwise the project will turn into

a renovation project. Expert Interview,

Implementing Agency



68

• Late signing of the agreement with the MoES.

Understandably, the lengthy negotiations with the Government on these different aspects of G2G funding

had to some extent a negative impact on the Project progress. It should be noted that the Project partners

were not involved in these discussions despite their respective work being affected by the G2G. UNICEF,

however, undertook an intermediary role between the MoLSA and USAID, eventually brokering an

acceptable solution around which negotiations are still ongoing.

With respect to UNICEF and IP contributions to the overall Project funding and resources, as of July 2017

they had collectively committed over 0.5 million USD to the overall Project in addition to the USAID

funding, mostly in the form of human resources and Project support through their ongoing activities (such

as UNICEF’s campaign for inclusion of children with disabilities or WV’s ADP level social work). As of

mid-July, over 50% of the overall Project budget has been spent, with most of UNICEF and IP

contributions committed (see Table 9 below for details).

Table 15: Project Budget and Spending

Total Project Budget $ 3, 481,625,00

UNICEF and IP contribution for Years 1-3 $553,769.00

Total Spent to Date (Sept 2014-July 2017) $1,804, 867.00

% Total Spent against Budget 52%

UNICEF and IP contribution to Date (Sept 2014-July 2017) $ 472,975.00

However, as noted above, most of the tangible accomplishments as of June 30, 2017 were attributable to

BoH’s interventions that were completed in two regions (Lori and Syunik) and had spent its budget of

over 500, 000 USD. Other significant accomplishments include training of over 250 case managers and

social service providers that exceeded the initial target of 145, as well as adoption of secondary legislation

resulting from UNICEF in Armenia’s advocacy efforts and laying foundations for future progress.

Budget wise, CSCF has expended over 74% of its funds by June 2017, with the amount to be almost fully

expended by December when their contract expires. A small unspent portion is due to the pending

assessment of 140 children from two night care institutions in Armavir and one in Dilijan, which did not

happen due to the termination of the Project. SC has frozen the unspent portion of its budget pending the

adoption of the Family Code and subsequent activities, while WV also faces an unexpended budget balance

due to delays in its planned work. By June 2017 UNICEF has spent only 34% of its budget, i.e. $180, 328

with 342, 994 USD still remaining unspent. With respect to this state of affairs, it is worth emphasizing

again that although the Project was designed as a 5-year intervention with very ambitious plans of

activities, the start-off phase was slow, especially with regard to recruitment of project staff, contractual

arrangements with partners, developing MOUs, assessments, getting the Government stakeholders on

board, clarifying their roles and getting their buy-in. It took over one year to bring both the Government

and the project partners on board and fully start the Project implementation.

And finally, one more aspect of Project efficiency is related to how the State budget funds are being

utilized in support of the reforms, more specifically their distribution through the two line Ministries -

MoLSA and MoES. As noted throughout this report, child care and protection institutions and respective

CFSCs fall under the MoLSA, while the special schools (and their spinoff RPPCSs) fall under the MoES.



69

The assessment of the children from the first type of institutions is emphasizing their social needs, while

the latter is focused on educational needs. However, special schools are also boarding facilities and most

of the residing children come from families with at times complex social and poverty problems, especially

given the fact that the majority is placed there due to social and poverty related considerations. All

interviewed experts agree that there should be one unified model for children/families not to go to

different places for meeting their different needs. However, due to the conflicting interests of the two

Ministries, mostly related to the inflow of funds from the State budget, the two processes continue to be

separate, without much coordination and with use of different approaches. As one of the key informants

noted, “The funds from these sources could come to one place and serve the children in a more targeted

way”.


Finding 11. UNICEF and MoLSA have successfully joined efforts to mobilize all ongoing interventions

related to a child’s rights to protection and social systems reforms to support the Project activities and

interventions.

From 2010 onward UNICEF Armenia has been heavily involved in the GoA’s social services reforms

guided by its belief that family vulnerability including issues related to child rights protection should be

addressed holistically by a single system with integrated services. With technical and financial support

from UNICEF, USAID and WB, Armenia is currently implementing comprehensive reforms in the area

of social protection, including child protection. Introduction of the Integrated Social Services (ISS) is the

main pillar of these reforms emphasizing a “one window” model for provision of cash and non-cash

benefits, co-location of different service providers (responsible for pensions, social assistance, employment

and disability certification), introduction of the concept/system of case management and project

management through local social planning efforts. These developments have important implications for

the Project since with thousands of children out of institutions and returning to their biological or foster

families, case managers’ workload is expected to increase. Thus, throughout the Project UNICEF has

emphasized enhancing case manager understanding of the deinstitutionalization processes and their

ability to support children leaving institutions in their reunification with their families and communities

of origin. UNICEF contracted CSCF to train case managers and other social service providers on foster

care and deinstitutionalization responses in general. A thirty (30)-hour training were delivered to over

100 case managers (instead of the planned 45) and 192 service providers (instead of planned 100) in Syunik,

Lori, Yerevan, Armavir and Ararat. The trainings were developed after conducting needs assessment of

case managers (through in-depth interviews and focus groups) and followed by the development and

delivery of the training modules. The insert below summarizes the topics included in the case management

modules:28

Modules Included in Case Management Manual

28 The information source include summary data provided by CSCF based on its reports and UNICEF QR 3 and 4,

2016



70

1. Case management in the RA within the context of current reforms;

2. Integrated provision of social services as a modern system and case management as part of it;

3. Deinstitutionalization and alternative care: Role of the case manager;

4. Child protection from violence and referral procedures in the context of the child’s best interest;

5. Specificity of work with children that need special protection and case management.

Besides, 30-hour 5-day Foster trainings to over 78 case managers and service providers (instead of planned

60) were delivered in Syunik, Lori, Yerevan, Armavir and Ararat regions. The training topics included a

more general background approach of care options: institutional care, adoption, kinship and foster care

and narrowing down to the specifics of foster care and the logical steps based on the best interest of the

child. The insert below summarized the topics included in the foster care modules:

Modules Included in Foster Care Trainings

1. Main forms of foster care for children in difficult life situations;

2. Foster Care procedure and stages of implementation;

3. Specifications of working with foster parents;

4. Specifications of care in the context of child rearing;

5. Specifications of care for children with special needs in the context of foster care.

As noted above, in total, 344 families were assessed by case managers in Lori and Syunik and an additional

52 in Kotayk. As a follow up to the needs assessment, case managers have developed individual social

action plans and worked to implement those partnering with WV whenever the involved families have

been eligible for assistance. Some targeted case managers also were trained on assessing foster families,

making decisions on placements and supervising the work of foster families.

The Project interventions allowed building on UNICEF’s other capacity building activities targeting case

managers and mobilizing them around the current deinstitutionalization and IE reforms. As agreed by

experts and stakeholders, case managers have learnt how to follow up the needs assessments and remain

continuously engaged with the families. They have also collaborated with other stakeholders such as the

police, health and education providers, and others, thus adopting the culture of institutional cooperation

for addressing children’s issues and needs. Generally, the case managers’ involvement in the Project

allowed them to apply in practice case management as a tool and understand its uses. It also had some

broader implications since the case managers and other social service providers, as key stakeholders of the

ongoing reforms, improved their understanding of their role thus also mitigating any resistance that

existed. All interviewed case managers that had attended CSCF trainings (5) assessed them as very useful

and of high quality.

Their approach is very interesting…we go to families together, conduct assessments, they guide us how to

assess…We have regular contact, call them with our questions. They are very supportive. In-depth interview,

Case manager

If there were trainings like this, I would like to attend again. Their presentation of how to manage cases was very

interesting…In-depth interview, Case manager



71

The competencies and techniques acquired by case managers were successfully tested during the

Noubarashen reorganization. It was an example of a good cooperation model among ISS case managers,

employment services, MOLSA, NGOs, etc. to leverage resources in support of the children being reunified

with their families. In addition to Project-led efforts, it should be emphasized that several stakeholders

noted how effectively MoLSA mobilized case managers to support children released from Noubarashen

who were returning to their communities of origin. Top level ministry officials worked closely with case

managers requiring monthly reports from them on the situation of these children. Both WV and CSCF

also worked closely with the case managers, especially in Syunik, where there is more for support. Some

quotes included below provide evidence to the effectiveness of these efforts.

Ideally, case managers should have been involved earlier, since the children were out of the institutions without

any response in place. However, MoLSA’s close supervision of the process at that point was very helpful. Expert

interview, Implementing Agency

Two-sided assessments were implemented in Lori to understand the placement options for each child, Following

this, the Marz administration decided to visit the families again, and then expanded discussions of cases were

conducted. Expert interview, Implementing Agency

… the Marz administration requested that we report on 30 cases of children that left institutions. We provide

those and know that all 30 children are enrolled in public schools. In-depth interview, Case manage

Despite these significant positive developments, both experts and case managers point to some persisting

issues that needs to be addressed, including:

• Perception of the case managers’ role as mostly that of cash benefit distributor rather than a case

manager since the same professional cannot effectively implement both functions. This issue was

especially visible in Lori, where several stakeholders noted that case managers’ involvement in

deinstitutionalization support has been suspended due to the Marz administration’s position that

those two functions are incompatible.

• The case managers’ involvement in assessing of the biological families of the children, but not

children themselves, which was done by other organizations, as well as limited cases of foster

family placements did not allow case managers to practice their skills with respect to this

alternative services.

• Lack of support services and other resources remains a constant limitation to effective

implementation of the case manager’s function.

However, it should be emphasized that the overall progress with strengthened application of case

management practice is unquestionably positive. It is largely due to UNICEF in Armenia’s support to

reforms and policies with respect to the ISS model and case management in the past decade, and testing

and enhancement of the system through projects implemented by UNICEF. This include the EU-funded

“Mitigating Social Consequences of Labour Migration and Maximizing Migrants’ Involvement in Local

Development” project and this USAID-funded Project. .

As to Territorial Social Plans (TSPs), which is part of the ISS reforms, they were embedded in the 2010 -

2014 key legislation and elaborated in the new Law on Social Assistance due to UNICEF in Armenia’s

effective advocacy efforts. Development of the LSP Concept, Methodology and Financing Models with an

eventual passage of a joint MoLSA and MTA decree on LSP introduction and reflection of related



72

provisions in the amended Law on Social Assistance (2014) allowed to formalize this effective vehicle for

identifying service gaps and establishing lacking social services in the communities. However, these plans

should be supported by the adequate financial resources. While the new Law on Social Assistance includes

several provisions related to institutionalization of LSP, provisions for financial resources in support of

social projects are still ambiguous and the State budget as a source of funding is not included. This may be

one of the reasons, why the TSPs were not fully utilized for the Project implementation.

To summarize, UNICEF and MoLSA have consistently and successfully capitalized on other critical

reforms initiatives, namely introduction of ISS and case management to support the ongoing

deinstitutionalization and IE reforms. This approach has been especially effective in the current situation

when the communities of origin face a lack of services and social workers, and case managers have been

closing that gap to the extent possible.

Finding 12. To a certain extent, both UNICEF, WV and CSCF are all involved in the deinstitutionalization

component of the Project. As a result, there is more expertise and targeted efforts invested on the one

hand, but somewhat confusing role distribution and coordination on the other.

UNICEF’s role as the lead agency responsible for the coordination of the Project implementation was

defined in the Project Proposal and further implementation agreements. However, in addition, UNICEF

also undertook several other responsibilities with respect to “the strengthening of the legislative and policy

framework for quality social protection of vulnerable children and their families…assisting MoLSA in

developing the strategy on deinstitutionalization and establishment of alternative services… ensure

capacity building activities for case managers and social workers in all aspects related to de-

institutionalization”29. According to the initial Project setup World Vision was responsible for establishing

alternative community based services (including the CFSC model, costing, staff training, etc.), though as

discussed throughout this report, it eventually became involved in the assessments of children, provision

of assistance, support of families reunited with children, and other activities. FAR’s CSCF that was brought

into the Project slightly later by UNICEF eventually became involved in child assessments, and supporting

case managers’ work with families. Consequently, a number of functions were replicated by these different

organizations. To illustrate:

• WV’s role in developing the model for the CFSC assumed a certain involvement in the

deinstitutionalization strategy and policy including costing of services and financial implications.

However, as the lead coordinating partner, it was UNICEF that was involved in policy level discussions

and communication with the Government partners. The ownership of the model and follow up with

the Government was at times ambiguous leading to hold up of the model for several months.

• WV was involved in the assessment of children in Noubarashen and Vanadzor. Given the delays in

reorganizing these institutions, their assessments became outdated and a need for re-assessment

emerged. While in the case of Noubarashen, WV committed additional resources to this, it is assumed

that in Vanadzor FAR’s CSCF will take over. Interestingly enough, it appeared during the interviews

that WV did not explicitly acknowledge its removal from the assessment process, but according to

29 Project Proposal



73

UNICEF the change was due to discontent by MoLSA about the company that WV outsourced in-

depth child assessments to. CSCF’s involvement was added by UNICEF based on the need to involve

case managers in child assessment and referral process, and CSCF’s respective capacity and experience

in this field.

• In Kapan (Syunik), UNICEF has worked directly with the institution through trained case managers

supported by CSCF, who assessed the social needs of the children and developed individual care plans.

However, throughout the interviews and FG discussions, both WV and other stakeholders indicated

their social workers’ involvement in both assessment and support to families.

You do one thing, then time passes, your work loses its relevance, and then the same thing should be done over.

Expert Interview, Implementing Agency

…. We provided case by case response by social workers, developing the individual plans and working with

families. Expert Interview, Implementing Agency

We signed a separate agreement with UNICEF for case manager training and under it we also worked on training

foster families. Expert Interview, Implementing Agency

Interestingly, while WV perceives some of its work under the Project as outside its agreed scope, in reality

the respective activities have been supported through the WV budget lines, including ADP salaries, as

noted by UNICEF in Armenia staff.

Nevertheless, duplication of efforts has definitely resulted in some inefficient use of resources, since while

World Vision has expended its entire budget for in-depth child assessments and even committed

additional resources (around 18,000 USD instead of planned 12, 000), UNICEF has also provided Project

funds to CSCF for similar activities. In addition, there were no clear distribution of roles as to who and

how supporting family integration and later monitoring of the children would take place. While the case

managers’ general role is more of a referral and consolidation of resources, the social workers have the

intimate knowledge of the beneficiaries/clients and are the resource persons available locally to address

day to day issues of children and their families. Under the Project, the case managers were trained to

expand their scope and provide services to beneficiaries under the supervision of CSCF experts, but the

process is ongoing and needs to be further enhanced, especially given that the service area under a case

manager is much larger than that covered by a community social worker. While social service NGOs like

CSCF, COAF and other have a handful of social workers, WV is the only organization that has a large base

of social workers across the country and they became the first point of contact for the reunited families in

those areas where they are present (e.g. Kanaker - Zeytoun district of Yerevan or Syunik Marz).

However, the positive aspect of this situation is that all involved organizations have vested interest in

promoting child welfare reforms and supporting families and alternative services as part of their mission.

Despite the occasional ambiguity of roles and responsibilities, they all mobilize available resources when

needed to serve the child’s best interest. Oftentimes, it is accomplished as part of their ongoing activities

that are not part of the Project, but supports the same goals and objectives (e.g. WV and CSCF activities

through their regular work).

As noted by the benefiting institutions, though the assessment of the children in the institutions was

implemented separately by three-four different actors, the results were triangulated, discussed and

decisions made jointly. To illustrate, the children residing in the institutions and their families have been



74

assessed by at least two groups in terms of their social situation, including home visits by the work group

of the Institution, by the staff of the FWCRP department and World Vision (first by the hired

organization, then by the WV relevant staff). In some cases, the case managers were mentioned to be of

great help during the assessments (in Yerevan especially). While assessments were implemented

separately, afterwards join discussions were held and decisions made on each individual case, jointly.

According to the Director of Yerevan CFS Center, the assessment results mostly coincided and were

discussed and as such lead to joint decisions.

Foster Care

Finding 13: The progress and the future of the foster care component of the Project was challenged by the

delays of the approval of the Family Code30 that would enable children in difficult life circumstances to

benefit from foster care services and the fact that in 2017 no funds from the state budget were committed

to increase the number of foster families, thus challenging foster family motivation.

Since 2008, the Introduction of Foster Family Institute has been supported by the RA State Budget, which

annually committed funding for foster care of 25 children deprived of parental care. As part of the Project,

it was planned that 80 children should be placed in foster care. Among those placements 10 would be

children with disabilities and 20 would be placed for a shorter term, with 80% of those children staying

in the same family for at least 6 months. However, new placements did not take place due a number of

reasons:

- According to the Project priority diversified foster placements were to be made for children in target

institutions. During project implementation phase reorganisation/transformation plan included mainly

night care institutions and special schools for children in difficult life circumstances, including those with

disabilities, whereas the acting Family Code enabled foster care placements only for children with legal

status of “deprived of parental care”.

- In 2017 as well, no additional funds were committed to increasing the number of foster families, which

affected the Project efficiency since the foster families’ trainings happened in 2017 without any immediate

placement of children in families.

However, some positive developments are expected in this regard in 2018. The 2018 Child Rights

Protection Program approved by the Government on September 29, 2017 plans to increase the number of

children in foster families to 120. The State Budget will fund these items of the Program by committing

166343.5 thousand drams. As one of the program experts noted, “we have identified and trained families,

but they are not being paid. USAID/UNICEF could have supported these families before the allocations

from the state budget were made”.

It should be noted that the approval of the Family Code is essential in this respect since the proposed

amendment will allow also qualifying children in difficult life circumstances for foster care thus increasing

the potential number of children to be placed. However, despite some progress in this direction, the

passage of legal documents is still pending31. As noted above, on March 2016 the RA Government approved


31 As the Introductory Note to this report specifies, the package of legal documents related to the amendments of the

Family code was submitted to the National Assembly in November 2017.



75

the Concept for Placing Children in Difficult Life Circumstances in Foster Care. The Concept outlined

those amendments that should be made to existing legislation, including those to the RA Family Code,

Civil Procedure Code, Administrative Code, RA Law on Protection of Child’s Rights and others. In July

2017, the RA Ministry of Justice has circulated the proposed changes on the e-draft platform. The essence

of proposed changes is summarized below:

• While according to the existing legislation only children deprived of parental care can be placed in

foster care, the amended Family Code will make children in difficult life circumstances eligible for

foster care starting from 2018. Under aged single mothers could also be placed in foster care.

• Training will become mandatory for foster families.

• The following 4 types of foster families will function- specialized (with more intense capacity building

planned for this group), crisis response, vacation, and general. In addition, age limitations are also set.

Thus, none of the foster parents should be older than 55 at the moment of placement, while the age

difference of foster parents should not be less than 18 years older than children under their care and

more than 50.

Monitoring of the foster families is also in the focus of these legislative amendments. If currently the

Guardianship and Trusteeship bodies are responsible for the selection and monitoring of foster families,

the proposed amendments transfer this authority to Marz administrations, Yerevan municipality and

Territorial Office of Social Services.

To summarize, significant efforts were invested by the Project to ensure that legal changes required for

the effective functioning of the foster care are approved and enforced. However, the delays in the passage

of the overall legislative package are currently challenging the real functioning of the system and

demotivating the potential foster families that underwent training and got accreditation as foster families,

are factors that are starting to have negative impact.

Inclusive Education

Finding 14: Despite significant knowledge about inclusive education, somewhat favorable perception by

schools and the entire community, lack of special conditions and resources needed has been voiced as a

critical challenge to implementing inclusive education. Additionally, the changes in the financing of the

system also affect its efficiency and effectiveness.

While the soft interventions under the Project have proceeded intensively and effectively, the physical

improvements of the schools involved in the IE system envisioned by the G2G agreement have not

followed. Several informants at policy making and school level note this as one of the key challenges to

inclusive education since the lack of ramps, special toilets and other aspects of physical environment

diminish accessibility of schools. In some of the schools there are no vacant classrooms to be designated

for working with children with SEN, or the needed items in such classrooms are missing (e.g. the mirror

for the speech therapist).

In addition to physical resources, lack of specialized professionals has also been noted as one of the

impediments to implementing inclusive education. There is a confusion about the multi-disciplinary team



76

and the teacher assistant. Some of the informants believe that the existence of the team was a good resource

for the IE, while relying only on teacher assistants is not sufficient since they are not involved in

assessment of children, which is supported by the RPPSCs and are working with a very large number of

children.

With respect to RPPSCs, their work is considered useful and supportive. In Syunik, the RPPSCs are fully

functional, involved both in the assessment of children and support to schools. In Lori, Spitak RPPSC is

more advanced than Vanadzor, which is still not fully operational. The average value of the collaboration

of schools with these centers (RPPSCS) is 3.9 (out of 5) based on the survey results. The RPPSCs are mainly

involved in needs assessment of children with SEN, supporting the schools in assessment and in working

with the children. The RPPSC staff should visit the schools to provide assistance, but they do not have

sufficient transportation means (only one car) or are under time pressure. As one of the Center heads

noted, “we did all of this in the past as well. Now our staff is to go all around the region to do the work,

but in the past, they were all in one place. That was easier and more effective”. However, it should be

noted that timetables of activities for children with SEN were available in all schools and they included

the center interventions as well (based on observations).

The RPPSC staff should also provide services in the centers, but several of them claimed that the former

special schools are not adapted for that. Some opinions were also voiced about the accessibility of the

centers since they are in the locations of the former special schools, which were intentionally put outside

the central areas. However, with current functionality, the centers should be easily reachable by the

beneficiaries, both children and professionals.

The Centers also lack specialists, such as speech therapists, special educators, and psychologists. Many of

RPPSC staff are now involved in distant learning at the Armenian State Pedagogical University or other

pedagogical universities to gain this new qualification, since in the past they used to be subject teachers at

special schools.

There is a dire need for specialists, we deal with specialists with no relevant education (psychologists, social

educators) …however, according to the Government decision they should have University education in that area.

In Goris, when they realized how serious the issue was, they opened a graduate program at the University to train

people…In-depth interview, State Agency

And finally, throughout interviews and discussions concerns were raised with respect to financing

mechanism of the inclusive education system, which has affected both the motivation for implementing

it and the involvement of the specialists. In the past, when inclusive education was the focus of the

reforms, the school was financed by the number of assessed children with SEN receiving several times

more per-child funding for these students. However, since the introduction of the concept of the overall

inclusiveness, only children with mid- to a high degree of problems receive additional funding. These

changes also affect the availability of the specialists. If under the first model the multi-disciplinary team

consisting of different specialists was required to support inclusive education, currently the institute of

teacher assistants is introduced. As discussed throughout the report, one teacher assistant is covering over

100 children without real ability to focus on their issues. Consequently, the motivation to implement

inclusive education in different regions varies. Thus, in Lori where they just introduced the system and

are using the old formula, the school principals are motivated to have more children with SEN assessed



77

and identified. In, Syunik, which has shifted to overall inclusiveness, the attitude is different, there is less

interest to work with the RPPSCs on initial assessment.

One of the worst things that could be done is that the multi-disciplinary team is not financed because of introducing

the teacher assistant. It is assumed that the assistants should be specialists like speech therapists or special teachers,

but the position description does not require this…in the regions the lack of specialists is overwhelming Expert

Interview. Independent Expert

To summarize, efficiency of IE interventions within the project is directly related to additional injections

into the system to support the ongoing process. The Project has successfully utilized its funds to train

thousands of professionals and establish RPPSCs as a resource base for IE implementation. As a result,

positive perceptions of inclusion of children with SEN and knowledge of basic IE concepts are becoming

integrated into the educational system. However, all these achievements are not sufficient for successful

implementation of the IE. Additional investments are needed in the physical infrastructure (through G2G)

and further capacity building of professionals involved both at schools and RPPSCs. State funding

mechanisms should also provide financial motivation for the schools to implement the system. Thus,

ensuring full efficiency of the IE component is dependent on a good balance between soft interventions

and infrastructure improvements, donor assistance and state commitment. All these factors are also critical

for further sustainability of IE integration into the mainstream education system.

SUSTAINABIILTY32

Finding 15: Sustainability of the Project interventions in support of reforms efforts depends on a number

of overarching and interlinked factors ranging from availability of alternative care mechanisms to capacity

of the professionals involved in delivery of services to public awareness and others. While the Project has

successfully established preconditions for some, others still need to be initiated and implemented.

Sustainability guarantees of these multiple and complex interventions are equally diverse and manifold.

They include laying down the foundations of the prerequisite legal framework, availability of financial

resources, availability of professional human resources to implement the reforms at local levels such as

social services, schools, PPSCs, others, availability of alternative care services, specific guides and

regulations about service operations, public awareness and acceptance of reforms, etc. Some of these

sustainability triggers have been incorporated into the initial Project design and were more or less

successfully ensured during the past 2,5 years, including:

• Advocacy with the Government of Armenia and the line ministries to adopt much the needed

programs, normative acts, decrees and decisions that will lay the foundations for pushing the reforms

forward. 2014 was a turning point in this respect since both Amendments to the Law on General

Education and the new Law on Social Assistance were adopted laying the legal foundations for

32 This finding covers both general Project related and more component-specific sustainability aspects of the Project.

Given the current state of the progress under the Project, sustainability under IE and Foster Care is combined with

findings on efficiency, while in case of deinstitutionalization, there is no standalone sustainability finding given how

its different aspects are considered within this finding.



78

inclusive education and introducing the key elements of the Integrated Social Services (such as case

management, local social planning and institutional cooperation among social service providers).

These key developments were to a great extent attributable to UNICEF’s longstanding efforts in both

directions through its Education and Social Protection programmes.

As discussed in several parts of this report, despite some challenges with respect to Family Code and

reorganization of Lori and Syunik Marzes child care and protection institutions, several important

normative legal acts were passed by both the Government and the two line ministries in support of

the reforms. These included the 2017-2021 National Strategy on Child Protection, the Action Plan

and Timetable for Provision of Total Inclusive Education System, The Concept for Foster Care of

Children in Difficult Life Circumstances, several decrees and decisions issued to implement institution

reorganization, to approve methodologies, etc., relevant to inclusive education.

• Capacity building of different groups of professionals has also been an important aspect of almost all

Project components, including the planned training of reorganized institution staff (which happened

in September-October 2017), training of foster families, training of the staff of special schools and

RPPSCs, large scale training of teaching staff, training of social service providers, including case

managers, etc. However, despite involving these different groups and thousands (including school

teachers) of professionals in capacity building activities, these interventions were mostly one time

and/or short term in nature, whereas long term consistent efforts are required to change the attitudes

and professional behaviors of these specialists.

• The situation is less favorable with respect to availability of alternative services, especially at the

communities of origin of the children leaving the institutions. This aspect of the Project, as discussed

throughout the report, has been one of the most problematic, especially with respect to social services.

While Noubarashen institution was reorganized into a CFSC (2016), the transformation of Syunik and

Lori institutions is still pending. With respect to Noubarashen, several of the children that left the

institution returned to their communities of origin, where service provision is almost nonexistent. As

the WV representative put it, “If WV does not work there, then no support is provided to them. WV

does some work in Lori and Syunik Marzes, but children return to different places and in this case,

they should integrate both into the family and the education system.” To address the above mentioned

situations and to overcome the existing service gaps, UNICEF mobilized its local and international

partners (such as SOAR, SOS-villages, CSCF) and the ISS case managers to provide the needed support

to children reunified with their families.

• Well-developed and approved regulations, guides, manuals and other tools are also an essential part

of rolling out new systems and services that provide consistency and direction to reforms

implementation by relevant institutions/players. While the reorganization decrees set out the

functions of new service centers or RPPCs, there is still a lack of specific guidelines and methodologies

as to how these new functions should be implemented. To illustrate, while several teachers raised the

issue of not knowing how to work with individual cases/children, NIE experts pointed out that they

had sufficient understanding of the general approach to tackle each child specifically. In case of foster



79

care, comprehensive guides and manuals have been developed, but their application is still pending

on the adoption of the amendments to Family Code33.

With respect to financial resources, as noted above, the approach that combined the Project and G2G

resources has been perceived as an effective mechanism for ensuring that the Government reforms have

sufficient donor support. Once these funds are provided and are matched with Project interventions, the

reforms progress should be more visible. In addition, UNICEF has provided technical assistance to the

Government in costing the reforms and redirecting funds from reorganization of institutions toward

supporting families and establishing alternative care services.

And finally, there are two aspects of the reforms that, as agreed by interviewed experts, need significant

improvement if sustainability of reforms is to be ensured. These are 1) public awareness building about

the reforms and 2) working with parents (both of the children to be reunified with their families and

larger parents’ groups affected by the introduction of inclusive education into mainstream public schools).

There is a shared belief among donors, implementers and experts that the Project has not emphasized

these aspects of the reforms despite their importance for the success of the Project and the reforms. Some

quotations and reflections on these issues are presented below.

The Education component of the current project hasn’t foreseen awareness rising, though it is a very important

aspect to remove the invisible barriers at schools for children with disabilities. BoH Final Report

On the 6th day of the in-service inclusive education training School the principal/teacher will conduct an expanded

seminar for the parent community at the school and introduce the overview of inclusive education. BoH Final

Report

…there should be daily engagement with the community, awareness building, but no one is doing that. Smaller

efforts, but not for the bigger reforms. Neither the staff, nor the families know what is happening. Expert Interview,

Independent Expert

… We identified potential foster parents, introduced them to regional authorities, but they did not

support these parents. Expert Interview, Implementing Agency

One of the weaknesses of the Project is that public awareness is not included in the Project. Expert Interview,

Implementing Agency

CROSS CUTTING ISSUES

Finding 16. While the Project has emphasized the gender aspect of its interventions requesting data

disaggregation by gender and general sensitivity to gender issues, there seems to be no visible differences

in the way deinstitutionalization or inclusive education is affecting boys and girls. However, with respect

to the specialists involved in both reforms, female professionals have a prevailing presence.

At the stage of the Project Proposal, UNICEF in Armenia and partners’ joint analysis indicated that” the

institutionalization affects both boys and girls and their number in institutions is generally equal. The only

exception is that the proportion of girls with disabilities to boys in specialized orphanages (55% boys with

33 As noted in the introductory note to this report, the amendments to the Family Code were submitted to the

National Assembly in November 2017 and adopted in December 2017.



80

disabilities vs 45% girls) is higher than the same proportion in the families (68% boys with disabilities vs

32% girls).”34 Table 16 below confirms the initial assumption that the number of boys who have been

certified for SEN is slightly higher. However, throughout the evaluation no obvious differences in how

deinstitutionalization and inclusive education reforms is affecting boys and girls were observed.

Table 16: Assessed Children by Gender

Number of

assessed

children

Total No.

of children

No of

Certified

children

according

to 2017

First Level assessment Size of Support

No SNE Have SNE

Mild Modera

te Severe

Compoun

d

Girl

Bo

y Girl

Bo

y Girl

Bo

y

Gir

l

Bo

y Little

Modera

te

High

Freque

nt

Total or

Regular

Number of

First level

assessed

children

640

67

0 75

12

1 11 12 64 106

1310 196 23 170 41 34 34 57

Number of II

level assessed

children

Total No.

of children No SNE Have SNE Mild

Modera

te Severe

Compoun

d

38 57 0 2 37 56 5 8 13 17 14

2

9 4 3

95 2 93 13 30 43 7

Source: BoH Final Report

With respect to inclusive education, as both quantitative and qualitative research showed, adaptation of

girls and boys involved in the inclusive education system is conditioned mostly by the issue they are facing

regarding SEN and not primarily by their gender. There were some opinions voiced during the qualitative

interviews that girls with issues had more complexes and found it difficult to overcome their situation or

that sometimes parents of girls are more inclined to avoid assessment supposedly not wanting to stigmatize

their children. However, as already noted, the informants always emphasized the fact that that seriousness

of the condition faced by the child is the main factor affecting adaptation rather than gender.

One gender-specific attitude toward children was expressed during FGs with foster parents who indicated

that it is preferable to adopt girls since there is the perception that girls are easier to deal with.

However, the situation is quite different in case of professionals working with children, the predominant

majority of whom are women with few exceptions where men are heads of RPPSCs or public schools. This

situation is indicative of the general trend in Armenia’s education sector. Thus, 92.3 % of all surveyed

teaching staff members are female with only 7.7 % being male (mostly PT teachers).

34 Project Proposal.



81

Similarly, the staff of child care and rights protection institutions are also mostly female. It has also been

observed during the visits to these institutions that it is mostly female family members (mothers and

grandmothers) that are accompanying children to the CFSCs, while fathers are present when the child

with disability needs physical support.

An interesting gender trend is observed in case of the foster care generally. Since it is not required that a

foster family comprise of both a father and a mother, in those cases when the foster care giver is not a full

family, it is almost always a woman and these are the majority of cases. However, there are cases when

two women are registered as a foster pair, and very few cases when a male is foster parent without a

woman pair. Chart 4 below summarizes the gender dynamics under foster care.

Chart 4: Gender Distribution across Foster Families

Source: List of trained foster families provided by SC/CSCF

However, it should also be noted that since only one of foster parent’s presence at trainings is sufficient

for qualifying the family for foster care, most of trained foster family care givers were women.

30

1

19

2

Female Male M/F F/F



82

CHAPTER IV: CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS

CONCLUSIONS

Over the last two decades UNICEF and development partners have successfully advocated with the

Government of Armenia for the child’s right to live in family environment as being in the child’s best

interest. Several study visits, expert and situational analysis/assessments, costing analysis with respect to

the financial burden that institutions present for the state budget and other efforts have resulted in the

Government of Armenia’s declaring deinstitutionalization reforms a priority back in 2011. From there on,

momentum for the reforms was irreversibly established as evidenced throughout this report by

referencing several key policy and legal documents (including a number of decrees and decisions for

reorganizing specific institutions). Most importantly, several key national programs and strategies such as

the National Strategy for Child Rights Protection 2013- 2016 and 2017-2021, the Government Program

for 2017-2022 and Human Rights Action Plan for 2017-2022 further consolidated commitments to

transforming the child welfare system and prioritizing de-institutionalization reforms.

Similarly, following the amendment of the Law on General Education in 2014 and the approval of the

most recent State program for Education Development 2030, inclusive education has become one of the

key priorities of the Armenian Government.

Both reform directions have been supported through several pilot initiatives helping the Government to

observe and assess the practical implications and implementation modalities of the reforms.

Thus, by 2014 when the Project was being designed, the momentum for the reforms in both directions

was irreversibly established and the Project intended to support and sustain that momentum.

As emphasized in this report, the Project almost identically mirrored the key aspects of these two reforms

directions by focusing on improving the pre-requisite legal framework, supporting transformation of

existing institutions and establishment of alternative services, engaging into capacity building of

professionals and specialists required for implementation of the reforms and other relevant efforts. Each

of Project components was tasked to a competent organization with reputation and experience of working

in the relevant areas (World Vision, Bridge of Hope, Save the Children, FAR’s Child Support Foundation)

with UNICEF acting as the lead partner and coordinating all Project efforts. However, due to this initial

distribution of these roles and responsibilities, coordination between partners and UNICEF as well as

UNICEF and USAID, and the Project and Government partners has been at times challenging and

ineffective.

Two key considerations are:

• Inclusion of two major reforms directions (with their different aspects) into one Project complicated

coordination efforts. While both areas of reforms deal with child’s rights and transformation of

institutions involved in child care and protection, they have different stakeholders and beneficiaries

both at the Government, regional and local level (MoLSA and MoES, child care institutions and special

schools, alternative services and public schools, different Marz departments, etc.).

• UNICEF’s role is to be an advocate for policy making in the best interest of children. However, due to

the complexities of the Project, there were at times some confusion around the role of UNICEF as the

coordinator and implementer of the Project. This situation is most visible in the relations between the



83

Government, USAID and UNICEF, where the Project implementation was in parallel with the

interests of G2G support between USAID and the Government, but perceived as the same process and

not two different relationships between the three parties, which at times led to misunderstandings

about who was accountable for what.

The Project has reached some key results including reorganization of a number of institutions (both child

care, child rights protection and special schools, establishment of one CFSC and 4 RPPSCs, needs

assessment and placement of hundreds of children), training of thousands of teaching staff under the IE

component, as well as other professionals, development and endorsement of legal acts, etc. However, there

have also been pushbacks such as the delays with the transformation model in Noubarashen, no visible

progress with setting up of alternative community based services, delays with the amendments to the

Family Code and subsequent challenges with the placement of the children in foster care, and others.

While some of these interruptions are attributable to the Government’s uneven actions with respect to

their reforms agenda (i.e. situation with the Family Code)35, others have been due to the internal Project

dynamics (i.e. non-clarity with child assessment and follow up family support. PSAs and documentary on

foster care were produced but not publicly released yet, upon the decision by all parties to go public once

the Family Code is adopted).

The Project has also been at disadvantage given the non-inclusion of a rigorous national-level public

awareness campaign in all its directions, i.e. deinstitutionalization and foster care as well as inclusive

education. The public at large is mostly uninformed about its general goal, objectives and directions,

though efforts have been made to inform and engage key stakeholders and at times beneficiaries through

local meetings and forums.

The future of the Project interventions is very much dependent on the availability of a professionally

trained cadre of specialists to implement the reforms locally as part of newly established services (CFSCs,

RPPSCS) and those supporting inclusive education in mainstream schools. While the Project has designed

and implemented capacity building interventions, they are mostly one time and short term in nature

which is a concern expressed by different Project stakeholder groups in terms of continuity and quality.

Overall, given the complex and sensitive nature of the reforms supported by the Project, more time is

needed for effective implementation of some or its aspects (such as establishing CFSCs and making them

operational, putting in place professionals, etc.). As the situation with inclusive education component

indicates, the fact that Bridge of Hope has been involved in similar interventions for almost over a decade

and is replicating its pilot project at a larger scale through the current Project, the implementation of this

component turned out to be more comprehensive and effective.

And finally, while assessment of the Project impact was not directly targeted by this evaluation given its

focus on mid-term results, it should be noted that even at this point there is confidence about the impact

of the Project interventions, especially with respect to legal foundations and strategic priorities of the

reforms. As discussed throughout the report, adoption of several decrees, concept papers, State programs

coupled with informal statements have cemented the commitment of the RA Government to the ongoing

reforms. Specific steps have been taken to close or downsize institutions (both child care and special

schools) and reunite thousands of children with their families. As such, there is confidence among




84

stakeholders that deinstitutionalization and inclusive education will be further implemented in Armenia,

in the best interest of the children affected. As overwhelmingly agreed by the evaluation informants,

UNICEF Armenia’s role in this process has been instrumental.

LESSONS LEARNT

Some lessons learnt that can be derived from the result of this evaluation include the following:

• Putting two reforms and directions under on one Project umbrella has complicated the Project

structure and coordination creating pressures with respect to communication with partners and

stakeholders, involving them in advocacy in different areas, undertaking responsibility for ensuring

results in too many directions.

• UNICEF in Armenia’s role as a major international organization involved in policy advocacy on child

rights protection is very important and as much as possible should not be mixed up with the role of

implementer, thus challenging its involvement with national policy makers as a policy leader and a

watchdog.

• A national level public awareness campaign should have been incorporated in the design of the Project

and its standalone components especially given its focus on sensitive reforms touching upon thousands

of lives, including children and their families, as well as requiring involvement and coordination of

different stakeholders from different levels in society.

• Acceptance of a model for reorganization of the institutions should have been more rigorously

pressured and promoted to the Government of Armenia allowing for a specific approach to closing

down the institutions and establishing alternative services across the Project-targeted institutions.

Similarly, community based alternative services should have been established in the communities of

origin of returning children. UNICEF in Armenia could have worked more intensively with the NGOs

present in the regions to involve them in the follow up of the reunified children and their families (as

was the case with SOS villages, which was mobilized at later stages of the Project to support its

implementation throughout its ongoing activities).

RECOMMENDATIONS

Recommendations included in this part of the report build up on the Findings and Conclusions resulting

from this evaluation and are also grouped by relevant evaluation criteria.

Relevance

1. USAID as the Project donor and UNICEF as the lead Project partner should continue their efforts to

bring to fruition the Government’s reforms with respect to child welfare and inclusive education.

While currently the Project serves as the main locomotive in support of the reforms agenda, and as

such is important, UNICEF in Armenia should also prioritize its policy expertise in terms of child

rights protection and use all available channels to build on the current momentum for these reforms,

which was to a large degree achieved due to UNICEF in Armenia’s continuous efforts over the years.

Effectiveness and Efficiency



85

2. If possible, as a tactics moving forward, USAID should consider separating the two reform directions

(child welfare and inclusive education) into two standalone projects while UNICEF’s relevant sections

could take over responsibility for each direction (i.e. Education section taking over Inclusive

Education given its longstanding experience with work in that area)..While the current structure has

been designed to ensure harmonization of the two reforms and their key directions, coordination

across Government partners, implementing partners, and different stakeholders has been challenging.

If the separation is still under the auspices of UNICEF, the harmonization could be furthered.

3. In the future, while undertaking funding of a Project in support of ongoing reforms, USAID should

consider a more structured approach, especially when tying soft interventions to G2G support. One

alterative is starting small and laying the foundations of a bigger project, then expanding it and adding

budget support components once confident about the progress. In case UNICEF undertakes the role

of a Project implementer, it should prioritize pilot projects that showcase the benefits of the specific

approaches of reforms implementation and serve as a model to be replicated by the Government. To

illustrate, with the current child welfare and deinstitutionalization reforms, there seems to be a need

for a reorganization and transformation model and focusing on establishing and making functional

one CFSC could have served as such a pilot. Another alternative is piloting all possible child and family

support services in one region, including CFSCs, RPPSs, strengthening case managers, social workers

and other social services to demonstrate the interconnections between these key institutions and

services and demonstrate how effectively their cooperation could be shaped in support of children in

difficult life circumstances.

4. All Project stakeholders, including USAID as donor, MoLSA and MoES as well as UNICEF should be

aware of any duplications between the roles and responsibilities of different implementing partners

and should define the tasks of partners based on their core competencies. More involvement from the

implementing partners in policy related communication with their respective Government

stakeholders could also result in improved Project coordination.

Sustainability

5. As noted throughout the report, sustainability of the Project efforts is very much dependent on

availability of resources and such factors as availability of community based family support services,

professionally trained specialists, guidelines and manuals in support to service provision and others.

This said, all key players in the reforms, including the Government of Armenia, USAID, UNICEF and

other implementing partners should use available resources and capabilities to ensure these critical

gaps are closed. While each player may utilize different models, methodologies and mechanisms to

ensure further sustainability of the reforms, coordination of efforts will remain a critical priority.

6. In the future, USAID, UNICEF and IPs should consider including a national level public awareness

campaign in similar interventions that support large scale and sensitive reforms. Given UNICEF’s

competencies with respect to designing and implementing such campaigns, it can become one of the

key intervention areas led and managed by UNICEF.

7. Both the Government of Armenia and UNICEF should continue efforts directed to capacity building

of professionals needed for the successful implementation of this reforms both through the Project

and any other channels/initiatives available. In this respect, it is important to identify the key partner

organizations that should be strengthened to support child welfare reforms. While the IE and BoH

experience provides a good example of how the NIE and the Republican PPC became key Project

partners involved in both teacher trainings and capacity building of RPPSC specialists, the role should



86

be taken over by a State agency and not fall to an NGO active in that area. With respect to the child

welfare reforms, MoLSA should not take over this function given its role as a policy setter but the

National Institute of Labour and Social Research could be considered as a possible candidate for this

role.

8. MoLSA and UNICEF should continue using case managers in support of the reforms implementation

and enhancing their professional experience and resources to work further with children reunified

with their families and other children in need of support. Community social workers should also be

brought in to support both child welfare and inclusive education reforms implementation. While a

case manager’s involvement in the ongoing Project and reforms is commendable and necessary, they

are not present in the communities on daily basis and are mostly involved in referrals. Similarly,

regional/Marz administration level departments and their staff are focused on decisions in regards to

placement, documentation processing and other related activities. This creates a gap of specialists on

the community level, and social workers (as evidenced by WV model) could become that important

link in the reforms implementation.

9. Working with parents and children should be an important part of reforms that are touching their

lives and their future. As such, the Project interventions would have benefited from more intensive

engagement with both groups. While this can be partially achieved through a public awareness

campaign, targeted activities are also needed.

And finally, the Government of Armenia (both MoLSA and MoES) should undertake a more robust

leading and coordinating role with respect to all aspects of the reforms, ranging from legislative changes

to capacity building to coordination between different stakeholders including government partners,

donors and implementers. Timely and relevant delivery of all the inputs required for moving forward the

reforms process should be ensured by relevant stakeholders, including adoption of decrees and

commitment of state budget resources (e.g. for foster care or IE) and be responsible and accountable of the

overall reforms outcomes, both positive and negative.

Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,

Family Support and Inclusive Education Services as part of Child Care Reform” project

UNITED NATIONS CHILDREN'S FUND in Armenia

ANNEX 1 TERMS OF REFERENCE

1. Program Information:

Program (Outcome WBS & Name): 0260/AO/05/882. Outcome 2. Disadvantaged families

Project (Output WBS & Name): 0260/AO/05/882/005 Output 2.5. Family Environment

Activity (Activity WBS & Name): 0260/AO/05/882/005/001 Activity: 001 – Child Care System Reform

UNICEF is mandated by the United Nations General Assembly to advocate for the protection of children's

rights, to help meet their basic needs and to expand their opportunities to reach their full potential. The UNICEF

staff and experts/consultants should act in accordance with the UN Code of Conduct and UNICEF Mission.

2. Background, Object and Rationale of the Evaluation:

While Armenia ratified the CRC in 1992, which stipulates the right of the child to live in a family environment,

the country remains significantly reliant on institutional care for children in difficult life circumstances. In 2009,

the Guidelines for Alternative Care of Children welcomed by the UN General Assembly were adopted to

implement care provisions as per the CRC that strongly recommended for children to be brought up in family

environment, and clearly stated that socio-economic status of the family should not be a reason for separating

the child from his/her family.

The overarching goal of the Child Welfare Reform is to ensure that the child care system in Armenia

significantly reduces reliance on large-scale institutions through establishment of a network of community and

family based services responding to the needs of children and their families. UNICEF Armenia, with USAID

support is implementing ”Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family

Support and Inclusive Education Services as part of Child Care Reform” program in partnership with Bridge of

Hope NGO, Children’s Support Center Foundation, World Vision Armenia office and Save the Children. It

supports the Government of Armenia to achieve this goal through the following objectives:

Institutional mechanisms for the functioning of social service system are set up;

Alternative community-based family support services are accessible to vulnerable children and

families;

Family substitution service system is strengthened with the diversification of types of foster care,

establishment of monitoring and evaluation system;

Inclusive education system is strengthened to provide quality learning for all children through set-

up of pedagogical-psychological support centers and mainstream school trainings;




An enabling legal and regulatory framework is established to support child-care reform;

Social norms regarding children with disabilities have changed favorably towards inclusion.

The project is subject to mid-term evaluation in accordance with the AID-111-IO-14-00004 agreement with

USAID. The mid-term evaluation is to be conducted for the period of October 2014 until June 2017. Provided

that the duration of the project is 5 years and the closing date for the Grant Agreement is 09/08/2019, the mid-

term review of the project is due.

Mid-term evaluation will assess the progress in meeting the project goals and outcomes (as defined in the M&E

Plan). They will provide early lessons learnt and recommendations as well as identify significant discrepancies

between expected results and actual achievements, including an analysis behind the reasons for discrepancies

between actual and projected indicators, if any.

The primary users of the evaluation are UNICEF (Country Office, Regional Office, Headquarter divisions),

USAID and Government counterparts (primarily Ministry of Labour and Social Affairs, Ministry of Education

and Science, Ministry of Territorial Administration and Development, regional administrations in target areas)

as well as Implementing partners (“Bridge of Hope” NGO, “Children's Support Centre” Foundation, Fund for

Armenian Relief, Save the Children, World Vision Armenia). Other national and international partners,

primarily those involved in the field of child and social protection in Armenia are also seen as part of the

audience of the report and key stakeholders. Evaluation findings, conclusions and recommendations will be

made available for the use of above-mentioned audience for further consideration of better responding to the

needs of vulnerable children and their families in Armenia. In addition, other UN agencies working in Armenia

may use evaluation findings for their strategic planning and/or review processes.

In sum, the evaluation will inform both the UNICEF and USAID, and other stakeholders on further actions in

the areas of child protection within the context of deinstitutionalization reform.

3. Purpose and Objective

The overall purpose of the Mid-Term Evaluation of the project is to assess the relevance, effectiveness,

efficiency as well as impact and sustainability of interventions under the “Toward Social Inclusion of

Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as part of

Child Care Reform” project.

The specific objectives of the mid-term evaluation are to assess the relevance, effectiveness, efficiency, impact

and sustainability of project interventions in the following directions:




Family and community based alternative services and their role in prevention of institutionalization;

Institutional mechanisms for the functioning of social service system;

The process of strengthening the family substitution service system with the diversification of types of

foster care;

The process of strengthening the inclusive education system to provide quality learning for all children;

Legal and regulatory framework developed to support child-care reform;

Institutional cooperation and coordination between implementing partners/key actors (both government

and donor community) in responding to vulnerable families and children in the frames of the project

(including financial);

Coherence and ordination and compatibility of the congruence between six objectives/components of

the project to meet the overarching goal.

4. The Scope of the Evaluation and Limitations

Given that this is a mid-term evaluation, two of the evaluation criteria this project will be evaluated against,

namely impact and sustainability, are to be considered to the extent possible.

The geographical coverage of the evaluation includes Lori and Syunik Marzes (regions) and capital city

Yerevan. The evaluation will primarily focus on the beneficiaries of various project interventions that will

include beneficiary families, teachers, case managers, local, regional and national authorities.

The main guiding document for the evaluation will be the Project Monitoring and Evaluation Plan, which states

the logic of interventions with clearly set objectives, results, activities and corresponding indicators on output

and outcome levels, means of verification, risks and assumptions. These will be thoroughly reviewed (in terms

of validity and limitations) by the independent evaluation team during the inception phase.

The limitations of the indicators are conditioned by the fact that some of them need to be viewed together with

the identified risks and external conditions, as stipulated in the M&E plan. The M&E plan also includes data

limitations such as absence of common definition of the parameter of “a reform”, program self-reporting being

a subject to reliability and accuracy, etc.

Taking into account that the evaluation is a mid-term and will be followed by the summative evaluation at the

end of the Project, this evaluation needs to ensure the appropriate quality and comparability of baseline data,

including disaggregation by gender, age, income quintile, disability status, etc.

The evaluation will be conducted by a team of independent evaluators (an organization) in close cooperation

and coordination with UNICEF in Armenia, USAID, as well as relevant Ministries and Implementing partners.




5. Evaluation Questions

Core evaluation criteria such as relevance, effectiveness, efficiency, sustainability and impact (the latter - to the

extent possible) will be analyzed. Key evaluation questions will include but are not limited to the following:

Relevance

Are the project activities/components relevant to the actual/defined needs of the vulnerable families

with children? Were the objectives clear and feasible? Do the main components of the project contribute

to the planned objectives and logically interlinked? How well are they linked to each other?

Is the project in line with the current priorities of the country? Is the Government committed (both in

terms of timing and financially) to the project? How the project is aligned with and supports the national

development plans, strategies and national plans of action?

Has the project involved relevant stakeholders through consultative processes or information-sharing

during its preparation phase? Was the needs assessment/analysis carried out at the beginning of the

project reflecting the various needs of different stakeholders? Are these needs still relevant? Have there

any new, more relevant needs emerged that the project should address?

To what extent were the reform initiatives informed by the needs and interests of diverse groups of

stakeholders (also from gender equality perspective)?

What are the beneficiaries’ views about the services provided? What are their views about improving

services (if there is a need to improve)?

Effectiveness

How effective has the project been in establishing ownership by the stakeholders? Can the project

management and implementation considered as participatory. If yes, does this approach contribute to

achievement of the project objectives? Has the project been appropriately responsive to political, legal,

economic, institutional (and other) changes?

Is the project making sufficient progress towards its planned objectives? What are the key achievements,

challenges and implementation lessons? Will the project likely achieve its planned objectives upon

completion and demonstrate success?

Are the strategies appropriate and effective to achieve the planned objectives? What, if any, alternative

strategies would have been more effective in achieving its objectives?

How effectively the family support services are provided in regional (and community) level and how

they are in line with actual needs?




Efficiency

To what extent has UNICEF and implementing partners made good use of the human, financial and

technical resources, and has used an appropriate combination of tools and approaches to pursue the

achievement of project results in a cost-effective manner?

Was there a clear distribution of roles and responsibilities of key actors involved, including staff,

implementing partners and governmental partners towards the achievement of project objectives?

To what extent budgetary allocations by the donor between the partners were efficient? To what extent

did UNICEF and implementing partners capitalize on other complementary initiatives to the project to

reinforce the results of the project?

Has the project done appropriate financial planning and reporting? Has the management of funds (and

reporting) been done in an efficient manner? Has the co-funding/contribution from different project

partners been made at the expected level? Are there established procedures for checks and controls in

the day-to-day financial management, procurement of good and services and decisions regarding

resource allocation for the project?

Have project funds and activities been delivered in a timely manner? If not, what were the bottlenecks

encountered? Were the procedures put in place for staffing, systems (administration, accounting, other)

and reporting efficient?

Sustainability

To what extent have the UNICEF country office and project partners been able to support the

government and beneficiaries in developing capacities and establishing mechanisms to ensure

ownership and the durability of effects under the reform initiatives?

Is there a need to scale down the project (i.e. if the project duration is shorter than planned)? If so, do

project objectives and strategies have to be adjusted?

Has the project successfully built or strengthened an enabling environment (laws, policies, people's

attitudes etc.)?

How does the government intend to maintain the provision of family support services in future? Are

there any specific/envisioned plans/projects?

What are possible sustainability mechanisms for each of the project component? Can the project

approach or results be replicated or scaled up by national partners or other actors? Is this likely to

happen? Can any unintended or unexpected positive or negative effects be observed as a consequence

of the project's interventions?

Impact

Has the project contributed to or is likely to contribute long-term social changes for vulnerable families

and children? What difference has it made, including policy changes? What are the realistic long-term

effects of the project on de-institutionalization of children?




Has the project increased the satisfaction with family and community based alternative services?

The evaluation questions will be refined with the independent evaluation team and finalized during the inception

phase with a goal to explore other aspects, such as coordination between the partners and cooperation.

6. Duty station: Yerevan, Armenia

7. Supervisor:

The evaluation team will be supervised and report to the UNICEF Child Protection (CP) Specialist with regular

de-briefing with UNICEF Deputy Representative, UNICEF Representative and UNICEF M&E and Child

Rights Systems Monitoring Specialist about the progress of the evaluation. The independent evaluation team

will work on daily basis with UNICEF Country Office Child Protection (CP) Unit. The CP team will support

the selected evaluation team in contacting with the government partners and stakeholders during the fieldwork

(if need be).

8. Evaluation Methodology

In this evaluation mixed method approach will be applied by combining qualitative and quantitative components

to ensure complementary strengths and non-overlapping weaknesses. The independent evaluation team will

collect data from desk review and verify them with soft data from field visits, in-depth interviews and focus

groups. The analysis will be built on triangulating information collected from different stakeholders (project

staff, project partners, stakeholders and beneficiaries) through different methods including secondary data and

documentation review and primary data. It should critically examine the information gathered from the various

sources, and synthesize the information in an objective manner. If contradictory information is obtained from

different stakeholders, an effort should be made to understand the reasons for such information, including any

gender-based differences.

The evaluation team will review the following documents before conducting any interviews or field trips: project

documentation, progress reports, work plans, mission reports, monitoring data, workshop reports, minutes,

country data, policies, legal documents, etc.

Quantitative methods will include the conduct of a survey among teachers benefited from the project. In-depth

qualitative interviews will be conducted with implementing government and non-government partners,

including international/donor organisations.

During the report writing phase, the evaluation team should pay close attention to formulating good

recommendations, clearly prioritizing 5-10 recommendations for concrete action. The following are most likely

to lead to good recommendations:




- Key stakeholders are consulted during the development of recommendations, which heightens

programmatic and technical relevance.

- An appropriate sequencing in the implementation of recommendations is noted, especially when one

part of the response is contingent upon a prior action being completed.

- The recommendations clarify where change is needed to solve problems and also where positive aspects

should be continued or enlarged.

- Recommendations are referenced at the point where the finding evidence and analysis is made, to show

the logical connection. The full recommendations can be fully presented in a concluding chapter, but

referencing them within the document is helpful.

- The specific organizations that the recommendation is directed to should be noted, so there is no mistake

about who should respond.

The evaluation approach and data collection and analysis methods should also focus on gender and rights

aspects, be responsive and appropriate for analyzing the gender equality, human rights issues including child

rights identified in the scope. Gender, equity and human rights considerations need to be further elaborated by

the evaluation team during the inception phase and addressed across the final report.

To the extent possible data should be disaggregated by gender, age, minority and socially excluded groups, as

well as encompass urban/rural divide.

8.1 Evaluation approach, data collection methods and instruments

The independent evaluation team will identify key stakeholders/informants (including but not limited to project

implementers, decision makers, direct and indirect beneficiaries, etc.), and appropriate data collection methods

for each informant category (such as semi-structured or in-depth interviews, expert interviews, focus groups).

A combination of these methods should be proposed by the independent evaluation team in the Implementation

Plan and Methodology (Inception Report), and revisited, if necessary, during the fieldwork preparation.

In close cooperation with UNICEF project team, the independent evaluator will also be responsible for the

development of appropriate instruments, including questionnaires, interview and focus group guides,

observation check-lists for each of the methods selected, in Armenian and English. All materials should be

gender-competent in language and presentation, as well as take into consideration human rights and equity

angles, if possible. While there may be overlaps in the topics and items covered for the different informant

categories, the guides/protocols should be customized appropriate to each informant category to be able to

extract the relevant information from each group, and address the key process questions listed earlier.

Interviewers/facilitators involved in this project by the independent evaluator must have relevant qualifications

and be adequately trained/consulted prior to fieldwork (including gender-competency knowledge). Prior to




fieldwork the independent evaluator will be responsible for an interviewer training and pre-testing of the

instruments to be utilized.

8.2 Fieldwork Approach

UNICEF Armenia requires the independent evaluator to propose a fieldwork schedule and approach that will

ensure the accuracy and reliability of information gathered through the effective use of methods, staff, funds

and time. The independent evaluator will develop a preliminary fieldwork plan for coordinating the evaluation

efforts. This plan should focus on the following:

- Draft Schedule of fieldwork activities;

- Draft Schedule and approach to conducting the survey among teachers benefitted from the project;

- Number of interviews/qualitative and quantitative/ in each target group/region/community, criteria for

selecting interview respondents;

- Number of focus group (FG) discussions in each target group/region/community, criteria for selecting

FG participants;

- Approach to the site-visit/interview/focus group protocols and transcripts.

It is envisioned that evaluation fieldwork will cover different stakeholder groups, including decision makers,

those who are implementing and overseeing the implementation and those who are supposed to benefit from it.

9. Major tasks to be accomplished:

The evaluation process will consist of 4 key phases.




Major tasks RESPONS

IBLE

COORDINA

TION

DEADLIN

E

Des

ign

ph

ase

Desk review of reference material

- UNICEF Armenia CP Unit will support the

evaluation team to compile the list of most

important background material, documents, and

reports for review.

- The project M&E plan will be reviewed by the

evaluation team along with other key documents.

Stakeholder mapping

- The evaluation team will prepare a mapping of

stakeholders relevant to the evaluation. The

mapping will include ministries, regional and local

authorities, implementing partners, service

providers, direct and indirect beneficiaries,

development partners.

Implementation plan and methodology

- Evaluation matrix will be developed based on

mixed method design.

- Questionnaires (qualitative and quantitative) will

be developed.

- The set of evaluation questions will be finalized.

- Field work schedule and approach will be

drafted.

Inception Report

- Inception report will be developed and presented

to UNICEF Armenia with all the document/tasks

listed above for this phase embedded.

Independen

t

Evaluation

team

UNICEF

Child

Protection

Section; M&E

Specialist,

MoLSA,

MoES

July/August,

2017

Fie

ld p

hase

Data collection and analysis

- Collection of evaluation data (primary and

secondary) will be carried out through different

techniques, including survey, in-depth, informal

and semi-structured interviews, focus group

discussions and observations.

Independen

t

Evaluation

team

UNICEF

Child

Protection

Section; M&E

Specialist,

August/Sept

ember, 2017




- Survey instrument will be pretested based on the

methodology developed by the evaluation team.

- Data collection report will be submitted covering

the following pieces: documentation of survey

implementation, quality control, survey findings,

survey final instrument, dataset, users’ manual for

dataset, evaluation guides and summary of

protocols.

- The analysis will be based on detailed

protocols/transcripts of interviews, focus groups

and survey results, comparative analysis.

Debriefing

- Debriefing meeting will be organized for

UNICEF Armenia on the preliminary findings,

testing elements of conclusions and tentative

recommendations.

MoLSA,

MoES

Rep

ort

ing p

hase

Evaluation report

- The mid-term evaluation report will be drafted.

- Comments provided by UNICEF Armenia and

other key stakeholders (government, donor and

other development partners) will be addressed.

- Validation meeting will be organized with

partners and stakeholders to present evaluation

findings.

- The Final Mid-Term Evaluation Report will be

submitted.

Independent

Evaluation

team

UNICEF

Child

Protection

Section; M&E

Specialist,

USAID,

MoLSA,

MoES

September/

October,

2017

Dis

sem

ina

tio

n a

nd

Foll

ow

-up

Presentation and Dissemination

- The Evaluation Report presentation will be

organized for government partners and key

stakeholders.

- The mid-term evaluation report will be

disseminated (to stakeholders, development

partners, and Regional Office).

Follow-up

- Preparation of the management response.

Independent

Evaluation

team

UNICEF

UNICEF

Child

Protection

Section;

Management

October,

2017




The evaluation will follow the principles of the UN Evaluation Group’s norms and standards in particular with

regard to independence, objectiveness, impartiality and inclusiveness and will be guided by the UN ethics

guidance as guiding principle to ensure quality of evaluation process, especially apropos conflict of interest,

confidentiality of individual informants, sensitive to beliefs, manners and customs, discrimination and gender

equality, to address issues of vulnerable population, particularly families with children that are disadvantaged

and excluded.1

10. Deliverables:

The evaluation team is expected to produce and submit the following deliverables with detailed description what

each of the deliverable should cover:

- Inception Report (August)

Implementation plan and methodology including but not limited to: a) work plan; b) a stakeholder map; c)

the evaluation matrix/guides (including the final set of evaluation questions listed by domains and

indicators), d) survey instrument (questionnaire) with pre-test methodology, e) the overall evaluation design

with a detailed description of the data collection plan for the field phase;

- Draft Mid-Term Evaluation Report (first week of October)

a. Data collection report, including documentation of survey implementation, quality control, survey

findings (variable aggregation and descriptive report), survey final instrument, dataset, users’ manual for

dataset, evaluation protocols/transcripts.

b. Debriefing document (two-three pages overview) synthesizing the main preliminary findings, conclusions

and recommendations of the evaluation, to be presented and discussed with UNICEF Armenia management

during the debriefing meeting planned at the end of the field work phase.

- Validation workshop with government partners and other stakeholders (mid-October)

Validation workshop brief report to be included into the Final Mid-Term Evaluation report as an annex.

- Final Mid-Term Evaluation Report (end of October)

a. Final report with all the comments addressed. The report with the Executive Summary should be

maximum 50 pages, excluding annexes.

b. Presentation of the mid-term evaluation findings for key stakeholders (Power Point presentation).

The Evaluation Report is proposed to have the following structure, to be reviewed once the contractor is

selected:

1 http://www.uneval.org/document/detail/102




LIST OF ACRONYMS

ACKNOWLEDGEMENTS

EXECUTIVE SUMMARY

CHAPTER I: INTRODUCTION TO THE EVALUATION, OBJECT AND METHODOLOGY

1.1 Introduction

1.2 Object of the Evaluation

1.3 Background of the Action and Context

1.4 Logic of Intervention

CHAPTER II. EVALUATION PURPOSE, OBJECTIVES AND SCOPE

CHAPTER III. EVALUATION METHODOLOGY

3.1. Qualitative Data Collection: Approach and Implementation

3.2. Quantitative Data Collection: Approach and Implementation

3.3. Major Limitations

3.4. Ethical Considerations, Human Rights and Gender

CHAPTER IV. ANALYSES AND FINDINGS

4.1. Relevance

4.2. Effectiveness

4.3. Efficiency

4.4. Sustainability

4.5. Impact

4.6. Cross-cutting topics

CHAPTER V. CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS

5.1. Conclusions

5.2. Lessons Learnt

5.3. Recommendations

ANNEXES

1. Terms of Reference

2. Desk Review and Background Documents

3. List of Respondents by Data Collection Method

4. Detailed Methodology

5. Interview Guides and Survey Instrument




6. Information on Evaluation team

7. Evaluation matrix

8. Results framework

9. Output tables

10. Other documents, if relevant.

All deliverables will be drafted in English. All reports should follow the structure and detailed outlines discussed

and agreed with UNICEF in Armenia. The final report will be translated into Armenian.

11. Time-Frame:

The evaluation is expected to take place during the period of July 2017 – October 2017 with overall 50 days of

level of effort.

The selected institution (independent evaluation team) will work for the period of 50 work days within 3 months.

With consideration of reconstructing suitable basis for evaluation and quantitative survey, the level of effort of

some of the experts, as well as interviewers/field supervisors may well exceed 60 days.

It is envisaged that the evaluation will cover the capital city of Yerevan and Lori, Syunik regions including

number of communities. The exact schedule of the activities will be agreed with the institution (independent

evaluation team) based on the consultancy implementation progress. The deadline for submission of final

deliverables to UNICEF in Armenia is October 31, 2017.

12. Qualifications or specialized knowledge/experience required:

The team will consist of a team leader and three experts.

The team leader will be responsible for managing the evaluation and delivering the final report. The experts/team

members will be responsible, inter alia, for the evaluation design, data collection, analysis of some sections of

the report, implementation and logistics.

Team Leader to act as an evaluation coordinator and to work with UNICEF Armenia:

Required Qualifications:

Advanced university degree and/or academic background in Sociology, Economics, Public Policy

or a related field;

At least 10 years of proven record in managing project/program evaluations in child protection

and/or other social sectors;




Extensive experience in designing evaluations and household surveys, conducting qualitative

analysis and surveys, data analysis and report writing;

Proven very good knowledge of child protection reform processes in Armenia;

Ability to work within the international and multicultural environment;

Very good communication and presentation skills with government and community members;

High analytical and conceptual skills and ability;

Good knowledge of computer applications;

Fluency in written and spoken English.

Experts/team members (up to 3 experts proficient in quantitative and qualitative methods).

Required Qualifications:

Advanced university degree and/or academic background in Sociology, Economics or a related

field;

At least 7 years of experience in program evaluation, particularly in conducting qualitative analysis

in child rights/protection and/or other related fields;

Good knowledge of child protection reforms processes in Armenia.

Experience in data collection and conducting surveys (including household surveys);

Demonstrated ability to prepare interview protocols and working with databases;

Previous experience in evaluation report writing;

Demonstrated ability to work in multicultural teams;

Demonstrated gender competency;

Excellent communication skills;

Fluency in written and spoken Armenian and English.

To facilitate gender sensitive focus group discussions, the independent evaluation team’s gender balance is an

advantage. The team leader’s experience with the United Nations or other development agencies is an asset.

13. Procedures and logistics:

UNICEF does not provide or arrange health insurance coverage for the members of evaluation team.

UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if

work/outputs is incomplete, not delivered or for failure to meet deadlines (fees reduced due to late submission:

1 month - 20%; 2 months - 50%; more than 2 months – payment withhold). All materials developed will

remain the copyright of UNICEF and that UNICEF will be free to adapt and modify them in the future. This

ToR is an integral part of the contract (SSA) signed with the contractor (independent evaluation company).




14. Terms of the application

Interested organizations are requested to express their interest to UNICEF in Armenia within 2 weeks from

the date of the publication of this announcement, specifying:

Implementation Plan and Methodology for the completion of the assigned tasks;

A detailed Work-plan with activities and timeframe;

Detailed Budget for completing the assignment (including travel);

CVs of proposed experts;

Past experience in fulfilling similar tasks and the outcome of at least one presented as an

attachment.

The proposal with all necessary documents should be submitted in a sealed envelope, either in MS Word or

Adobe PDF format in 3 (three) hard copies in English, to the following address: 14 Petros Adamyan str.,

Yerevan, Armenia, UNICEF in Armenia with the following subject: Mid-Term Review of “Toward Social

Inclusion of Vulnerable Children in Armenia” project.

The final selection of the evaluation institution will be conducted by UNICEF in Armenia based on previous

experience in implementing similar assignments; competence in conducting similar activities; evaluation

institution’s overall access to expertise and qualifications required for performing this task and fees proposed.

The overall score for each of the proposals is calculated based on a ratio of 70% - 30% between the technical

and financial proposals.

Maximum Points

Technical Proposal Score 70 points

Financial Proposal Score 30 points

Overall Proposal Score 100 points

Technical proposals received will be evaluated against the set evaluation criteria (total 70 points). The

maximum number of points (30) will be allotted to the lowest financial proposal that is opened and compared

among those invited institutions which obtain the threshold points in the evaluation of the technical

component (49 points). All other financial proposals will receive points in inverse proportion to the lowest

price.




15. Prepared by:

Hayk Khemchyan

Child Protection Specialist Signature and Date:

Lusine Yeremyan

Monitoring and Evaluation, Child Rights Systems Monitoring

Specialist Signature and Date:

16. Reviewed by:

For Operations related

UNICEF Operations Manager Signature and Date:

Gayane Avanesyan

17. Approved by:

OIC for UNICEF Representative in Armenia Signature and Date:

Liv Elin Indreiten

Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project


ANNEX 2 LIST OF DESK REVIEW AND BACKGROUND MATERIALS

ENGLISH SOURCES

1. Amendment number four to assistance agreement between the Government of the United States of America and

Government of the Republic of Armenia for improving the Effectiveness of Health and Social Protection Services

(September 27, 2013)

2. BoH Budget3. BoH Project Final Proposal4. BoH revised Budget to UNICEF for reprogramming (June 05, 2015)5. BoH: Program Proposal Outline for Programme Cooperation Agreement6. BUDGET: SAVE THE CHILDREN ARMENIA. "Expanding Alternative Care, Family Support and Inclusive

Education Services as part of Child Care Reform” project (Revised 26/06/2015)7. Budget_NILSR 2016-20178. CPD 2016-20209. Draft Government decision on approving 2018 Program and Action Plan on Child Rights Protection10. GoA Decision N 831-N, dated as of August 11, 2016 on Reorganization of the “Yerevan Child Care and

Protection Boarding Institution N 2” into “Child and Family Support Center” SNCO11. Handbook on legal regulations12. Joint Workplan UNICEF Armenia and World Vision Armenia13. Legislative reform related to the Convention on the rights of the Child. National case studies: Armenia, Barbados

and Ghana, 200814. Monitoring and Evaluation Plan: Revision1, June 201615. Program Proposal Outline for Programme Cooperation Agreement with Save the Children International NGO16. Project Indicator Definition Table 17. Project Logframe. Project title - ”Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,

Family Support and Inclusive Education Services as part of Child Care Reform”18. Project Proposal: Prepared by UNICEF Armenia in cooperation with WV, SC and BoH 19. Rapid Assessment of Child Care and Protection Institutions under the Ministry of Labor and Social Affairs,

Marina Galstyan, commissioned by UNICEF under USAID funding, Yerevan 201420. Report on local best practices of case management

21. "Toward Social Inclusion of Vulnerable Children in Armenia" Program Quarterly Progress Updates QR 1-11

22. "Toward Social Inclusion of Vulnerable Children in Armenia: Expanding Alternative Care, Family Support and

Inclusive Education Services as Part of Child Care Reform" Monitoring & Evaluation Plan (Revision 1, June

2016)23. "Toward social inclusion of vulnerable children: Expanding alternative care, family support, and inclusive

education services as a part of child-care reform"

https://www.google.am/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0ahUKEwjcgozci53XAhVGJsAKHdVZCB8QFggwMAE&url=https%3A%2F%2Fsearchengines.guru%2Fshowthread.php%3Ft%3D680142&usg=AOvVaw0QVzdCalTzQB3jVczf3C6e








24. UNICEF Armenia’s Request for Proposals to conduct Mid-term Evaluation of the ‘‘Toward Social Inclusion of

Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as part of

Child Care Reform’’ project (LRFP-2017-9132946).

25. UNICEF Child Protection Revised budget 0209

26. UNICEF Child Protection WVA Budget Allocation (Revised 09/10/14)

27. UNICEF In Armenia’s CPDs, annual reports and Project quarterly reports are used as sources

28. UNICEF, BoH, WV quarterly reports are used as a source

29. WV Project Proposal Outline

ARMENIAN SOURCES

1. Դեպք վարողների համար մոնիթորինգի մեթոդական ուղեցույց2. .Դեպքի վարում Մեթոդական ձեռնարկ դեպք վարողների համար3. 2018 Երեխայի իրավունքների պաշտպանության թվականի տարեկան ծրագիր և

, միջոցառումների ցանկ 2017 28- ՀՀ կառավարության թվականի սեպտեմբերի ի նիստի

որոշմամբ

4. Երեխաների պաշտպանության պրակտիկային վերաբերող չափորոշիչները

5. Իրազեկման նյութ՝ խաչվող կետեր․ Ապաինստիտուցիոնալացում և ինտեգրված

ծառայություններ

6. « »Խնամատար ընտանիք ․ Ձեռնարկ դեպք վարողների համար

7. -Խնամատար ընտանիքների և խնամատարության ծրագրի մոնիտորինգի ձեռնարկ

ուղեցույց

8. « »Խնամատար ընտանիքների և խնամատարության ծրագրի մոնիթորինգ ․

-Մեթոդական ձեռնարկ ուղեցույց

9. « »Խնամատար ընտանիք ․ ,Մեթոդական ուղեցույցներ ծնողների համար

, մասնագետների համար Հանրային իրազեկման համար)

10. « ».Խնամատար ընտանիք Մեթոդական ուղեցույց վերապատրաստվողների համար

( Մոդուլներ ծնողների համար

11. « »Խնամատար ընտանիք ․Մ . եթոդական ուղեցույց Հանրային իրազեկման արշավի

կազմակերպման առանձնահատկությունները

12. « ». Խնամատար ընտանիք Մեթոդական ուղեցույց վերապատրաստվողների համար

( )Մոդուլներ ։ « . ,Դեպի երեխաների սոցիալական ներառում այլընտրանքային խնամքի

ընտանեկան աջակցության և ներառական կրթության ծառայությունների

» զարգացումը՝ որպես երեխաների խնամքի բարեփոխում ծրագիր



13. « »Խնամատար ընտանիք ․ Մեթոդական ձեռնարկ վերապատրաստողների համար։

« . , Դեպի երեխաների սոցիալական ներառում այլընտրանքային խնամքի ընտանեկան

աջակցության և ներառական կրթության ծառայությունների զարգացումը՝ որպես

» երեխաների խնամքի բարեփոխում ծրագիր14. « »Խնամատար ընտանիք ․ Մեթոդական ձեռնարկ պոտենցիալ խնամատար ծնողներին

վերապատրաստելու համար։ « .Դեպի երեխաների սոցիալական ներառում

, այլընտրանքային խնամքի ընտանեկան աջակցության և ներառական կրթության

» ծառայությունների զարգացումը՝ որպես երեխաների խնամքի բարեփոխում ծրագիր

15. « »Խնամատար խնամքի սկզբունքներ և չափորոշիչներ ․ -Մեթոդական ձեռնարկ

ուղեցույց

16. -Խնամատար ընտանիքների և խնամատարության ծրագրի մոնիտորինգի ձեռնարկ

ուղեցույց17. Կյանքի դժվարին իրավիճակներում հայտնված երեխաներին խնամատարության

,հանձնելու ընթացակարգի բարելավման հայեցակարգ 2016 .ՀՀ կառավարության թ

10- 9 մարտի ի թիվ արձանագրային որոշմամբ18. (9 , 2004 .)Հայաստանի Հանրապետության Ընտանեկան օրենսգիրք նոյեմբեր թ19. « Հայաստանի Հանրապետության ընտանեկան օրենսգրքում փոփոխություններ և

», « լրացումներ կատարելու մասին Վարչական իրավախախտումների վերաբերյալ

»,Հայաստանի Հանրապետության օրենսգրքում լրացումներ կատարելու մասին

« Հայաստանի Հանրապետության քաղաքացիական դատավարության օրենսգրքում

» « » փոփոխություն կատարելու մասին և Երեխայի իրավունքների մասին Հայաստանի

»Հանրապետության օրենքում փոփոխություններ և լրացում կատարելու մասին

, Հայաստանի Հանրապետության օրենքների նախագծեր https://www.e-draft.am/projects/313

20. Հայաստանի Հանրապետության կառավարության որոշում « Գյումրու Երեխաների

խնամքի և Պաշտպանության N1 » Գիշերօթիկ Հաստատություն Պետական Ոչ

« Առևտրային Կազմակերպությունը Շիրակի Մարզի Երեխայի և Ընտանիքի

» Աջակցության Կենտրոն Հիմնադրամի վերակազմավորելու և գույք տրամադրելու

մասին

21. Հայաստանի Հանրապետության կառավարության որոշում « Դիլիջանի Երեխաների

»,խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն «Բյուրեղավանի

» Երեխաների խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն Պետական Ոչ

Առևտրային Կազմակերպությունները լուծարելու և Հայաստանի Հանրապետության

2007 26- Կառավարության թվականի հուլիսի ի N 890- Ն որոշման մեջ փոփոխություններ

կատարելու մասին





Երեխաների խնամքի և Պաշտպանության Գիշերօթիկ « » Հաստատություն Պետական

Ոչ Առևտրային Կազմակերպությունները լուծարելու և Հայաստանի

2007 26- Հանրապետության Կառավարության թվականի հուլիսի ի N 890- Ն որոշման մեջ

» փոփոխություններ կատարելու մասին Հայաստանի Հանրապետության

Կառավարության որոշման նախագծի ընդունման վերաբերյալ



Երեխաների խնամքի և Պաշտպանության Գիշերօթիկ « » Հաստատություն Պետական

Ոչ Առևտրային Կազմակերպությունները լուծարելու և «Հայաստանի

2007 26- Հանրապետության Կառավարության թվականի հուլիսի ի N 890- Ն որոշման մեջ

» փոփոխություններ կատարելու մասին Հայաստանի Հանրապետության

կառավարության որոշման նախագծի վերաբերյալ ստացված առաջարկությունների

ամփոփաթերթ

24. Հայաստանի Հանրապետության կառավարության որոշում « Կապանի Երեխաների

» խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն Պետական Ոչ Առևտրային

« Կազմակերպությունը Սյունիքի Մարզի Երեխայի և Ընտանիքի Աջակցության

» Կենտրոն Հիմնադրամի վերակազմավորելու և գույք տրամադրելու մասին

25. Հայաստանի Հանրապետության կառավարության որոշում « Վանաձորի Երեխաների

խնամքի և Պաշտպանության N 1 » Գիշերօթիկ Հաստատություն Պետական Ոչ

« Առևտրային Կազմակերպությունը Լոռու Մարզի Երեխայի և Ընտանիքի

» ,Աջակցության Կենտրոն Հիմնադրամի վերակազմավորելու և գույք տրամադրելու

« » Վանաձորի Մանկատուն Պետական Ոչ Առևտրային Կազմակերպությունը լուծարելու

և 2002 28- NՀայաստանի Հանրապետության կառավարության թվականի նոյեմբերի ի

1906-Ն որոշումն ուժը կորցրած ճանաչելու մասին

26. . . Մոդուլ Սուպերվիզիան սոցիալական պաշտպանության ոլորտում նպատակներ և

գործառույթներ

27. ( 2016)Շտեմարան Երևան

28. Սոցիալական դեպքի վարում․ Մեթոդական ձեռնարկ մասնագետների համար



29. Սոցիալական Ծառայությունների Արձանագրային նախագիծ Հայաստանի

Հանրապետության Լոռու Մարզի Երեխաների Շուրջօրյա Հաստատությունների

վերակազմավորման վերաբերյալ ծրագրին և ծրագրի իրականացման միջոցառումների

ժամանակացույցին հավանություն տալու մասին։

ONLINE SOURCES

1. http://www.unicef.am/en/articles/aboutunicef 2. https://armenia.savethechildren.net/sites/armenia.savethechildren.net/files/library/book-eng.pdf 3. http://armstat.am/file/article/poverty_2016a_2.pdf 4. http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=45. http://www.uneval.org/normsandstandards/index.jsp 6. http://www.unEvaluation.org/ethicalguidelines

http://www.unEvaluation.org/ethicalguidelines

http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4

http://armstat.am/file/article/poverty_2016a_2.pdf

https://armenia.savethechildren.net/sites/armenia.savethechildren.net/files/library/book-eng.pdf

http://www.unicef.am/en/articles/aboutunicef



ANNEX 3. LIST OF REPONDENTS BY DATA COLLECTION METHOD

List of Experts InterviewedName Position Method

1. Robert Stepanyan Head of Development Programs and Monitoring Department of the Ministry of Education and Science of the RA

Expert interview

2. Anahit Muradyan Chief Specialist of the General Education Department of the Ministry of Education and Science of the RA

Expert interview

3. Anna Safaryan Assistant to the Minister, Ministry of Labor and Social Affairs RA

Expert interview

4. Lena Hayrapetyan Head of Child Rights Protection Unit, Ministry of Labor and Social Affairs RA

Expert interview

5. Armen Abrahamyan Director of Nork Center, Ministry of Labor and Social Affairs RA

Expert interview

6. Varduhi Katinyan Open Government Partnership Expert/Center for Pedagogical Psychological- Assessment ofYerevan/ Republican Pedagogical and Psychological Support Center

Expert interview

7. Anahit Bakhshyan Former Deputy Director of NIE MOES of the RA CJSC, inclusive education specialist

Expert interview

8. Eduard Israyelyan Head of Child Rights Protection Division at Human Rights Defender's Office of the Republic of Armenia

Expert interview

9. Marine Aghajanyan Director, Yerevan Child and Family Support Center Foundation, under the RA Ministry of Labour and Social Affairs

Expert interview

10. David Tumasyan Legal Expert, Head of Legal and Statistics Department, Investigative Committee of RA

Expert interview

11. Armen Hovsepyan Head of General Education Management and Content of Norms Development Division of NIE MOES of the RA

Expert interview

12. Gnel Tigranyan NIE regional structures in Lori Director, Ministry of Education and Science, National Institute for Education

Expert interview

13. Magda Gevorgyan, Marine Mirzoyan

NIE regional structures in Kapan Director, Coordinator, Ministry of Education and Science, National Institute for Education

Expert interview

14. Meline Grigoryan Inclusive Education Expert Expert interview

15. Susanna Tadevosyan, Hripsime Nazaretyan

BoH DirectorProject coordinator, Bridge of Hope

Expert interview

16. Grigori Grigoryants Child Protection Expert, Save the Children Expert interview17. Mira Antonyan Director of FAR Child Support Center Expert interview18. Voskan Ghazaryan Project Coordinator, World Vision Armenia Expert interview19. Hayk Khemchyan Child Protection Officer at UNICEF. Expert interview20. Hasmik Arakelyan Program Officer at U.S. Fund for UNICEF Expert interview

https://www.facebook.com/pages/US-Fund-for-UNICEF/108188819203296?timeline_context_item_type=intro_card_work&timeline_context_item_source=1793010174



Armenia21. Artak Shakaryan Child Protection Officer at UNICEF Armenia Expert interview22. Lusine Yeremyan M&E Officer at UNICEF Armenia Expert interview23. Liv Elin Indreiten Deputy Representative, UNICEF Armenia Expert interview24. Ani Manukyan USAID/Armenia Project Management

SpecialistExpert interview

25. Aida Muradyan World Vision, Child Protective Specialist Expert interview

List of Key Informants InterviewedName Position Method

1. Maga Ter- Hovhannisyan Social worker, Trainer, FAR Child Support Center

Individual interview

2. Anahit Sahakyan Lecturer, Trainer, Yerevan State University Individual Interview3. Anahit Galstyan Yerevan, No 160 School Principal Individual interview4. Anush Abelyan Yerevan, No 100 School Principal Individual Interview5. 2 biological family parents, Yerevan Parents of a former Noubarashen child

resident Individual interview

6. Erjanik Sargsyan, Hayk Harutyunyan

Department of Education, Culture and Sport of the Syunik region of the RA

Individual Interview

7. Anna Voskanyan Head of the Family, Women and Children’s Division ,Syunik Marzpetaran

Individual interview/Small group interview

8. Rouzanna Hovhannisyan Director ,Kapan Child Care and Protection Boarding Institution SNCO, under the RA Ministry of Labour and Social Affairs (to be reorganized into Syunik Child and Family Support Center Foundation)


9. Artour Hakobyan Director of the Kapan Territorial EmploymentCenter of State Employment Agency under the Ministry of Labour and Social Affairs

Individual Interview/Small group interview

10. Araik Hayrapetyan Director of the Kapan Territorial EmploymentCenter of State Employment Agency under the Ministry of Labour and Social Affairs

Individual Interview/Small Group Interview

11. Hermine Shalunc Director of Goris Pedagogical and Psychological Support Center


12. Hermine Davtyan Director of Sisian Pedagogical and Psychological Support Center


13. Liana Grigoryan Ofelya Hovakimyan

Director of Kapan Pedagogical and Psychological Support Center, Deputy Director


14. Karmen Karapetyan Syunik City School, Goris School No.3 School Principal


https://www.facebook.com/UnicefArmenia/?timeline_context_item_type=intro_card_work&timeline_context_item_source=529897924



15. Anahit Petrosyan Syunik Rural School, Angeghakot School Principal


16. Anna Parsadanyan Mother of Arustamyan Mariam, Kapan, Syunik Marz


17. Susanna Hovhannisyan Mother of Hovhannisyan Alvina, Kapan, Syunik Marz


18. Gayane Abelyan Mother of Abelyan Gor, Kapan, Syunik Marz Individual Interview

19. Shaghat School Mother of Hayk Stepanyan, Shaghat, Syunik Marz /an inclusive child/


20. Narek Sargsyan Head of Staff of Lori Marzpetaran Individual Interview

21. Sasun Harutyunyan Director of Former Special school of Vanadzor, Vanadzor Pedagogical and Psychological Support Center


22. Hasmik Khachatryan Director of Former Special school of Spitak, Spitak Pedagogical and Psychological Support Center


23. Martirosyan Lori, City School, Deputy Director of School No. 16 in Vanadzor


24. Abazyan Lori, Rural School Lernapat Secondary School Principal


25. Marine Yarmaloyan Lori, City School, Spitak No 5 School Principal


26. Hasmik Nerkararyan Former Director,Vanadzor Child Care and Protection BoardingInstitution SNCO, under the RA Ministry of Labour and Social Affairs (to be reorganized into Lori Child and Family Support Center Foundation)

Individual Interviews/Small GroupInterview

27. Mariam Aperyan Acting Director, Vanadzor Child Care and Protection BoardingInstitution SNCO, under the RA Ministry of Labour and Social Affairs (to be reorganized into Lori Child and Family Support Center Foundation)

Individual interviews/Small group interview



28. Lernuhi Gevorgyan Secretary of Vanadzor Guardianship and Trusteeship Commission, Vandzor Mayor’s Office

Individual interviews

29. Anahit Poghosyan Qnqush Harutyunyan

Vanadzor Social Support Regional Service Individual interviews

30. 4 Candidate Foster Families and 1 experienced foster family

Vanadzor, Lori Marz Individual interviews

31. Ruzanna Yazeryan Case manager, Yerevan, Shengavit SSTD


32.Ruzanna Karapetyan Case manager, Yerevan, Shengavit SSTD


33.Garik Galstyan Case manager, Lori, Vanadzor SSTD Individual interviews

34.Qristine Alaberqyan Case manager, Lori, Vanadzor SSTD Individual interviews

35.Ruzanna Khachartyan Case manager, Syunik, Kapan SSTD Individual interviews

36.Gohar Zaqaryan Case manager, Syunik, Meghri SSTD Individual interviews

List of Focus Group Discussions Conducted Institution Participants Method

1. Yerevan Child and Family Support Center Foundation, Yerevan

Social Workers – 6 respondents all female FGD

2. Yerevan, Ararat and Armavir Marzes 6 Candidate Foster Families FGD3. Kapan No 5 School Parents -10 FGD4. Kapan Local Psychological

Pedagogical Support CenterStaff – 6 FGD

5. Goris No 3 School Teachers – 9 FGD6. Goris No 3 School Parents –4 FGD7. Angeghakot School Teachers – 11 FGD8. Kapan Child Care and Protection

Boarding Institution SNCO, under theRA Ministry of Labour and Social Affairs

Social workers, social educators and psychologists – 7 respondents 6 female, 1 male

FGD

9. Vanadzor No 16 School Parents-9 FGD



10. Vanadzor No 16 School Teachers– 6 FGD11. Spitak former special school, current

Local Psychological Pedagogical Support Center

Staff – 6 FGD

12. Lernapat School Teachers – 13 FGD13. Lernapat School Parents – 4 FGD14. Spitak No 5 School Teachers – 7 FGD

Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project


ANNEX 4: SURVEY INSTRUMENT & INTERVIEW GUIDES

Quantitative questionnaire for interviews with teachers

Questionnaire N _________

Dear teacher,VISTAA Expert Center is conducting a survey commissioned by UNICEF in Armenia for the purposes ofevaluating introduction of inclusive education in Armenia, particularly in the Syunik region.Your participation in the survey is voluntary. However, with your candid answers you can greatly contribute notonly to the implementation of the survey but also the implementation of system reforms in the country in thecorrect direction. Your name will not be mentioned anywhere, your answers will not be presented individuallybut will be analysed aggregately along with the responses given by all participants. Thank you for your cooperation

Identify the answers to the questions below in advance and fill out the following for all interviews carried out at thegiven school:

Number of children in inclusive education __________________________

Number of children from special schools registered at the given school _______________________________

Number of children from special schools enrolled in the given school _______________________________

Section 1. General education

A. Overall awareness

1. Are there children enrolled in the inclusive education system at your school?

1. Yes

2. No

98. Difficult to say

2. What is inclusive education? It is a system of organizing education, according to which:

1. …children with special education needs study in a separate institution

2. Children with special education needs study at general education schools together with their peers while receiving additional services

3. Children with special education needs study at general education schools in separate classrooms




3. I will read out some options for you to select if these children are considered children with special educationneeds.

3.1. Who do you think should be included? 3. Considered

as such3.1. Should be included

1. Children of socially insecure families 1. Yes 2. No 1. Yes 2. No2. Children whose parents work and are unable to tend to the education of their child/children

1. Yes 2. No 1. Yes 2. No

3. Children with physical impairments 1. Yes 2. No 1. Yes 2. No4. Children with visual impairments 1. Yes 2. No 1. Yes 2. No5. Children with speech and language impairments 1. Yes 2. No 1. Yes 2. No6. Children with hearing impairments 1. Yes 2. No 1. Yes 2. No7. Children with mental health issues 1. Yes 2. No 1. Yes 2. No8. Children without parents 1. Yes 2. No 1. Yes 2. No9. Children from large families 1. Yes 2. No 1. Yes 2. No10. Children with mental disorders 1. Yes 2. No 1. Yes 2. No11. Children who are deaf and speech-impaired 1. Yes 2. No 1. Yes 2. No

12. Blind children

13. Other /please note/ 1. Yes 2. No 1. Yes 2. No98. Difficult to say

4. Are the following statements accurate?

Yes No Difficult to say

The medical model of disability is at the basis of inclusive education

1 2 3

Every teacher must have the skills for child observation andassessing learning capabilities of the student.

1 2 3

Teachers of respective subjects do not take part in thedevelopment and implementation of individual education plans.Only the parents, the class teacher, the special teacher and thepsychologist are involved in the process.

1 2 3

The individual education plan is developed for 1 school year orin response to a short-term issue.

1 2 3



B. Attitude

5. Do you think Armenia is ready for the introduction of the inclusive education system?

1. Yes, absolutely

2. To some extent yes

3. To a certain extent, no

4. Not ready at all


6.On a scale of 1 to 5 (where 1 is the lowest score and 5 is the highest) please rate thereadiness for the introduction of inclusive education of:

1 2 3 4 5

The members of your community, parents (attitude) 1 2 3 4 5

Your school (the building and other amenities) 1 2 3 4 5

Your colleagues (attitude, knowledge, skills) 1 2 3 4 5

7. To what extent do you personally accept the idea of introducing the inclusive education system?1. I absolutely accept it2. I accept it to a certain degree 3. I do not accept it to a certain degree4. I do not accept it at all98. Difficult to say

7.1. Why do you think so?

__________________________________________________________________________________________________________________________________________________________



8. Please note whether the introduction of the inclusive education system has a positive or a negative role forthe following groups/institutions.

Absolutely negative

Som

ewhat negative

Neither

positive,

nor

negative

Som

ewhat positive

Absolutely positive

Difficult to say

1. Children with physical impairments 1 2 3 4 5 98

2. Children with mild mental impairments 1 2 3 4 5 983. Children with severe mental impairments 1 2 3 4 5 984. Children of socially insecure families 1 2 3 4 5 98

5. Children excluded from the inclusive educationsystem

1 2 3 4 5 98

6. Parents of children excluded from the inclusiveeducation system

1 2 3 4 5 98

7. Parents of children enrolled in the inclusiveeducation system

1 2 3 4 5 98

8. Teachers 1 2 3 4 5 989. School administration 1 2 3 4 5 9810. Community 1 2 3 4 5 9811.Society/country 1 2 3 4 5 9812. Stakeholders interested in the introduction of

the system /ministry of education, NationalInstitute of Education, other…/

1 2 3 4 5 98

9. What do your colleagues think of the children in inclusive education? They say that … (do not read out theoptions. Only up to 3 options can be accepted)

1. They are no different from other children2. They disturb the class3. Children in inclusive education pay more attention to classes4. It is impossible to control them5. Working with them is a pleasure



6. Working with them is very difficult7. It would be better to have relevant specialists working with them8. They make significant progress9. It would be better if they attended relevant educational institutions10. It is very good that they study together with all other children11. Other __________________________________

10. What does a teacher need for working more effectively in the inclusive education system? (Do not read outoptions but note only 1).

1. Extra payment, salary

2. Didactic materials

3. Verbal encouragement

4. Promotion

5. Other _____________________________________


C. Practice

11.Does your school have children from special schools?

1. Yes, if yes, how many? ___________________________

2. No


12. Are there children enrolled in the inclusive education system in the classes you teach? 1. Yes 2. No /Move on to Section 2/98. Difficult to say /Move on to Section 2/

13. I will read out a number of observations. Please state which one you agree with and which one you do not.

Absolutel

y agree

Som

ewha

t agree

Som

ewha

t disagree

Do

not

agree

atall

Difficult

to say



I spend most of my time in the classroom working with thechild/children in the inclusive education system than with others

1 2 3 4 98

I work with children in the inclusive education system outside themain school hours as well

1 2 3 4 98

I prepare for the work with the children in the inclusiveeducation system for much longer than for working with theother children.

1 2 3 4 98

14. What is the attitude of the classmates towards children in the inclusive education system?

1. Very good

2. Good

3. Neutral

4. Not so good

5. Poor


15. How often have you had problems/difficulties in dealing with children in the inclusive education system?1. Very often2. Often3. Sometimes4. Rarely5. Never 98. Difficult to say

16. How do you decide where the child from the inclusive education system sits in the classroom?1. The needs of the child are taken into account

2. The needs of the class are taken into account3. The child sits randomly. Nothing special is taken into account98. Difficult to say

17. Whom do the children in the inclusive education system mostly interact with in the classroom/school? /Noteup to 3 answers/

1. All children in classroom2. Only with one another3. With children of neighbours’ attending the same school4. With relatives attending the same school5. With children who are nice to them6. With no one7. _______________________8. _______________________



18. Do you cooperate with your colleagues while working with children in inclusive education?1. Yes, always2. Sometimes yes3. Mostly no /Move on to Q 20/4. No, never /Move on to Q 20/

19. How often do you cooperate with regards to the issues of children in the inclusive education system?V

eryoften

Often

Som

etim

es

Rarely

Never

With the teacher assistant or members of the multidisciplinary team 1 2 3 4 5

With employees of the pedagogical and psychological support center 1 2 3 4 5

With the representatives of the National Institute of Education 1 2 3 4 5

With the parents of children in the inclusive education system 1 2 3 4 5

With the parents of other children 1 2 3 4 5

20. Do you think your work has affected the following…?

Yes Somewhat yes

Somewhat no

No Difficultto say

The attitude of children towards those in the inclusive educationsystem has changed

1 2 3 4 98

The attitude of children in the inclusive education system haschanged towards other children

1 2 3 4 98

The behaviour of children in the inclusive education system haschanged

1 2 3 4 98

The performance of children in the inclusive education systemhas changed

1 2 3 4 98

Section 2. Relevance and efficiency of classes

21. Have you participated in a training course on inclusive education in the last 2 years?1. Yes2. No /Move on to Section 4/



22. When and where was the course? How long was it and who was it organized by?

N When/year Where1. At the school2. At a differentschool in theprovince3. Local NIEoffice4. Local NIEoffice

Duration/howmany days

Organized by1. Administration of the given school2. Members of the multidisciplinary team3. Staff of NIE Yerevan office4. NIE regional office staff5. Bridge of Hope NGO6. Staff of national pedagogical and psychologicalsupport center7. Staff of local pedagogical and psychologicalsupport center8. Other

1

2

3

23. Did you receive a certificate upon completion of the training course implemented by your schoolrepresentative and the NIE staff?

1. Yes

2. No

24. Did you find the trainings useful? 1. Absolutely 2. Somewhat useful 3. Not so useful 4. Not useful at all

25. How was your attendance in the trainings? 1. I did not miss a training2. I attended more than half 3. I attended half4. I attended less than half of the trainings 5. I barely attended the trainings

26. How many of your school’s teachers do you think still need trainings on inclusive education?1. All teachers2. More than half of teachers3. Half of teachers4. Fewer than half of teachers5. Several teachers6. No one /Move on to Q28/7. Difficult to say

27. /If in answer to Q 26 it was mentioned that there were teachers who need additional training, clarify what type ofschool teachers are they?

ALL following questions are about the training courses organized by Bridge of Hope NGO, whichwere implemented by the school director and the NIE employee.



1. Elementary school teachers 2. Middle school teachers3. High school teachers4. Other /please note / ____________________________

28. Do you personally apply the knowledge you gained during trainings in class? 1. Yes, always2. Mostly yes3. Mostly no4. No, never

29. One a scale of 1 to 5 (with 1 being the lowest and 5 being the highest score) please rate the following for theimplemented trainings:

1 2 3 4 5… effectiveness 1 2 3 4 5…usefulness to you 1 2 3 4 5…adequate organization (venue, period, duration of training, hospitality, etc.) 1 2 3 4 5

…quality of provision of materials (was it done in a comprehensible fashion in clearand understandable language? Were interactive techniques used, etc.?)

1 2 3 4 5

…contents of materials (were all questions discussed? Did the material includenovelty, etc.?)

1 2 3 4 5

Section 3. Availability of auxiliary resources

30. Do you conduct a needs assessment of students?1. Yes2. No /Move on to Q32/

31. If yes, then whom do you cooperate with for this purpose?1. Relevant specialists from the school2. Local NIE staff3. Staff of local pedagogical and psychological support center4. Other _______________________________________

32. Do representatives of the pedagogical and psychological support center work with your school?1. Yes2. No /Move on to Section 5/98. Difficult to say /Move on to Section 5/

33. If yes, then please rate how effective the cooperation was on a scale from 1 to 5.

1 2 3 4 5 98



34. What issues do they cooperate with the school on /note all possible answers/?1. They carry out a needs assessment for the child2. They provide consultations on issues of children in the inclusive education system3. They develop individual education plans4. They develop a support plan for the child5. They work with children in the inclusive education system6. They don’t do anything7. Other /please note/_____________________________________________98. Difficult to say

35. Does the introduced assessment system allow the objective assessment and identification of children withissues?

1. Yes, absolutely2. Mostly yes3. Mostly no4. Not at all98. Difficult to say

36. How the efficiency of cooperation between the center and the school be improved?

__________________________________________________________________________________________________________________________________________________________________________

Section 4. Social, demographic, and other characteristics of respondent

37. Age _________________________

38. Gender1. Male2. Female

39. You are a …1. Teacher

2. Teacher’s assistant3. Social pedagogue4. Speech therapist 5. Psychologist6. Other ________________________________

40. What subject do you teach? ____________________

1. I do not teach



41. How long have you been working in the education sector? ______________________________

42. How long have you been working at the given school? ________________

43. Have you ever worked at a special education school? 1. Yes 2. No

THANK YOU

Telephone: _____________________________________________________________

(When asking for the telephone number, notify them that it is only for the verification of the interviewer’s work and formaking further clarifications later, which does not contradict with the requirements of anonymity.)

Is there anything you would like to add? _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Fill out after finishing the interview

Area of residence 1. Urban 2. Rural

Community ____________________________________________

School number ________________________________________________

Interviewer __________________________

Date entry by_______________________



Questionnaire for in-depth interviews with school principals

Establishing contact Presenting survey objectives, principles of data summary and anonymity

Relevance

A1. Perceptions on the necessity for the introduction of the inclusive education system

1. How long has your school been involved in the process of introducing inclusiveeducation?

2. How do you generally view the idea of inclusive education? Why do you thinkso?

3. How effective is the introduction process of that system? Why do you think so?Do you think the Armenian society, the parents, the teachers and the children areready for inclusive education?

4. Is the introduction of inclusive education in the best interests of the children, thesociety, the parents, the teachers, the government and the implementers? Why doyou think so?

5. Do you think inclusive education reflects the needs of such children?6. Overall, how would you describe children enrolled in inclusive education? How

are they different from other students?7. Based on the assessment system, children with different needs have been

enrolled. How are they different from one another? How different are the needsof girls and boys?

8. How many children of each group are there at your school? How many girls andhow many boys? What are their peculiarities /needs and so on/?

9. How would you describe the process of introduction of inclusive education inArmenia? What are the obstacles? What are contributing factors?

A2. Participation in trainings on inclusive education

10. Have you participated in trainings on inclusive education? When and where andhow long? How would you rate those trainings overall?

11. Rate the trainings according to:a) The need/relevance of the topic b) Contentsc) Preparedness of training specialistsd) Duration (appropriate or not)e) efficiency and applicability, correspondence to Armenian reality

12. What changes have occurred as a result of those trainings?13. Have you applied the knowledge you gained from those trainings? How?



14. Have you passed the knowledge gained from the training on to your colleagues?How was the training of other teachers done?

15. What was the time ratio for training and for being trained? Do you think that theassigned time /30 hours/ is enough for passing on the knowledge gain in thesame time period?

16. What would you suggest to improve the effectiveness of the process oftransferring the knowledge /changing the way of organizing the courses,assigning higher roles to pedagogical and psychological centers …/.

Effectiveness

B1. Transfer of knowledge on inclusive education to teachers

Organization of training courses on inclusive education17. How did you get ready for the training of other teachers? Did you prepare

beforehand or you just passed on the knowledge you had gained from thecourse?

18. How long was the training that you had organized? What were the objectives andthe outcomes?

19. What sorts of problems did you have to face in your work with teachers aimed atthe introduction of inclusive education?

20. What sorts of changes have you noticed as a result of your work in the attitudesand work styles of teachers?

21. How do you as a principal take part in and control the process of introducinginclusive education at your school?

22. How well did the teachers of your school comprehend the idea of inclusiveness?To what extent do they use or display an inclusive approach?

23. How do other individuals, organizations, institutions participate in theintroduction of the inclusive education system? How willing are they? Howinterested are they to do so?

Availability of supporting resources; establishment of pedagogical andpsychological centers

24. Are there special education schools in your community? (If it did exist and wasturned into a pedagogical and psychological center, then) do you thinkpedagogical and psychological schools successfully replace special schools?Why do you think so?

25. How do you collaborate with the center? Please describe the procedure. Whatactivities do they do and how frequently do you meet?

26. What should be done for increasing the effectiveness of cooperation between thecenter and the school?

B2. Perceptions of effectiveness of work with children in the inclusive education system

27. How does inclusive education help children enrolled in the system? Does it havea positive or a negative impact on them? Is there a difference for boys and girls?Why do you think so?

28. What kind of changes have you noticed since the school has become inclusive?Is there a difference in terms of girls and boys? Why?

29. What group of children enrolled in the inclusive education system is the lattermore effective? Why do you think so? Are the boys more easily adapted or thegirls?

30. What would help the introduction of inclusive education be more effective?Efficiency

C1. Assessment of conditions

31. Has the school been adapted for all children /ramp, toilets, classrooms/? Istransportation provided? Does the food provided at school help enhanceinclusiveness?

32. Does the school have the required professional staff?



required for working withchildren in the inclusive

education system

33. What do you need to have the children in the inclusive education system fullyengaged? Is there a difference between the needs of boys and girls?

34. What do the children need that is not provided and the lack of which hinderstheir inclusion?

35. What sort of support is needed to increase the efficiency of the inclusiveeducation system?

Sustainability

D1. Perceptions of sustainability of the project

36. In your opinion, how effective is the introduction of inclusive education deemedby parents of schoolchildren and parents of children in inclusive education?What kind of feedback do you get?

37. Are the children ready for inclusiveness? What about the parents? Why do youthink so? Are the boys more easily adapted or the girls?

38. How do you personally rate the impact of activities you carried out? How arethey perceived by the parents?

39. How do you think the introduction of inclusive education generally impactscommunities?

40. Does inclusive education replace special education schools whilst filling theirnegative gaps? What sorts of problems does it solve?

41. How would you describe the impact of inclusive education in your owncommunity? What visible changes are there?

42. What should be changed in the inclusive education system?43. Do you think the current Armenian schools and the society are ready for the

introduction of inclusive education? Why?44. What should be done to have a smooth introduction of inclusive education

accepted by all?



Questionnaire for the focus group discussion with teachers

Establishing contact Introduction of the objectives if the survey, principles of anonymity and data summary by the moderatorSelf-introduction of participants (profession, subject taught and position, number ofyears worked at the school, previous work experience with children in the inclusive education system)

Relevance

A1. Perceptions on the need for the introduction of the inclusive education system

How do you generally view the idea of inclusive education? Why do you believe so?How efficient is the process of introduction of that system in Armenia? Why do you think so? Do you think the Armenian society, the parents, the teachers and the children are ready for inclusive education?Is the introduction of inclusive education in the best interests of the children, the society, the parents, the teachers, the government and the implementers? Why do you think so?How would you describe children enrolled in inclusive education? How are they different from other students? How do the needs of boys and girls differ?Based on the assessment system, children with different needs have been enrolled. How are they different from one another? How different are the needs of girls and boys?How many children of each group are there at your school? How many girls and how many boys? What are their peculiarities /needs and so on/?

A2. Participation in trainings on inclusive education

Have you participated in trainings on inclusive education? When and where and how long? How would you rate those trainings overall?

Rate the trainings according to: a) The need/relevance of the topic b) Contentsc) Preparedness of training specialistsd) Duration (appropriate or not)e) efficiency and applicability, correspondence to Armenian reality

Did the training course meet your needs? Are there teachers who, according to you, needed the training more than others did? If yes, then who? Why do you think so?What changes have occurred as a result of those trainings?Have you applied the knowledge you gained from those trainings?Have you passed the knowledge gained from the training on to your colleagues? How?Do you think the knowledge and skills you have now are enough for applying an efficient inclusive approach? If no, then why? What kind of additional knowledge and skills do you think you still need?Were monitoring visits paid to these trainings? If yes, then by whom? How did that go? What would you recommend to increase the efficiency of the training courses?

Effectiveness

Observation of children in the inclusive education system and identification of their learning abilitiesDo you have individual education plans for working with the children? How realistic are those plans?When you first walk into a classroom, do you give or receive more information /observing, meeting/?How do you identify the children that need to be enrolled in the inclusive education system?Describe how you conduct a needs assessment. How do you decide which method you should use for working with children?



B1. The process of working with children enrolled in the inclusive education system

Availability of supporting resources; establishment of pedagogical and psychological centersAre there special education schools in your community? (If it did exist and was turned into a pedagogical and psychological center, then) do you think pedagogical and psychological schools successfully replace special schools? Why do you think so?How do you collaborate with the center? Please describe the procedure. What activities do they do and how frequently do you meet?Does the center work with children with inclusive education, their families and teachers?How do the children and their families see the role of the center? What is the attitude of the community towards the activities of the center?What should be done for increasing the efficiency of cooperation between the school andthe center?

Development and implementation of individual plans for the studentsHow do you develop the individual education plan? Who participates in the process? How is that carried out?Is the plan accepted by the parents? How is the implementation of the individual student plan carried out?How do other people, organizations and institutions contribute to the process of providing support to the child? How willing are they to do so? And how motivated are they for doing so?

B2. Perceptions of effectiveness of work withchildren in the inclusive education system

To what extent does inclusive education help children? Does it have a positive or a negative impact on them? Why do you think so?When you started working in the class that had a child from the inclusive education system, what kind of problems were you faced with? Are the issues of girls different from those of the boys? How do you deal with various problems? What are the problems that remain without solutions?What kind of changes have you noticed in your work?What group of children in inclusive education benefits the most from your work? Is therea difference in terms of boys and girls? Why do you think so? What would help to make the “education for all” and “education for everyone” approaches more effective?

Efficiency

C1. Assessment of conditions required for working with children in the inclusive education system

Do you think you have the knowledge needed to work with children in the inclusive education system?Do you have relevant the relevant technical means for working with the children?Has the school been adapted for all children /ramp, toilets, classrooms/? Is transportation provided? Does the food served at school contribute to the enhancement of inclusiveness?Does the school have the relevant professional staff?What do you see is lacking for the full engagement of children in the inclusive education system? Is there a difference between the needs of boys and girls?What sort of support is needed to have the performance of the inclusive education systembe more effective?

Sustainability

In your opinion, how effective is the introduction of inclusive education deemed by parents of schoolchildren and parents of children in inclusive education? What kind of feedback do you get?Are the children ready for inclusiveness? What about the parents? Why do you think so?




Are the boys more easily adapted or the girls?How do you personally rate the impact of activities you carried out? How are they perceived by the parents?How do you think the introduction of inclusive education generally impacts communities?Does inclusive education replace special education schools whilst filling their negative gaps? What sorts of problems does it solve?How would you describe the impact of inclusive education in your own community? What visible changes are there?What should be changed in the inclusive education system?Do you think the current Armenian schools and the society are ready for the introduction of inclusive education? Why?What should be done to have a smooth introduction of inclusive education accepted by all?



Questions for the expert interviews (generalized questionnaire)

Deinstitutionalization, introduction of inclusive education and all related functions

1. Please tell us a little about the ongoing reforms

2. How are they conducted?

3. How does your organization take part in the processes? What international organizations do you cooperate withduring the implementation of those activities? How do you see your cooperation with other organizations?

4. How effective are the ongoing reforms? What are their strengths and weaknesses?

5. How are the ongoing reforms related to state policy, priorities and the interests of children?

6. Are the actions undertaken by UNICEF coordinated and brought in line with the reforms carried out by thegovernment? Why? Why not?

7. What are the main directions and stages of reforms since the launch of the project? What activities are foreseenfor the coming years?

8. What are the challenges during the implementation of the reforms?

Legal

Financial

Organizational

Content

9. What are the contributing and hindering factors for the reforms?

10. How do various organizations i.e. schools, governor’s office, special education schools and others cooperate withone another?

11. Two types of integration is done: with the school and with the family. How are these processes going? How doesthe government respond to challenges arising on the way?

12. How do you see the future of the reforms without the financial, methodological and other forms of supportprovided by UNICEF and other donors? What will the government do? How do you see the sustainability of thereforms?

13. What are the lessons learned?

14. What would you change in the implementation of the project?

15. Do you know about the feedback from beneficiaries? How is it?



Questionnaire for in-depth interviews with family members

/Biological family, foster family – with and without experience/

Adapted for each case

Establishing contact Introduction of survey objectives, principles of anonymity and data summary

Self-introduction/ introducing the child’s story

Please tell us about your child a little – age, gender, which institution he/she attends or used to attend, causes for attending the institution

When did he/she stop going? What was the reason (a question for the biological family)?

How and why did you get involved in the foster family system? What kind of foster experience do you have (a question for the family with foster care experience)?

How and why did you want to get involved in foster services (a question for the family with no foster care experience)?

Relevance

A1. Perceptions of the need for the implementation of the reforms

1. What do you think of the new approach?2. Are you ready for the new approach? What about your child? 3. How ready are the child’s friends, teachers, social workers or other supporting

organizations for the new approach?4. How has your situation (social, psychological, domestic, financial) changed /or will

change/ following the reform? What about the child?5. Does the new approach reflect your child’s actual needs? Is it good or bad for you

personally?A2. Participation in training courses related tothe reforms

6. Did you take part in training courses within the frameworks of the reform? When,where and how long was the training? How would you assess the courses in general?

7. Rate the training courses according to: a) Relevance/necessity of topic b) Contentsc) Level of preparedness of training specialistsd) Duration (long enough or not)



e) usefulness and applicability, correspondence to Armenian reality8. Did this training course meet your needs?9. What kind of changes occurred due to the training? 10. What other types of knowledge/skills do you feel you still need to acquire?11. What would you recommend to make the training courses more effective?

Effectiveness

B1. Process of working with children

Observation of children and identification of needs12. Are there institutions or people who work with your family/child? Who are they?

Please tell us a bit about how they support or help you. How do you see caring foryour child without such help?

13. Does your child attend any other institution?14. Do you receive support from the state? What kind? How has the support provided by

the state changed following the reform? Does the state support more now thanbefore? If no such support existed, how would you be able to care for your child?

15. How is the cooperation done in the following chain: parent-child-social worker-casemanager-institution-state agency?

16. What should be done to make the cooperation between various organizations andstakeholders more effective?

B2. Perceptions of effectiveness of work withchildren involved in the reforms

17. What do you think is the best place for the child? The family, the institution, theschool …?

18. Is the child better off now? What is better and what is worse now (If there are nochanges yet, ask what will happen if there are changes)?

19. Are you personally better off now? What is better and what is worse? What changedin your family for the better and what changed for the worse (if no changes havehappened, ask what will happen if there are changes).

20. What is better for the future of the child?21. What difficulties do you face/will you face after the change? How do you overcome

difficulties? 22. What would help you overcome difficulties more easily?

Efficiency

C1. Assessment of conditions required for the reforms

23. Do you think the organizations or individuals working with children have adequateknowledge and skills? What is their attitude? Have you noticed attitude changestowards the child? If yes, then what do you think that has to do with?

24. Do you think the institutions your child attends have adequate conditions to meet theneeds of the child? What do these institutions need to meet the needs of your child ina better way?

25. Do you have adequate conditions (time, knowledge/skills, finances, domesticconditions, resources, etc.) for meeting the needs of the child and ensuring a normallife? What would you need to better be able to meet the needs of the children?

26. What kind of assistance is needed to make the system more effective?Sustainability


27. Do you think the current approach is better than the old one? Why do you think so? 28. Are people around you (other children, specialists, institutions, etc.) ready for the

shift to the new system?29. What has changed? How? 30. What should be changed in the process of implementation of the reforms?



A question for the foster family

31. What kind of cultural and legal obstacles are there?



Questionnaire blocks for the focus group discussion with staffs of institutions (boarding school, special education school, care center, pedagogical and psychological center)

Adapted for each case

Establishing contact Introduction of the objectives if the survey, principles of anonymity and data summary by the moderator

Self-introduction of participants (profession, occupation and position, work experience with children)

Relevance

A1. Perceptions on the need for the introduction of the inclusive education system

1. How do you generally view the idea of reforms in the sector (for each case thename of the corresponding reform is read out, i.e. deinstitutionalization,introduction of inclusive education and other related activities)? Why do youthink so?

2. How efficient is the process of introduction of that system in Armenia? Why do youthink so? Do you think the Armenian society, the parents, the teachers and thechildren are ready for inclusive education?

3. Is the introduction of inclusive education in the best interests of the children, thesociety, the parents, the teachers, the government and the implementers? Why do youthink so?

4. How would you overall describe the process of implementation of the reforms interms of the situation of children before and after the reform? How has the situationof children changed?

5. What happened to your organization as a result of the reforms?6. In your opinion, how are the implemented reforms related to state policy, priorities

and the interests of children?7. Are the actions undertaken in the frameworks of the reforms coordinated? How do

various parties cooperate? A2. Participation in trainings related to the reforms

8. Have you participated in trainings carried out in the frameworks of the reform?When, where and how long was the training? How would you rate those trainingsoverall?

9. Rate the trainings according to: a) The need/relevance of the topic b) Contentsc) Preparedness of training specialistsd) Duration (appropriate or not)e) efficiency and applicability, correspondence to Armenian reality

10. Did the training course meet your needs? Are there specialists who, according toyou, needed the training more than others did? If yes, then who? Why do you thinkso?

11. What changes have occurred as a result of those trainings?12. Have you applied the knowledge you gained from those trainings?13. Do you think the knowledge and skills you have now are enough for working

efficiently? If not, then why? What kind of additional knowledge and skills do youthink you still need to acquire?

14. Were monitoring visits paid to these trainings? If yes, then by whom? How did thatgo?

15. What would you recommend to increase the efficiency of the training courses?Effectiveness



B1. The process of working with children

Observation of children and identification of needs16. Do you have mechanisms for the observation of children and the identification of

their needs? What are those and are they effective or not? Why?17. How has your work changed before and after the reform in terms of different

procedures, styles, etc.?18. What organizations do you cooperate with in your work? Please describe main

directions of cooperation. What sorts of challenges do you face during thatcooperation? How do other people, organization and institutions contribute to theprocess of providing support to the child? How willing and motivated are they?

19. Two types of integration is done: with the school and with the family. How are theseprocesses going? How does the government respond to challenges arising on theway?

20. How is the cooperation maintained in the parent-child-institution-state agency chain?21. What should be done to increase the effectiveness between different organizations

and stakeholders?


22. How have the implemented reforms supported children in that system? Have theyhad a positive or negative impact on them? In terms of the impact, does it matter ifthe child is a girl or a boy? Why do you think so?

23. Where do the unfavorable conditions for the child lie in the institution-foster family-biological family-school chain?

24. What problems did you have to face as a result of the implementation of thereforms? How do you deal with the issues that come up? What are the problems thatstill require solutions?

25. What changes have you noticed in your work?26. What group of children involved in the reforms benefits the most from your work?

Do the girls adapt more easily than the boys?27. What would help to make your work more effective?

Efficiency

C1. Assessment ofconditions required for

the reforms

28. How well do you think you possess the knowledge required for working withchildren in the new system?

29. Do you have the corresponding technical means to work with the children?30. Has the institution been adapted for all children /ramp, toilets, classrooms/? Is

transportation provided?31. Does your institution have relevant professional staff?32. What do the children need that is not provided and the lack of which hampers their

efficient wellbeing? Is there a difference between girls and boys in terms of thoseneeds?

33. What kind of support is need to increase the level of efficiency of the reformssystem?

Sustainability


34. How effective do you think the society /community, parents, children, staff workingin the system/ deems the introduction of the new system? What sorts of feedback doyou get?

35. Are the children ready to switch to the new system? What about the parents, thespecialists and the institutions? Why do you think so?

36. How do you personally rate the impact of the activities you carried out? How do themembers of the community perceive that?

37. Overall, how do you think the implementation of the reforms impacts the



communities? What sorts of visible changes are there in your own community oramong the children?

38. What should be changed in the process of implementation of the reforms?39. Do you think the current Armenian society is ready for the undertaken reforms?

Why?40. What should be done to have the reforms implemented smoothly and in a manner

accepted by all?



Questionnaire for in-depth interviews with case managers and social service providers

Adapted for every case

Establishing contact Introduction of survey objectives, principles of anonymity and data summary

Relevance

A1. Perceptions of the need for the implementation of the reforms

1. What do you think of the reforms in the sector in general (for each case the title ofthe corresponding reform is given, i.e. deinstitutionalization, introduction ofinclusive education and other related activities) Why do you think so?

2. How effective is the introduction of that system in Armenia? Why do you think so?Do you think the Armenian society, the parents, the teachers and the children areready for that change?

3. Are the social service providers /case managers ready for those reforms?4. How would you describe the process of implementation of the reforms and the

situation of children before and after the reforms? How has the situation of thechildren changed?

5. What changed in your activities due to the reforms? What functions did you stopcarrying out and what new functions did you receive?

6. How do you think the reforms in implementation are related to state policy, prioritiesand the interests of children?

7. Are the actions undertaken in the frameworks of the reforms coordinated well? Howdo the various parties cooperate?

A2. Participation in training courses related tothe reforms

8. Did you participate in training courses in the frameworks of the reforms? When,where and how long were the courses? How would you rate those trainings ingeneral?

9. Rate trainings according to: a) Relevance/necessity of topic b) Contentsc) Level of preparedness of training specialistsd) Duration (long enough or not)

e) usefulness and applicability, correspondence to Armenian reality

10. Did that training course meet your needs?11. What changes occurred due to the training?12. Have you applied the knowledge gained from the training?13. Do you think your current knowledge and skills are adequate for working

effectively? If not, then why? What sorts of additional knowledge/skills do you thinkyou still need?

14. What would you recommend to increase the effectiveness of training courses?

Effectiveness

Observation of children and identification of needs15. Do you have monthly or annual plans of case management with families? How

realistic are those plans?16. Do you have mechanisms for the observation of children and identification of needs?

What are those? Do they work? If not, then why?17. What organizations do you cooperate with in your work? Please describe the main

directions of cooperation. What kind of problems do you face in the course of thesecollaborations? How do other people, organizations and institutions get involved in



B1. Process of working with children

the process of providing support to the child? How willing and motivated are they?18. Two types of integration is done: with the school and with the family. How are these

processes going? How does the government respond to challenges arising in theprocess?

19. How is the cooperation done in the following chain: parent-child-social worker-casemanager-institution-state agency?

20. What should be done to increase the effectiveness between various organizations andstakeholders?


21. How do the implemented reforms help children enrolled in the system? Do they havepositive or negative impacts on the children? Is there a difference between theimpacts on boys and on girls? Why do you think so?

22. Where are the most favorable conditions for the child in the following chain:institution-foster family-biological family?

23. What issues did you have to face in relation to your work as a result of the reforms?How do you deal with these issues? What type of issues still need solutions?

24. What kind of changes have you noticed resulting from your work? 25. What group of children in the system benefits the most from your work? Why do you

think so? Do girls adapt more easily than boys?26. What would help you to make your work more effective?

Efficiency

C1. Assessment ofconditions required for

the reforms

27. Do you think you have the adequate knowledge required for working with childrenunder the new system?

28. What types of technical resources do you need for the work with children/transportation, etc./?

29. What do the children need that is not provided to them and the lack of which hinderstheir effective wellbeing? Is there a difference between boys and girls in terms ofneeds?

30. What kind of assistance is needed to increase the efficiency of the system ofreforms?

Sustainability


31. How effective is the introduction of the new system deemed by the society/community, parents, children, system staff/? What kind of feedback do you get?

32. Are the children ready for the shift to the new system? What about the parents, thespecialists and the institutions? Why do you think so?

33. How do you personally rate the impact of your activities? How are they perceived bythe members of the community?

34. How do you think the implementation of reforms affects the communities? Whatsorts of visible changes have you noticed among the children in your community?

35. What should be changed in the process of implementation of the reforms?36. Do you think the current Armenian society is ready for these reforms? Why? 37. What should be done to have a smooth implementation of reforms carried out in a

fashion acceptable by all?

Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services aspart of Child Care Reform” project UNITED NATIONS CHILDREN'S FUND in Armenia

ANNEX 5: EVALUATION MATRIX

Rel

evan

ce

# Are the project activities/components relevant to the actual/defined needs of the

vulnerable families with children? Were the objectives clear and feasible? Do the

main components of the project contribute to the planned objectives and logically

interlinked? How well are they linked to each other?

FG: With parents of vulnerable childrenIn depth Interviews with:

1. Families with vulnerable children 2. Foster families 3. Case managers 4. Staff and representatives of institutions, including special schools 5. USAID, UNICEF, Implementing Partners 6. RA MoE, MoLSA

# the Government committed (both in terms of timing and financially) to the

project? How the project is aligned with and supports the national development

plans, strategies and national plans of action? Is the project in line with the current

priorities of the country?

Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional

administrations, regional psychological and pedagogical supportcenters, child and family support centers

2. USAID, UNICEF, Implementing Partners

# Has the project involved relevant stakeholders through consultative processes or

information-sharing during its preparation phase? Was the needs

assessment/analysis carried out at the beginning of the project reflecting the various

needs of different stakeholders? Are these needs still relevant? Have there any new,

more relevant needs emerged that the project should address?

Expert Interviews and/or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional


2. USAID, UNICEF, Implementing Partners3. Foster families4. Case managers

Document review# To what extent were the reform initiatives informed by the needs and interests of

diverse groups of stakeholders (also from gender equality perspective)?



2. USAID, UNICEF, Implementing Partners In depth Interviews

3. Families with vulnerable children 4. Case managers 5. Trained teachers

# What are the beneficiaries’ views about the services provided? What are their FG


views about improving services (if there is a need to improve) effectiveness? 1. Families with vulnerable children 2. Teachers

Quantitative Survey 1. Teachers

In depth Interviews or a specially utilized method (such as essays ordiscussions)

1. Children

Eff

ecti

ven

ess # How effective has the project been in establishing ownership by the stakeholders?

Can the project management and implementation considered as participatory. If yes,

does this approach contributes to achievement of the project objectives? Has the

project been appropriately responsive to political, legal, economic, institutional (and

other) changes?




Document review

# Is the project making sufficient progress towards its planned objectives? What are

the key achievements, challenges and implementation lessons? Will the project

likely achieve its planned objectives upon completion and demonstrate success?

# Are the strategies appropriate and effective to achieve the planned objectives?

What, if any, alternative strategies would have been more effective in achieving its

objectives?



2. USAID, UNICEF, Implementing Partners FG

1. Families with vulnerable children 2. Teachers


In depth Interviews or a specially utilized method (such as essays ordiscussions)

1. Children# Are the strategies appropriate and effective to achieve the planned objectives? What, if any, alternative strategies would have been more effective in achieving its objectives?




# How effectively the family support services are provided in regional (and

community) level and how they are in line with actual needs?

FG1. Teachers 2. Parents of vulnerable children

In depth interviews


1. Families of vulnerable children 2. Case managers 3. Children of institutions 4. Family and community based alternative services

Eff

icie

ncy

# To what extent has UNICEF and implementing partners made good use of the

human, financial and technical resources, and has used an appropriate combination

of tools and approaches to pursue the achievement of project results in a cost-

effective manner?

Expert interviews or in depth interviews 1. MoE, MoLSA2. UNICEF, USAID, Implementing partners

Document review

# Was there a clear distribution of roles and responsibilities of key factors involved,

including staff, implementing partners and governmental partners towards the

achievement of project objectives?



2. USAID, UNICEF, Implementing Partners Document review

# To what extent budgetary allocations by the donor between the partners were

efficient? To what extent did UNICEF and implementing partners capitalize on

other complementary initiatives to the project to reinforce the results of the project?




# Has the project done appropriate financial planning and reporting?

Has the management of funds (and reporting) been done in an efficient manner?

Has the co-funding/contribution from different project partners been made at the

expected level? Are there established procedures for checks and controls in the day-

to-day financial management, procurement of good and services and decisions

regarding resource allocation for the project?




# Have project funds and activities been delivered in a timely manner?

If not, what were the bottlenecks encountered? Were the procedures put in place for

staffing,

systems (administration, accounting, other) and reporting efficient?





Su

stai

nab

ilit

y # To what extent have the UNICEF country office and project partners been able to

support the government and beneficiaries in developing capacities and establishing

mechanisms to ensure ownership and the durability of effects under the reform

initiatives?



2. USAID, UNICEF, Implementing Partners Document reviewsFG

1. Teachers 2. Families of vulnerable children

In depth interviews 1. Families of vulnerable children 2. Case managers 3. Staff and representatives of institutions 4. Family and community based alternative services

# Is there a need to scale down the project (i.e. if the project duration is shorter than

planned)? If so, do project objectives and strategies have to be adjusted?



2. USAID, UNICEF, Implementing Partners Document reviews FG

1. Teachers 2. Families with vulnerable children



# Has the project successfully built or strengthened an enabling environment (laws,

policies, people's attitudes etc.)?



2. USAID, UNICEF, Implementing Partners Document reviews FG

3. Teachers


4. Families with vulnerable children Quantitative Survey

2. Teachers In depth interviews

5. Families of vulnerable children 6. Case managers 7. Staff and representatives of institutions

1. Family and community based alternative services# How does the government intend to maintain the provision of family support

services in future? Are there any specific/envisioned plans/projects?




# What are possible sustainability mechanisms for each of the project component?

Can the project approach or results be replicated or scaled up by national partners or

other actors? Is this likely to happen? Can any unintended or unexpected positive or

negative effects be observed as a consequence of the project's interventions?



2. USAID, UNICEF, Implementing Partners Document reviewFG



In depth interviews 8. Families of vulnerable children 9. Case managers 10. Staff and representatives of institutions

1. Family and community based alternative services

Imp

act # Has the project contributed to or is likely to contribute long-term social changes

for vulnerable families and children? What difference has it made, including policy

changes? What are the realistic long-term effects of the project on de-

institutionalization of children?




Document reviewFG





# Has the project increased the satisfaction with family and community based

alternative services?

FG1. Teachers 2. Families with vulnerable children






ANNEX 6 RESULTS FRAMEWORK: M&E PLAN

Toward Social Inclusion of Vulnerable Children in Armenia: Expanding Alternative Care, Family Support and Inclusive Education

Services as Part of Child Care Reform

Monitoring & Evaluation Plan

Revision 1June 2016

2Monitoring & Evaluation Plan, Revision 1

Table of Contents

Toward Social Inclusion of Vulnerable Children in Armenia: Expanding Alternative Care, Family Support and Inclusive Education Services as Part of Child Care Reform 1List of Acronyms 31. Overview 42. Summary of the Program and Objectives 4

Program Goal and Objectives 63. Monitoring Component 6

Indicators 7Baselines and Targets for Performance 7Data Disaggregation 7Data Quality Reviews (DQR) 7

4. Progress, Annual and Final Reports 85. Evaluation Component: Mid Term and Final Evaluations 96. Implementation and Management of M&E 10

Responsibilities 107. Review and Revision of the M&E Plan 118. Coordination of M&E Data Gathering 11

Annex 1. Theory of Change (ToC) 13

Annex 2. Performance/Indicator Tracking Table 15

Annex 3 Indicator Definition Table 26


List of Acronyms

AOR Agreement Officer’s Representative

CRC Convention on the Rights of the Child

DPO Disabled People’s Organization

DQA Data Quality Assessment

DQR Data Quality Review

HACT Harmonized Approach to Cash Transfer

ICF International Classification of Functioning, Disability and Health

IDT Indicator Definition Table

ITT Indicator Tracking Table

M&E Monitoring and Evaluation

NGO Nongovernmental Organization

NSS National Statistical Service

PCR Program Completion Report

QA Quality Assurance

ToC Theory of Change

UNICEF United Nations Children’s Fund

USAID United States Agency for International Development

USG United States Government


1. Overview

Monitoring and Evaluation (M&E) is essential for UNICEF’s resultsbased approach to program management towards realization of child rights, particularly the most disadvantaged, vulnerable, and excluded. UNICEF’s continuum for results is designed to foster learning and accountability and serve as a model for dialogue about development results among government and development partners and other stakeholders. This is a key component in UNICEF in Armenia’s work Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as Part of Child Care Reform (hereinafter “the Program”). It remains incorporated into all facets of the Program cycle, from design through program closeout.

The M&E Plan/Framework serves the following functions:

Explains in detail how UNICEF in Armenia and USAID will monitor the Program to determine whether they are achieving their intended results and to measure larger impacts over time through evaluations.

Outlines any M&E requirements that UNICEF in Armenia and implementing partners, namely Save the Children, World Vision, and Bridge of Hope, must meet during the implementation of the Program.

Serves as a guide for the Program implementation and management, so that UNICEF, beneficiaries, and other stakeholders understand the objectives and targets they are responsible for achieving under this Program and are aware of their progress towards those objectives and targets during implementation.

Establishes a process to alert implementers, stakeholders, and UNICEF to any problems in Program implementation and provides the basis for any needed program adjustments.

This M&E plan is considered a binding document for UNICEF in Armenia and implementing partners Save the Children, World Vision, and Bridge of Hope. The listed organizations are required to provide respective data under components they are responsible for including the indicator tracking table which needs to be completed on a quarterly basis.

Any significant changes to the approved M&E plan will require approval of USAID.

2. Summary of the Program and Objectives

Although Armenia ratified the Convention on the Rights of the Child (CRC) in 1992, which stipulates the right of the child to live in a family environment, the country remains reliant on institutional care for children in difficult life circumstances. In 2009, the UN Guidelines for alternative care were established to implement care provisions per the CRC that strongly recommend for children to be brought up in family environments and clearly state that socioeconomic status of the family should not be a reason for separating a child from his/her family.


Developing a system for the protection of children’s rights ranks high on the Government of Armenia’s agenda. Deinstitutionalization of children from residential care institutions in 2007 was an important achievement in the reform of child protection services: it led to restructuring of 17 boarding schools and reintegration of approximately 4,000 boarding school students into their biological families. In 2011, the government reaffirmed its dedication to the deinstitutionalization of children and declared it a government priority. Despite this progress, Armenia currently has 40 residential institutions, including 8 childcare institutions (6 state and 2 nonstate), 9 care and protection boarding institutions (8 state and 1 nonstate), and 23 special educational institutions. This translates into 4,000 institutionalized children, approximately 1,500 of whom are in state institutions, 300 in private care institutions, and 2,200 in special educational institutions. The majority of those institutionalized children have at least one living parent. Children in special schools largely lack access to inclusive education and diverse social and rehabilitation services in their communities of origin and, therefore, are enrolled in special schools as a last resort.

Primary reasons identified reasons for child institutionalization are the lack of communitybased services to assist families and children in need, especially children with disabilities from the families with poor economic conditions; the lack of educational programs able to accommodate varied developmental capacities; an absence of community services for children with special needs; the lack of information for parents on the conditions of children in special schools; and limited coverage for the recently introduced system of diversified family assistance.

The 2013 Concluding Observations of the Committee of the Rights of the Child recommend that the state ensures institutional care is used only as a last resort and that adequate safeguards and clear needsbased and best interests of the child criteria.1 Furthermore, it recommends mobilizing efforts and resources to strengthen families, promote family reunification, and limit further separation of children from families to a necessary minimum. The strategy ensures that the most vulnerable families have access to basic universal services and social assistance programmes2 through reallocation of funds from residential care to community and familybased services3 and proper case management work for vulnerable children and their families. In parallel, it is imperative to strengthen the intersectoral cooperation on child issues where the child and his/her rights are viewed as a holistic matter for policy making.

Ensuring the rights of children with disabilities to family life and community, social, and educational inclusion should be a priority. According to a recent survey conducted by UNICEF in Armenia4, 13% of children with disabilities live in residential institutions (either orphanages or special boarding schools5). There are few communities where residential institutions offer rehabilitation services, education support, daytime care, development services, and social support for the families of disabled children. In this regard, the Committee of the Rights of the Child recommends that the state takes measures to deinstitutionalize children with disabilities and

1 Concluding observations of the Committee of the Rights of the Child, combined third and fourth periodic reports of Armenia, 27 May 14 June 20132 This is one of the recommendations of the Committee of the Rights of the Child (2013)3 Elena Andreeva (2010), Towards Alternative Child Care Services in Armenia: Costing Residential Care Institutions and Community Based Services. UNICEF4 It’s about Inclusion, UNICEF, 2012, p.405 It’s about Inclusion, UNICEF, 2012, p.24


provide them alternative familybased care options and education in mainstream schools as well as allocate adequate human, technical, and financial resources to ensure early detection and rehabilitation services.

According to UNICEF’s analysis, although institutionalization affects both boys and girls equally, gender discrimination is apparent, as 68% of boys with disabilities stay with families as opposed to only 32% of girls. Additionally, the fact that boys are preferred over girls when it comes to adoption further leads to an undervaluing of girls in society. To ameliorate this situation, gender sensitive approaches need to be applied for girls with disabilities. Finally, along with several reported cases of misuse of financial resources at residential institutions, it is also a costly and inefficient service for a country with a very limited social budget and high rates of poverty.

Program Goal and Objectives

The overall goal of the program is to support the Government of Armenia to ensure that children realize their rights to live in a family environment and access communitybased social and educational services. The project will ensure that the child care system in Armenia significantly reduces reliance on largescale institutions through the establishment of a network of communitybased services responding to the needs of the children, families, and community. The Program will achieve the overall goal through the following objectives (summarized in the Program Theory of Change, See below):

Institutional mechanisms for the functioning of social service system are improved. Alternative communitybased family support services are accessible to vulnerable

children and families. Family substitution service system is strengthened with the diversification of types of

foster care establishment of monitoring and evaluation system. Inclusive Education System is strengthened to provide quality learning for all children

through set up of pedagogical psychological support centers and mainstream school trainings.

An enabling legal and regulatory framework is established to support childcare reform. Legal norms and social perceptions regarding children with disabilities have changed

favorably towards inclusion.


Figure 1. Theory of Change (ToC)


General public supports childcare reform to expand and adapt basic services for

most vulnerable children and adolescents

Community heads, town hall representatives, and city councils support vulnerable children to

have access to communitybased social and educational services

Accessible alternative community‐based family support services to vulnerable children and families

Strengthened family substitution service system with the diversification of types of foster care, establishment of monitoring and evaluation system

UNICEF Core Roles System Change Mediumterm changes Long term changes

fa

Inputs and Outputs Outcomes Impact

Voice

Leveraging

National Dialogue

M&E

Knowledge Exchange

Policy Advice and TA

Modelling

Children’s rights to live in a family environment and access communitybased social and

educational services are realized

Change in legal and social norms towards foster care and inclusion

Established national framework for institutional cooperation between services with sample protocols of

cooperation developed Established or expanded family

support services

Developed and approved foster care policy

Accessible Foster Care services for children in diverse, difficult life

circumstances

Approved by the Government and by the National Assembly recommendations for improving legal and regulatory framework

Established pedagogical support centers to support inclusive schools in

their geographical area Inclusion principles accepted by

teachers and directors for mainstream schools to provide education to all

children

Children claim and exercise their rights to access basic health, education and social/child protection services in their communities

An enabling legal and regulatory framework to support child‐care reform and improved institutional mechanisms for the functioning of social service system

NGOs, think tanks, civil society leaders, and media advocate and

influence public opinion to fulfil child rights to

live in a family environment and

strengthen inclusion

Strengthened inclusive education system to provide quality learning for all children

Legal norms and social perceptions regarding CWD changed favorably towards inclusion


3. Monitoring Component

Program and Activity performance will be monitored systematically and on an ongoing basis through the regular indicator tracking system, in line with UNICEF HACT requirements; programmatic visits; spot checks; and audits. Analysis of the information derived from these monitoring mechanisms will inform needed programmatic adjustments with the view towards improving the overall impact of this Program.

Indicators Lower level indicators (outputs) and their targets will mainly be drawn from program joint work plans developed with Program partners. Higher level indicators (outcome) and their targets will be directly linked to the analysis to be conducted during the Program implementation to estimate the impacts of the interventions (Indicator Tracking Table (ITT), Annex 2). The data for the analysis will be collected through surveys/evaluations.

The Indicator Definition Table (IDT, Annex 3) included in the M&E Plan will provide a detailed definition of each indicator, unit of measurement, source of data, method of data collection, frequency of data collection, and the entity responsible for collecting the data. Indicator Tracking Table that is part of the M&E Plan will ensure tracking of interim progress towards goals. All baselines and targets relevant to specific activity will be analyzed and specified.

Baselines and Targets for PerformanceThe baselines and targets for each indicator are shown in the Performance Tracking Tables. Baselines for some of the indicators will be collected by program partners. Outputs and their targets are mainly derived from Program joint work plans. Outcomes and their targets are linked to the analysis to be conducted during the program implementation to estimate the impacts of the interventions. The data for the analysis will be also collected through ad hoc studies, surveys, and evaluations.

Data DisaggregationProgram performance indicators will be disaggregated by gender where appropriate and feasible. This way, the Program implementers will assess program effectiveness at supporting gender balance in its activities. Disaggregation by age, disability status, ethnicity, and income quintile can be performed on need basis during the implementation.

Data Quality Reviews (DQR) – by UNICEF

The goal of the DQR is to conduct an independent review of the quality of indicators and data under the Program through verification of the quality and the consistency of performance data over time, across different implementers/partner organizations and reporting institutions (i.e. administrative data, if relevant). Such data quality reviews will also identify cases in which the highest degree of data quality is not possible given the realities of the data collection circumstances. These assessments will cover data reported from Program partners


and other data sources, such as administrative (i.e. data provided by Ministries, NSS official data, etc.), as necessary.

The particular objectives for the data quality reviews will be identification of the following parameters: i) what proportion of the data collected during the Program implementation has quality problems (completeness, conformity, consistency, accuracy, duplication, integrity); ii) which of the records in the dataset are of unacceptably low quality; iii) what are the most predominant data quality problems within each field provided by Program partners.

The DQR should be conducted once during the life of the Program (the date to be agreed with USAID), focusing on verifying the universally accepted aspects of data quality (such as accuracy, reliability, timeliness, objectivity, and relevance) through review of all components of data collection, processing, analyses, and reporting processes. UNICEF together with Program partners will contract the data quality reviewers to perform the task. UNICEF M&E specialists, the Child Rights Systems Monitoring Specialist, and the Child Protection Specialist will also regularly check the quality of data collected and provided by partners. USAID AOR will be present at the DQR and review the data systems firsthand.

Data Quality Assessments (DQA) – by USAID

USAID requires that a DQA must occur for all externally reported indicators sometime within three years of data collection and before being reported. The USAID AOR is responsible for conducting the DQAs for indicators in the M&E Plan that will be reported to USAID/Washington. Implementing Partners and third party M&E contractors will often be involved and engaged in conducting DQAs. If the AOR does not conduct the DQA, he/she is responsible for certifying the DQA, once done, and for addressing findings with the implementer and ensuring that corrective actions are taken.

Data should reasonably meet these five standards of data quality: validity, integrity, precision, reliability, and timeliness.

4. Progress, Annual and Final Reports

Program partners will produce quarterly reports (Progress Reports) starting from the second quarter (Feb/Mar 2015). Progress Reports will track the progress of the Program by including data on the indicators described in the Monitoring Component, methodology of data collection, and analysis of the data alongside with other information related to program success, any challenges and lessons learnt. These documents will also discuss field visits performed by implementing partners to observe the performance and identify the cases of special interest (if any). UNICEF/Armenia’s Child Protection Specialist will review and approve the reports.

Based on progress report data submitted by program partners, the UNICEF Child Protection Specialist will compile quarterly progress updates and share with USAID/Armenia. The Quarterly Progress Updates shall collectively discuss the work of the program highlighting:

Achievements during the quarter;


Update indicators; Timeline of events; Summary of obstacles and issues and how they are being addressed; Program activities planned for the next quarter.

The quarterly updates shall cover US Government fiscal quarters and be due within 15 days after each fiscal quarter.

The Annual Supplementary Reports will be prepared by Program partners by end of quarter four and provide additional information on accomplishments and developments of Program implementation as well as lessons learnt to complement UNICEF annual country office reporting. Some of the highlights of Annual reports can be made available to the public through UNICEF website and other means of communication as they become available.

UNICEF will also prepare a final report Program Completion Report (PCR) to be submitted to USAID within 90 days of the agreement end date. All three partners (Save the Children, World Vision, Bridge of Hope) will contribute to the development of this report. The PCR should provide: A concise description of the Program from proposal to completion including

objectives, strategies and activities undertaken within the program implementation and corresponding results;

A preliminary assessment of the Program’s outcomes; Identification of beneficiaries including relevant characteristics, such as gender, age,

and income level, and degree of participation; Identification of stakeholders supporting the Program; A summary of problems/obstacles encountered during the implementation and how

the obstacles were addressed; An executive summary of the accomplishments and results achieved; A preliminary assessment of the Program’s sustainability that is, its likelihood to

reach the future monitoring targets established as a measure of the programs’ sustainability;

Lessons learned, including changes that might have been made in M&E criteria, policies, procedures and practices related to the program.

1. Evaluation Component: Mid Term and Final Evaluations

Evaluation is an essential element for UNICEF in Armenia. Midterm and/or final evaluations will be conducted during the life of the Program to improve program management and provide lessons for future program implementation as well as assess the effectiveness, efficiency, relevance, impact, and longterm sustainability of the Program. UNICEF will engage independent evaluators to conduct evaluations.

Midterm evaluation planned in 2017 will assess progress in meeting the Program goals and outcomes. They will provide early lessons learned and identify significant discrepancies between expected results and actual achievements, including an analysis behind the reasons for discrepancies between actual and projected indicators.

In 2019/2020, UNICEF, jointly with the Government of Armenia, will arrange for final evaluation (endofproject evaluation) to address the relevance, effectiveness, efficiency,


impact and sustainability of family and community based alternative services and their role in prevention of institutionalization. The evaluation findings will inform decisionmakers on alternative care, family support mechanisms and inclusive education as well as contribute to learning and knowledge sharing on social inclusion of vulnerable families with children.

In addition, the following issues will be covered under the evaluation:

Effectiveness of the Program at meeting goals; Reasons behind the success or failure to achieve goals, objectives and targets; Unintended results of the program (positive and negative); Longterm sustainability of results; Lessons learned and recommendations.

Both midterm and end of project evaluations will be conducted with the involvement of external evaluation experts selected through an open tender process. The entity selected to conduct the evaluation will decide the overall methodology to be used in the evaluation based on a prior, agreedupon statement of work developed by UNICEF.

A sound quality assurance (QA) system for evaluation is critical to UNICEF’s mandate and to enhance the organization’s reputation as a knowledge leader on children, ensure scarce resources are not wasted, and to protect the children, women, and others who participate in the research, and whose lives are affected by its outcomes. Therefore, all evaluations commissioned and/or conducted by UNICEF will go through the quality assurance system and be subject to assessment by the UNICEF Office of Research.

2. Implementation and Management of M&E

Prior to starting implementation of the program activities, UNICEF will orient Program implementers on how program performance is to be measured and will provide necessary support to comply with the M&E Plan. UNICEF may make adjustments to the M&E Plan as needed, provided any modification or amendment of the M&E Plan (including ITT) and communicate the changes to the implementing partners.

Data StorageUNICEF will share the M&E plan including Indicator Performance Table, Indicator Definition Table and Indicator Tracking Table with the Implementing Partners. Each partner will collect the information they are responsible for according to Indicator Performance and Tracking tables. Partners will use the same approach while collecting the data to ensure the reliability, validity and integrity of data. UNICEF will consolidate the data provided by each IP in a database.

Implementing partners will share with UNICEF the reports, policy documents, program implementation documents the copies of which will be stored at Child Protection section.Filing and record keeping systems for both hard copies and digitized information will be set up.

Responsibilities

UNICEF in Armenia assumes general M&E responsibilities, including the following:


Guide the establishment of the M&E system, including datacollection, dataanalysis and reporting systems;

Ensure that the M&E Plan is modified and updated as improved information becomes available;

Ensure that findings are disaggregated by gender, age, and income, as applicable; Support to design midterm and final evaluations and surveys; Participate in monitoring through site visits, review of program reports and secondary

data; Facilitate learning exchanges and information dissemination; Organize and oversee regular independent data quality reviews.

UNICEF Armenia M&E specialist and Child Protection Specialist, in cooperation with USAID Agreement Officer’s Representative, will ensure realization of the referenced tasks.

3. Review and Revision of the M&E Plan

The M&E Plan is designed to evolve over time, adjusting to changes in program activities and improvements in performance monitoring and measurement. In the fourth quarter of every year or as necessary, the M&E, Child Rights Systems Monitoring Specialist, and the Child Protection Specialist will review how well the objectives set in the M&E Plan are accomplished. The review is intended to ensure that the M&E Plan measures program performance accurately and provides critical information on the need for changes in project design. The review will ensure that the M&E plan developed for this Program:

Show whether the logical sequence of intervention outcomes are occurring; Check whether indicator definitions are precise and timely; Check whether M&E indicators accurately reflect program performance; Update indicator targets, as allowed; Add indicators, as needed, to track hitherto unmeasured results.

The M&E Plan will be revised by UNICEF, in agreement with USAID, when the need for change has been identified in the review. Any significant changes to the approved M&E plan will require additional approval of USAID. UNICEF will communicate the revised M&E Plan to the Program partners.

4. Coordination of M&E Data Gathering

UNICEF will receive data disaggregated by gender (further disaggregation, such as age, disability status, ethnicity, and income quintile can be taken into consideration at later stages when need be) from Program implementers (Save the Children, World Vision, and Bridge of Hope) responsible for tracking respective indicators on quarterly/annual basis as well as the National Statistical Service (NSS) and respective lineministries. The Indicator Tracking Table of the M&E Plan will be updated by UNICEF each quarter, once the data is compiled and reported by partner organizations. The following diagram6 (See Diagram 1) displays the flow of information from these organizations to UNICEF in Armenia7.

6 The diagram is meant to depict the flow of information, not the supervisory relationship of the organizations.


Diagram 1. The flow of information from different agencies to UNICEF in Armenia

National Statistical Service

Ministry of Labor and Social

Affairs

Ministry of Education

Ministry of Territorial

Administration

7 The Indicator Definition Table provides the definition of each indicator, unit of measurement, methodology and frequency of data collection as well as outline the information that will be collected and reported by each institution, methodology and frequency of data collection.

Save the Children

World Vision

USAID in Armenia

Bridge of Hope

UNICEF in Armenia


Annex 2.Performance/Indicator Tracking Table

Program Goal Targets

Objectives Indicators Units Baseline(2014)

Year 1 Year 2 Year 3 Year 4 Year 5

Goal: Children realize their rights to live in a family environment and access communitybased social and educational services.

Percent decrease of children among children in targeted residential care facilities

Percentage 0 0 10% 20% 35% 50%

Number of children from target institutions reunified with families and remaining in families for at least six months in target communities

NumberCumulative

40(state

deinstitutionalization program)

40 250 450 800 1100

Number of children from targeted institutions placed in alternative family environment in target communities remaining in alternative families for at least six months

NumberCumulative

64 64 120 250 400 700

Number of children removed from residential care and from target communities using communitybased social services

NumberCumulative

2400 2400 2800 3200 3900 4500

Number of children and families using family support services at least 3 times over a year

Number cumulative

150 300 450 600

Number of children with special educational needs benefitting from communitybased inclusive education at least one year

NumberCumulative

3000 3000 3600 4500 5000 5500

Objectives Indicator Units Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Implementing


(2014) 2015 2016 2017 2018 2019 partnerOUTCOME #1: Institutional mechanisms for the functioning of social service system are set up

Child protection is a part of integrated social services

Annual number of referrals of child cases to social services

NumberCumulative

0 0 370 700 1200 1800 UNICEF in Armenia

Output 1.1: Case managers are fully functional in the target regions, equipped to perform social needs assessments and followup the cases of children

1.1.1 Training package for the case managers is developed

Training module developed Date Aug 2015

UNICEF in Armenia

1.1.2 Respective bodies identified and trained, including regional level child protection units, case managers involved in foster care processes

Number of bodies trained NumberCumulative

5 25 35

Output 1.2: A national framework for institutional cooperation between services is established and a guideline on its implementation is developed and approved

1.2.1 A national framework for institutional cooperation between services on child care and protection is established

A government decree regulating institutional cooperation among services adopted

Date Feb 2016

UNICEF in Armenia1.2.2 Guideline on implementing

institutional cooperation is developed and approved

Developed guideline is submitted to MOLSA

Date Sep 2016

Developed guideline is approved by MOLSA

Date Nov 2016

Output 1.3: Territorial social plans addressing the needs of the socially vulnerable populations are in place

1.3.1 Amendments in territorial social plans addressing the needs of the socially vulnerable populations are approved of regional administration

Amendments in territorial social plans approved

Date Date 2015 (Lori)

TBD TBD TBD TBD

Objectives Indicator Units Baseline (2014)

Year 12015

Year 22016

Year 32017

Year 42018

Year 52019

Implementing partner


OUTCOME #2: Alternative communitybased family support services are accessible to vulnerable children and families

2.1 Increased access to community based alternative services in targeted communities

Number of communitybased services accessible to children and their families in difficult life circumstances

NumberCumulative

0 0 4 8 10 12 World Vision

2.2 Enhanced knowledge of staff on newly introduced services

Percent increase of the knowledge of the staff on newly introduced/expanded services

Percentage8 0 N/A 15% 15% 15% N/A

Output 2.1: Family support services (medical and social rehabilitation, afterschool care, psychosocial support, etc.) are established or expanded

2.1.1 Residential institutions closed based on closure plan of each institution

Number of institutions closed Number Cumulative

0 1 2 4 6

World Vision2.1.2 Residential institutions transformed based on the transformation plan of each institution

Number of residential institutions transformed into alternative services based on transformation plan

NumberCumulative

0 4 8 12 16

2.1.3 Availability of trained staff with the capacity to provide services based on newly introduced and/or expanded service/s

Number of staff trained to deliver new services

NumberCumulative

0 0 20 40 60

2.1.4 Social needs of the child and the family assessed

Number of institutionalized childcases assessed on social needs

NumberCumulative

0 450 1200 2600

2.1.5 Individual care plans developed and followedup

Number of social cases conducted by case managers

NumberCumulative

0 450 1200 1800

Number of social cases with demonstrated progress

Number Cumulative

0 200 600 900

8 Note that the presented percentages are not cumulative as the participants in each training group will be different. The percentage increase is going to be calculated compared to its baseline which will be estimated via pretest.


Percent of social cases solved by case managers

Percentage 0 10% 35% 60%

2.1.6 Family support services, including medical and social rehabilitation, afterschool care, psychosocial support, and other services established/expanded in selected regions

Number of community based services established classified by type

NumberCumulative

24 24 26 28


Year 12015

Year 22016

Year 32017

Year 42018

Year 52019


OUTCOME #3: Family substitution service system is strengthened with the diversification of types of foster care, establishment of monitoring and evaluation system3.1 Expansion and diversification of foster care policy in Armenia to be approved by the Government

Foster care policies revised and approved

Date Oct 2015 Save the Children

3.2 Enhanced knowledge of families on being a foster family

Percent increase of the knowledge of trained families to be a foster family

Percentage9 0 15% 15% 15% N/A N/A Save the Children

Output 3.1: A comprehensive foster care policy is developed and approved by the government

3.1.1 A comprehensive foster care policy is developed and approved by the government

Approved policy on foster care

Date

3.1.2 Existing foster care system/framework is analyzed and compared to international standards

Foster care system in Armenia analyzed and compared to international standards

Date Feb 2015

Save the Children

3.1.3 Full foster care model in line Full foster care Date Sep



with UNICEF pilot/modelling guidelines developed and customized to the Armenian context

model/framework developed 2015

3.1.4 Foster care policy package is drafted

Foster care policy drafted Date Jun 2015

3.1.5 System for monitoring and supervision of foster care placements is introduced

Foster care placement monitoring and supervision document adopted

Date Jan 2017

3.1.6 System for monitoring and supervision of foster care placements is operational

Foster care placement monitoring and supervision established

Date Mar 2017

Output 3.2: Foster Care services are available for children in diverse, difficult life circumstances

3.2.1 Database for foster families developed

Database of foster families developed

Date Jan 2017

Save the Children

Number of foster families registered in the database

Number Cumulative

30 45 80 95

3.2.2.Database for foster families linked to staterun database and maintained

Database of foster families linked to NORK database

Date Oct 2017

3.2.3 Awareness raising activities to promote foster care are organized

Number of TV programs broadcasted on local TV stations

NumberCumulative

0 0 6 9 9 9

3.2.4 Families for different types of foster care are identified and trained

Number of potential foster families trained

Number Cumulative

30 45 80 95

3.2.5 Children from residential care facilities in targeted communities removed and placed in foster

Number of children placed in foster care

NumberCumulative

15 30 50 80


families in cooperation with the Government of Armenia

Percent of children remaining in foster care after 6 months of being placed.

Percentage 0 80% 80% 80%


Year 12015

Year 22016

Year 32017

Year 42018

Year 52019


OUTCOME #4: Inclusive Education System is strengthened and expanded to provide quality learning for ALL children in targeted regions

4.2 Increased access to communitybased inclusive schools in selected regions

Number of children from 6 to18 transferred from special educational institutions to community based inclusive schools

NumberCumulative

0 0 200 500 800 1100

Percent of transferred children successfully completing at least one academic year

Percentage 0 0 40% 50% 55% 70%

Number of out of school and home schooled children in target regions decreased

NumberCumulative

1000 900 800 600 500 300

4.3 Enhanced capacity of the staff of Pedagogical support centers in conducting ICF based assessments

Percent increase in the knowledge of the staff of Pedagogical Support Centers trained in supporting inclusive

Percentage10 0 15% 15% 15% 15% 15%



education and conducting ICF based assessments

Output 4.1: Pedagogical support centers are established to support inclusive schools in their geographical area

4.1.1 Enhanced capacity of communitybased inclusive schools in selected regions

Percentage of inclusive schools in targeted regions receiving methodological and practical support from Pedagogical Support Centers

Percentage 0 5% 10% 25% 35% 50% Bridge of Hope

4.1.2 The capacity of newly established pedagogical support centers is enhanced to support inclusive schools in their geographical area

Number of staff of Pedagogical Support Centers trained in supporting inclusive education and conducting ICF based assessments

NumberCumulative

0 40 80 160 200 240

Number of trained Pedagogical Support Centers staff conducting ICF based assessments

Number Cumulative

0 0 40 80 160 200

Output 4.2: Teachers and directors for mainstream schools (in the programme areas) understand and accept the principles of inclusion and have the capacity to provide education to ALL children4.2.1 Mainstream schools in targeted regions have the capacity to provide education to children with disabilities

Number of mainstream schools in targeted regions trained on inclusive education practices

Number Cumulative

139 209 299 469 590 764 Bridge of Hope


Year 12015

Year 22016

Year 32017

Year 42018

Year 52019


OUTCOME #5: An enabling legal and regulatory framework is established to support Childcare Reform

5.1. Local legislation is adjusted to respond to childcare reform directions

Number of legal recommendations submitted to the Government of Armenia

Number Cumulative

2 6 7 9 0 0

UNICEF in


Armenia5.2 Legal and policy framework is set up to allow outsourcing of social services to nonstate providers

Number of Social Assistance legal/policy amendments adopted

Number Cumulative

2 2 4 0 0 0

5.3. Master Plan for transformation of residential care institutions is developed

Rapid assessment of residential care institutions conducted

Date 0 Jan 2015

Output 5.1: Recommendations for improving legal and regulatory framework are approved by the Government and by the National Assembly, as appropriate

5.1.1 Family code and bylaws are adjusted to provide framework for transparent adoption process

Number of legal recommendations on adoption approved by the government of RA

Number 0 5 15

UNICEF in Armenia

5.1.2 Law on Social Assistance and by laws are amended to allow outsourcing of state social services to nonstate providers

Number of communitybased NGOs outsourced by state for providing service to vulnerable children and their families

NumberCumulative

2 5 10 15 18 22

5.1.3 Policy document on deinstitutionalization, transformation of residential institutions and types of alternative services developed

Types of alternative services defined

Date Mar2015

OUTCOME #6: Social norms regarding children with disabilities have changed favorably toward inclusion

6.1 People in Armenia have favorable attitude towards children with physical and intellectual disabilities

Percent increase in the positive attitude toward children with physical disabilities

Percentage 0 TBD UNICEF in Armenia


Percent increase in the positive attitude toward children with intellectual disabilities

Percentage 0 20%


USAID Standard indicators M&E Plan

Indicator Description Units Baseline

Year 1

Year 2

Year 3

Year 4

Year 5

Indicator #1

Number of social protection policy reforms drafted, adopted or implemented with USG support

Number 4 1 2 TBD TBD

Indicator #2

Number of vulnerable people benefiting from USGsupported social services

Total Number 1246 2200 2654 2046 TBD

Number of men Number 250 400 465 446 TBD

Number of women Number 373 801 697 670 TBD

Vulnerable children Number 623 999 1312 930 TBD

Indicator #3

Number of service providers trained who serve vulnerable persons

Total Number 3330 4712 5265 5265 TBD

Number of men Number 999 1885 1580 1580 TBD

Number of women Number 2331 2872 3685 3685 TBD

Indicator #4

Number of USG assisted organizations and/or service delivery systems

TotalNumberCumulative

112 176 130 130 TBD

Local NGO Number 2 5 4 4 TBD


strengthened who serve vulnerable populations

(nonDPO)

Disabled People's Organizations (DPO)

Number 1 1 1 1 TBD

Schools Number 102 162 125 125 111

Other Number 7 8 TBD TBD TBD


Annex 3Indicator Definition Table

Goal Indicators

Program Goal Indicator Definition of Indicator Units Source/ Responsible Entity

Methodology Frequency of Data

CollectionChildren realize their rights to live in a family environment and access communitybased social and educational services.

Percent decrease of children among children in targeted residential care facilities

Percent decrease of children (disaggregated by sex and age) among children in targeted residential care facilities (by type).

Percentage Line ministries, NSS, UNICEF in Armenia

Administrative data, program implementation documents, Final evaluation

Annual

Number of children from target institutions reunified with families and remaining in families for at least six months in target communities

Number of children reunified with their families in target communities (disaggregated by age, sex and type of disability, if relevant)

Number Cumulative

Line ministries, NSS, UNICEF in Armenia


Annual

Number of children from targeted institutions placed in alternative family environment in target communities remaining in alternative families for at least six months

Number of children placed in alternative families (disaggregated by type – kinship, foster, adoption) in target communities.

Number Cumulative

Official registrar, National Statistical Service, UNICEF in Armenia


Annual

Number of children removed from residential care and from target communities using communitybased social services

Number children and their families in difficult life circumstances accessing communitybased services

Number Cumulative

Line ministries, Service delivery units, NSS


Annual

Number of children and families using family support services at least 3 times over a

Number of children and families benefitting from family support services on

NumberCumulative

Line ministries, service delivery units, UNICEF in

Administrative data, program implementation

Annual


year average annually Armenia documents, Final evaluation

Number of children with special educational needs benefitting from communitybased inclusive education at least one year

Number of children (disaggregated by age and sex) with special educational needs benefitting from communitybased inclusive education

Number Cumulative

Institutions providing inclusiveeducation, UNICEF in Armenia


Quarterly

Program outcome and output level indicators

Purpose Indicator Definition of Indicator Units Source/ Responsible Entity

Methodology Frequency of Data

CollectionOUTCOME #1: Institutional mechanisms for the functioning of social service system are set up

Child protection is a part of integrated social services

Annual number of referrals of child cases to social services

Number of cases referred to case managers

Number cumulative

Regional administration, UNICEF in Armenia


Annual

Output 1.1: Case managers are fully functional in the target regions, equipped to perform social needs assessments and followup the cases of children

1.1.1 Training package for the case managers is developed

Training module developed

Training modules are developed for case managers and foster families

Date UNICEF in Armenia Training module package

Once

1.1.2 Respective bodies identified and trained, including regional level child protection units, case managers involved in foster care processes

Number of bodies trained

Number of regional level child protection units, case managers that are involved in foster care processes trained.

Number Cumulative

UNICEF in Armenia Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents

Twice a year

Output 1.2: A national framework for institutional cooperation between services is established and sample protocols of cooperation are developed.


1.2.1 A national framework for institutional cooperation between services is established

A government decree on institutional cooperation among services adopted

A government decree regulating institutional cooperation among services is developed and adopted

Date UNICEF in Armenia Government decree, program implementation documents

Once

1.2.2 Guideline on implementing institutional cooperation is developed and approved

Developed guideline is submitted to MOLSA

Guideline developed and submitted to MOLSA for review

Date UNICEF in Armenia Copy of guideline, program implementation documents

Once

Developed guideline is approved by MOLSA

Developed guideline is approved by the government decree

Date UNICEF in Armenia Government decree, program implementation documents

Once

Output 1.3: Territorial social plans addressing the needs of the socially vulnerable populations are in place1.3.1 Amendments in territorial social plans addressing the needs of the socially vulnerable populations are approved by the regional administration

Amendments in territorial social plans approved

Amendments in territorial social plans addressing the needs of the socially vulnerable populations are approved

Date UNICEF in Armenia Copy of approval, Administrative data, program implementation documents

Once


Methodology Frequency of Data Collection

OUTCOME #2: Alternative communitybased family support services are accessible to vulnerable children and families

2.1 Increased access to communitybased alternative services in targeted communities

Number of communitybased services accessible to children and their families in difficult life circumstances

Number of communitybased social services utilized by the target communities and by children in residential care

Number Cumulative

Program partner/World vision


Annual

2.2 Enhanced knowledge of staff on newly introduced services

Percent increase of the knowledge of the trained staff on newly

Percent increase of the knowledge of the trained staff on newly

Percentage World Vision Pre/post test Annual


introduced/expanded services

introduced/expanded services

Output 2.1: Family support services (medical and social rehabilitation, afterschool care, psychosocial support, etc.) are established or expanded

2.1.1 Residential institutions closed based on the closure plan of each institution

Number of institutions closed

Residential institutions that are closed according to their closure plan

Number Cumulative

Program partner/ World vision

Administrative data, program implementation documents

Annual

2.1.2 Residential institutions transformed based on the transformation plan of each institution

Number of residential institutions transformed into alternative services based on transformation plan

Residential institutions that are transformed into alternative services based on transformation plan

Number Cumulative


Administrative data, program implementation documents

Annual

2.1.3 Availability of trained staff with the capacity to provide services based on newly introduced and/or expanded service/s

Number of staff trained to deliver new services

Number of newly hired and/or old staff that are trained to deliver services

Number Cumulative


Prepost training evaluation, Service contracts with trainers, training modules, Program implementation documents

Annual

2.1.4 Social needs of the child and the family assessed



Number Cumulative

Program partner/World Vision

Program implementation documents, Copy of assessed cases

Annual

2.1.5 Individual care plans developed and followedup

Number of social cases conducted by case managers

Number of social cases managed by case managers

Number Cumulative

Regional administration, World Vision

Administrative data, Program implementation documents, case reports

Annual

Number of social cases with demonstrated

Number of cases which registered advancement

Number Cumulativ

Regional administration,

Administrative data, Program

Annual


progress e World Vision implementation documents, case reports

Percent of social cases solved by case managers

Number of cases completed Percent Regional administration, World Vision

Administrative data, Program implementation documents, case reports

Annual

2.1.6 Family support services, including medical and social rehabilitation, afterschool care, psychosocial support, and other services established/expanded in selected regions

Number of community based services established classified by type

Number of community based services established that are classified by type (at least one of each type)

Number Cumulative

Regional administration, World Vision

Administrative data, Program implementation documents, reports

Annual



OUTCOME #3: Family substitution service system is strengthened with the diversification of types of foster care, establishment of monitoring and evaluation system

3.1 Expansion and diversification of foster care policy in Armenia to be approved by the Government

Foster care policy revised and approved

Foster care policy revised and approved

Date Program partner/Save the Children

Policy document; administrative data, program implementation documents, Final evaluation

Once

3.2 Enhanced knowledge of families on being a foster family



Percentage Program partner/Save the Children

Pre/post test Annual 11

11 The responsible organization will conduct data collection before and after each training through pre and post training evaluations.


Output 3.1: A comprehensive foster care policy is developed and approved by the government

3.1.1 A comprehensive foster care policy is developed and approved by the government

Approved policy on foster care

Developed and approved policy on foster care which includes status clarification of children eligible for foster care, criteria for foster parents and monitoring and supervision of foster care


Policy document, Program implémentation documents

Once

3.1.2 Existing foster care system/framework is analyzed and compared to international standards

Foster care system in Armenia analyzed and compared to international standards

Foster care system analyzed and compared to international standards


Copy of the report Once

3.1.3 Full foster care model in line with UNICEF pilot/modeling developed and customized to the Armenian context

Full foster care model/framework developed

Foster care model/framework developed in line with UNICEF pilot/modeling guidelines and customized to the Armenian context


Program implémentation documents, report

Once

3.1.4 Foster care policy package is drafted Foster care policy drafted Foster care policy is drafted Date Program

partner/Save the Children

Admin data Once

3.1.5 System for monitoring and supervision of foster care placements is introduced

Foster care placement monitoring and supervision document adopted

Foster care placement monitoring and supervision document/plan is developed and introduced


Monitoring and supervision plan, Program implementation documents

Once

3.1.6 System for monitoring and supervision of foster care placements is operational

Foster care placement monitoring and supervision established

Foster care monitoring and supervision is established and operational


Monitoring and supervision plan, Program implementation documents

Output 3.2: Foster Care services are available for children in diverse, difficult life circumstances


3.2.1 Database for foster families developed




Report extracted from database, Program implementation document

Once

Number of foster families registered in database

Number of foster families who are registered in database

Number Cumulative

Program partner/Save the Children

Report extracted from database, program implementation document

Annual

3.2.2 Database for foster families linked to staterun database and maintained

Database of foster families linked to Nork database

Linkage of foster families’ database to staterun Nork database


Once

3.2.3 Awareness raising activities to promote foster care are organized

Number of TV programs broadcasted on local TV stations Number of userfriendly brochures for wider public developed on amended family code and diversified family care

Number of TV programs broadcasted on local TV stations and number of userfriendly brochures for wider public developed on amended family code and diversified family care

Number Cumulative


Copy of brochures and broadcasted programs (copied in a udrive)

Annual

3.2.4 Families for different types of foster care are identified and trained

Number of potential foster families trained

Number of families trained to be foster

Number Cumulative


Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents

Quarterly

3.2.5 Children from residential care facilities in targeted communities removed and placed in foster families in cooperation with the

Number of children placed in foster care Number of children

(disaggregated by gender and age) placed in foster care

Number Cumulative


Administrative data, Program implementation documents

Twice a year


Government of Armenia

Percent of children remaining in foster care after 6 months of being placed

Number of children (disaggregated by gender and age) remaining in foster care after 6 months of placement

Percentage Program partner/Save the Children

Administrative data, Program implementation documents



OUTCOME #4: Inclusive Education System is strengthened to provide quality learning for ALL children

4.1 Increased access to communitybased inclusive schools in selected regions

Number of children from 6 to18 transferred from special educational institutions to community based inclusive schools

Number of children (girls and boys) from 6 to18 transferred from special educational institutions to community based inclusive schools in target regions

Number Cumulative

Bridge of Hope Administrative data, Prepost training assessment, Program implementation documents, Final evaluation

Annual

Percent of transferred children successfully completing at least one academic year

Number of transferred children who completed at least one school year

Percentage Bridge of Hope Administrative data of line ministries, Program implementation documents, Final evaluation

Annual



Number Bridge of Hope Administrative data of line ministries, Program implementation documents, Final evaluation

Annual

4.2 Enhanced capacity of the staff of Pedagogical support centers in conducting ICF based assessments

Percent increase in the knowledge of the staff of Pedagogical Support Centers trained in supporting inclusive

Percent increase in the knowledge of the staff of Pedagogical Support Centers trained in supporting inclusive education and

Percentage Bridge of Hope Pre/post test Annual


education and conducting ICF based assessments

conducting ICF based assessments

Output 4.1: Pedagogical support centers are established to support inclusive schools in their geographical area

4.1.1 Enhanced capacity of communitybased inclusive schools in selected regions

Percent of inclusive schools in targeted regions receiving methodological and practical support from Pedagogical Support Centers

Percent of inclusive schools in targeted regions receiving methodological and practical support from Pedagogical Support Centers over the total number of inclusive schools at that point of time

Percentage Bridge of Hope Administrative data, Program implementation documents

Annual

4.1.2 The capacity of newly established pedagogical support centers are enhanced to support inclusive schools in their geographical area



Number Cumulative

Bridge of Hope Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents

Quarterly

Number of trained Pedagogical Support Centers staff conducting ICF based assessments.

Number of Pedagogical Support Centers’ staff who are trained and conduct ICF based assessments

Number Cumulative

Bridge of Hope Program implementation documents

Annual

Output 4.2: Teachers and directors for mainstream schools (in the programme areas) understand and accept the principles of inclusion and have the capacity to provide education to ALL children4.2.1 Mainstream schools in targeted regions have the capacity to provide education to children with disabilities

Number of mainstream schools in targeted regions trained on inclusive education practices

Number of mainstream schools in targeted regions trained on inclusive education practices of the total number of mainstream schools in targeted regions at that point of time

Number Cumulative

Bridge of Hope Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents

Quarterly

Program Goal Indicator Definition of Indicator Units Source/ Methodology Frequency of


Responsible Entity

Data Collection

OUTCOME #5: An enabling legal and regulatory framework is established to support Childcare Reform

5.1. Local legislation is adjusted to respond to childcare reform directions



Number Cumulative

UNICEF in Armenia Final report with legal recommendations attached

Annual

5.2 Legal and policy framework is set up to allow outsourcing of social services to nonstate providers



Number Cumulative

UNICEF in Armenia Report on adopted social assistance legal/policy amendments

Annual

5.3. Master Plan for transformation of residential care institutions is developed



Date UNICEF in Armenia Assessment report Once

Output 5.1: Recommendations for improving Legal and regulatory framework are approved by the Government, and by the National Assembly, as appropriate

5.1.1 Family code and bylaws are adjusted to provide framework for transparent adoption process

Number of legal recommendations on adoption approved by the Government of RA

Number of legal recommendations on adoption approved by the Government of RA

Number UNICEF in Armenia Program implementation documents/reports

Annual

5.1.2 Law on Social Assistance and by laws are amended to allow outsourcing of state social services to nonstate providers



Number UNICEF in Armenia Administrative data, program implementation documents

Annual

5.1.3 Policy document on deinstitutionalization, transformation of residential

Types of alternative services defined

Policy document on types of alternative services defined

Date UNICEF in Armenia Policy document, program implementation

Once


institutions and types of alternative services developed

reports

OUTCOME #6: Social norms regarding children with disabilities have changed favorably toward inclusion

6.1 People in Armenia have favorable attitude toward children with physical or intellectual disabilities

Percent increase in the positive attitude towards children with physical disabilities

Percent increase in the positive attitude towards children with physical disabilities (in comparison with the attitudes in 2013)

Percentage UNICEF in Armenia Survey data Once

Percent increase in the positive attitude towards children with intellectual disabilities

Percent increase in the positive attitude towards children with mental disabilities (in comparison with the attitudes in 2013)

Percentage UNICEF in Armenia Survey data Once

USAID Standard indicators

Indicator Definition Linkage to LongTerm Outcome or Impact

Indicator Type

Unit of Measure

Use of Indicator

Data Source and Reporting Frequency

Known Data Limitations

Baseline Timeframe

Disaggregate(s)

#1 Number of social protection policy reforms drafted, adopted or implemented with USG support

Number of social protection policy reform policies drafted, adopted, and/or implemented as a result of

This output provides the necessary foundation work for outcome level changes in social services

Output Number of policy reforms drafted, adopted or implemented

Can be used to monitor progress

Review of Project reports/Program review results Documents/Unit count of policiesAnnual reporting

A “reform” is not a defined unit of measure that permits one to meaningfully conclude that implementing twelve “reforms” is four times better than

Baseline timeframe needs to be established by the Operating Unit

None


(3.3.113) USG technical support/input. (Higher = Better) Provides information about progress in the development of a country’s social protection policies (policies drafted). It also provides proxy measures for social protection regulations (policies adopted) and for systems (policies implemented).

policies. implementing three “reforms”—or even better at all. Moreover, there is no common definition of how to define the parameter of “a reform”. Missions should include a narrative when reporting on this indicator explaining the relevance and impact of any reform.

#2 Number of vulnerable people benefitting

Number of vulnerable people receiving services from

Provides an essential link to higher order outcome

Output Number of people

Monitor Progress

Review of Project reports/Program review results Documents/Reviews of service records

Program self reporting is always subject to variability of inter rater

Baseline timeframe needs to be established by the

• Sex• Vulnerable children • Other targeted vulnerable


from USGsupported social services (3.3.28)

programs funded in whole or in part by the USG. (Higher = Better)Simple output measure to enable the roll up of vulnerable people assisted through USG resources.

measure Annual reporting reliability Operating Unit

people• Victims of torture• War victims• Persons with disabilities• Males with disabilities• Females with disabilities

#3 Number of service providers trained who serve vulnerable persons (3.3.213)

Number of service providers trained who serve vulnerable persons. (Higher = Better) Tracks improvement in the capacity to provide social assistance and social service

This measure provides an initial basis for getting to higher order outcomes.

Output Number of service providers

Tracks capacity improvement

Review of Project reports/Program review results Documents/Unit count from training recordsAnnual reporting

Program self reporting is subject to error


Sex


programs. Also serves as an indicator of a government’s commitment and capacity to protect its vulnerable populations.


#4 Number of USGassisted organizations and/or service delivery systems strengthened who serve vulnerable populations (3.3.215)

Number of organizations or service delivery systems strengthened (providing better or more efficient services) with USG resources. (Higher = Better) Tracks improvement in the capacity to provide social assistance and social service programs. Also serves as an indicator of a government’s commitment and capacity to protect its vulnerable populations.

Improvement in systems delivery is essential to long term outcome achievement

Output Number of organizations and/or delivery systems

Tracks progress towards outcomes

Review of Project reports/Program review results Documents/Unit count from project recordsAnnual reporting

Program self reporting is subject to error


• Local NGO (nonDPO); • DPO;• Schools;• Other.



ANNEX 7 OUTPUT TABLES GENERATED FROM QUANTITATIVE SURVEY

General Awareness on Inclusive Education

Table 1. Knowledge of the definition of inclusive education according to type of community (village,town) Community type Total

Urban RuralWrong answer 7 12 19Correct answer 137 156 293Total 144 168 312

Awareness coefficient 0.90 0.86 0.87

Table 2. Knowledge of the definition of inclusive education according to regions (Kapan, Goris,Sisian) Region Total

Kapan Goris SisianWrong answer 7 5 7 19Correct answer 141 63 89 293Total 148 68 96 312Awareness coefficient 0.91 0.85 0.85 0.87

Table 3. Knowledge of teachers on which children are considered as having special education needs

Correctanswer

Wronganswer

Total Index

1. Children of socially insecure families 86.90% 13.10% 100%

Kapan 139 9 148 0.88Goris 50 18 68 0.47Sisian 82 14 96 0.712. Children whose parents work and cannotengage with the child(ren)’s education

88.50% 11.50% 100%

Kapan 134 14 148 0.81Goris 57 11 68 0.68

Sisian 85 11 96 0.773. Children with physical disabilities 90.10% 9.90% 100%

Kapan 135 13 148 0.82Goris 57 11 68 0.68Sisian 89 7 96 0.854. Children with visual impairments 76.90% 23.10% 100%

Kapan 105 43 148 0.42Goris 57 11 68 0.68Sisian 78 18 96 0.63



5. Children with speech impairments 89.40% 10.60% 100%

Kapan 126 22 148 0.70Goris 65 3 68 0.91Sisian 88 8 96 0.836. Children with hearing impairments 92.00% 8.00% 100%

Kapan 133 15 148 0.80Goris 63 5 68 0.85Sisian 91 5 96 0.907. Children with psychological disorders 92.00% 8.00% 100%

Kapan 142 6 148 0.92Goris 64 4 68 0.88Sisian 81 15 96 0.698. Children without parents (orphans) 86.50% 13.50% 100%

Kapan 135 13 148 0.82Goris 56 12 68 0.65

Sisian 79 17 96 0.659. Children from large families 91.70% 8.30% 100%

Kapan 143 5 148 0.93Goris 60 8 68 0.76Sisian 83 13 96 0.7310. Children with mental disorders 95.50% 4.50% 100%

Kapan 146 2 148 0.97Goris 64 4 68 0.88

Sisian 88 8 96 0.8311. Deaf and hard of hearing children 93.30% 6.70% 100%

Kapan 143 5 148 0.93Goris 62 6 68 0.82

Sisian 86 10 96 0.7912. Blind children 91.30% 8.70% 100%

Kapan 142 6 148 0.92Goris 63 5 68 0.85Sisian 80 16 96 0.67Average index 0.78Kapan 0.83Goris 0.76Sisian 0.75

Table 4: Correct or incorrect answers to presented statement



Statements Correctanswer

Wronganswer

Difficult to say

Total

The medical model of disability is at the basis of inclusiveeducation

19.60% 75.00%

5.40% 100%

Every teacher should have the skills to identify theobservation and learning abilities of the student

98.10% 1.60% 0.30% 100%

Subject teachers do not participate in the developmentand implementation of the individual education plan.Only the parents, the class teacher, the special pedagogueand the psychologist are involved in the process

69.20% 29.80%

1.00% 100%

The individual education plan is developed for one schoolyear or based on the short-term issue.

91.70% 5.10% 3.20% 100%

Table 5: Calculation of awareness indexes according to regions

Index of knowledge

Average Kapan Goris Sisian

The medical model of disability is at the basis of inclusive education

-0.56 -0.69 -0.59 -0.4

Every teacher should have the skills to identify theobservation and learning abilities of the student

0.96 1 1 0.88

Subject teachers do not participate in thedevelopment and implementation of the individualeducation plan. Only the parents, the class teacher,the special pedagogue and the psychologist areinvolved in the process

0.39 0.41 0.47 0.29

The individual education plan is developed for oneschool year or based on the short-term issue.

0.83 0.85 0.88 0.77

Table 6: Index of general knowledge according to regions

Kapan Goris Sisian0.66 0.67 0.63

Attitude Table 7: Is Armenia ready for the introduction of the inclusive education system?

Answer %

Yes, absolutely 19 6.1%To some extent yes 202 64.7%To some extent no 51 16.3%Not ready at all 35 11.2%Difficult to say 5 1.6%Total 312 100.0%Table 8: Average score given by teachers (on a scale of 1 to 5) about the level of readiness toimplement inclusive education by the following entities/groups:



Community members, parents (attitude) 3.0Their Schools (building and amenities) 2.8Their colleges (attitude, knowledge, skills) 4.1

Table 9: How do you personally accept the idea of introducing the inclusive education system?

Answer %

I absolutely accept it 102 32.7I somewhat accept it 151 48.4I somewhat do not accept it 32 10.3I do not accept it at all 18 5.8Difficult to say 9 2.9Total 312 100.0

Table 10: Why do you think so? Answer %

Positive arguments

Respect for the rights of all children 169 54.2

The inclusive system contributes to the integration of the society, has aneducational function

17 5.4

Children feel good; psychological aspect 12 3.8Negative arguments

Negative impact on classroom; the learning process (the lesson) is disrupted 34 10.9

No corresponding conditions (physical, adequate attention during thelesson, mentality of other children’s parents)

35 11.2

Those children require individual approach 18 5.8In some cases special schools convey more useful knowledge thanmainstream schools

7 2.2

They stand out anyway and their issue is emphasized 5 1.6Inefficient spending of resources (finances, time) 5 1.6

Difficult to say 10 3.2Total 312 100.0

Table 11. Teachers’ perception about which children should be considered as having special educationneeds

1. Yes 2. No Total



9. Children from large families 16.0% 84.0%

100%

2. Children whose parents work and cannot engage with theireducation

17.3% 82.7%

100%

1. Children of socially insecure families 21.5% 78.5%

100%

8. Children without parents (orphans) 23.4% 76.6%

100%

4. Children with visual impairments 79.8% 20.2%

100%

7. Children with psychological disorders 81.1% 18.9%

100%

12. Blind children 81.1% 18.9%

100%

11. Deaf and hard of hearing children 84.0% 16.0%

100%

10. Children with mental disorders 84.3% 15.7%

100%

6. Children with hearing impairments 88.5% 11.5% 100%

5. Children with speech impairments 89.7% 10.3%

100%

3. Children with physical disabilities 90.4% 9.6% 100%

Table 12: Impacts of the introduction of the inclusive education system on the following groups/structures:

Ab

solu

tely

n

egat

ive

Som

ewh

at n

egat

ive

Nei

ther

pos

itiv

e n

or n

egat

ive

Som

ewh

at p

osit

ive

Ab

solu

tely

pos

itiv

e

Dif

ficu

lt t

o sa

y

Tot

al

1. Children with physicalimpairments

1.6% 5.8% 6.7% 43.6% 41% 1.3% 100%

2. Children with mild mentalimpairments

3.5% 7.1% 11.5% 42.6% 34.3% 1% 100%

3. Children with severe mentaldisorders

17.3% 10.9% 22.1% 22.8% 23.4% 3.5% 100%

4. Children of socially insecurefamilies

4.8% 3.6% 37.8% 26.9% 24% 2.9% 100%

5. Children excluded from theinclusive education system

14.1% 20.9% 22.4% 21.5% 19.6% 1.5% 100%

6. Parents of children excludedfrom the inclusive educationsystem

14.7% 22.4% 24% 24% 11.5% 3.4% 100%

7. Parents of children in theinclusive education system

2.6% 2.6% 7.9% 31.1% 54.8% 1% 100%



8. Teachers 3.5% 11.6% 19.2% 29.5% 34.9% 1.3% 100%

9. School administration 0.3% 4.8% 15.4% 26.3% 48.7% 4.5% 100%

10. Community 0.6% 2.9% 20.5% 32.1% 39.7% 4.2% 100%

11. Society/country 1.3% 3.2% 19.9% 33% 37.5% 5.1% 100%

12. Stakeholders interested in theintroduction of the system/Ministry of Education, NationalInstitute of Education, other …/

0.3% 1.6% 8.3% 18.6% 63.8% 7.4% 100%

Table 13: What do your colleagues think of the children in inclusiveeducation? (up to 3 options for answers were available)

Answer %

It is very good that they study with all other children 117 19.6%Working with them is very difficult 98 16.4%It would be better if relevant specialists worked with them 95 15.9%

They disrupt the lesson 80 13.4%They make significant progress 67 11.2%They are in no way different from other children 55 9.2%It would be better if they attended corresponding educational institutions 27 4.5%

Working with them is a pleasure 18 3.0%They pay more attention in the classroom 11 1.8%It’s impossible to supervise them 11 1.8%The human value of the child increases 5 .8%We all have to help them 4 .7%Special plans are needed 3 .5%Difficult to say 3 .5%There are no relevant conditions 2 .3%The student exploits the fact of having an impairment 1 .2%Total 597 100.0%

Practice

Table 14: What does a teacher need to work more effectively withinthe inclusive education system?

Answer %

Didactic materials 87 27.9Corresponding knowledge (professional education), skills andtrainings

61 19.6

Personal qualities: patience, strong will power 33 10.6Additional payment, salary 32 10.3Permanent availability of a corresponding specialist 28 9.0



Time 14 4.5Methodological guideline 13 4.2Experience 11 3.5Material and technical base, facilities, additional classroom 10 3.2

Verbal rewards 4 1.3Fewer number of students in class 3 1.0Promotion 2 .6Difficult to say 14 4.5Total 312 100.0

56.1% of respondents have noted they had a child from the inclusive education system in their class, while 43.9% saidthey did not. The rest of the questions were answered by those who taught children with special education needs

Table 16: Which of the following do youagree with?

Completelyagree

Somewhatagree

Somewhatdisagree

Do notagree atall

Difficult tosay

Total

I spend a bigger part of the lessonworking with the child(ren) with specialeducation needs, others

10.9%

14.8% 29.1% 44.6% 0.6% 100%

I work with children with specialeducation needs also outside of the mainschool hours

14.3%

20.6% 10.3% 53.1% 1.7% 100%

I spend more time preparing for the workwith the children with special educationneeds, than for work with other children

33.1%

32.6% 13.1% 20.6% 0.6% 100%

Table 17: In the classroom the seat of the child enrolled in the inclusive education system isdecided according to:

%

The child’s needs 76.0The child’s free will 17.7The needs of the class 5.1Difficult to say 1.1Total 100.0

Table 18: Attitude of peers towards children in the inclusive education system %

Very good 22.9Good 58.3Poor 9.7Very poor 8.6Difficult to say .6



Total 100.0

Table 19: Frequency of issues with children enrolled in the inclusive education system %

Very often 4.6Often 17.1Sometimes 48.6Rarely 14.3Never 15.4Total 100.0

Table 20: Who do children enrolled in the inclusive education system mostly interact with inclass/school?

%

All children in class 56.5%Only among themselves 2.2%With neighbor’s children attending the same school 6.9%With relatives attending the same school 6.5%With children who treat them nicely 26.7%With no one 1.3% Total 100.0%

Table 21: Do you collaborate with your colleagues when working with children in the inclusive education system?

%

Yes, always 74.9

Sometimes yes 23.4

Mostly no 1.7

Total 100.0

Table 22: How frequently do you collaborate in relation to issues of children in the inclusive educationsystem?

Veryfrequently

Frequently

Sometimes

Rarely Never Total

With the Teacher’s Assistant or members ofthe multidisciplinary group

29.7% 33.1% 25.0% 3.5% 8.7% 100%



With staff of pedagogical and psychologicalsupport center

7.6% 22.7% 36.0% 13.4% 20.3% 100%

With representatives of the NationalInstitute of Education

4.1% 10.5% 25.6% 16.9% 43.0% 100%

With parents of children enrolled in theinclusive education system

21.5% 41.3% 27.3% 6.4% 3.5% 100%

With the parents of other children 10.5% 22.7% 28.5% 15.7% 22.7% 100%

Table 23: Changes due to introduction of inclusive educationYes Somewha

t yesSomewhat no

No Difficult to say

Total

The attitude of children towards theirpeers included in the inclusive educationsystem has changed

64.0%

30.8% 0.6% 4.0%

0.6% 100%

The attitude of children enrolled in theinclusive education system towards otherchildren has changed

50.9%

37.7% 1.1% 6.9%

3.4% 100%

The behavior of children in inclusiveeducation has changed

42.3%

49.7% 2.9% 5.1%

0.0% 100%

The performance of children in inclusiveeducation has changed

35.4%

50.3% 7.4% 6.3%

0.6% 100%

Relevance and Efficiency of Trainings



Table 24: Have you participated in trainings on inclusive education during the last 2 years?

%

Yes 76.9No 23.1Total 100.0

Respondents who said they participated in trainings have answered the rest of the questions (total number n=240):

Table 25: Date of the training (year)

Total

Trainings of teachers Training of teachingassistants

Difficult to say

Difficult to say 4 0 1 5

2015 139 9 2 1342016 93 15 0 842017 50 26 1 47Total 233 33 4 2701

Respondents who said they took part in trainings organized by BoH (240 persons) noted whether they received certificates

Table 26: Received a certificate %

Yes 80.4

No 19.6

Total 100.0

Table 27: Were the trainings useful for you? %

Very useful 70.0Somewhat useful 27.1Not so useful 1.7Not useful at all 1.3Total 100.0

Table 28: Which part(s) of the training did you attend? %

All of it 96.7

More than half 2.9

Half .4

1 Respondents could give more than 1 answer, that is why the total quantity is 270



Total 100.0

Table 29: Which part of the teachers still need trainings on inclusive education? %

All of them 12.5

More than half 7.5Half 5.0Fewer than half 14.6Some of them 20.8No one 29.2Difficult to say 10.4Total 100.0

Table 30: Do you utilize the knowledge gained from the trainings? %

Yes, always 40.0Mostly yes 44.6

Mostly no 5.8

No 9.6

Total 100.0

Table 31: Average score (on a scale of 1 to 5, where 5 is the highest and 1 is the lowest mark) to thefollowing:Efficiency of the training 4.2Usefulness of the training for teacher 4.2Level of organization of training (venue, time period, duration of training, friendliness, etc) 4.3Quality of teaching (whether things were explained in a proper, simple and comprehensiblemanner, use of interactive techniques, etc.)

4.6

Training contents (whether all topics were discussed, if training material containednovelties, etc.)

4.5

Availability of Supporting Resources

Table 32: Do you conduct students’ needs assessments? Answer %

Yes 189 60.6No 123 39.4Total 312 100.0



Table 33: If yes, then who do you collaborate with for that purpose (opportunity for severaloptions)?

%

With corresponding specialists of school 76.7%

With local NIE staff 5.3%With staff of local pedagogical and psychological support center 29.1%

Total 111.1%

Table 34: Do representatives of Pedagogical and Psychological Support Center work withthe school?

%

Yes 72.1No 21.8Difficult to say 6.1Total 100.0

Table 35: What issues do you collaborate on? %

Conducting a needs assessment for the child 23.4%Working with children enrolled in the inclusive education system 23.1%

Providing consultancy to the school on issues related to children enrolled in the inclusiveeducation system

22.1%

Developing individual education plans 15.1%Developing support plan for the child 14.0%Difficult to say 2.3%They don’t do anything .2%Total 100.0%

Table 36: Does the assessment system that has been introduced allow for the objectiveidentification and assessment of children with issues?

%

Yes, absolutely 23.6Mostly yes 56.4Mostly no 12.9Not at all 3.6Difficult to say 3.6Total 100.0

Table 37: Recommendations on improving cooperation between theschool and the Center

Answer %



Frequent and regular meetings, consultancy 86 27.6

Organization of joint seminars 44 14.1Allocation of work, mutual arrangements 12 3.8Availability of relevant conditions at school /specialists, classroom,didactic materials/

9 2.9

Working with parents and other students 9 2.9Engagement of all teachers and not just teaching assistants 7 2.2

Further cooperation on working with children 7 2.2

Longer and more detailed assessment of children 7 2.2

There is already a high level of cooperation 6 1.9A bigger number of professional specialists at the center 5 1.6Implementing supervision 1 .3Difficult to say 119 38.1Total 312 100.0

Table 38: Gender of respondent %

Male 7.7Female 92.3Total 100

Table 39: What subject do you teach? %

I don’t teach 8.3

Elementary school teacher 21.5

Languages 30.4

Humanities/social science, religion, history, geography, military science 12.8

Physics and mathematics/ computer science, physics, mathematics, geometry 16.7

Science/chemistry, biology, environment 5.4

Physical education, labor, technology, music, chess 4.8

Total 100

(RFP Number: LRFP-2017-9132946) Mid-term Evaluation Final ... · Services as Part of Child Care...

Documents

Transcript of (RFP Number: LRFP-2017-9132946) Mid-term Evaluation Final ... · Services as Part of Child Care...