Written by ICF

February 2016

European Social Fund (ESF)

2007 ndash 2013 ex-post

evaluation Supporting the

integration of disadvantaged

groups into the labour market

and society

Executive Summary

EUROPEAN COMMISSION

Directorate-General for Employment Social Affairs and Inclusion

Directorate G mdash Audit and evaluation

Unit EMPL G4 - Evaluation amp Impact Assessment

E-mail EMPL-G4-UNITeceuropaeu

European Commission

B-1049 Brussels

EUROPEAN COMMISSION

Directorate-General for Employment Social Affairs and Inclusion

European Social Fund (ESF)

2007-2013 ex-post

evaluation Supporting the

integration of disadvantaged

groups into the labour

market and society

Executive Summary

February 2016

LEGAL NOTICE

This document has been prepared for the European Commission however it reflects the views only of the authors and the Commission cannot be held responsible for any use which may be made of the information

contained therein

More information on the European Union is available on the Internet (httpwwweuropaeu)

Luxembourg Publications Office of the European Union 2016

copy European Union 2016

Reproduction is authorised provided the source is acknowledged

Please quote this report as ICF (2016) 2007-2013 Ex-post evaluation on supporting the integration of disadvantaged groups into the labour market and society for the European

Commission Directorate-General Employment Social Affairs and Inclusion

The opinions expressed are those of the Contractor only and do not represent the Commissionrsquos official position

Europe Direct is a service to help you find answers to your questions about the European Union

Freephone number ()

00 800 6 7 8 9 10 11

() The information given is free as are most calls (though some operators phone

boxes or hotels may charge you)

EUROPEAN COMMISSION

Directorate-General for Employment Social Affairs and Inclusion

Directorate G mdash Audit and evaluation

Unit EMPL G4 - Evaluation amp Impact Assessment

E-mail EMPL-G4-UNITeceuropaeu

European Commission

B-1049 Brussels

EUROPEAN COMMISSION

Directorate-General for Employment Social Affairs and Inclusion

European Social Fund (ESF)

2007-2013 ex-post

evaluation Supporting the

integration of disadvantaged

groups into the labour

market and society

Executive Summary

February 2016

LEGAL NOTICE

This document has been prepared for the European Commission however it reflects the views only of the authors and the Commission cannot be held responsible for any use which may be made of the information

contained therein

More information on the European Union is available on the Internet (httpwwweuropaeu)

Luxembourg Publications Office of the European Union 2016

copy European Union 2016

Reproduction is authorised provided the source is acknowledged

Please quote this report as ICF (2016) 2007-2013 Ex-post evaluation on supporting the integration of disadvantaged groups into the labour market and society for the European

Commission Directorate-General Employment Social Affairs and Inclusion

The opinions expressed are those of the Contractor only and do not represent the Commissionrsquos official position

Europe Direct is a service to help you find answers to your questions about the European Union

Freephone number ()

00 800 6 7 8 9 10 11

() The information given is free as are most calls (though some operators phone

boxes or hotels may charge you)

EUROPEAN COMMISSION

Directorate-General for Employment Social Affairs and Inclusion

European Social Fund (ESF)

2007-2013 ex-post

evaluation Supporting the

integration of disadvantaged

groups into the labour

market and society

Executive Summary

February 2016

LEGAL NOTICE

This document has been prepared for the European Commission however it reflects the views only of the authors and the Commission cannot be held responsible for any use which may be made of the information

contained therein

More information on the European Union is available on the Internet (httpwwweuropaeu)

Luxembourg Publications Office of the European Union 2016

copy European Union 2016

Reproduction is authorised provided the source is acknowledged

Please quote this report as ICF (2016) 2007-2013 Ex-post evaluation on supporting the integration of disadvantaged groups into the labour market and society for the European

Commission Directorate-General Employment Social Affairs and Inclusion

The opinions expressed are those of the Contractor only and do not represent the Commissionrsquos official position

Europe Direct is a service to help you find answers to your questions about the European Union

Freephone number ()

00 800 6 7 8 9 10 11

() The information given is free as are most calls (though some operators phone

boxes or hotels may charge you)

February 2016

LEGAL NOTICE

This document has been prepared for the European Commission however it reflects the views only of the authors and the Commission cannot be held responsible for any use which may be made of the information

contained therein

More information on the European Union is available on the Internet (httpwwweuropaeu)

Luxembourg Publications Office of the European Union 2016

copy European Union 2016

Reproduction is authorised provided the source is acknowledged

Please quote this report as ICF (2016) 2007-2013 Ex-post evaluation on supporting the integration of disadvantaged groups into the labour market and society for the European

Commission Directorate-General Employment Social Affairs and Inclusion

The opinions expressed are those of the Contractor only and do not represent the Commissionrsquos official position

Europe Direct is a service to help you find answers to your questions about the European Union

Freephone number ()

00 800 6 7 8 9 10 11

() The information given is free as are most calls (though some operators phone

boxes or hotels may charge you)

February 2016 1

Key findings EU-27 (as of end 2013)

Between 2007 and 2013 the ESF contributed to EU objectives in the social

inclusion field and also played a role in countering the effects of the economic crisis

euro115 billion of EU and national funding were allocated to the Social

inclusion field or 10 of total ESF investment in all OPs

Thus far euro71 billion of certified eligible expenditure paid by beneficiaries

(actual expenditure) a 612 average financial implementation rate (actual

expenditureallocated expenditure) varying between 21 and 93 across the Member States This compares to an overall rate for all ESF expenditure of

575

ESF plays an important role in enhancing and modernising labour market policy delivery mechanisms for disadvantaged groups and contributed to

achievement of EU2020 priorities

61 million participations were reported of which 53 were women

53 had a low level of educational attainment (ISCED 1-2)

47 of participants were unemployed ndash of which 22 were long-term unemployed and 34 were inactive

24 were young people 11 were classified as migrants 9 as minorities 16 as disabled and 21 as lsquoother disadvantagedrsquo (covering a broad range of

disadvantages defined at nationalregional level)

When calculated as a share of the overall population of disadvantaged groups in the EU their coverage in ESF social inclusion priority axes ranged from

21 for the long term unemployed to 03 for disabled individuals

At least 13 million results in ESF SI activities in 27 Member States have been reported to December 2013 Of these 499000 secured employment and

244000 achieved a qualification while 557000 achieved other positive

results

The overall success rate was 33 when comparing only the participations covered by the result indicators This is a partial picture as not all results

could be aggregated Notably this excludes soft results which are

particularly relevant given the distance to labour market of disadvantaged groups

February 2016 2

Background to the evaluation and Social Inclusion (SI) target groups

This evaluation covered the time period until the end of 2013 with eligibility of

expenditure continuing until the end of 2015 It included the analysis of expenditure

outputs and results across 27 Member States taking account of available evaluations and a detailed study of a total of 58 interventions in eight countries1

The ESF Regulationrsquos social inclusion priority aims at reinforcing the social inclusion of

disadvantaged people with a view to their sustainable integration in employment and

combating all forms of discrimination in the labour market (notably through employability measures access to vocational education and training) As such these

measures were intended to contribute to the key targets of the Lisbon Strategy and

subsequently the EU 2020 goals linked to the reduction of poverty and social

exclusion

In addition the 2008 Commission Recommendation on the Active Inclusion of People Excluded from the Labour Market2 called on Member States to lsquodesign and implement

an integrated comprehensive strategy for the active inclusion of people excluded from

the labour market combining adequate income support inclusive labour markets and

access to quality servicesrsquo

Overall alignment of the OPs with the Lisbon and EU 2020 priorities the 2008

Recommendation and the Country Specific Recommendations was high

The 2007-2013 ESF Operational Programmes were designed during a period of

improved economic performance and rising employment levels in the EU whereas implementation took place during a period of economic downturn The crisis

particularly impacted upon the most disadvantaged groups in the labour market

The scale and policy contribution of ESF investment to social inclusion

An initial allocation of euro115 billion (of EU and national funds) was identified in OPs

with dedicated SI Priority Axes This amounted to 10 of total ESF investment The highest share of resources allocated to SI interventions from the total ESF allocation

was found in competitiveness regions with an average of 22 whereas in

convergence regions it reached 16

Member States had significant leeway over defining the overall architecture of ESF programming where and how social inclusion activities were included and how much

funding was dedicated to this priority field As a result this summary uses data

only for OPs with dedicated SI Priority Axes thus underestimating SI activity

supported by the ESF3

At the end December 2013 total certified ESF SI public expenditure amounted to

euro71 billion (a financial implementation rate of 612 but with some interventions

still ongoing) This implementation rate is higher than the overall implementation rate

(575) which is significant given the complexity of the theme

Countries where the ESF had the largest impact on SI investment were in BG LT and RO During the crisis in 12 countries changes to OPs were requested to shift

greater resources to disadvantaged groups including the long term unemployed

facing significant labour market integration challenges disabled individuals migrants

1 Austria Cyprus Finland Lithuania Luxembourg the Netherlands Romania and the UK The period covered by the evaluation was from 2007 ndash 2013 It relied on data and evaluations provided by Member

States and the European Commission as well as original research This ex-post evaluation builds on the

results of preparatory study (see httpwwwlseacukbusinessAndConsultancyLSEConsultingpdfESF-2007-2013pdf) the EEN and the previous 2000-2006 ESF ex-post evaluation 2 Commission Recommendation of the 3 October 2008 on the Active Inclusion of People Excluded from the Labour Market (2008867EC) httpeur-lexeuropaeulegal-

contentENTXTHTMLuri=CELEX32008H0867ampfrom=EN 3 Other OPs are covered by the two other thematic evaluations

February 2016 2

Background to the evaluation and Social Inclusion (SI) target groups

This evaluation covered the time period until the end of 2013 with eligibility of

expenditure continuing until the end of 2015 It included the analysis of expenditure

outputs and results across 27 Member States taking account of available evaluations and a detailed study of a total of 58 interventions in eight countries1

The ESF Regulationrsquos social inclusion priority aims at reinforcing the social inclusion of

disadvantaged people with a view to their sustainable integration in employment and

combating all forms of discrimination in the labour market (notably through employability measures access to vocational education and training) As such these

measures were intended to contribute to the key targets of the Lisbon Strategy and

subsequently the EU 2020 goals linked to the reduction of poverty and social

exclusion

In addition the 2008 Commission Recommendation on the Active Inclusion of People Excluded from the Labour Market2 called on Member States to lsquodesign and implement

an integrated comprehensive strategy for the active inclusion of people excluded from

the labour market combining adequate income support inclusive labour markets and

access to quality servicesrsquo

Overall alignment of the OPs with the Lisbon and EU 2020 priorities the 2008

Recommendation and the Country Specific Recommendations was high

The 2007-2013 ESF Operational Programmes were designed during a period of

improved economic performance and rising employment levels in the EU whereas implementation took place during a period of economic downturn The crisis

particularly impacted upon the most disadvantaged groups in the labour market

The scale and policy contribution of ESF investment to social inclusion

An initial allocation of euro115 billion (of EU and national funds) was identified in OPs

with dedicated SI Priority Axes This amounted to 10 of total ESF investment The highest share of resources allocated to SI interventions from the total ESF allocation

was found in competitiveness regions with an average of 22 whereas in

convergence regions it reached 16

Member States had significant leeway over defining the overall architecture of ESF programming where and how social inclusion activities were included and how much

funding was dedicated to this priority field As a result this summary uses data

only for OPs with dedicated SI Priority Axes thus underestimating SI activity

supported by the ESF3

At the end December 2013 total certified ESF SI public expenditure amounted to

euro71 billion (a financial implementation rate of 612 but with some interventions

still ongoing) This implementation rate is higher than the overall implementation rate

(575) which is significant given the complexity of the theme

Countries where the ESF had the largest impact on SI investment were in BG LT and RO During the crisis in 12 countries changes to OPs were requested to shift

greater resources to disadvantaged groups including the long term unemployed

facing significant labour market integration challenges disabled individuals migrants

1 Austria Cyprus Finland Lithuania Luxembourg the Netherlands Romania and the UK The period covered by the evaluation was from 2007 ndash 2013 It relied on data and evaluations provided by Member

States and the European Commission as well as original research This ex-post evaluation builds on the

results of preparatory study (see httpwwwlseacukbusinessAndConsultancyLSEConsultingpdfESF-2007-2013pdf) the EEN and the previous 2000-2006 ESF ex-post evaluation 2 Commission Recommendation of the 3 October 2008 on the Active Inclusion of People Excluded from the Labour Market (2008867EC) httpeur-lexeuropaeulegal-

contentENTXTHTMLuri=CELEX32008H0867ampfrom=EN 3 Other OPs are covered by the two other thematic evaluations

February 2016 3

and specifically also young people In another 3 countries changes were made in the

prioritisation of target groups towards those significantly affected by unemployment as

a result of the crisis

It was possible to classify the wide variety of actions supporting the labour market

integration of disadvantaged individuals

including wider systemic measures into 4

clusters

Actions to support labour market integration

included preventative initiatives such as

actions to reduce drop-out rates of disadvantaged

young people In addition supporting and enabling actions were provided to address

issues such as specialist support for individuals

with alcohol and drug issues confidence building

measures for individuals who have been out of

the labour market for a long time and support for families affected by long-term social exclusion

Furthermore basic and vocational training

measures for disadvantaged groups were

supported by in-depth counselling (most Member States) and targeted hiring subsidies or sheltered

employment offered Often such measures were combined into holistic pathways

thus reflecting the increased emphasis on tailored approaches for disadvantaged

job seekers

Systemic actions focused on the upgrading of the operation of labour market

institutions addressing discrimination and awareness raising on equal opportunities

and building the development infrastructure to support intermediate labour markets

and the social economy

Some of the headline figures for ESF SI investment across the EU-27 Member States

are as follows

61 million participations were supported with ESF SI funding linked to

dedicated SI OP priority axes4 According to Annex XXIII data5 a total of 18

million individuals from disadvantaged groups were supported by activities under all OPs and priority axes showing that a significant share of social

inclusion interventions were lsquomainstreamedrsquo in access to employment or human

capital interventions

Of the 61 million participants 53 were women (66 across Convergence OPs and 44 across the Regional Competitiveness OPs) 24 of participants

were young people aged between 15 and 24 and 6 were aged 55-64

Upon entering the measures 47 of participants were unemployed and

34 were inactive6 Of the unemployed 22 were long-term unemployed

53 of participants had a low level of educational attainment (ISCED 1-

2) 28 had achieved ISCED level 3 8 ISCED level 4 and 10 had a higher

education qualification (ISCED 5-6)

4 ESF data effectively records lsquoparticipationsrsquo so they may be some multiple counting of individuals taking

part in measures 5 This Annex to the ESF Regulation specifies how data on participants in ESF funded actions should be

recorded Amongst other things it requires a breakdown of participants by vulnerable groups (eg migrants minorities disabled etc) and labour market status 6 The remainder were previously employed

Cluster 1 ndash supporting and enabling

actions (including measures to create conditions to support participation eg

debt counselling language training for

migrants etc)

Cluster 2 ndash actions with an employment

objective (split between advice guidance

and training ndash cluster 2a actions with employment as an output ndash cluster 2b

and actions to sustain employment ndash cluster 2c)

Cluster 3 ndash pathway approaches (linking

advice guidance training ongoing employment support etc)

Cluster 4 ndash systemic measures influencing systems institutional or

cultural contexts (eg anti-discrimination

measures expanding or upgrading labour market institutions)

February 2016 4

11 of participants were classified as migrants 9 as minorities

(including Roma) 16 as disabled and 21 as lsquoother disadvantagedrsquo

However these figures have to be treated with caution because of probable

under-reporting of disadvantaged profiles This relates to different national definitions applied by the Member States and to legal obstacles to collecting

data in Member States The actual number of disadvantaged individuals

supported is therefore potentially significantly higher

Concerning outputs for entities according to OP-specific indicators at least 70200 enterprises and 2100 entities delivering public services were

supported while 1500 structures in the local economy were created or

supported7

Effectiveness efficiency of ESF SI interventions

At least 13 million employment qualification and other categories of positive results were achieved by participants in ESF SI activities to December 20138 Of

these 499000 secured employment and 244000 achieved a qualification While some

individuals could achieve multiple results often interventions did not measure results

beyond participation The aggregated results ratio was 33 (228 for employment and 355 for qualifications) This is based on aggregated results divided by

participations excluding the priority axes where no results were aggregated Looking

across the clusters of activity defined for the evaluation findings from the 58

interventions reviewed9 in-depth showed that 42 of participants secured employment 8 qualification and 26 another positive result reflecting their more

advanced stage of implementation and characteristics It should be noted that on

average participants in the in-depth actions were less likely to be classified as

disadvantaged relative to participants in all relevant SI actions across the EU

For many of the participants facing greatest disadvantage and at most distance from

the labour market lsquosoftrsquo results represent a key achievement of the ESF

provision These include personal (eg improved confidence and motivation)

attitudinal (eg more positive view of further training and employment opportunities)

and behavioural results (eg more active job search) However only around 25 of the interventions reviewed in-depth indicated that soft results had been measured

Among these interventions between 35 and 80 of participants had achieved such

soft results

The experience of the implemented interventions shows that long-term unemployed individuals without other significant personal challenges benefit most from (short) re-

training measures and the delivery of job search skills Disabled participants are often

successfully supported through sheltered employment and follow-up support as well

as the delivery ICT or workplace support structures (depending on the nature of their disability) Migrants lacking core languages skills are often provided with language

training which is in some cases combined with the delivery of vocational skills

recognised in nationallocal labour market lsquoOther disadvantaged groupsrsquo with complex

problems (eg substance abuse issues mental or physical health issues etc) benefit

most from holistic interventions which combine outreach (peer) mentoring with targeted personal and vocational support and follow-up support post job placement

7 OP-specific available indicators only capture a fraction of all outputs achieved for entities This type of

indicator was not collected for most OPs in addition some indicators were very specific to the OP and

cannot be included in one of the categories identified (number of enterprises supported number of entities delivering public services supported number of structures of the local economy created or supported) 8 This is based on information from 48 of available indicators which it was possible to aggregate Other

indicators did not fall into the selected categories or could not be associated with participant numbers This number therefore underestimates the total results achieved 9 These in-depth interventions constitute 10 of allocated SI expenditure in the eight in-depth countries

February 2016 5

Subsidy measures can also be shown to be effective for a range of target groups

including disabled individuals and long-term unemployed job seekers

Across all Member States for ESF SI the average cost per participation was

around euro1200 whereas among the interventions assessed in depth the average cost was euro1300 This slightly higher cost could again be linked to higher financial

implementation rates to date

Sustainability

Few of the interventions reviewed in-depth measured the sustainability of results for participants (eg after 6 or 12 months following an ESF intervention) Where such

information was available results varied considerably from over 70 sustained after 6

months to below 20 depending on the context of the groups targeted and the socio-

economic settings in which they were located

For the interventions themselves sustainability is relatively high with just over 80 either continuing or intending to continue their activities Of these around 65

continuedexpected to continue to deliver support to disadvantaged groups with a

mixture of ESF and national funding with the remaining continuingintending to

continue with national local or regional funding

Gender sensitivity of ESF SI interventions

Despite the increased emphasis on gender mainstreaming considerations in this

programming period the availability of meaningful gender sensitivity

assessments (notably of gendered result indicators) or evidence of their

implementation was variable Gender assessments were limited to indicating that interventions lsquosupported gender equalityrsquo or were lsquogender neutralrsquo Around 75 of the

interventions assessed took this approach In terms of participation there was an

almost equal representation of men and women in ESF SI funded interventions (53

female) Similarly 54 of participants who had achieved a positive result and whose gender is known were women

Community Added Value of ESF SI investments

Volume effects allow Member States to add to existing SI actions and support

a greater number of participants than would have been possible otherwise

They played a particular role in the economic context to counter the impacts of the

economic crisis as previously indicated

Volume aspects were the most clearly identifiable among SI interventions

assessed in depth These were reported for 35 of the 58 interventions studied and

allowed Member States to add to existing SI actions and support a greater number of participants than would have been possible otherwise

Scope effects revolved around the ability to offer more tailored services to

specific target groups (28 of the 58 interventions) which would otherwise have

access only to mainstream services not tailored to their specific ndash and often more intensive - needs Examples include support to address gambling or substance abuse

issues targeted health or family support or the delivery of tailored training (in co-

ordination with a specific employer)

Role effects were visible in projects developing supports or activities which acted as a model for other local provision or were subsequently mainstreamed (17 of the 58

interventions)

Process effects (14 of the 58 interventions) mainly revolved around the

establishment of successful partnerships at the local level to assist disadvantaged

groups

February 2016 6

ESF was used as core funding for reforms (in CZ EL and HU) or supplementary

funding to reforms (in EE FR MT and PL) linked to the improvement of delivery

systems and methods and the testing of new approaches

The socio-economic impact for ESF social inclusion investment

Impacts were most visible at the micro level where 61 million individuals were

supported with ESF SI funding linked to dedicated SI OP priority axes At the

intermediate level systemic actions helped to boost the capacity of labour market

institutions This included support for mainstream institutions such as PES but also for the establishment of new structures (such as assessment systems to test the work

capability of disabled individuals) and assistance to build better networks and

structures in intermediate labour markets in order to support more effectively job

seekers with complex problems

ESF supported some significant national programmes and reforms to promote the integration of disadvantaged groups into the labour market (including through the

development of institutions and support systems such as the intermediate labour

market) This was mainly achieved by providing the infrastructure and resources

necessary to deliver individualised tailored services to some of the most vulnerable jobseekers

A statistically significant correlation can be observed between the change in

the share of the people at risk of poverty or social exclusion as well as the

share of low-skilled at risk of poverty and social inclusion and the ESF SI investment at the country level This suggests that countries where the

significance of ESF in the national context is greater made more progress in reducing

the number of people at risk of poverty of social exclusion10

Key Lessons

This report draws lessons for policy and target group choices effective programming and implementation as well as robust monitoring and evaluation These include the

following

During the crisis ESF support proved sufficiently flexible to adjust to emerging

policy requirements Targeted tailored interventions which address the additional barriers to

labour market entry facing disadvantaged groups with different local expert

agencies working together (eg public employment services local authorities

non-governmental organisations etc) prove to be effective for ESF SI target groups

Follow-up support to stabilise new employment relationships achieved is also

critical and should be factored into delivery planning

Effective approaches to outreach and engagement particularly for the most

disengaged target groups and those suspicious of public agencies (such as addicts and ex-offenders) were key as was the provision of often intensive

support throughout the intervention

In a number of countries where systemic approaches are emphasised ESF SI

investment plays an important role in enhancing and modernising labour market policy delivery mechanisms for disadvantaged groups

Considering the importance of soft results for disadvantaged groups in the

labour market a comprehensive assessment of the benefits of ESF investment

requires the development of consistent approaches to assess the achievement of such results by participants in ESF interventions This is of particular

10 This relates primarily to the Central and Eastern European countries However this correlation does not imply a causality as many other factors beyond ESF investment have an impact on the evolution of poverty

risk at Member State level

February 2016 7

relevance for SI interventions where direct employment results are often more

remote because of the nature of the target group

Monitoring systems and evaluation strategies should be enhanced to

provide more robust and comparable results The 2014-2020 ESF regulations and guidance already address some elements (eg strengthening

requirements for data collection and evaluation)

doi102795625537

[KR-0

2-1

6-2

10-E

N-N

]

R-02-16-210-EN-N

February 2016 4

11 of participants were classified as migrants 9 as minorities

(including Roma) 16 as disabled and 21 as lsquoother disadvantagedrsquo

However these figures have to be treated with caution because of probable

under-reporting of disadvantaged profiles This relates to different national definitions applied by the Member States and to legal obstacles to collecting

data in Member States The actual number of disadvantaged individuals

supported is therefore potentially significantly higher

Concerning outputs for entities according to OP-specific indicators at least 70200 enterprises and 2100 entities delivering public services were

supported while 1500 structures in the local economy were created or

supported7

Effectiveness efficiency of ESF SI interventions

At least 13 million employment qualification and other categories of positive results were achieved by participants in ESF SI activities to December 20138 Of

these 499000 secured employment and 244000 achieved a qualification While some

individuals could achieve multiple results often interventions did not measure results

beyond participation The aggregated results ratio was 33 (228 for employment and 355 for qualifications) This is based on aggregated results divided by

participations excluding the priority axes where no results were aggregated Looking

across the clusters of activity defined for the evaluation findings from the 58

interventions reviewed9 in-depth showed that 42 of participants secured employment 8 qualification and 26 another positive result reflecting their more

advanced stage of implementation and characteristics It should be noted that on

average participants in the in-depth actions were less likely to be classified as

disadvantaged relative to participants in all relevant SI actions across the EU

For many of the participants facing greatest disadvantage and at most distance from

the labour market lsquosoftrsquo results represent a key achievement of the ESF

provision These include personal (eg improved confidence and motivation)

attitudinal (eg more positive view of further training and employment opportunities)

and behavioural results (eg more active job search) However only around 25 of the interventions reviewed in-depth indicated that soft results had been measured

Among these interventions between 35 and 80 of participants had achieved such

soft results

The experience of the implemented interventions shows that long-term unemployed individuals without other significant personal challenges benefit most from (short) re-

training measures and the delivery of job search skills Disabled participants are often

successfully supported through sheltered employment and follow-up support as well

as the delivery ICT or workplace support structures (depending on the nature of their disability) Migrants lacking core languages skills are often provided with language

training which is in some cases combined with the delivery of vocational skills

recognised in nationallocal labour market lsquoOther disadvantaged groupsrsquo with complex

problems (eg substance abuse issues mental or physical health issues etc) benefit

most from holistic interventions which combine outreach (peer) mentoring with targeted personal and vocational support and follow-up support post job placement

7 OP-specific available indicators only capture a fraction of all outputs achieved for entities This type of

indicator was not collected for most OPs in addition some indicators were very specific to the OP and

cannot be included in one of the categories identified (number of enterprises supported number of entities delivering public services supported number of structures of the local economy created or supported) 8 This is based on information from 48 of available indicators which it was possible to aggregate Other

indicators did not fall into the selected categories or could not be associated with participant numbers This number therefore underestimates the total results achieved 9 These in-depth interventions constitute 10 of allocated SI expenditure in the eight in-depth countries

February 2016 5

Subsidy measures can also be shown to be effective for a range of target groups

including disabled individuals and long-term unemployed job seekers

Across all Member States for ESF SI the average cost per participation was

around euro1200 whereas among the interventions assessed in depth the average cost was euro1300 This slightly higher cost could again be linked to higher financial

implementation rates to date

Sustainability

Few of the interventions reviewed in-depth measured the sustainability of results for participants (eg after 6 or 12 months following an ESF intervention) Where such

information was available results varied considerably from over 70 sustained after 6

months to below 20 depending on the context of the groups targeted and the socio-

economic settings in which they were located

For the interventions themselves sustainability is relatively high with just over 80 either continuing or intending to continue their activities Of these around 65

continuedexpected to continue to deliver support to disadvantaged groups with a

mixture of ESF and national funding with the remaining continuingintending to

continue with national local or regional funding

Gender sensitivity of ESF SI interventions

Despite the increased emphasis on gender mainstreaming considerations in this

programming period the availability of meaningful gender sensitivity

assessments (notably of gendered result indicators) or evidence of their

implementation was variable Gender assessments were limited to indicating that interventions lsquosupported gender equalityrsquo or were lsquogender neutralrsquo Around 75 of the

interventions assessed took this approach In terms of participation there was an

almost equal representation of men and women in ESF SI funded interventions (53

female) Similarly 54 of participants who had achieved a positive result and whose gender is known were women

Community Added Value of ESF SI investments

Volume effects allow Member States to add to existing SI actions and support

a greater number of participants than would have been possible otherwise

They played a particular role in the economic context to counter the impacts of the

economic crisis as previously indicated

Volume aspects were the most clearly identifiable among SI interventions

assessed in depth These were reported for 35 of the 58 interventions studied and

allowed Member States to add to existing SI actions and support a greater number of participants than would have been possible otherwise

Scope effects revolved around the ability to offer more tailored services to

specific target groups (28 of the 58 interventions) which would otherwise have

access only to mainstream services not tailored to their specific ndash and often more intensive - needs Examples include support to address gambling or substance abuse

issues targeted health or family support or the delivery of tailored training (in co-

ordination with a specific employer)

Role effects were visible in projects developing supports or activities which acted as a model for other local provision or were subsequently mainstreamed (17 of the 58

interventions)

Process effects (14 of the 58 interventions) mainly revolved around the

establishment of successful partnerships at the local level to assist disadvantaged

groups

February 2016 6

ESF was used as core funding for reforms (in CZ EL and HU) or supplementary

funding to reforms (in EE FR MT and PL) linked to the improvement of delivery

systems and methods and the testing of new approaches

The socio-economic impact for ESF social inclusion investment

Impacts were most visible at the micro level where 61 million individuals were

supported with ESF SI funding linked to dedicated SI OP priority axes At the

intermediate level systemic actions helped to boost the capacity of labour market

institutions This included support for mainstream institutions such as PES but also for the establishment of new structures (such as assessment systems to test the work

capability of disabled individuals) and assistance to build better networks and

structures in intermediate labour markets in order to support more effectively job

seekers with complex problems

ESF supported some significant national programmes and reforms to promote the integration of disadvantaged groups into the labour market (including through the

development of institutions and support systems such as the intermediate labour

market) This was mainly achieved by providing the infrastructure and resources

necessary to deliver individualised tailored services to some of the most vulnerable jobseekers

A statistically significant correlation can be observed between the change in

the share of the people at risk of poverty or social exclusion as well as the

share of low-skilled at risk of poverty and social inclusion and the ESF SI investment at the country level This suggests that countries where the

significance of ESF in the national context is greater made more progress in reducing

the number of people at risk of poverty of social exclusion10

Key Lessons

This report draws lessons for policy and target group choices effective programming and implementation as well as robust monitoring and evaluation These include the

following

During the crisis ESF support proved sufficiently flexible to adjust to emerging

policy requirements Targeted tailored interventions which address the additional barriers to

labour market entry facing disadvantaged groups with different local expert

agencies working together (eg public employment services local authorities

non-governmental organisations etc) prove to be effective for ESF SI target groups

Follow-up support to stabilise new employment relationships achieved is also

critical and should be factored into delivery planning

Effective approaches to outreach and engagement particularly for the most

disengaged target groups and those suspicious of public agencies (such as addicts and ex-offenders) were key as was the provision of often intensive

support throughout the intervention

In a number of countries where systemic approaches are emphasised ESF SI

investment plays an important role in enhancing and modernising labour market policy delivery mechanisms for disadvantaged groups

Considering the importance of soft results for disadvantaged groups in the

labour market a comprehensive assessment of the benefits of ESF investment

requires the development of consistent approaches to assess the achievement of such results by participants in ESF interventions This is of particular

10 This relates primarily to the Central and Eastern European countries However this correlation does not imply a causality as many other factors beyond ESF investment have an impact on the evolution of poverty

risk at Member State level

February 2016 7

relevance for SI interventions where direct employment results are often more

remote because of the nature of the target group

Monitoring systems and evaluation strategies should be enhanced to

provide more robust and comparable results The 2014-2020 ESF regulations and guidance already address some elements (eg strengthening

requirements for data collection and evaluation)

doi102795625537

[KR-0

2-1

6-2

10-E

N-N

]

R-02-16-210-EN-N

February 2016 5

Subsidy measures can also be shown to be effective for a range of target groups

including disabled individuals and long-term unemployed job seekers

Across all Member States for ESF SI the average cost per participation was

around euro1200 whereas among the interventions assessed in depth the average cost was euro1300 This slightly higher cost could again be linked to higher financial

implementation rates to date

Sustainability

Few of the interventions reviewed in-depth measured the sustainability of results for participants (eg after 6 or 12 months following an ESF intervention) Where such

information was available results varied considerably from over 70 sustained after 6

months to below 20 depending on the context of the groups targeted and the socio-

economic settings in which they were located

For the interventions themselves sustainability is relatively high with just over 80 either continuing or intending to continue their activities Of these around 65

continuedexpected to continue to deliver support to disadvantaged groups with a

mixture of ESF and national funding with the remaining continuingintending to

continue with national local or regional funding

Gender sensitivity of ESF SI interventions

Despite the increased emphasis on gender mainstreaming considerations in this

programming period the availability of meaningful gender sensitivity

assessments (notably of gendered result indicators) or evidence of their

implementation was variable Gender assessments were limited to indicating that interventions lsquosupported gender equalityrsquo or were lsquogender neutralrsquo Around 75 of the

interventions assessed took this approach In terms of participation there was an

almost equal representation of men and women in ESF SI funded interventions (53

female) Similarly 54 of participants who had achieved a positive result and whose gender is known were women

Community Added Value of ESF SI investments

Volume effects allow Member States to add to existing SI actions and support

a greater number of participants than would have been possible otherwise

They played a particular role in the economic context to counter the impacts of the

economic crisis as previously indicated

Volume aspects were the most clearly identifiable among SI interventions

assessed in depth These were reported for 35 of the 58 interventions studied and

allowed Member States to add to existing SI actions and support a greater number of participants than would have been possible otherwise

Scope effects revolved around the ability to offer more tailored services to

specific target groups (28 of the 58 interventions) which would otherwise have

access only to mainstream services not tailored to their specific ndash and often more intensive - needs Examples include support to address gambling or substance abuse

issues targeted health or family support or the delivery of tailored training (in co-

ordination with a specific employer)

Role effects were visible in projects developing supports or activities which acted as a model for other local provision or were subsequently mainstreamed (17 of the 58

interventions)

Process effects (14 of the 58 interventions) mainly revolved around the

establishment of successful partnerships at the local level to assist disadvantaged

groups

February 2016 6

ESF was used as core funding for reforms (in CZ EL and HU) or supplementary

funding to reforms (in EE FR MT and PL) linked to the improvement of delivery

systems and methods and the testing of new approaches

The socio-economic impact for ESF social inclusion investment

Impacts were most visible at the micro level where 61 million individuals were

supported with ESF SI funding linked to dedicated SI OP priority axes At the

intermediate level systemic actions helped to boost the capacity of labour market

institutions This included support for mainstream institutions such as PES but also for the establishment of new structures (such as assessment systems to test the work

capability of disabled individuals) and assistance to build better networks and

structures in intermediate labour markets in order to support more effectively job

seekers with complex problems

ESF supported some significant national programmes and reforms to promote the integration of disadvantaged groups into the labour market (including through the

development of institutions and support systems such as the intermediate labour

market) This was mainly achieved by providing the infrastructure and resources

necessary to deliver individualised tailored services to some of the most vulnerable jobseekers

A statistically significant correlation can be observed between the change in

the share of the people at risk of poverty or social exclusion as well as the

share of low-skilled at risk of poverty and social inclusion and the ESF SI investment at the country level This suggests that countries where the

significance of ESF in the national context is greater made more progress in reducing

the number of people at risk of poverty of social exclusion10

Key Lessons

This report draws lessons for policy and target group choices effective programming and implementation as well as robust monitoring and evaluation These include the

following

During the crisis ESF support proved sufficiently flexible to adjust to emerging

policy requirements Targeted tailored interventions which address the additional barriers to

labour market entry facing disadvantaged groups with different local expert

agencies working together (eg public employment services local authorities

non-governmental organisations etc) prove to be effective for ESF SI target groups

Follow-up support to stabilise new employment relationships achieved is also

critical and should be factored into delivery planning

Effective approaches to outreach and engagement particularly for the most

disengaged target groups and those suspicious of public agencies (such as addicts and ex-offenders) were key as was the provision of often intensive

support throughout the intervention

In a number of countries where systemic approaches are emphasised ESF SI

investment plays an important role in enhancing and modernising labour market policy delivery mechanisms for disadvantaged groups

Considering the importance of soft results for disadvantaged groups in the

labour market a comprehensive assessment of the benefits of ESF investment

requires the development of consistent approaches to assess the achievement of such results by participants in ESF interventions This is of particular

10 This relates primarily to the Central and Eastern European countries However this correlation does not imply a causality as many other factors beyond ESF investment have an impact on the evolution of poverty

risk at Member State level

February 2016 7

relevance for SI interventions where direct employment results are often more

remote because of the nature of the target group

Monitoring systems and evaluation strategies should be enhanced to

provide more robust and comparable results The 2014-2020 ESF regulations and guidance already address some elements (eg strengthening

requirements for data collection and evaluation)

doi102795625537

[KR-0

2-1

6-2

10-E

N-N

]

R-02-16-210-EN-N

February 2016 6

ESF was used as core funding for reforms (in CZ EL and HU) or supplementary

funding to reforms (in EE FR MT and PL) linked to the improvement of delivery

systems and methods and the testing of new approaches

The socio-economic impact for ESF social inclusion investment

Impacts were most visible at the micro level where 61 million individuals were

supported with ESF SI funding linked to dedicated SI OP priority axes At the

intermediate level systemic actions helped to boost the capacity of labour market

institutions This included support for mainstream institutions such as PES but also for the establishment of new structures (such as assessment systems to test the work

capability of disabled individuals) and assistance to build better networks and

structures in intermediate labour markets in order to support more effectively job

seekers with complex problems

ESF supported some significant national programmes and reforms to promote the integration of disadvantaged groups into the labour market (including through the

development of institutions and support systems such as the intermediate labour

market) This was mainly achieved by providing the infrastructure and resources

necessary to deliver individualised tailored services to some of the most vulnerable jobseekers

A statistically significant correlation can be observed between the change in

the share of the people at risk of poverty or social exclusion as well as the

share of low-skilled at risk of poverty and social inclusion and the ESF SI investment at the country level This suggests that countries where the

significance of ESF in the national context is greater made more progress in reducing

the number of people at risk of poverty of social exclusion10

Key Lessons

This report draws lessons for policy and target group choices effective programming and implementation as well as robust monitoring and evaluation These include the

following

During the crisis ESF support proved sufficiently flexible to adjust to emerging

policy requirements Targeted tailored interventions which address the additional barriers to

labour market entry facing disadvantaged groups with different local expert

agencies working together (eg public employment services local authorities

non-governmental organisations etc) prove to be effective for ESF SI target groups

Follow-up support to stabilise new employment relationships achieved is also

critical and should be factored into delivery planning

Effective approaches to outreach and engagement particularly for the most

disengaged target groups and those suspicious of public agencies (such as addicts and ex-offenders) were key as was the provision of often intensive

support throughout the intervention

In a number of countries where systemic approaches are emphasised ESF SI

investment plays an important role in enhancing and modernising labour market policy delivery mechanisms for disadvantaged groups

Considering the importance of soft results for disadvantaged groups in the

labour market a comprehensive assessment of the benefits of ESF investment

requires the development of consistent approaches to assess the achievement of such results by participants in ESF interventions This is of particular

10 This relates primarily to the Central and Eastern European countries However this correlation does not imply a causality as many other factors beyond ESF investment have an impact on the evolution of poverty

risk at Member State level

February 2016 7

relevance for SI interventions where direct employment results are often more

remote because of the nature of the target group

Monitoring systems and evaluation strategies should be enhanced to

provide more robust and comparable results The 2014-2020 ESF regulations and guidance already address some elements (eg strengthening

requirements for data collection and evaluation)

doi102795625537

[KR-0

2-1

6-2

10-E

N-N

]

R-02-16-210-EN-N

February 2016 7

relevance for SI interventions where direct employment results are often more

remote because of the nature of the target group

Monitoring systems and evaluation strategies should be enhanced to

provide more robust and comparable results The 2014-2020 ESF regulations and guidance already address some elements (eg strengthening

requirements for data collection and evaluation)

doi102795625537

[KR-0

2-1

6-2

10-E

N-N

]

R-02-16-210-EN-N

doi102795625537

[KR-0

2-1

6-2

10-E

N-N

]

R-02-16-210-EN-N