Evaluation in European Foundations:Trends and Perspectives

Andrew BarnettUK Branch Director

Calouste Gulbenkian Foundation

Outline

1. Two key trends2. Our recent experience of designing an

evaluation system3. What is evaluation for?

Lifecycle of a programme

1

•Scoping: research and consultation to identify most effective intervention in response to an issue/problem

2

•Objectives and outcomes: developing a plan of activity to maximise beneficial impact including determining what success might look like.

3

•Implementation: might include funding pilot projects.

4

•Evaluation: assessing impact and discerning learning.

5

•Dissemination: targeted communication of the learning to those who can make a difference, can change systems, scale or replicate successful initiatives.

6

•Exit: concluding the programme

Trend 1: Focus on ‘effectiveness’

Traditional giving helps one person or

organisation at a time by providing support for immediate needs. Strategic philanthropy focuses on systemic change and builds for the future.Centre for Effective Philanthropy

Foundation sector in Europe is growing dynamically

• 110,000 ‘public-benefit foundations’ in the EU

• 43% set up as recently as the early 1990s (many of these small and associated with ‘new wealth’)

• Foundations in Europe spend between €83 billion and €150 billion annually, over twice as much as the US foundation sector

• Direct full-time employment: between 750,000 to 1 million people in the EU

Some important players – but not really a movement

• Bertelsmann Foundation (Germany)

• Compagnia di San Paolo (Italy)• King Baudouin Foundation

(Belgium)• Bernard van Leer Foundation

(Netherlands)International network of strategy philanthropy (2001-2005)Publication: Rethinking Philanthropic Effectiveness (2005)

Foundations are magpies. They rarely stick with one way of approaching evaluation.

Gerry Salole, EFC

Trend 2: the quest to measure value

Helping all organisations identify, measure and evaluate their organisational outcomes would be hugely valuable as a whole... Funders and commissioners have a vital role to play in incentivising good outcomes measurement – funders need to incorporate evaluation data into subsequent rounds of grant giving in order for organisations to see a return for their efforts, and commissioners need to put money aside in contracts specifically for the evaluation of projects.

Measuring Social Value Demos 2010

Examples

UK Players:New Economics FoundationNew Philanthropy CapitalNCVOCharities Evaluation ServicesCabinet Office

Principles of SROI

• Involve stakeholders• Understand what changes• Value the things that matter. • Only include what is material.• Do not over claim. • Be transparent. • Verify the result

Our experience: drivers and lessons• Need to maximise impact• Need to tell a compelling story to partners and

collaborators and to the wider sectors• Need to extract learning from individual

activities• Desire to set an example and to lead: to be at

the forefront of thinking and practice

Key stages in the journey

• Develop a strategy and operationalise the strategy

• Go live as soon as possible – ‘retrofit’ existing projects and apply system to new or early stage projects

• Use an external consultant to draw strands together and co-devise a system with the team – codify the process, make it explicit.

• Review and iterate. And keep on doing it.

Our strategy

• One overriding purpose which links to• 3 main strategic aims which link to• 3 objectives under each aim which reflect• Time-limited programmes and activities• 1 cross-cutting aim concerned with capacity

A vital planning and management tool: the evaluation cycle

From implicit to explicit

Main lessons learned

• Evaluation is the means not the end. • It is part of your planning tool box.• At the most back level, an evaluation system consists of

making the implicit explicit.• Everyone needs to own any evaluation system.• A good evaluation system should not constrain but

should provide greater freedom. • Be realistic about measuring long term top-level impact • The system should focus on the programme level and

above

Live design by users

Our emerging ‘theory of change’

• Scoping• Coalition building• Persuading• Demonstrating• Learning and improvement

Concluding questions

• What is evaluation for?• What do you believe to be the main drivers?• What’s your journey like?• Questions for me.

For more information

andrew.barnett@gulbenkian.org.uk

www.gulbenkian.org.uk