supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Measuring OutcomesHow do we understand the changes we are making …

Workshop for Funding Fair 2013

supporting voluntary

andcommunity

action

VAW’s Just For You project

Using mainly the Charities Evaluation Services (CES) outcomes-focussed approach to planning, monitoring and evaluation to:

1. Make your work more effective.2. Improve how you communicate the

outcomes of your services (especially to funders).

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

What’s the story?

1. Clear about the changes (outcomes and impact) that we are seeking to achieve.

2. Clear about the relationship between these and what we do.

3. Identifying quantitative and qualitative indicators of changes we make.

4. Robust measurement methods.5. Clear and concise reporting.

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Outcomes Planning Triangle

  

Overall Aim or

Mission

Specific Aims: This is why we do our

work

Objectives:This is what we do

Need: a problem or issue, or situation where something needs to change to make things better, for a person, a group of people, an environment or an organisation.

Outcomes: the changes, benefits, learning or other effects that happen as a result of our work.

Inputs: the resources put into a project to carry out an activity.

Outputs: the detailed activities, services and products we do or provide.

Impact: lasting change brought about by our work

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Task – 5-10 minutes

• Think about the Outcome or Outcomes (change) that motivate you most – use the triangle if it helps.

• Share with your neighbour

• Not a test! – rather a way for us all to focus our thinking on the changes that we each individually are involved in bringing about.

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Feedback

supporting voluntary

andcommunity

action

Need: children are falling behind at school because of problems at home

THEORY OF CHANGE The beginning of making a difference Angela Kail, Tris Lumley NPC April 2012

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Making sure that outcomes are realistic

• Many charities have grand aims, such as ‘the alleviation of poverty’. But aims like these are too large for a charity to achieve on its own, so it is not sensible to think about how to measure them.

• A plan for change helps charities focus on concrete, defined aims and outcomes, which are potentially measurable.

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Understanding progress towards the final goal

• Some final goals cannot easily be measured. They involve change that happens too gradually or change that happens in the lives of people who are difficult to track.

• Planning intermediate steps that lead to the end goal can help organisations work out whether they are making a difference towards that end goal, by measuring the intermediate steps.

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Measuring outcomesOutcome How will we know?Reduced isolation for older people

There will be a change in the number of older people stating that they feel isolated

Reduced vandalism in the area There will be a change in the number of incidents reported

Small voluntary organisations have a better understanding of income generation

There will be a change in the level of understanding of income generation within small organisations in the area (and longer term a change in income)

http://www.biglotteryfund.org.uk/funding/funding-guidance/applying-for-funding/

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Outcome indicators

• Outcome indicators are well defined pieces of information that can be assessed or measured to show whether outcomes have been achieved.

• Outcomes indicators enable us to measure progress towards planned outcomes

• They are signs or clues that you monitor and can be qualitative or quantitative.

• Each outcome has at least one indicator - some have many indicators.

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Outcomes indicators – outcome ‘people are ready for work’

• An example of a quantitative indicatormight be the number and type of qualifications gained by people attending a training centre. • An example of a qualitative indicator mightbe the change in confidence of those attending the centre (whether it increased, decreased or stayed the same).

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Robust measurement methods 1

• Knowing from the start what the need is, what change you seek to make (aim), the change you will make (outcome), the ways that change - and any others - will be demonstrated and measured (outcome indicators) …. you already know the what and why.

• So what is left is HOW?

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Robust measurement methods 2

• Collect only the information you need and can use

• Use methods that are sensitive to the people you want to collect information from

• Involve others in the choice and design of your information collection methods

• Fit this info collection into other work processes as much as possible.

supporting voluntary

andcommunity

action

• Interviews• Self-completion tools (surveys, tests, forms,

diaries)• Observation• Keeping records and notes• Group activities – focus groups, group

interviews, • Visual/audio methods – photos, video, voice

recordings• Social media – blogs, podcasts, twitter

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Task

Identify with your neighbour one way each that you are familiar with or have used recently to collect information about your work.• Did the information demonstrate a change? • If so, what was that change?• Was it quantitative or qualitative info?• How robust was it – is it open to challenge,

will it ‘fall down’ if questioned?

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Feedback

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Assessing Change: Developing & using outcomes monitoring tools *

1. Planning your outcomes monitoring2. Selecting appropriate methods3. Developing monitoring tools

• Designing your own tools• Finding and using off-the-shelf tools

4. Preparing to collect outcomes information5. Collecting outcomes information*By Diana Parkingson and Avan Wadia for CES Oct 2010http://

www.ces-vol.org.uk/Publications-Research/publications-free-downloads/publications-free-assessing-change

http://www.biglotteryfund.org.uk/funding/funding-guidance/applying-for-funding/

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Overall aim

Specific aims:This is why we do our work

Objectives:This is what we do

Clear and concise reporting

1) Clear purpose

2) Defined aims

Outcomes: 4) Demonstrate results

Outputs: 3) Coherent activities

5) Evidence

6) Lessons learned

C) Evaluation

A) Planning B) Monitoring

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

Discussion and

Questions

supporting voluntary

andcommunity

action

supporting voluntary

andcommunity

action

THANK YOUEmma Plouviez

Organisational Development OfficerVoluntary Action Westminster

eplouviez@vawcvs.org