Análisis coste beneficio CBA

Greater Manchester Cost Benefit Analysis: Technical Specification

March 2013This publication is supported by the European Union Programme for Employment and Social Solidarity - PROGRESS (2007 – 2013).

CONTENTS

1 Introduction .......................................................................................................................3

2 Objective of analysis ..........................................................................................................6

3 Background to approach....................................................................................................8

4 Cost Benefit Analysis Inputs.............................................................................................10

5 Cost Benefit Analysis Outputs..........................................................................................12

6 Costs – method of calculation .........................................................................................14

7 Benefits – method of calculation.....................................................................................19

8 Assumptions and unit benefits for each outcome ..........................................................30

9 Overall analysis approach ................................................................................................75

10 Further information .....................................................................................................77

2

1 Introduction

1.1 Development of this document This document details the functional specification for the cost‐benefit analysis (CBA) model

that partnerships across Greater Manchester (GM) are using in order to appraise and

evaluate interventions, and support decision making on investment approaches across

public, private and third sector agencies.

Development of this document has involved significant support and input from various

central government analysts from a number of departments including HM Treasury,

Department of Work and Pensions, Department for Business, Innovation and Skills,

Department of Health, Department of Communities and Local Government, Department for

Education, Ministry of Justice, Home Office and the Cabinet Office who form the Technical

Advisory Group for the CBA methodology.

The current status of this document is draft and it is proposed in the near future to get a

final review of the document by the Technical Advisory Group in order to confirm that the

approach is the most robust and up‐to date as is possible at the present time. The

document is intended to be a living document, and as such will be updated further as better

evidence, costings and methodologies are developed.

1.2 Purpose of the document The primary audience for this document is CBA practitioners. By reading this document,

they will gain a clear understanding of the technical approaches GM is using to demonstrate

the cost‐effectiveness of public policy interventions. CBA experts will note the

commonalities between GM’s approach and the methods recommended by official

guidance notes. They will also understand the primary and secondary data and evidence we

have reviewed and upon which we will call in order to translate outcomes into monetary

values.

Policy makers and generalists will also have use for this document. Firstly, the document

provides an introduction to CBA for those who have not previously encountered the

methodology. Secondly, it will generate confidence in GM’s ability to commission

interventions that are forecast/proven to be cost‐effective; and in GM’s capacity and

willingness to continue to monitor and evaluate projects once funding has been secured.

3

1.3 Structure and contents of the document The document is structured as follows:

o Chapter 2 explains what benefits GM will realise from possessing a cost‐benefit

model and the points at which we will apply the model to proposals and projects.

o Chapter 3 lists the existing guidance and best practice upon which we have drawn

when building the model.

o Chapter 4 summarises the two main inputs that will be needed to run the model:

data on project costs; and evidence of project outcomes.

o Chapter 5 summarises the two main outputs that will be produced by the model:

benefit‐cost ratios for individual proposals/projects; and forecasts of total savings if

a proposal/project was mainstreamed across GM.

o Chapter 6 provides more detail on how we will calculate the cost figures that will be

inputted into our model, explaining how we will identify the marginal cost of

projects and how we will work out costs for particular public agencies.

o Chapter 7 provides more detail on how we will track outcomes from GM projects,

how these will be translated into net outcomes via the use of comparator areas, and

our general approach to placing values (monetising) on these outcomes.

o Chapter 8 sets out the evidence and assumptions we will use in order to track and

monetise individual outcomes for individual government departments. To aid

government department’s use of this document, each outcome section includes a

list of the government agencies whose work relates to that outcome.

o Chapter 9 introduces a methodology to calculate social value for wellbeing

outcomes, and sets out the metrics to use for each of the elements of wellbeing.

o Chapter 10 switches the focus to consider how we will treat the outputs from the

model, in terms of developing benefit‐cost ratios for projects, up‐scaling findings to

the GM level and accounting for risk and uncertainty.

o Chapter 11 provides links to further sources of information and support relevant to

our work.

4

1.4 Document version status This document is version 2.0 of the Cost Benefit Analysis Technical Specification. The key

updates since version 1.0 of the document are:

Additional outcomes for new areas of modelling – e.g. crime, and alcohol and drug

dependency.

Updates to existing approaches to monetising outcomes based on more up to date

research.

Further guidance as to certain aspects of the methodology, e.g. calculating

deadweight, thinking about cashability, assigning benefits across agencies.

An additional chapter on a methodology to include social value in the overall

assessment of value for money.

Updates to this document are planned to extend the Technical Specification into a

framework for monitoring and evaluating as wide a range of innovative interventions and

projects as possible. Thus, it is not possible to predict all the interventions to which this

model will be applied. Changes in political and/or economic contexts may necessitate a

revisiting of this model to incorporate new outcomes and new assumptions around the

value of these outcomes.

Furthermore, it is the intention of the GM team to update and improve our cost‐benefit

models as projects and programmes report primary data on the outcomes they achieve and

the value of these outcomes to the taxpayer and the wider economy (see chapter 2 for

more detail on this iterative approach).

Further versions of this document will be published in due course and made available on our

Knowledge Hub site (see chapter 11) and via the New Economy website.

For further information about this document and the GM methodology, please contact

Julian Cox ([email protected]), David Morris

([email protected]) or Rupert Greenhalgh

([email protected]).

5

mailto:[email protected]



2 Objective of analysis

2.1 Stages at which costbenefit analysis can be applied CBA offers valuable information at multiple stages in the project lifecycle:

o At the pre‐delivery stage CBA can inform ex‐ante appraisals of what return on

investment a proposal may be expected to deliver. Based on this information,

commissioners may choose to allocate scare public funds towards those proposals

that are expected to deliver the greatest return.

o During the delivery stage regular CBA, based on up‐to‐date project management

data, can tell us whether a project is achieving, or is likely to achieve, its forecast

return on investment. If a project is found to be failing to meet expectations, project

managers can redesign delivery or, in some cases, stop delivery and reallocate funds

to better performing projects.

o Post‐delivery CBA is one way of judging whether a project has been worthwhile

undertaking. Ex‐post CBA provides lessons for commissioners. It also gives us

information with which to update and improve the assumptions that are used in

ex‐ante appraisals of future projects (for example, based on past CBA, we assume

that achieving outcome X will be worth amount y).

2.2 Ongoing development of the GM costbenefit model In recognition of the multiple roles that CBA can play, GM is committed to developing a

flexible CBA model that can be used before, during and after project delivery. We have

started by designing an ex‐ante, appraisal model. This produces benefit‐cost ratios (BCR) of

individual projects based on:

o estimates of how many residents these projects will benefit;

o forecasts of what projects will cost the taxpayer; and

o assumptions around what the value of project benefits will be to the taxpayer/wider

economy.

As projects begin to deliver across GM (and some are now at this stage), we will ask project

managers to collect and report data and information on:

o the numbers of people they are working with;

o how many are achieving specified outcomes; and

o how much it is costing the taxpayer to run the project.

This information will be extrapolated to cover the lifetime of the project and fed into our

ex‐ante CBA model to produce a new BCR.

6

Once a project has completed, a final evaluation report will be produced, setting out total

outputs and outcomes and total project costs. Based on this information our CBA model will

be transformed into ex‐post, evaluation tools, with real‐life, evidenced figures for

engagement and impact rates and the fiscal/economic value of achieving specific outcomes.

A third and final BCR will be generated and we will update the values and assumptions in

our ex‐ante CBA model to produce more accurate, primary evidence based analysis.

If this approach is successful it will mean that, year‐on‐year, GM partners are able to use

past evidence and knowledge to make more accurate forecasts in relation to future

propositions.

The corollary of developing a CBA model that will be updated and improved over time is the

need to acknowledge that the benefit cost ratios produced by such a model will never be

100% accurate estimates of a project‘s value for money. There will be assumptions and/or

reliance upon secondary data which weaken our ability to draw broad policy lessons from

individual project or programme reviews.

In recognition of this weakness, GM makes two proposals:

o Firstly, all outputs from our CBA model will be subjected to a range of risk and

sensitivity tests (see chapter 9) in order to understand more about the degree of

confidence with which we should treat the outputs from our model.

o Secondly, when GM partners commission and evaluate interventions they will be

advised to consider more than just the benefit‐cost ratio of the project. They will be

advised to consider interventions from a range of perspectives, including qualitative

feedback, strategic contribution and GM’s capacity to delivery, alongside the BCR.

7

3 Background to approach

3.1 Reference texts Our model, the processes it follows and the assumptions it makes, are based, wherever

possible, on best practice, as defined in existing guidance. This means that a CBA expert will

not need a detailed understanding of the GM CBA model or the local context in order to

interpret our results. It also means that a lay reader will have multiple sources from which

to gain an understanding of our work.

The key document that has informed the development of the Greater Manchester CBA

model is HM Treasury’s Green Book.1 From the Green Book we have drawn guidance on

technical details such as:

o The importance of agreeing the objectives of any intervention, the indicators which

will be used to measure achievement, and the timescale over which outputs and

outcomes will be monitored;

o the task of considering, and where possible agreeing the value of, three types of

outcomes from public sector interventions: fiscal, economic and social outcomes;

o the need to count all the costs that the public sector incurs in order to deliver a

project, including those that are borne by agencies other than the direct sponsor of a

project;

o the discount rate that should be applied to costs and benefits that will be realised in

future years; and

o the need to adjust CBA outputs to account for risk and uncertainty.

Individual government departments have their own CBA models. These models often serve

to add extra ‘modules’ to the Green Book; for example, DWP provides further guidance on

how researchers should model the costs and benefits associated with moving somebody off

benefits and into sustained employment. In addition to the Green Book, we have drawn on

the following guidance notes:

o DWP – The Department for Work and Pensions Social Cost‐Benefit Analysis

framework.2

o BIS – Additionality guidance.3

1 The Green Book : Appraisal and Evaluation in Central Government, HM Treasury 2003

2 “Social Cost‐Benefit Analysis Framework”, Daniel Fujiwara, DWP, 2010

3 “BIS Occasional Paper No. 1 – Research to improve the assessment of additionality”, 2009

8

o Cabinet office/New Economics Foundation – A guide to Social Return on Investment:

we have reviewed the advice contained in this document around methodologies for

valuing social outcomes, and used these principles as the basis for the social value

methodology in Chapter 9.

o Volunteering England – Volunteer Investment and Value Audit: our models follows

Volunteering England’s advice on how to count and value the in‐kind costs

associated with voluntary/third sector input into public sector projects.

The Technical Specification highlights where our model draws on existing guidance.

3.2 Analysis structure The diagram schematically shows the structure of the analysis carried out. In the following

chapters we outline the detailed approach we will take at each stage of the analysis.

9

4 Cost Benefit Analysis Inputs

There are two key inputs into our model: costs and benefits (or outcomes).

4.1 Types of costs Our model aims to capture all the costs associated with delivering a specific service or

intervention within a project4 area. The model sums three different types of costs:

Capital costs – one off investments, such as new/refurbished buildings and facilities;

Revenue costs ‐ costs which tend to fluctuate in relation to the amount of project

activity being undertaken, such as staff salaries;

In‐kind costs ‐ those inputs which are needed in order to make a project a success

but which the public purse will not have to pay for, such as a charity providing their

facilities for free. These are counted because there will be an opportunity cost

associated with using these resources for project activities.

Some projects will be designed to make immediate cost savings through streamlining

services and avoiding duplication. These savings should be subtracted from a project’s

delivery costs, rather than included as a benefit.

4 We have used the word “project” to refer to any initiative or intervention. This could range from a small

neighbourhood pilot to a large scale change in delivery of public services across the whole of Greater

Manchester.

10

11

4.2 Types of benefits Our model seeks to place a value on three different types of benefits (or outcomes)

associated with delivering a specific service or intervention within a project area. These are:

Fiscal benefits – savings to the taxpayer that are due to a specific project – for

instance, reduced health service, police or education costs;

Economic benefits – gains which accrue to individuals – for instance, increased

earnings – or the whole economy – for instance, increased GVA due to more people

being employed;

Social benefits – gains which accrue to society – for instance, improved health and

wellbeing or increased satisfaction with the community.

4.3 Identifying marginal costs and marginal benefits The approach taken to appraising/evaluating the success of projects across GM is to

compare the additional outcomes achieved by the project with the additional costs of

delivering the project.

In order to do this it is important to:

1. have a comprehensive view of both the costs of providing the services offered by the

project, and also the outcomes predicted/achieved from the project; and

2. make an assessment of the costs and outcomes that would result if the project was

not to take place (known in technical terms as accounting for deadweight).

The methodology and assumptions behind calculating these additional costs and benefits

are covered in more detail in chapters 6 and 7.

4.4 Analysis time frame The GM CBA model’s primary analysis timeframe is a five year assessment of costs and

benefits. This timescale has been chosen to reflect the need to identify short term savings

of a project to the public sector. However, the GM CBA model can be modified in order to

assess the benefits over any time frame that agencies wish to consider.

5 Cost Benefit Analysis Outputs

There are three key outputs from the model – benefit cost ratios, net present efficiencies

and payback periods. These tell us whether projects provide value for money; the level of

overall savings from an initial investment; and how long it takes to see a return on the initial

investment (ROI).

5.1 Benefit Cost Ratios The BCR measure is a ratio of the monetised value of the outcomes of the project (the

benefits) to the cost of implementing the project. It is calculated as follows:

BCR = Benefits/Costs

This measure provides an indication of the ROI achieved by the project.

A BCR<1 indicates that the project has cost more than it has achieved and therefore is not

economically beneficial and should be considered for decommissioning. A BCR of 1 is the

break even point at which the benefits achieved exactly match the costs incurred. It could

be argued that any scheme which has a benefit cost ratio >1 should therefore be

progressed. However, this is impractical if funding is limited and decisions have to be made

as to where to best allocate funds. This is where the BCR can be a powerful decision support

tool, as projects can be prioritised based on their BCR. Those projects with a high BCR

should be progressed first, and projects with a BCR just above 1 given a lower priority.

In order to provide a consistent measure of costs and benefits now and into the future,

future costs and benefits are discounted to produce a Net Present Value (NPV) (see section

10.1 for further details). These NPVs are then used in the BCR calculation.

12

The GM model can produce individual BCRs for fiscal benefits, economic benefits and social

benefits. Further detail on the definitions of each of these types of benefits can be found in

section 7.2.3.

5.2 Net present efficiencies BCRs are not always meaningful ways of comparing projects. For example, a project may be

streamlining services, and reducing the overall level of costs. For this project the marginal

cost would be negative, and therefore the BCR would be a negative value. To layperson,

such an outcome might suggest that the project was not worth continuing, when in fact the

project has made a positive contribution in terms of reducing delivery costs.

This example illustrates the importance of having a second way of assessing the success of

the projects, namely their net present efficiency, which we have defined as follows:

Net present efficiency = Net present value of the benefits – Net present value of the costs

Projects with a higher net present efficiency should normally be chosen first. However, as

noted, any funding constraints need to also be taken into account.

5.3 Payback period When investing in a project it is important to know how quickly a return on that investment

is achieved (so that plans can be made to reinvest savings elsewhere at a certain point in the

future). The payback period provides this information and is defined as follows:

Payback period = Year in which the cumulative net present value of the benefits > cumulative

net present value of the costs

5.4 Other outputs The CBA tool can also produce graphical outputs. These include pie charts showing the split

of costs and benefits, and column charts showing the agencies who are investing in a project

and those who benefit from reduced future reactive spending.

5.5 Use of the CBA model outputs in decision making The figures and charts produced by the CBA model give support to decision makers deciding

whether to commission or decommission services. Due to the current financial pressures on

public services, the most useful analysis is likely to be the outputs relating to fiscal benefits –

i.e. the fiscal Benefit Cost Ratio, and the fiscal net present efficiency (to give an indication of

cashable savings). The economic and social results will provide the broader picture as to the

additional benefits that can be achieved.

However, all outputs from the CBA model should be used with care, with a full

understanding of the limitations that may exist with the data and the assumption upon

which the analysis is based. In recognition of these points, we recommend that CBA is used

as a decision support tool rather than a decision making tool.

13

6 Costs – method of calculation

A pre‐requisite to CBA (and to meaningful evaluation in general) is the ability to accurately

identify the costs that are associated with the project which is being assessed.

For CBA, we are interested in the marginal cost of the new delivery model (NDM) for the

intervention, compared to business as usual (BAU). (N.B. we will use a similar approach for

benefits where we take account of deadweight – what would happen anyway under

business as usual).

For each intervention being assessed, the CBA model requires two cost figures: one which

relates to the cost of delivering services without any intervention (the reference case); and

one which relates to the cost of delivering services with the intervention.

6.1 Reference case costs The reference case is the BAU approach to dealing with a specific issue. We have developed

a framework that helps project teams to break down, calculate and report the costs that the

public purse has been incurring to deal with these issues (such as worklessness or lack of

school readiness amongst young children) prior to the design of the intervention.

The framework starts by asking project teams to break down how services and support is

currently delivered at four stages in the client journey. These four stages are:

Contact and engagement – when agencies are identifying who needs support;

Assessment – establishing the exact needs of these residents and planning a

response;

Interventions – the delivery of support;

14

Reviewing – as participants move towards the end of their engagement there is the

need to review achievements.

It then asks project teams to take each of these stages in turn and:

List the agencies who currently incur costs;

Finally, it requires project teams to:

List the types of costs each agency incurs (with total costs split into revenue, capital

and in‐kind).

By undertaking this three stage approach, project teams should be able to produce a

detailed profile of which agencies are incurring what types of cost and when this occurs.

Where possible, reference case costs for each project are compared to the reference case

costs being reported by other projects to check that they seem accurate.

6.2 Intervention case costs The approach used to calculate intervention case costs is similar to that used to calculate

reference option costs. Project teams are asked to consider which agencies, under the NDM,

will incur what types of costs at each of the four stages in the customer journey. From this,

the model can build a bottom‐up model of intervention case costs to be measured against

the reference option cost in order to demonstrate the net cost of an intervention.

The initial NDM cost figure will be an estimate, calculated before the project has completed

all its activities. Within this estimate there will be several costs which cannot be forecast

accurately. For instance:

A project team cannot definitively state how much will need to be spent on

delivering support until the project starts operating on the ground;

A project team cannot accurately forecast the amount of funding they may save as a

result of more efficient staff working until the new delivery model has had time to

get going.

The CBA model fills in these knowledge gaps with estimates and assumptions of how many

participants a project will engage and what it will cost to support each participant based on

previously evidenced cost figures.

In the longer‐term, the model provides a mechanism for gathering more cost data as

interventions begin to deliver on the ground. Thus, over time, the model’s estimates of the

NDM costs, and hence marginal cost, should become more accurate.

15

6.3 Calculation of marginal costs based on new delivery model The method the CBA models uses to calculate the marginal cost of each intervention varies

depending on the NDM being adopted. GM has identified two broad categories of NDM:

Provision of additional service(s)

Where new services are being provided which are not currently provided, the identification

of marginal cost is relatively straightforward. The project cost is the cost of providing the

additional service(s), identified using the framework described above. The reference cost

can be assumed to be zero, because the service(s) are currently not being provided.

Reorganising services to reduce duplication and provide a more streamlined approach

Calculating the marginal cost under a reorganising services model is more complex.

The Technical Specification recommends that projects undertake a two step process:

The first step is to review all the agencies involved in BAU provision of the services

and determine their unit cost per resident/client/beneficiary/time period etc;

The second step is to assess the NDM, with the aim of identifying the same unit cost

metric(s).

If agencies are being brought together in a combined team, project costs are calculated

based on the costs of providing this combined service. The reference costs are calculated by

assessing the overall costs of each agency included in the project, and apportioning costs to

the project area based on the caseload in the project area compared to the overall caseload

for the agency. An example is shown below:

JobCentre Plus costs example ‐ The local job centre covers not just the

intervention neighbourhood, but also two other neighbourhoods:

The total costs of providing all JC+ services from the job centre are £3m/year.

The case load in the pilot area is just over half that of the whole area.

Number of individuals on benefits in pilot area = 1,200

Number of individuals on benefits in whole JC+ area = 2,000

Reference costs for JC+ = £3m x 1200/2000 = £1.8m/year.

16

6.4 Approach used to identify costs by agency As noted earlier, projects often involve input from a range of departments and agencies. It is

important to capture all the costs incurred by all partners both under BAU and the NDM.

Below we provide more detailed guidance on our approach to working out costs for key

partners.

6.4.1 Direct agency costs Project teams will need to report revenue, capital and in‐kind costs. Of these, revenue costs

are likely to be largest and will cover the following:

Staff salaries (pro‐rata for those staff who spend a % of their working week on

activities not related to the project);

On‐costs – National Insurance and pension costs for these staff (again pro‐rated for

staff who do not work full‐time on the project);

Overhead costs – to account for rent and utility costs. This can be calculated by an

average percentage addition to total staff costs;

Staff travel and subsistence expenses – especially important for those staff whose

role involves outreach activities.

Projects should identify where a specific agency department will incur non‐staffing related

costs due to project activity. For instance, Children’s or Adult’s Services departments may

provide venues for project activity. Costs can be assessed by using information on hire costs

for such venues and multiplying these costs by the amount of time projects will use such

venues. Other non‐staffing costs may relate to the production of leaflets and materials with

which to recruit participants and to accompany the delivery of support.

6.4.2 Referrals to other services It is very important that CBA modelling captures all the costs related to project referrals to

other services. For example, a family may have a case worker engaged who provides the

day‐to‐day support, and whose costs can be calculated as above. However, in order to

achieve the outcomes of the project, other services may be needed such as mental health,

alcohol treatment or debt advice. These would generally be referred onto by the case

worker.

As effective referral is often key to the success of a project, it is important that referral costs

are accurately identified. Ideally this would be by a detailed analysis of the costs of

supporting agencies for the cohort. However, in many cases this is not possible, in which

case a ready reckoner approach should be used.

17

For example, NICE Clinical Guideline 1155 sets out typical costs for different types of alcohol

dependency treatment. This provides an average unit cost for psychological interventions

received by people with mild alcohol dependence of £741.67, which is a mean cost of the three

main types of therapy recommended: cognitive behavioural therapies, behavioural therapies or

social network and environment‐based therapies. By multiplying the number of people in a

cohort needing treatment for alcohol dependency by this unit cost, a typical cost of

providing these referred services can be calculated.

6.4.3 Voluntary sector Projects often include input from the voluntary/third sector, in some instances given for

free or at below market rates. Projects should identify whether this is the case for their

intervention and if it is, quantify and value the amount of support they receive from

voluntary/third sector agencies.

The main types of support from the voluntary sector, and the quantification approach we

will use for each, are set out below:

Volunteer staff time – the model recommends using Volunteering England’s advice

on how to value the time given by volunteers to a project;

Free use of community facilities/venues – projects should estimate/record the length

of time over which they will benefit from these services and multiply this figure by

an hourly rental rate.

5 Alcohol‐use disorders: alcohol dependence – Costing report, National Institute for Health and Clinical

Excellence, 2011

18

7 Benefits – method of calculation

7.1 Outcomes

For each initiative, the analysis is broken down into the impact on a number of specific

outcomes.

The outcomes that the CBA is currently able to assess against are:

Reduced number of individuals claiming Job Seekers Allowance (JSA) in the project

area;

Reduced number of individuals claiming Incapacity Benefit (IB) or Employment and

Support Allowance (ESA);

Reduced number of individuals claiming Lone Parent Income Support (LPIS);

Increased number of people holding Level 2 and Level 3 skills;

Reduced number of Children in Poverty;

Improved school readiness (increased number of pupils achieving 78+ points at Early

Years Foundation Stage);

Reduced parental mental health problems (number of individuals with anxiety and

depression);

Reduced avoidable A&E attendance;

Reduced number of Children in Poverty;

19

Reduced incidents of domestic violence;

Reduced number of anti‐social behaviour incidents;

Crime against individuals and households;

Reoffending;

Reduced housing evictions;

Reduced number of children in care;

Drug abuse

Alcohol dependency

Truancy

Exclusion.

Other outcomes that are targeted by partnerships will be assessed on a case‐by‐case basis

to identify whether they can be included in the CBA model without comprising the level of

robustness expected in the analysis.

Details of assumptions made for each outcome and the means of monetising the outcomes

are outlined in Chapter 8.

20

7.2 Analysis approach

7.2.1 Input data For each outcome, project data is needed to determine inputs into the CBA.

The following data is required to be inputted into the model spreadsheet:

The total population in the project area (e.g. total working age population);

At risk/targeted population at risk (e.g. number of individuals on IB/ESA);

Level of engagement with the target population (percentage individuals who

engaged with the services and who continue to be engaged until the intervention is

complete);

Scale of impact in changing the outcome (percentage success at achieving the

desired outcomes ‐ e.g. getting the individual into employment).

7.2.2 Deadweight When calculating the incremental benefits of a scheme, the change in outcomes that would

have happened anyway is known as the deadweight. Deadweight can be calculated in

various ways, depending on the type of CBA being undertaken:

1) When engaged in predictive CBA, deadweight is best determined by forecasting the

trends in outcome indicators over the duration of the project, taking account of

cyclical effects (e.g. for JSA volumes);

2) During or at the end of a project, CBA can take two approaches to calculating

deadweight:

a. For measures which are not affected by localised changes or the transience

of individuals into and out of the project area, by assessing the change in

outcome measures of comparator areas or cohorts (see below);

b. For other measures, by measuring changes in the outcomes of individuals

engaged in the project through a robust monitoring and evaluation plan.

7.2.2.1 Comparator areas and control groups

One way of assessing deadweight is to use comparator areas. These are areas that have

similar characteristics to the project area but are not implementing the NDM. For example,

the Early Years and Better Life Chances pilots used comparator areas which had been

matched based on the 2007 Index of Multiple Deprivation (IMD). Ideally, comparator areas

should closely match a project area in terms of demographics, economic make‐up, levels of

crime and antisocial behaviour, transport connectivity and so on.

21

Control groups are a more micro‐level alternative to comparator areas. Under this approach

to assessing deadweight, a project team tracks the same outcomes they hope to deliver

using the NDM for a group of people who are not receiving the NDM. Providing that the

intervention and control groups are similar in terms of make‐up (gender, age, economic

status etc), this approach provides a well‐regarded means of assessing project deadweight.

For more detail on comparison area and control group methods, readers are referred to

HMT’s Magenta Book guidance on how to undertake evaluation (http://www.hm‐

treasury.gov.uk/magentabook).

7.2.2.2 Individual monitoring

For some outcome measures, it may not be possible to identify a net impact that is

discernible from the background variation in the outcome measure. However, this does not

necessarily mean that the project has not delivered significant changes in outcomes for the

individuals involved. It is therefore desirable that project teams also collect information

about the change in outcomes for individuals. To understand the deadweight factor for each

individual, project teams can review an individual’s case history and make a judgement on

the likelihood of a change in outcome occurring without the project intervention. For

example, when assessing the interventions that will aid individuals to find employment,

projects should assess the working history of the individual over preceding years.

22

7.2.3 Assumptions for other impacts on additionality

7.2.3.1 Leakage

A project intervention may lead to changes in outcomes that benefit others outside the

project area or result in individuals moving out of the project areas. To date, the CBA model

has been applied to geographically small projects, such that the likelihood of benefits

remaining within the overall boundary of analysis is judged to be high. Therefore, the model

does not, at present, allow for leakage assumptions.

7.2.3.2 Displacement and substitution

Displacement and substitution refers to the extent to which increases in outcomes

(especially employment) are offset by reduction in outcomes elsewhere. At present it is

assumed that employment rates are supply side, rather than demand side, constrained, and

therefore displacement and substitution effects are assumed to be zero.

7.2.3.3 Multiplier effects

At present the model makes no assumption on whether increases in outcomes have knock‐

on effects elsewhere in the project area. For example, the model currently assumes that no

further economic activity occurs as a result of increased employment in the project area.

The Green Book recommends that appraisals and evaluations include consideration of

leakage, displacement, substitution and multiplier factors. Therefore, it is our intention in

future to produce versions of the model which allow project teams to include these factors.

7.2.4 Monetisation In order to turn outcomes of the projects into a financial benefit that can then be used in

the cost benefit analysis, each outcome needs to be monetised.

Each outcome is monetised under 3 categories:

Benefit Category Description

Fiscal Savings to central and local government agencies, resulting in reduced

overall government expenditure

Economic Economic benefits to individuals and companies in terms of increased

income

Social Social benefits to individuals and society, e.g. increased health, well

being and community cohesion

N.B. In order to realise fiscal savings, decommissioning will be required; otherwise, other

demands on services will backfill available capacity and remove the opportunity to deliver

cashable savings. More detail on how the model addresses cashability is included under

section 7.5.

23

7.2.5 Indexation Monetisation values are recorded in the spreadsheet based on the year of the

study/analysis from which they are taken. The CBA model automatically converts all values

to current prices, using the GDP deflator indices produced by HM Treasury.6

6 http://www.hm‐treasury.gov.uk/d/gdp_deflators.xls

24

http://www.hm-treasury.gov.uk/d/gdp_deflators.xls

7.2.6 Optimism bias (OB) Evidence shows that commissioners and practitioners are often overly optimistic about the

outcomes that will be achieved by the project or programme and the amount of money that

will be needed to deliver these outcomes. It seems reasonable to assume that the degree of

over‐optimism will be greater when the data and evidence upon which CBA is based is

patchy, old, or weak. Therefore, the CBA model applies optimism bias correction factors in

response to the level of uncertainty in the data or assumptions used. The OB approach used

is based on the following confidence grade definitions:

Confidence Grade for Cost data

Confidence grade

Colour coding

Data source Age of data Known Data

error

Optimism bias

correction

1 Independently audited cost

data

Current Data (<1 year old)

+-2%

0%

2 Formal service

delivery contract costs

1-2 years old +-5% +5%

3 Practitioner monitored

costs 2-3 years old +-10% +10%

4

Costs developed from

ready reckoners

3-4 years old +-15% +15%

5 4-5 years old +-20% +25%

6 Uncorroborated

expert judgement

>5 years old +-25% +40%

The confidence grade which the CBA model applies the data is determined by the lowest

assessment in any of the descriptive columns. The optimism bias correction factor for the

data is then determined based on the lowest confidence grade found in relation to each

individual outcome and costs are increased by the corresponding percentage factor (shown

in the table above). Data in the spreadsheet is colour‐coded to enable a quick visual

assessment of the quality of the cost data inputs.

25

Confidence Grade for Benefits data

Confidence grade

Colour coding

Population/ Cohort Data

Evidence base (engagement / impact)

Monetisation evidence

Age of data /

analysis

Known data error

Optimism bias

correction

1 Figures taken from

agency data systemsRandomised Control

Trial in UK

Independent primary analysis

Current Data (<1 year old)

+-2%

0%

2 Figures derived from

local stats

International Randomised Control

Trial

Practitioner primary analysis

1-2 years old

+-5% -5%

3 Figures based on

national analysis in similar areas

Independent monitoring of outcomes with a

robust evaluation plan

National Cost Benefit

Framework

2-3 years old

+-10% -10%

4 Figures based on generic national

analysis

Practitioner monitoring of outcomes with a

robust evaluation plan

Secondary National analysis

3-4 years old

+-15% -15%

5 Figures based on

international analysis

Secondary evidence from a similar type of

intervention

International analysis

4-5 years old

+-20% -25%

6 Uncorroborated

expert judgement Uncorroborated expert

judgement

Uncorroborated expert

judgement

>5 years old

+-25% -40%

26

For project benefits, the same approach is taken but this time the forecast benefits are

reduced by the confidence grade percentage.

7.3 Benefit Calculation The maximum potential monetary benefit for each outcome is calculated using the

following formula:

Maximum Potential Benefit = at risk/targeted population* %engagement* %impact* unit

benefit* (1‐ optimism bias correction)

7.4 Lag/drop off By completing the stages in the CBA model described above, project teams will be able to

produce an estimate of the maximum potential fiscal, economic and social benefit that

would arise if all their project beneficiaries all immediately achieved the desired outcome

(e.g. entered employment) and were able to sustain this outcome for 5 years (e.g. no

beneficiary is made unemployed again in the next 5 years).

Such an estimate would not be realistic. This is because:

A project is unlikely to have the capacity to work with all clients from day 1 – instead,

it may enrol clients on the project in phases;

The desired outcomes from a project will almost definitely not be achieved on day 1

and may take more than a year to materialise;

Over subsequent years some of the project beneficiaries are likely to revert back to

their pre‐intervention status.

To make its analysis as realistic as possible, the CBA model requires project teams to

forecast when a desired outcome is likely to occur and for how long that outcome will be

sustained across the beneficiary group. Below we describe this approach using the example

of a project which helps people into employment.

27

Lag and dropoff calculations – case study A project has the capacity to work with 100 residents, with the primary outcome

being to support these residents off JSA and into sustained employment. Using the

CBA model, the team calculates the maximum potential annual benefit to be:

100 x £8,308 = £800,308 x 5 years = £4m

However, the team only has the capacity to support 50 residents per year. It is also felt that half of the residents supported each year face extensive barriers to entering employment and are therefore highly unlikely to re‐enter employment in year 1. The team considers that:

20% of the maximum annual benefit will be achieved in year 1 (i.e. just under half the initial cohort enter employment), 65% in year 2 and 80% in year 3.

After year 3 some of the initial beneficiaries are likely to return to unemployed status due to natural change in the labour market. The team agrees that:

75% of the cohort are likely to be in employment in year 4, falling to 70% in year 5.

Based upon these percentages, the 5 year actual benefit = £2.48m

For each outcome category the Technical Specification aims to provide guidance on lag and

drop‐off rates (see section 8). However, the lack of longitudinal data relating to outcomes

such as entry into employment or reoffending makes it hard for the Technical Specification

to provide robust guidance on this topic. In the absence of robust data, project teams are

advised to combine the type of outcomes profile given above with their own expert

judgement.

7.5 Cashability Along with considering lag and drop‐off in outcomes, CBA analysis should (ideally) also

consider how cashable specific outcomes are. The term cashability refers to what extent the

achieving of a specific outcome (e.g. one fewer reoffending incident) will result in an

absolute reduction is fiscal expenditure (e.g. lower budgets for police and other criminal

justice agencies).

For some outcomes, such as benefits payments, cashability is almost 100% ‐ i.e. if somebody

enters employment the DWP no longer pays out JSA/ESA. For other categories of benefits

(in particular health and criminal justice outcomes) cashability is a lot lower, because

decommissioning a prison or a hospital wing requires reduction in service demand at scale;

and over a longer period of time.

28

Where possible, the Technical Specification provides guidance on the rates of cashability

associated with specific outcomes and hence what percentage of the total outcome should

be subtracted for the overall analysis. Where guidance is not available, project teams should

ask themselves questions such as:

Under what contractual basis is the reactive cost currently incurred – is it on a call off

basis, or is it under a long‐term outsourcing agreement?

Is there considerable infrastructure (e.g. a building) related to the outcome, which

makes it harder to decommission spend in the short‐term?

How many agencies are involved in delivering the intervention(s) that achieves the

outcome? If the answer is several, it may be harder to get lots of agencies to agree

to stop devoting resources to the same thing.

Is the monetary value of the outcome fairly small, thereby making it less likely that it

will be prioritised for decommissioning?

Is it politically controversial to decommission activity relating to this outcome?

The decision of whether to present CBA with, or without, a consideration of cashability

should be related to how the CBA analysis is going to be used. If a partnership has a pressing

need to stop doing ‘x’ activity in order to fund ‘y’, then cashability has to be considered. But

if a partnership is not too concerned with immediate savings, then they may choose to

ignore considerations of cashability for the time being.

29

8 Assumptions and unit benefits for each outcome This chapter sets out, for each outcome currently included in the CBA model, the evidence

and assumptions that have been built into Excel spreadsheet (readers who will not be

involved in undertaking CBA may wish to skip to chapter 9).

The chapter has sub‐sections for each outcome currently included in the CBA model. Each

sub‐section identifies the agencies that are likely to have a role in delivering and/or

benefitting from the specific outcome; specifies the ways in which outcomes should be

measured; and explains how the model monetises the fiscal, economic and social impact

associated with each outcome and how it apportions benefit across agencies, over time etc.

For each outcome the chapter provides some guidance on how cohort engagement and

impact percentages can be calculated. However, the precise method for working out these

two important percentages is largely dependant upon how information on project

performance is going to be collected (e.g. here analysts should summarise what monitoring

information they will be collecting and when, and define what will constitute the

achievement ‐or impact‐ of the outcome).

More sub‐sections will be added to this chapter as more outcomes are researched and

incorporated into the CBA model.

30

8.1 Reduced Benefits claimants – Job Seekers Allowance (JSA) Key partners ‐ DWP, DH, MoJ, HO, Local Authorities.

Metric – Reduction in the number of individuals claiming Job Seekers Allowance (JSA) in the project area.

At risk/targeted population – Individuals on JSA in the project cohort.

Calculation – Number of individuals on JSA as determined from NOMIS for large cohorts or

by local data collection for smaller cohorts. Where not otherwise available, gender and age

split determined from NOMIS data – assumed to be consistent across benefit types.

Engagement – Dependant on intervention.

Impact – Dependant on intervention. Note that the monetisation figures below are based

on entry to employment for a 12 month continuous period. Not all individuals entering

employment will stay in a job indefinitely, and therefore an assessment of the length of

employment should be included when calculating the impact of an intervention.

Monetisation – Using DWP Total Place Cost Benefit Framework guidance7.

The monetisation values per individual finding employment are outlined in the table below:

These figures are based on 2009/10 prices.

Monetised benefit per year per extra

individual in employment

Type of claimant Fiscal Economic Social

Reduced benefits payments JSA £7800

Improved health – savings to

NHS

JSA £516

17‐24 Male £532 £842 £230

25+ Male £266 £421 £115

17‐24 Female £130 £206 £56

Reduced crime

25+ Female £43 £69 £19

Increased income £3307

Fiscal benefits

7 Total Place Cost Benefit Framework ‐ Adam Robinson, EG Partnerships Division, DWP 2010

31

Benefits payments ‐ Reduced benefits claimants will result in savings to the exchequer

managed by the DWP (Annually Managed Expenditure).

Health ‐ The calculation to determine the impact on health service costs is based on a

reduction in NHS services of 33 percent as outlined in the DWP guidance document. These

savings will fall to the NHS. However, decommissioning of services is also necessary in order

to cash these savings. These costs relate to general NHS costs, and are not broken down to

include acute issues such as teenage pregnancy, drink and substance misuse. If these

outcomes are targeted, they should be assessed as specific outcomes and monetised

individually.

Crime ‐ For the reduced crime impact of employment, we have assumed that employment

provides a 48% increase in income compared to benefits. This is based on analysis across

Greater Manchester of off‐flows from benefits into different professions, assuming

beneficiaries earn 75% of the average net wage for these professions.

The DWP guidance is based on studies considering the impact of income on the propensity

to commit property crime. The recommendation in the guidance is to assume a 0.6%

reduction in property crime for each 1% increase in income. Using this approach results in a

29% reduction in the cost of crime committed by the individual on average. A detailed

summary of the DWP approach, including references to studies into the link between crime

and income, can be found in the DWP Social Cost‐Benefit Analysis Framework.8

Cost of crime figures have been reassessed using the DWP framework approach, splitting

the costs down into fiscal, economic and social benefits, using the latest cost of crime

figures (2009/10 prices).

Due to the differences in assessing the cohort for benefits payments/improved health

(determined by benefit claim type) and reduced crime (determined by age), this outcome is

split in the excel model. When determining the proportion of fiscal benefits to individual

agencies, the following percentages should be used:

Benefits Payment/improved health ‐ 94% DWP (AME), 6% NHS

Reduced crime – 35% Police, Probation 25%, 36% other CJS spend, 4% NHS

Economic benefits ‐ Increased income – the increase in earnings assumed per individual is

based on the same analysis as has been used to determine the crime impact above – see

Appendix 2 for details. To determine the increase in income for the individual employed

compared to being out of work, an element to cover the travel to work costs (£433 as

suggested in the DWP guidance) has been subtracted. Childcare costs have not been

subtracted from the increased earnings, as these can be thought of as a transfer payment


32

from one individual to another within Greater Manchester. We have also ignored the value

of any lost leisure time through entering employment, as the majority of individuals are in

involuntary employment and as such the value of this lost leisure time will be compensated

for by the improved personal wellbeing achieved through employment (see non‐monetised

benefits below).

Non monetised benefits – other benefits may accrue from individuals entering employment that have not been monetised. These could include personal wellbeing improvements gained through employment, and the impact on the wider community through increased employment, especially through being role models for others seeking employment. It is not possible at the present time to directly calculate the social value of employment as this will vary widely among cohorts. It is recommended that increases in confidence and self esteem are directly measured as outcomes and social value created calculated as outlined in chapter 9.

Deadweight:

o For appraisal – where no more specific deadweight value for the cohort can be

calculated, deadweight should be valued at 26.3% of the benefits. This is based on

an analysis of 16 People and Skills based evaluations carried out by BIS. 9

o For evaluation – outcomes and deadweight will be determined based on individual

case history before, during and at the end of the project. The number of months

worked in the previous 2 year period prior to the project intervention should be

assessed. Employment outcomes following the intervention should then be tracked

and the additionality calculated as the difference in work history prior to and

following the intervention.


33

8.2 Reduced Benefits Claimants – Incapacity Benefit (IB) Key associated government agencies ‐ DWP, DH, MoJ, HO, Local Authorities.

Metric – Reduction in the number of individuals claiming Incapacity benefit.

At risk / targeted population – Individuals on IB in the project cohort.

Calculation – Number on IB as determined from NOMIS10 for large cohorts or by local data

collection for smaller cohorts. Where not otherwise available, gender and age split

determined from NOMIS data – assumed to be consistent across benefit types.











Reduced benefits payments IB £8160


NHS

IB £1031

17‐24 Male £532 £842 £230

25+ Male £266 £421 £115

17‐24 Female £130 £206 £56

Reduced crime

25+ Female £43 £69 £19


10 www.nomisweb.co.uk


34

http://www.nomisweb.co.uk/


Fiscal benefits

Benefits payments ‐ Reduced benefits claimants will result in savings to the to the

exchequer managed by the DWP (Annually Managed Expenditure).

Health ‐ The calculation to determine the impact on health services costs is based on a






individually.

Crime ‐ for the reduced crime impact of employment, we have assumed that employment


Greater Manchester of off flows from benefits into different professions, assuming







and income can be found in the DWP Social Cost‐Benefit Analysis Framework.12

Costs of crime figures have been reassessed using the DWP framework approach, splitting







Benefits Payment/improved health – 89% DWP (AME), 11% NHS


Economic benefits


35

Increased income – the increase in earnings assumed per individual is based on the same

analysis as has been used to determine the crime impact above – see Appendix 2 for details.

To determine the increase in income for the individual employed compared to being out of

work, an element to cover travel to work cost (£433 as suggested in the DWP guidance) has

been subtracted. Childcare costs have not been subtracted from the increased earnings, as

these can be thought of as a transfer payment from one individual to another within

Greater Manchester. We have also ignored the value of any lost leisure time through

entering employment, as the majority of individuals are in involuntary employment and as

such the value of this lost leisure time will be compensated for by the improved personal

wellbeing achieved through employment (see non monetised benefits below).

Non monetised benefits – other benefits may accrue from individuals entering employment that have not been monetised. These could include personal wellbeing improvements gained through employment, and the impact on the wider community through increased employment, especially through being role models to others seeking employment. It is not possible at the present time to directly calculate the social value of employment as this will vary widely among cohorts. It is recommended that increases in confidence and self esteem are directly measured as outcomes and social value created calculated as outlined in chapter 9.

Deadweight:






worked in the previous 2 year period prior to the intervention should be assessed.

Employment outcomes following the intervention should then be tracked and the

additionality calculated as the difference in work history prior to and following the

intervention.


36

8.3 Reduced Benefits Claimants –Employment Support Allowance (ESA) Key associated government agencies ‐ DWP, DH, MoJ, HO, Local Authorities.

Metric – Reduction in the number of individuals claiming ESA.

At risk / targeted population – Individuals on ESA in the project cohort.

Calculation – Number on ESA as determined from NOMIS14 for large cohorts or by local data













Reduced benefits payments ESA £8500


NHS

ESA £1016

17‐24 Male £532 £842 £230

25+ Male £266 £421 £115

17‐24 Female £130 £206 £56

Reduced crime

25+ Female £43 £69 £19


14 www.nomisweb.co.uk


37

http://www.nomisweb.co.uk/


Fiscal benefits

Benefits payments ‐ Reduced benefits claimants will result in savings to the to the

exchequer managed by the DWP (Annually Managed Expenditure).







individually.

Crime ‐ for the reduced crime impact of employment, we have assumed that employment










Costs of crime figures have been reassessed using the DWP framework approach, splitting









Economic benefits


38

Increased income – the increase in earnings assumed per individual is based on the same



work, an element to cover travel to work cost (£433 as suggested in the DWP guidance) has







Non monetised benefits – other benefits may accrue from individuals entering employment that have not been monetised. These could include personal wellbeing improvements gained through employment, and the impact on the wider community through increased employment, especially through being role models to others seeking employment. It is not possible at the present time to directly calculate the social value of employment as this will vary widely among cohorts. It is recommended that increases in confidence and self esteem are directly measured as outcomes and social value created calculated as outlined in chapter 9.

Deadweight:









intervention.


39

8.4 Reduced Benefits Payments – Lone Parent Income Support (LPIS) Key associated government agencies ‐ DWP, DH, MoJ, HO, Local Authorities.

Metric – Reduction in the number of individuals claiming Lone Parent Income Support

(LPIS).

At risk / targeted population – Individuals on LPIS in the project cohort.

Calculation – Number on LPIS as determined from NOMIS for large cohorts or by local data













Reduced benefits payments LPIS £6380


NHS

LPIS £508

17‐24 Male £532 £842 £230

25+ Male £266 £421 £115

17‐24 Female £130 £206 £56

Reduced crime

25+ Female £43 £69 £19



40


Fiscal benefits

Benefits payments ‐ Reduced benefits claimants will result in savings to the exchequer

managed by the DWP (Annually Managed Expenditure)..







individually.

Crime ‐ For the reduced crime impact of employment, we have assumed that employment










Cost of crime figures have been reassessed using the DWP framework approach, splitting









Economic benefits


41

Increased income – The increase in earnings assumed per individual is based on the same



work, an element to cover travel to work costs (£433 as suggested in the DWP guidance) has







Non monetised benefits – Other benefits may accrue from individuals entering employment that have not been monetised. These could include personal wellbeing improvements gained through employment, and the impact on the wider community through increased employment, especially through being role models to others seeking employment. It is not possible at the present time to directly calculate the social value of employment as this will vary widely among cohorts. It is recommended that increases in confidence and self esteem are directly measured as outcomes and social value created calculated as outlined in chapter 9.

Deadweight:









intervention.


42

8.5 Skills Key associated government agencies ‐ DWP, BIS, Local Authorities.

Metric – Increase in population qualified to Level 2 and Level 3 skills.

At risk / targeted population – Working age individuals in project areas with < level 3 skills.

Calculation – Total numbers based on Census 2001 figures are to be used to estimate the

skill levels in the project areas. As these figures are not up to date and trackable over the

lifetime of the projects, evaluation should track individual’s skill profiles as they are engaged

with the project.


Impact – Dependant on intervention.

Monetisation – Based on %age uplift in wages as identified in BIS and IER research

findings.21 22

Fiscal benefits –increased taxation revenue per annum

Increase from < Level2 to Level 2 – £787

Increase from Level 2 to Level 3 – £1391

Economic benefits – increased annual earnings per individual:

Increase from < Level2 to Level 2 – £1208

Increase from Level 2 to Level 3 – £1925

Social benefits ‐ £0

Non monetised benefits – There are potential additional well‐being impacts on individuals,

families and community. These should be measured separately as outlined in chapter 9.

Deadweight:

o For appraisal CBA, assumed zero deadweight, as individuals in project are not likely

to be engaged with training providers without support.

o For evaluation CBA, we will use comparator areas to determine deadweight.

21 BIS (2011): Returns to vocational qualifications. Research Paper 53 pg9‐10

22 IER (2008): Net Benefits of Training Study 2008 p62‐64

43

8.6 School Readiness Key associated government agencies ‐ DfE, Local Authorities.

Metric – Improved school readiness (increased number of pupils achieving 78+ points at

Early Years Foundation Stage). Potential additional well‐being impact on individuals, families

and community as outlined in chapter 9.

At risk / targeted population – 0‐4 year old children in project areas.

Calculation – Local authority calculations of population of 0‐4 year olds in project areas.



Monetisation – There is assumed to be a reduction in the cost of special education needs

provision at school resulting from an increase in school readiness. However, it is not

possible to derive a fiscal saving as funding mechanisms for schools are not based on the

level of low level special educational needs. There will be long term economic impacts for

individual children who have an improved education. This link is currently being researched

by the DfE and will be included in later versions of this framework.

Fiscal benefits – £0

Economic benefits – £0. Long term benefits for individuals due to improved educational

outcomes currently being researched. However no short/medium benefits are currently

included in the analysis.


Non monetised benefits – Improved long term educational, employment and earnings

potential. Reduction in crime and Anti‐Social Behaviour of teenagers.

Deadweight:

o For appraisal CBA – deadweight based on trends in Early Years Foundation Stage

(EYFS) scores in the project area (assuming the EYFS score is agreed as the metric to

monitor outcomes.

o For evaluative CBA – deadweight assessed using comparator area for the project.

44

8.7 Adult Mental health Key associated government agencies ‐ DH, Local Authorities.

Metric – Reduced adult mental health problems (number of individuals with anxiety and

depression).

At risk / targeted population – Parents in receipt of Incapacity Benefit for Mental Health

reasons. N.B. This will underestimate the number of people at risk in project areas. Work

to understand the levels and costs of mental health problems is underway and will be

incorporated in future versions of this framework.

Calculation – When estimating the number of early years parents with mental health issues,

the calculation would be: Total number of parents of 0‐4 year olds in project area * Number

of adults claiming Incapacity Benefit for Mental Health reasons / Working age population in

project area



Monetisation – Based on Kings Fund report 200823 which uses 2007 figures.

Overall costs Cost per person

Number of

people (£m)

Service

Costs (£b)

Lost earnings

(£b)

Total costs

(£b)

Service

Costs (£)

Lost

earnings (£)

Total

costs (£)

Depression 1.24 1.68 5.82 7.5 £1,355 £4,694 £6,048

Anxiety 2.28 1.24 7.7 8.94 £544 £3,377 £3,921

Total 3.52 2.92 13.52 16.44 £830 £3,841 £4,670

Service costs include prescribed drugs, inpatient care, GP costs, other NHS services,

supported accommodation and social services costs.

Fiscal benefits – Service cost per person – £830

Economic benefits – Lost earnings per person ‐ £3841


Non monetised benefits – There are potential additional well‐being impacts on individuals,

families and community. These should be measured separately as outlined in chapter 9..

Deadweight:

23 “Paying the Price ‐ The cost of mental health care in England to 2026” ‐ Kings Fund, 2008

45

o For appraisal – assume zero deadweight as individuals in project areas currently do

not receive specific support and are therefore unlikely to improve their outcomes.

o For evaluation – outcomes and deadweight determined based on individual case

history before and during the project.

46

8.8 A&E attendance Key associated government agencies ‐ DH, Local Authorities.

Metric – Reduced avoidable A&E attendance.

At risk / targeted population – Number of 0‐4 year olds attending A&E with no significant

treatment.

Calculation – Attendances at A&E based on figures provided by GM Commissioning Business

Services.



Monetisation – Based on the NHS tariff for “No investigation (Referred/Discharged)” – HRG

Code VO8

Fiscal benefits – Cost of basic A&E attendance with no investigation – £52

Economic benefits – £0


Non monetised benefits – None identified.

Deadweight:

o For appraisal CBA – deadweight should be calculated based on trends in A&E

attendance in the project area.

o For evaluation CBA – use comparator area to determine deadweight.

47

8.9

48

8.10 Domestic Violence Key associated government agencies ‐ HO, MoJ, Local Authorities

Metric – Reduced incidents of domestic violence.

At risk / targeted population – Parents of 0‐4 year olds suffering from domestic violence

incidents. N.B. Concern has been raised about the quality of data recorded for Domestic

Violence.

Calculation – Number of parents at risk of domestic violence = number of parents of 0‐4

year olds in project area * Number of domestic violence incidents in project area / total

population of project area.



Monetisation – Based on Silvia Walby report for the Dti/Women and Equality Unit, 200924,

and The Cost of Domestic Violence, 200425

Fiscal benefits – Policing/courts/NHS cost per incident – £13160

Split of fiscal benefits based on analysis for the Cheshire West and Chester Whole Place

Community Budget: Local Authority 35%, NHS 29%, Police 14%, Probation 10%, Other CJS

12%

Economic benefits – Cost to employers per incident due to absence ‐ £6553

Social benefits – The costs of human and emotional impact of domestic violence ‐ £764026


Deadweight:

o For appraisal CBA – use domestic violence trend data to determine deadweight.

o For evaluation CBA – use comparator area to determine deadweight.

24 “The Cost of Domestic Violence: Up‐date 2009” Silvia Walby, Lancaster University, 2009

25 The Cost of Domestic Violence: Sylvia Walby, University of Leeds, 2004

26 Costs range from £240 for threats or “common assault” such as pushing, holding or slapping, to over £750,000 for domestic homicide. The chosen cost is a midpoint and represents kicking, or hitting with first.

49

8.11 Antisocial Behaviour Key associated government agencies ‐ HO, Local Authorities

Metric – Reduced number of anti‐social behaviour incidents

At risk / targeted population – Individuals carrying out antisocial behaviour.

Calculation – Total number of incidents of Anti Social Behaviour as recorded by GM Police,

broken down into incidents which result in further actions, and those incidents which do not

result in any action being taken.



Monetisation – Based on LSE paper 200327, Home Office Online Report 30/0528

Fiscal benefits – Reduced costs of dealing with incidents – no further action £35, further

action necessary £500.

Split of fiscal benefits will be related to local arrangements. As suggested split would be

Police 40%, Local authority 30%, RSLs 30%


Social benefits ‐ Physical and emotional impact on direct victims of crime – lowest impact

chosen – that of ‘theft – not vehicle’ ‐ £118 incident

Non monetised benefits – Reduced fear of crime; improved desirability of locality.

Deadweight:

o For appraisal CBA – deadweight based on trends in ASB incidents in project area.

o For evaluation CBA – deadweight assessed by use of comparator area.

27 “The Economic and Social Costs of Anti‐Social Behaviour: A Review”, London School of Economics and

Political Science, 2003.

28 The economic and social costs of crime against individuals and households 2003/04, Home Office Online

Report 30/05

50

8.12 Crime Key associated government agencies ‐ MoJ, HO, Local Authorities

Metric – Reduced incidents of crime (all crimes).

At risk / targeted population – Individuals carrying out crime.

Calculation – Total number of incidents crime (as measured by the British Crime Survey).

Multipliers should be used to covert from recorded crime to actual crime. The average

multiplier for GM for 2011/12 was 5.24.



Monetisation – Updated figures based on Home Office Online Report 30/0529 and

Integrated Offender Management Value for Money Toolkit 30

Fiscal benefits – £3316 per crime. Agency split NHS 17%, Police 38%, Probation 3%, Other

CJS 42%.

Economic benefits – £3843 per crime.

Social benefits ‐ Physical and emotional impact on direct victims of crime = £8553 per crime.


Deadweight:

o For appraisal CBA – deadweight based on trends in crime incidents in project area.


29 The economic and social costs of crime against individuals and households 2003/04, Home Office Online

Report 30/05

30 Revisions made to the multipliers and unit costs of crime used in the Integrated Offender Management

Value for Money Toolkit, September 2011

51

8.13 Reoffending Key associated government agencies ‐ MoJ, HO, Local Authorities

Metric – Reduced reoffending.

At risk / targeted population – Previously convicted offenders.

Calculation – All offenders deemed at risk of reoffending


Impact – Dependant on intervention – Percentage of offenders who do not reoffend.

Monetisation – Greater Manchester Transforming Justice Pilot – Financial Incentive Model

Fiscal benefits – CJS costs ‐ £5731. Based on analysis using average interventions for short

term offenders and the metrics in the Financial Incentive model (assuming total costs, not

just cashable values). Police costs not currently included.

Economic benefits – £0 Not currently included.

Social benefits ‐ £0 Not currently included.


Deadweight:

o For appraisal CBA – deadweight based on average reoffending rates for similar

cohorts.

o For evaluation CBA – deadweight assessed by use of comparator area or propensity

score matching.

52

8.14 Reduced housing provider costs from evictions Key associated government agencies ‐ Housing providers, Local Authorities

Metric – Reduced housing evictions.

At risk / targeted population – Families at risk of eviction from social housing.

Calculation – Total number at risk identified by RSLs.



Monetisation – Based on Manchester City Council analysis from Eastlands Homes. The Real

Cost of Poor Housing, M Davidson, M Roys, S Nicol, D Ormandy and P Ambrose, BRE

Fiscal benefits – £8180. This is broken down into:

Average level of arrears at the point of an eviction ‐ £4,702.08.

Average cost to repair property ‐ £2,327 (average for all voids not just those

vacated from eviction which can often require more repairs).

Average rent loss per property whilst repairing/reletting ‐ £534.05.

Average court costs associated with an eviction ‐ £195.

Officer time associated with progressing an eviction ‐ £422.


Social benefits ‐ Reduced health impact of poor housing ‐ £187.5031

Non monetised benefits – Reduced homelessness.

Deadweight:

o For appraisal CBA – assumed to be zero.

o For evaluation CBA – trends in evictions in comparator area used to determine

deadweight.

31 A Feb 2010 study found that 4.8 million homes in England (22%) have what are called category 1 hazards

arising from defects as assessed using the Housing Health and Safety Rating System. The cost of these housing

hazards was £1.5bn per year, including costs to the NHS (£600m) and to the individual and to society through

loss of earnings associated with the health impacts of these hazards. The cost per household (excluding costs

to the NHS) is £187.50. This represents the value of avoiding treatment for ill health, and lost earnings. The

Real Cost of Poor Housing, M Davidson, M Roys, S Nicol, D Ormandy and P Ambrose, BRE

53

8.15 Children in Care Key associated government agencies ‐ DfE, Local Authorities

Metric ‐ Reduced numbers of children in care.

At risk / targeted population – Children identified as at risk of safeguarding – e.g. for FIP

programmes.

Calculation – Number of children as identified by project team.



Monetisation – Based on LA costs of children’s services.

Fiscal benefits – Removed cost of safeguarding – £42640 – average cost per child in foster

or residential care in Manchester City Council area.


Social benefits ‐ Reduced harm to children, based on the costs of human and emotional

impact of domestic violence ‐ £7,64032

Non monetised benefits – improved wellbeing and future opportunities of children.

Deadweight – analysis of trends in the project area of %ages of children who are identified

as at risk of safeguarding, who do not then reach the point at which safeguarding occurs.

32 Costs range from £240 for threats or “common assault” such as pushing, holding or slapping, to over £750,000 for domestic homicide. The chosen cost is a midpoint and represents kicking, or hitting with first.

54

8.16 Drug abuse Key associated government agencies ‐ NHS, MoJ, HO, Local Authorities

Metric – Numbers entering treatment for drug abuse.

At risk / targeted population – Individuals with drug or alcohol problems.

Calculation – Total numbers entering treatment

Engagement – Dependant on intervention

Impact – Dependant on intervention

Monetisation – Based on The Drug Treatment Outcomes Research study (DTORS): Cost‐

effectiveness analysis.

Fiscal benefits – NHS ‐ £1686, Criminal Justice system ‐ £10,145

Economic benefits – £8216 per year saving in not buying drugs.

Social benefits – Health impact £1,500 for drug abuse.33


Deadweight:

o For appraisal CBA – deadweight based on trends in drug abuse in project area.


33 The Drugs Treatment Outcomes Research Study suggests that drug treatment leads to 0.05 extra QALYs

compared to no drug treatment. A value of £30,000 per QALY is used in this analysis. See table 12.

55

8.17 Alcohol dependency Key associated government agencies ‐ NHS, Local Authorities

Metric – Numbers entering treatment for alcohol dependency.

At risk / targeted population – Individuals with alcohol problems.

Calculation – Total numbers entering treatment

Engagement – Dependant on intervention

Impact – Dependant on intervention

Monetisation – Nice Clinical Guidance 11534, Liverpool Public Health Observatory:

Prevention Programmes Cost‐Effectiveness Review: Alcohol, and The Drug Treatment

Outcomes Research study (DTORS): Cost‐effectiveness analysis.

Fiscal benefits – Savings to the NHS ‐ £1800 per year


Non economic benefits – Health impact on individual of entering treatment. This is

calculated as £699 for alcohol abuse35.


Deadweight:

o For appraisal CBA – deadweight based on trends in alcohol dependency in project

area.


34 Alcohol‐use disorders: alcohol dependence – Costing report, National Institute for Health and Clinical

Excellence, 2011

35 There are a range of studies looking at the impact of alcohol interventions. One analysis of brief

interventions delivered in GP surgeries found that they led to an additional 0.0233 QALYs (Quality Adjusted

Life Years) per person. A value of £30,000 per QALY is used in this analysis.

56

8.18 Truancy Key associated government agencies ‐ DfE, Local Authorities

Metric ‐ Reduced numbers of children absent from school for >15% of the time.

At risk / targeted population – Children identified who are not in school for >85% of school

year.

Calculation – Number of persistently truanting children as identified by project team.



Monetisation – Based on analysis of Manchester City Council truancy services costs, and

Misspent Youth, 200736.

Fiscal benefits – £476 per year. This is based on a £52 saving in health costs and a £424

saving to criminal justice agencies. Local authority initiatives around truancy are generally

proactive focussed on attendance rather than reactive as a result of children being at

school. Therefore they should be accounted for under the costs section of this

methodology.

Economic benefits – Increased earnings of £842 per year. N.B. This will only be relevant

when the child is of working age, and this may be beyond the timescale for the modelling.

Social benefits ‐ £0. Not measured directly with this outcome. Increases in wellbeing

should be recorded and valued separately – see section 9.

Non monetised benefits – improved wellbeing and future opportunities of children who are

not truants arising from less class disruption.

Deadweight – analysis of trends in the project area of truancy.

36 Misspent Youth: The costs of truancy and exclusion, New Philanthropy Capital, 2007

57

8.19 Exclusion Key associated government agencies ‐ DfE, Local Authorities

Metric ‐ Reduced numbers of children excluded from school.

At risk / targeted population – Children identified who are at risk of exclusion (may have

had temporary exclusions in the past).

Calculation – Number of children at risk of exclusion as identified by project team.



Monetisation – Based on Misspent Youth, 200737.

Fiscal benefits – £8247 per year. This is based on a £7181 cost per year of alternative

educational arrangements for excluded children (e.g. Pupil Referal Units), a £64 saving in

health costs and a £1002 saving to criminal justice agencies.

Economic benefits – Increased earnings of £529 per year. N.B. This will only be relevant

when the child is of working age, and this may be beyond the timescale for the modelling.



Non monetised benefits – improved wellbeing and future opportunities of children who are

not at risk of exclusion arising from less class disruption.

Deadweight – analysis of trends in the project area of exclusion.

37 Misspent Youth: The costs of truancy and exclusion, New Philanthropy Capital, 2007

58

8.20 Reduced Hospital admissions Key associated government agencies ‐ NHS.

Metric ‐ Reduced numbers of patients admitted to hospital.

At risk / targeted population – Population at risk of hospitalisation – especially the elderly.

Calculation – Number of admissions to hospital of the modelled cohort

(measured/predicted).



Monetisation – Median cost based on NHS Reference Costs 2010/1138.

Fiscal benefits – £2384 per year.

Economic benefits – £0. Not calculated for this outcome.



Non monetised benefits – reduced sickness absence, and related economic impact.

Deadweight – analysis of trends in the project area.

38 Department of Health, NHS 2010‐11 reference costs publication

59

8.21 Adult Social Services Residential Care Key associated government agencies ‐ Local Authorities

Metric ‐ Reduced numbers of adults needing residential care.

At risk / targeted population – Elderly populations who are at risk of needing residential

care, but could be supported better in the community.

Calculation – Number of adults at risk of needing residential care.



Monetisation – Based on PSSRU Unit Costs of Health and Social Care 2011.

Fiscal benefits – £309/week. Based on 2/3 of costs falling to Local Authorities.

Economic benefits – £155/week. Based on 1/3 of costs falling to private individuals.




Deadweight – analysis of trends in the project area.

60

9 Social Value

9.1 Introduction to Social Value As well as measuring the economic and fiscal benefits created by the interventions being evaluated, our CBA model also has the capacity to measure and value the social benefits created. The importance of measuring social value is increasingly recognised. Guidance on value for money in the Treasury Green Book states that ‘Wider social and environmental costs and benefits for which there is no market price also need to be brought into any assessment. They will often be more difficult to assess but are often important and should not be ignored simply because they cannot easily be costed.’39 More recently, the Office for National Statistics has started to measure subjective well-being among the UK population alongside more traditional indicators such as employment and household situation. Announcing the change, the Prime Minister outlined the importance of non-financial measures of success: ‘When a country is hit by an earthquake, that can increase GDP, because of the extra spending on reconstruction. When a city is torn apart by crime and disorder, that actually increases GDP, because we spend money on locks, and more people get employed in security. When someone falls seriously ill, that can increase GDP, because of the cost of buying the drugs and paying for care… So, destruction, crime, disease – in a very crude way all these things can amount to progress in terms of GDP… You’ve got to take practical steps to make sure government is properly focused on our quality of life as well as economic growth.’40 Conventional forms of CBA often exclude these wider social benefits. However disciplines such as Social Accounting and Audit have attempted to identify and quantify the social change created. More recently Social Return on Investment41 (SROI) has gone further by monetising this social change, and allowing the creation of a CBA framework that accounts for social, economic and environmental costs. This allows a more explicit analysis of the trade-offs between different types of value (for example, is a social benefit worth the environmental impact that the intervention will create) and helps create a broader understanding of value for money. 39 The Green Book : Appraisal and Evaluation in Central Government, HM Treasury 2003

40 David Cameron, 25 November 2010

41 For more information see the SROI guide, published by the UK Cabinet Office, and available here: http://www.thesroinetwork.org/publications/doc_download/51-sroi-guide-2009-for-printing-out

61

http://www.thesroinetwork.org/publications/doc_download/51-sroi-guide-2009-for-printing-out

SROI’s development in the UK has been driven by organisations such as the new economics foundation and the SROI Network, and has been funded by the UK Office for Civil Society and the Scottish Government (through the SROI Project).42 It is increasingly used to measure value-for-money and is recommended by the National Audit Office.43 This CBA incorporates many of the features of Social Return on Investment. It identifies a number of potential social benefits, monetises them, and outlines how they can be measured and incorporated into the CBA. This chapter is set out as follows:

1. Introduction to Social Value 2. The Social Outcomes to be included 3. Monetisation of Social Outcomes 4. Outcome measurement 5. Example measurement tools

9.2 The Social Outcomes to be included The potential social outcomes that may arise from the different interventions being evaluated has been discussed with a number of local stakeholders involved in delivering the interventions. Inevitably, the wide range of interventions is likely to lead to a wide range of social outcomes. In order to ensure comparability between different interventions a common outcomes (or benefits) framework has been used in this CBA model. Many of the social outcomes experienced by individuals are well-being outcomes. This CBA model draws on the well-being model developed by the new economics foundation’s Centre for Wellbeing, in the National Accounts of Well-being44 framework, as below.

42 http://www.socialimpactscotland.org.uk/about-/sroi-project-.aspx

43 See: www.nao.org.uk/sectors/civil_society/successful_commissioning/successful_commissioning/general_principles/value_for_money/vfm_and_tsos.aspx

44 New Economics Foundation, (2009) National Accounts of Well-being

62

http://www.socialimpactscotland.org.uk/about-/sroi-project-.aspx

http://www.nao.org.uk/sectors/civil_society/successful_commissioning/successful_commissioning/general_principles/value_for_money/vfm_and_tsos.aspx

http://www.nao.org.uk/sectors/civil_society/successful_commissioning/successful_commissioning/general_principles/value_for_money/vfm_and_tsos.aspx

The model shows three levels of well-being components. At the top, well-being is divided into personal and social well-being. Each of these is further divided into a number of well-being components, and personal well-being is further divided into sub-components. The five personal well-being components and the two social well-being components are used as the basis for the well-being outcomes in this cost-benefit analysis. The total list of social outcomes is defined below.

63

Social Outcomes

Outcome Type Outcomes / Benefits Description

Increased confidence / self-esteem

Drawn from the national accounts of well-being model (where it is described as resilience and self-esteem)

Reduced isolation Drawn from the national accounts of well-being model (where it is described as supportive relationships)

Positive functioning

Drawn from the national accounts of well-being model. This includes features such as autonomy and meaning and purpose

Improved well-being of individuals

Emotional well-being Drawn from the national accounts of well-being model

Reduced social impact of domestic violence

The human and emotional impact of domestic violence on the victim

Reduced social impact of anti-social behaviour

The human and emotional impact of anti-social behaviour on the victim

Reduced social impact of crime

The human and emotional impact of crime on the victim

Reduced health impact of alcohol

The health impact of alcohol abuse

Reduced health impact of drugs

The health impact of drug abuse

Improved health / well-being of individuals brought about by reduction of specific problems

Recued health impact of poor housing

The health impact of poor quality housing

Improved family relationships

Drawn from the national accounts of well-being model (where it is described as supportive relationships)

Improved family-well-being

Positive functioning Drawn from the national accounts of well-being model. This includes features such as autonomy and

64

meaning and purpose

Emotional well-being Drawn from the national accounts of well-being model

Confidence / self-esteem Drawn from the national accounts of well-being model (where it is described as resilience and self-esteem)

Improved children’s well-being

Reduced harm to children The human and emotional impact of domestic violence on the child

Sense of trust & belonging Drawn from the national accounts of well-being model


Drawn from the national accounts of well-being model. This includes features such as autonomy and meaning and purpose

Improved community well-being

Improved relationships Drawn from the national accounts of well-being model (where it is described as supportive relationships)

65

9.3 Monetisation of Social Outcomes The approach of this CBA model is to take one monetisation approach for the well-being outcomes drawn from the National Accounts of Well-being framework, and to draw on other sources for the other more specific health and well-being benefits. Measurement of subjective well-being is a relatively new discipline, and there have been relatively few attempts to value well-being. However, equating well-being with mental health allows us to use healthcare economics to monetise well-being. The Centre for Mental Health has attempted to put a cost on mental illness through the use of QALYs (Quality Adjusted Life Years).45 Their report looks at the average loss of health status in QALYs from a level 3 mental health problem, i.e. severe problem, (0.352 QALYs) and values this by using the NICE (National Institute for Health and Clinical Excellence) cost effectiveness threshold of £30,000 per QALY. Equating well-being with mental health therefore allows a valuation of overall well-being of 0.352 x £30,000 = £10,560 per year. The result is divided between different domains of well-being as shown below. Division of Well-being value

Well-being domain Proportion of overall value

Financial value

Confidence / self-esteem46 10% £1,056

Positive functioning 10% £1,056

Emotional well-being 10% £1,056

Vitality (not used in this model) 10% £1,056

Personal well-being

Satisfying life (not used in this model) 10% £1,056

Improved / supportive relationships, or reduced isolation

25% £2,640 Social well-being

Trust and belonging 25% £2,640

45 Centre for Mental Health, The economic and social costs of mental illness, June 2003, updated October 2010

46 Referred to as resilience / self-esteem in the National Accounts of well-being

66

Other outcomes have been monetised through a variety of methods. The table below shows the methodology for monetisation of all social outcomes in the Cost Benefit analysis. Financial values Outcome

Type Outcomes / Benefits

Description Value Source


£1,056 Well-being valuation model

Reduced isolation





Emotional well-being

See table on ‘division of well-being value’ above

£1,056

67

Well-being valuation model

Reduced social

pact of domestic violence

im

The costs of human and emotional impact of domestic violence. Costs range from £240 for threats or “common assault” such as pushing, holding or slapping, to over £750,000 for domestic homicide. The chosen cost is a midpoint and represents kicking, or hitting with first.

£7,640

Professor Sylvia Walby, University of Leeds:

The Cost of Domestic Violence, September 2004. See table 11.1

Reduced social impact of anti-social behaviour

The physical and emotional impact on

direct victims of a wide variety of crimes has been calculated. The lowest impact is for ‘theft – not vehicle’, and this has been selected to represent the social impact of anti-social behaviour

£118

Home Office Online Report 30/05: The economic and social costs of crime against individuals and households 2003/04

Improved health / well-being of individuals brought about by reduction of specific problems

Reduced social impact of

The social impact of crime is calculated by combining Physical and Emotional Impact on Direct Victims of different

£1480/ incident

Home Office Online Report 30/05: The economic and

crime types of crime. social costs of crime against individuals and households 2003/04

Reduced health impact of alcohol

There are a range of studies looking at the impact of alcohol interventions. One analysis of brief interventions delivered in GP surgeries found that they led to an additional 0.0233 QALYs (Quality Adjusted Life Years) per person. A value of £30,000 per QALY is used in this analysis.

£699

Liverpool Public Health Observatory: Prevention Programmes

Cost-Effectiveness Review: Alcohol


The Drugs Treatment Outcomes Research Study suggests that drug treatment leads to 0.05 extra QALYs compared to no drug treatment. A value of £30,000 per QALY is used in this analysis.

£1,500

The Drug Treatment Outcomes Research study (DTORS):

Cost-effectiveness analysis. (See table 12)


A recent (Feb 2010) study found that 4.8 million homes in England (22%) have what are called category 1 hazards arising from defects as assessed using the Housing Health and Safety Rating System.

The cost of these housing hazards was £1.5bn per year, including costs to the NHS (£600m) and to the individual and to society through loss of earnings associated with the health impacts of these hazards. The cost per household (excluding costs to the NHS) is £187.50. This represents the value of avoiding treatment for ill health, and lost earnings.

£187.50

M Davidson, M Roys, S Nicol, D Ormandy and P Ambrose, BRE: The Real Cost of Poor Housing

68





Improved family-well-being




Confidence / self-esteem



Improved children’s well-being

Reduced harm to children

The costs of human and emotional impact of domestic violence. Costs range from £240 for threats or “common assault” such as pushing, holding or slapping, to over £750,000 for domestic homicide. The chosen cost is a midpoint and represents kicking, or hitting with first.

£7,640

Professor Sylvia Walby, University of Leeds:

The Cost of Domestic Violence, September 2004. See table 11.1

Sense of trust & belonging




Improved community well-being

Improved relationships



9.4 Measuring Social Outcomes Measurement of social outcomes can often be more complicated than measuring fiscal outcomes. An individual either receives Job Seekers Allowance or not, and this is relatively easy to track over time. For the purposes of this CBA model, some of the social outcomes can be measured in the same way; for example, if we measure the number of people entering alcohol treatment, the monetary value above effectively makes a prediction as to their level of recovery and the corresponding health benefits.

69

However, credibly predicting the change in well-being outcomes such as self-esteem through the change in employment status is much more difficult. To measure these social outcomes rigorously, we need to use distance travelled measures to assess the extent to which someone’s self-esteem (for example) has increased. This model does not dictate exactly which outcome measures should be used, but some examples of pre-existing tools are given below in section 9.5. The essential features of such tools are that they measure the outcome on a scale, and that they can be used for multiple readings. Movement on these scales can then be inputted into the model. For example, if someone moves from point three to point five on a ten-point scale, then they receive two-ninths of the outcome.47 The different measurement approaches needed for each outcome are shown below. Examples are shown in section 9.5. Measurement approaches

Outcome Benefits Description


Distance travelled well-being measurement

Reduced isolation Distance travelled well-being measurement

Positive functioning Distance travelled well-being measurement




Reduced social impact of domestic

nce viole

Reduced incidents of domestic violence (see 8.2.6)

Reduced social act of anti-social

behaviour imp

Reduced number of anti-social behaviour incidents (see 8.2.7)

Reduced social act of crime imp

Reduced number of crimes (see 8.2.7B)

Improved health / well-being of individuals brought about by reduced domestic violence / ASB / crime / alcohol / drugs / poor Reduced health Numbers entering treatment (8.2.9B)

47 On a ten-point scale, there are nine possible steps that can be taken (one to two, two to three etc). If someone moves from three to five, they have moved two of these nine steps, and therefore take two-ninths of the value.

70

impact of alcohol


Numbers entering treatment (8.2.9B)

housing


Reduced housing evictions (see 8.2.8)



Positive functioning Distance travelled well-being measurement Improved family-well-being



Confidence / self-esteem

Distance travelled well-being measurement Improved children’s well-being Reduced harm to

children Reduced numbers of children in care (see 8.2.9)

Sense of trust & belonging


Positive functioning Distance travelled well-being measurement Improved community well-being

Improved relationships


Other necessary data Deadweight: The previous chapters outline how deadweight will be calculated for the different benefit types. The same level of deadweight will be used for social outcomes. So, for example, when deadweight is set at 26.3% at the appraisal stage for the fiscal and economic benefits of ‘Reduced Benefit Payments’, then we will also use a deadweight figure of 26.3% for any social outcomes that arise as a result. This assumes that the social, economic and fiscal outcomes are closely linked, which is likely to be the case. At the evaluation stage, the tracking of employment outcomes for individuals would be used to predict deadweight for social as well as economic and fiscal outcomes.

71

9.5 Example measurement tools This section gives examples of potential distance-travelled well-being questions that might be used. The table below shows the original European Social Survey questions and scales that were used to create the National Accounts of well-being48. Question scales were aligned, based on question wording and original numbering to invert scales where appropriate, so that a high score always represents a positive answer. The aligned scale used for each survey question is shown (with ‘inv’ indicating questions where the scale was inverted), and the value of the mean and standard deviation calculated according to the aligned scale Well-being questions

Well-being component

Question Aligned Scale49

Mean(on aligned scale)

Standard deviation


Taking all things together, how happy would you say you are?

0–10 7.27 1.92

Resilience and self-esteem (self-esteem)

In general I feel very positive about myself (agree – disagree)

1–5 (inv) 3.85 0.81

Positive functioning (competence)

Most days I feel a sense of accomplishment from what I do (agree – disagree)

1–5 (inv) 3.81 0.79

Supportive relationships

How much of the time spent with your immediate family is enjoyable?

0–6 5.10 1.05

Supportive relationships

There are people in my life who really care about me (agree – disagree)

1–5 (inv) 4.35 0.73

48 The questions can be seen, and the full survey taken, at www.nationalaccountsofwellbeing.org. The full survey contains around 50 questions, but this is likely to be far too many for effective evaluation of individual interventions so only a few have been selected here.

49 The scales for these questions are as follows. Emotional well-being: Extremely unhappy – 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, Extremely happy – 10, Don’t know. Agree – disagree scales: Agree strongly, Agree, Neither agree nor disagree, Disagree, Disagree strongly, Don’t know. Supportive relationships (time with immediate family): None of the time – 0, 1, 2, 3, 4, 5, All of the time – 6, Don’t know. Trust & belonging: Not at all – 0, 1, 2, 3, 4, 5, A great deal – 6, Don’t know

72

http://www.nationalaccountsofwellbeing.org/

Trust and belonging

To what extent do you feel that people treat you with respect?

0–6 4.43 1.16

Alternatively, a number of outcomes measurement tools have been developed that can help evaluate specific interventions. These are often designed to be primarily therapeutic tools, and can potentially be integrated more easily into every day practice. Triangle Consulting have created a range of Outcomes Stars50, which identify a range of outcomes for specific interventions and provide measurement tools for each outcome. These are typically on a ten point scale, but include a description of what each point looks like, so that it is far easier to asses where a person should be placed on a scale. There are also more specific tools for specific types of intervention that might be used, such as the Bridge Carer Support Outcome Profile (CSOP)51 for families and carers of substance misusers. Community well-being questions Measuring well-being outcomes for members of a local community is often much harder, as it is more difficult to engage and get reliable answers to questions. However, there are two sources that might be particularly useful. Over the page are some distance travelled well-being questions for members of the community put together by the Community Development Foundation and nef consulting (the new economics foundation).52 These ask about levels of well-being now, and three years ago, thereby getting ‘distance travelled’ with one reading. While this is less reliable than doing two separate measurements at different points in time, the difficulty of re-contacting the same people means it is often much more practical. Secondly, the government is now measuring well-being as part of its Integrated Household Survey, and at some point in the future local well-being data will be available. However, this isn’t available in the short term.53

50 www.outcomesstar.org.uk/

51 www.adfam.org.uk/docs/csop_impact.pdf

52 www.cdf.org.uk/web/guest/publication?id=362954

53 For initial, top-level data, see www.ons.gov.uk/ons/dcp171776_244488.pdf

73

http://www.outcomesstar.org.uk/

http://www.adfam.org.uk/docs/csop_impact.pdf

http://www.cdf.org.uk/web/guest/publication?id=362954

http://www.ons.gov.uk/ons/dcp171776_244488.pdf

Example community well-being questions

74

10 Overall analysis approach

10.1 Discounting In order to determine the net present value of the costs and benefits for use in calculations

of overall efficiency and benefit cost ratios, the values of future costs and benefits need to

be discounted to current prices. The CBA model uses a standard discount factor of 3.5%, as

stipulated in the Green Book.

10.2 Calculation of Benefit Cost ratios The approach to using benefit cost ratios is detailed in section 5.1.

The ratio is calculated by the following formula:

Benefit Cost Ratio = Net present value of the benefits / Net present value of the costs

Projects with a benefit cost ratio > 1 are economically viable. The higher the ratio, the

greater value for money provided by the approach. This allows prioritisation of projects for

future scale up.

Benefit cost ratios are calculated for each of the benefit categories – fiscal, economic and

social.

75

10.3 Calculation of net efficiency Net efficiency measures are described in section 5.2. It is a measure of the overall saving

through implementation of the scheme. This efficiency is simply calculated by the following

formula:

Net present efficiency = Net present value of the benefits – net present value of the costs

If this value is positive, a project is economically viable.

10.4 Distributional Analysis Distributional analysis using weighting of benefits to different sectors of society is not

carried out by GM CBA model. However, it is recommended that practitioners carry out a

high level assessment of which groups in society contribute and which groups in society

benefit from a project, in order to understand the distributional impact of implementing the

project that is being assessed.

76

11 Further information

11.1 Knowledge Hub site We have set up a GM CBA Knowledge Hub Group on the LGA Knowledge Hub website. This

site is the location for all current versions of the excel model to use for the CBA analysis,

along with this technical specification document, the cost database and other user guides.

It is also a location for analysts to share expertise, publish completed analysis and to ask

question and request assistance from other analysts. The site can be found by searching for

“Greater Manchester CBA Network” on the site or following this link:

https://knowledgehub.local.gov.uk/group/greatermanchestercostbenefitanalysisnetwork

Registration is required, but this is open to anyone who would like to use the GM CBA

model.

11.2 Evaluation Guides The audience for Greater Manchester’s CBA work is diverse. For example:

Central government AND local government;

Operational/delivery staff AND strategic/management staff;

CBA experts AND CBA novices.

In response, we have developed a range of tools and guides which explain our CBA

modelling work in a language and manner suited to the intended reader.

As noted in the introduction, the Technical Specification is aimed more at CBA experts than

novices. Accordingly, we have also produced ‘evaluation guides’ for use members of project

teams who may not yet possess a detailed understanding of CBA.

The evaluation guides are a project management tool. They explain how our CBA model

works and set out the core and secondary indicators, as listed in chapter 8, that a project

may wish to track in order to demonstrate achievements. For as many indicators as possible

we explain who collects the data, where it is reported and what projects need to do to

access this information. The evaluation guides also walk project teams through the process

they need to complete in order to identify their delivery costs.

The evaluation guides are designed to be as user friendly as possible, with suggested work

tasks and a worked example of how CBA can be applied to an intervention. The guides also

provide links to further sources of information and support.

The evaluation guides have been posted on our Knowledge Hub website.

77

11.3 Training and dissemination activities Alongside the evaluation guides, New Economy runs regular training events and seminars so

that a growing number of commissioners and project deliverers across GM and the UK are

able to gather and report data and information in a manner that fits with the GM CBA

model.

We also hold bi‐monthly networking sessions for analysts to share experience of using the

tools and techniques.

All GM’s work on CBA is reported on the GM CBA Knowledge Hub site. All are invited to join

the community, where members can download training materials, obtain contact details,

learn about upcoming conferences and events etc.

11.4 Bibliography In developing the GM CBA we have called on a wide range of secondary sources. These have

included:

o Research to improve the assessment of additionality – BIS, 2009

o A guide to Social Return on Investment – Cabinet office/New Economics Foundation,

2009

o “Costing A&E Attendances”, Dr Rod Jones, Healthcare Analysis and Reporting, 2008

o “The Economic and Social Costs of Anti‐Social Behaviour: A Review”, London School

of Economics and Political Science, 2003

o Green Book, HM Treasury

o Magenta Book, HM Treasury

o “Paying the Price ‐ The cost of mental health care in England to 2026” ‐ Kings Fund,

2008

o “The Cost of Domestic Violence: Up‐date 2009” Silvia Walby, Lancaster University,

2009

o “The cost of not ending child poverty ‐ How we can think about it, how it might be

measured, and some evidence”, Joseph Rowntree Foundation, 2008

o The Returns to Qualifications in England: Updating the Evidence Base on Level 2 and

Level 3 Vocational Qualifications – Centre for the Economics of Education report for

DCSF, 2007

o Total Place Cost Benefit Framework ‐ Adam Robinson, EG Partnerships Division,

DWP, 2010

78

79

o Volunteer Investment and Value Audit ‐ Volunteering England, 2007

As the scope of the model increases to consider new approaches and new outcomes this list

will be updated.

11.5 Model snapshots On the following pages we provide MS Excel snapshots of the GM CBA model. These serve

to illustrate how the model translates data on target/at risk populations into gross and net

outcomes and, ultimately, net cashable efficiencies at one and five years.

Project title: Peer Mentoring

Costs

RefCost category

Predicted costs (£)

Predicted costs notes/ assumptions Who pays?

Capital/ Revenue/ In-kind

Optimism bias correction

Total costs (5years)

% CostsYr1

% CostsYr2

% CostsYr3

% CostsYr4

% CostsYr5 Costs Yr1 Costs Yr2 Costs Yr3 Costs Yr4

C1Peer mentoring cost (Probation) £ 336,000

£1k per offender - Worksheet 2 and 3,

60% of costs - Worksheet 3

Probation Revenue 0% £ 336,000 50% 50% £ 168,000 £168,000 £ - £ -

C2Peer mentoring

cost (Local authority)

£ 224,000

£1k per offender - Worksheet 2 and 3,

40% of costs - Worksheet 3

Local Authority Revenue 0% £ 224,000 50% 50% £ 112,000 £112,000 £ - £ -

C3Mental Health

referals £ 138,600

£825 per person (Worksheet 3),

numbers based on benefits sheet

NHS Revenue 15% £ 159,390 50% 50% £ 79,695 £ 79,695 £ - £ -

C4Drug dependency

referals £ 679,168



NHS Revenue 15% £ 781,043 50% 50% £ 390,522 £390,522 £ - £ -

C5Alcohol

dependency referals

£ 94,976



NHS Revenue 15% £ 109,222 50% 50% £ 54,611 £ 54,611 £ - £ -

C6Training and

support of Peer Mentors

£ 60,000

Assumed £30000 annual cost for one

support/training worker

Local charity In-kind 40% £ 84,000 50% 50% £ 42,000 £ 42,000 £ - £ -

C7Reference costs -

existing drug dependency costs

-£ 67,917 10% of above costs, worksheets 2 and 3

NHS Revenue 15% -£ 78,104 50% 50% -£ 39,052 -£ 39,052 £ - £ -

C8Reference costs - existing alcohol

dependency costs-£ 9,498

10% of above costs, worksheets 2 and 3

NHS Revenue 15% -£ 10,922 50% 50% -£ 5,461 -£ 5,461 £ - £ -

C9 £ - £ - £ - £ - £ - C10 £ - £ - £ - £ - £ - C11 £ - £ - £ - £ - £ - C12 £ - £ - £ - £ - £ - C13 £ - £ - £ - £ - £ - C14 £ - £ - £ - £ - £ - C15 £ - £ - £ - £ - £ - C16 £ - £ - £ - £ - £ - C17 £ - £ - £ - £ - £ - C18 £ - £ - £ - £ - £ - C19 £ - £ - £ - £ - £ - C20 £ - £ - £ - £ - £ -

TOTAL £ 1,604,629 £ 802,315 £802,315 £ - £ -

80

Peer Mentoring

Benefits

Ref. OutcomesIncluded

outcome?Benefits

Who does benefit accrue

to?

Target population

Pop notes/ assumptions

Affected population / Predicted incidents

Affected pop notes/

assumptions

%age engagement

(effectiveness of reaching at risk

pop)

%age engagement

notes / assumptions

%age impact (effectiveness of changing skills /

attitudes / behaviours)

%age impact notes / assumptions

DeadweightDeadweight notes / assumptions

Unit fiscal benefit (£)

Unit economic benefit (£)

Unit non-economic benefit (£)

Monetisation evidence

financial yearGDP deflator

Unit cost notes /assumptions

Optimism bias correction

Annual Fiscal benefit

Annual Economic

benefit

Annual Non-economic

benefit

Assim

B1a Yes 420 JSA 70% 36% 24% £ 8,308 £ 3,307 2007-08 95.718DWP Total Place Cost Benefit

guidance note -40% £ 183,731 £ 73,134 £ -

B2a Yes 140 IB/ESA 70% 36% 24% £ 9,176 £ 2,947 2007-08 95.718DWP Total Place Cost Benefit

guidance note -40% £ 67,642 £ 21,724 £ -

B3a Yes 0 LPIS £ 6,888 £ 2,947 2007-08 95.718DWP Total Place Cost Benefit

guidance note -40% £ - £ - £ -

Yes 120 18-24 year old males 70% 36% 24% £ 235 £ 952 £ 261 2009-10 100

DWP Total Place Cost Benefit guidance note - reduction in crime based on increase in

income leading to a reduction in crime (2010 figures) police

costs only

-40% £ 1,419 £ 5,747 £ 1,576

Yes 9 18-24 year old females 70% 36% 24% £ 57 £ 232 £ 64 2009-10 100



costs only

-40% £ 26 £ 105 £ 29

Yes 401 25+ year old males 70% 36% 24% £ 118 £ 476 £ 130 2009-10 100



costs only

-40% £ 2,385 £ 9,621 £ 2,627

Yes 30 25+ year old females 70% 36% 24% £ 19 £ 77 £ 21 2009-10 100



costs only

-40% £ 29 £ 117 £ 32

B6 Mental health YesReduced health cost of interventions

NHS/Individuals

20 mentors, 7 clients per mentor,

4 sets of clients over 2 year period (6 months each)

16830% with mental health from Worksheet 2 70% 25% 5% £ 830 £ 3,841 2007-08 95.718

Reduced service costs and lost earnings - Kings Fund study (2008)

-40% £ 12,230 £ 56,628 £ -

B15Reduced drug dependency

YesReduced health and criminal justice costs

NHS, Police, CJS

560



11220% with drug dependency from Worksheet 2 70% 49% 5% £ 11,831 2009-10 100 Nigel Stott paper - DTORS -15% £ 347,692 £ - £ -

B16Reduced alcohol dependency

YesReduced health and criminal justice costs

NHS, Police, CJS

560



11220% with alcohol dependency from Worksheet 2

70% 60% 6% £ 1,800 2009-10 100 £1367 health -40% £ 45,723 £ - £ -

B17Reduced reoffending incidents

Yes

Reduced costs of courts, legal aid, custody and probation

Criminal Justice system

560



560All offenders at risk of reoffending 70% 85% 67% £ 5,731 2009-10 100 -25% £ 303,282 £ - £ -

TOTAL £ 964,159

B1b,2b,3b

Reduction in % WAP receiving

benefits (all types)

Decreased crime as people

move from unemployment

into employment

Police



Reduction in % WAP receiving

benefits (all types)

1) Fiscal benefit of moving people off

benefits and into work

2) Improved health outcomes

3) Increased income

1) DWP/HMT

2) DH

3) Individuals



560

560

560

81

Análisis coste beneficio CBA

Documents

Transcript of Análisis coste beneficio CBA