1

FILEY CASE STUDY: A GUIDE TO BENEFIT APPRAISAL

Introduction This case study provides the user with a “walkthrough” of the cost-benefit appraisal process.

Cost-benefit appraisal (CBA) is used by authorities to justify spending money on flood schemes. The ‘cost’ is the full price of a scheme and includes construction costs. The ‘benefit’ is how much a scheme reduces flood risk and prevents a flood causing damage (damage avoided).

Surface Water Flooding in Filey, North Yorkshire in 2007 affected hundreds of residential and a handful of commercial properties including an Ambulance Station. Depths of flooding were up to 1 to 1.5 metres, though many properties were flooded to just a few centimetres. Rescue and evacuation were an important part of Incident management.

Scarborough Borough Council and their consultants discussed options, with a short list as follows:

• Do Nothing • Do Minimum - Maintenance • Do Minimum - Demountable & Temporary Solutions • Do Something - Earth bund and storage option (without environmental enhancements) • Do Something - Earth bund and storage option (with environmental enhancements)

This benefit appraisal case study evaluates the benefits of the earth bund and storage option (without environmental enhancements).

This guide identifies 7 steps to complete the CBA process:

Step 1 - Receptor Data in the benefit area Step 2 - Modelling the flood for each return period Step 3 - Calculate the Annual Average Damages (AAD) Step 4 - Repeat Steps 2 -3 for the other scenarios (Do minimum, Do nothing) Step 5 - Robustness analysis: visualisation allows auditing of errors in modelling Step 6 - Benefit Cost Summary Step 7 - Visualisation of results for transparent decision making

Filey flood, July 2007 © George Gingell

2

Step 1 - Receptor Data in the benefit area The first step is to identify the benefiting area and the receptors at risk from flooding. In this example three benefiting areas are identified within Filey, North Yorkshire: Wooldale, Seadale and Whafedale (Figure 1)

Figure 1 – The three sub-areas of the Filey Case Study ©OpenStreetMap contributors – This data is available under the Open Database License

Within each area the receptors at risk are identified using the National Receptor Dataset (NRD), which georeferences all property, infrastructure and environmental assets (provided by The Environment Agency). In this example, we will focus solely on the direct impacts to residential property. Figure 2 illustrates the NRD data, visualised as a GIS output, where residential properties are categorised as a “dwelling”. Within the NRD, further detail is provided for each dwelling, such as ‘flat’, ‘semi-detached’ etc.

Wooldale

Seadale

Wharfedale

3

Figure 2 – An illustration of property points

In some cases, information for the property point is missing (this is shown as the blue rectangles in Figure 2). Semi-automated tools such as Google Streetview make auditing of data and gap filling fast and efficient.

The NRD does not contain the property threshold (the point at which flood damage begins, usually the door step or air brick where property has suspended floors). This needs to be estimated or measured during a field survey.

Within the NRD, each type of residential property is given an MCM code corresponding to a depth-damage curve (figure 3). It is important to ensure that the correct curve is attributed to the property in question.

MCM code Property Type 0 0.3 1.2 3

1 Residential Sector Average 1,031 21,598 32,106 50,598 11 Detached 937 30,433 45,144 72,059 12 Semi-detached 1,254 19,854 29,523 47,921 13 Terrace 1,089 18,161 26,933 42,162 14 Bungalow 875 28,583 43,228 71,672 15 Flat 731 19,645 28,965 41,289

Figure 3 – A small sample of an MCM residential depth-damage curve

These data are also available for all non residential data types for different charcacteristics of flooding (short and long duration/with and without flood warning). Other data is available on the MCM-Online for:

• Emergency services costs

4

• Evacuation costs and temporary accommodation costs • Damage to vehicles • Traffic disruption • Infrastructure Losses • Agricultural Losses • Recreation and ecosystem services

This case study will follow the process for evaluating residential property damages and link the results to other receptor losses and damages contributing significant further loss or damage.

One of the properties suffering minor flooding is 47 Unnamed Road off Scarborough Road. We will follow this property through the benefit appraisal process. #47 is a semi-detached property in the Seadale area; flooding here was less significant than in the two other neighbouring areas of Wooldale and Wharfedale. The corresponding depth-damage curve for this property would be the one highlighted in Figure 3.

At this stage of the process, you will know the charateristics of the receptor (property) in the benefit area (type, property threshold and its associated depth-damage curve). The next stage is to identify the flood depth at each property for all return periods (RPs).

Step 2 - Modelling the flood for each return period The 2007 flood was only one of a potential series of greater or lesser flood events. Mathematical modelling (in this case a product called InfoWorks) is used to simulate the flood depths for a range of these flood events. The threshold flood is where flooding commences and damages are zero. A range of sucessive floods allows the estimate of annual average damage for all floods over a long time period. Six such flood events were modelled for Filey.

Figure 4 – A 5-year flood with an annual probability of 20%

Flood Depth High (0.6m)

Flood Depth Low (0m)

#47

5

Figure 5 – A 10 year flood with an annual probability of 10%

Figure 6 – A 30 year flood with an annual probability of 3.33%

Flood Depth High (0.6m)

Flood Depth Low (0m)

Flood Depth High (1.0m)

Flood Depth Low (0m)

6

Figure 7 – A 50 year flood with an annual probability of 2%

Figure 8 - 100 year flood with an annual probability of 1%

Flood Depth High (1.4m)

Flood Depth Low (0m)

Flood Depth High (1.5m)

Flood Depth Low (0m

7

Figure 9 – A 200 year flood with an annual probability of 0.5%

Step 3 – Calculate the Annual Average Damages (AAD) For every property including #47 we now have:

• A property type. • A threshold level where flood water enters the property. • A flood depth for each of the modelled return periods, and as a result; • A damage value for every property for every return period.

Every input and calculation is stored against every property in the Environment Agency Standard Spreadsheet (Figure 10). At this stage, the first data checking process can begin. For example, an error can be found for the #12 Unnamed Road property, where the 30 year damage is greater than the 50 year damage.

Figure 10 – Individual ‘event’ property damage for each residential property under Do Nothing Scenario using EA standard spreadsheets (Seadale sub area)

Flood Depth High (1.5m)

Flood Depth Low (0m)

8

Property #47 has zero damages until the 200 year flood event. This may not be an error, as it may not have flooded in 2007. Unravelling such errors will be done later as part of a robustness analysis.

Damages are capped per property when the annual average damage accrued over the lifetime of any improvement scheme are greater than the market price of the property (taking average regional prices per property type). No property can have more damage than the total value of the property.

Within the same EA Standard Spreadsheet, the annual average damages (AAD) is automatically calculated. Present Value of Damage is £1.2 million using the fixed Treasury Discount rate and a 100 year time Horizon (Figure 11).

Figure 11 - Calculation of Annual Average Damages (Seadale) and Present value of Damages with other receptor damage/disruption included

The Loss Probability Curve plots the total damage for each return period. The area below this curve represents the annual average damages (£40,259) (Figures 11 and 12). The curve visualises a ‘good fit’. If the curve is not asymptotic to both axes then the annual average damages are more than likely to be overestimated.

Summary Annual Average Damage Sheet Nr.Client/Authority

Project name Option:

Project reference - Base date for estimates (year 0) Aug-11 First year of damage: 0 Prepared (date) 22/09/2011Scaling factor (e.g. £m, £k, £) £ Last year of period: 99 Printed 12/05/2014Discount rate 3.5% PV factor for mid-year 0: 29.813 Prepared by JC

Checked byApplicable year (if time varying) Checked date

Average waiting time (yrs) between events/frequency per year Total PV1 2 5 10 30 50 100 200 Infinity £

0.500 0.200 0.100 0.033 0.020 0.010 0.005 0Damage category Damage £Residential property - - 51,040 148,019 356,100 446,081 798,274 1,150,466 731,651 Ind/commercial (direct) - - - - - 7,578 7,578 8,588 3,464 Ind/comm (indirect) - - Traffic related - - Emergency services - - 2,858 8,289 19,942 25,405 45,128 64,907 41,166

- 3,025 12,705 37,510 58,685 64,130 69,575 75,020 145,044 Alt. Accommodation 3,348 6,078 12,156 25,546 52,326 85,801 119,276 91,820 Road Damage 2,376 12,277 51,390 85,973 100,848 121,407 141,966 187080Total damage £ - 8,749 84,958 257,364 546,245 696,367 1,127,762 1,560,223 Area (damagexfrequency) - 1,312 4,685 11,411 5,357 6,213 4,560 6,720

Total area, as above 40,259 PV Factor, as above 29.813Present value (assuming no change in damage or event frequency) 1,200,225 1,200,225 NotesArea calculations assume drop to zero at maximum frequency.Default value for the highest possible damage assumes continuation of gradient for last two points, an alternative value can be entered, if appropriate.One form should be completed for each option, including 'without project', and for each representative year if profile changesduring scheme life (e.g. sea-level rise)Residential property, Industrial / commercial (direct), and Other damages are itemised in Asset AAD sheet and automatically linked to this sheet

Scarborough Borough Council

Filey Flood Alleviation Strategy: Seadale only Do nothing

Extra Heating

9

Figure 12 – The loss-probability curve for Seadale, do nothing option.

Step 4 - Repeat Steps 2-3 for the other scenarios (Do minimum, Do nothing) The model is then run to establish the residual damages with each option as it is unlikely that any option will eliminate all flooding completely.

For schemes with 100 year or more standard of protection it is unlikely that residual damages will be more than 5% to 10% of Do Nothing damages. For simplicity in the case of Filey they are assumed to be zero.

Step 5 - Robustness analysis: visualisation allows auditing of errors in modelling The model is adjusted to eliminate obvious anomalies, picked up through visualisation of the results. The scenario illustrated in Figure 13 cannot happen in nature as the 30 year depth appears shallower than 10 year depth so model refinement was necessary.

Project: Filey Flood Alleviation Strategy: Seadale only Option: Do nothing

Frequency 0.000 0.500 0.200 0.100 0.033 0.020 0.010 0.005 0.000Damage £ 0 0 8748.5 84958.1027 257363.5571 546245.2198 696367.4288 1127761.92 1560223.33

0

200000

400000

600000

800000

1000000

1200000

1400000

1600000

1800000

0.000 0.100 0.200 0.300 0.400 0.500 0.600

Frequency

Damage £

10

Figure 13 – The initial modelling run showing erroneous results

Checking the modelled damages against reality is difficult if not impossible as by no means all modelled flood events have occurred, usually just one as in 2007 case. Using insurance claims and historic data can assist to some degree. Though insurance data uses financial replacement costs whilst FDGiA damages do not include VAT (a transfer payment) or betterment (it is assumed that all damaged goods are approximately half-way through their lives.)

Figure 14 - Selected insurance data for the 2007 flood

The following sources can be applied to ground truth the veracity of the modelling:

Indicates where 30-Year RP flood depths shallower than 10-year depths.

11

• Historic flood maps • Insurance data • Web sources (flicker, youtube) • Anecdotal evidence

Step 6 - Benefit Cost Summary Once all errors have been eradicated and final results obtained, the cost-benefit assessment can be completed using the EA Standard Spreadsheet. The cost of each Option (total scheme: Wharfedale, Wooldale and Seadale) needs also to be included. Costs include capital, maintenance and operational costs over a 100 year period (adhering to Government ‘long termism’ for major infrastructure schemes) discounted using Treasury discount rates. A 60% optimism bias is usually added at initial appraisal stage to reduce the difference between appraisal and outturn costs after the scheme is complete.

The Project Summary Sheet (Figure 15) automatically calculates the decision-making criteria (net present value, cost-benefit ratio and incremental cost-benefit ratio).

Property #47 is part of the Seadale sub area and Option 3c costs were divided equally between the 3 sub areas. The cost-benefit summary indicates that on its own using FDGiA rules the Seadale scheme is not cost beneficial for any of the options with a 0.9:1 ratio for Option 3c.

12

Figure 15 – Project Summary Sheet showing all Options for the Seadale case

Project Summary SheetClient/Authority Prepared (date) 22/09/2011

Printed 12/05/2014Project name Prepared by JC

Checked byProject reference Checked dateBase date for estimates (year 0) Aug-2011Scaling factor (e.g. £m, £k, £) £ (used for all costs, losses and benefits)Year 0 30 75Discount Rate 3.5% 3.00% 2.50%Optimism bias adjustment factor 60%Costs and benefits of options

Option number Option 1a Option 2a Option 2b Option 3c Option 3d

Option name Do NothingDo Minimum - Maintenance

Do Minimum - Demountable &

Temporay Solutions

g Earth bund and storage option

(without env.enhanceme

nts)

g Earth bund and storage option

(with env.enhanceme

nts)COSTS:PV capital costs 0 0 100,000 0PV operation and maintenance costs 0 1,308,281 1,308,281 0PV other 0 16,333 32,667 0Optimism bias adjustment 0 794,768 864,568 0PV negative costs (e.g. sales) 0 98,000 275,973 0PV contributionsTotal PV Costs £ excluding contributions 0 2,021,383 2,029,543 1,300,000 0Total PV Costs £ taking contributions into account 0 2,021,383 2,029,543 1,300,000 0BENEFITS:PV monetised flood damages 1,200,225 1,200,225 1,200,225 - PV monetised flood damages avoided 0 0 1,200,225 1,200,225PV monetised erosion damages 0 0 0 0 0PV monetised erosion damages avoided (protected) 0 0 0 0Total monetised PV damages £ 1,200,225 1,200,225 1,200,225 0 0Total monetised PV benefits £ 0 0 1,200,225 1,200,225PV damages (from scoring and weighting)PV damages avoided/benefits (from scoring and weighting)PV benefits from ecosystem servicesTotal PV damages £ 1,200,225 1,200,225 1,200,225 0 0Total PV benefits £ 0 0 1,200,225 1,200,225DECISION-MAKING CRITERIA:excluding contributionsBased on total PV benefits ( includes benefits from scoring and weighting and ecosystem services)Net Present Value NPV -2,021,383 -2,029,543 -99,775 1,200,225Average benefit/cost ratio BCR 0.0 0.0 0.9Incremental benefit/cost ratio IBCR

Highest bcr

Scarborough Borough Council

Costs and benefits £

Filey Flood Alleviation Strategy: Seadale only

13

Step 7 - Visualisation of results for transparent decision making It can sometimes be useful to convert a spreadsheet of figures into a format which is more visually interesting. This can be represented in graphs, but two alternative methods are heat maps (Figure 16) or damage stacks (Figure 17).

The darker colours in this heat map indicate higher damages, so we can see that the sub-area of Seadale shows the least damage per square metre and Wooldale the highest.

Figure 16 - Heat Maps of damage density for the 3 sub areas

Damage per square metre

£550

£275

£0

Wooldale Seadale

Wharfedale

14

The damage stack map below shows the flood extent and the concentration of damages. Simply, the higher the building stack, the higher the damages.

Figure 17 - Visualisation of the Do Nothing option, showing present value of damages for the three sub-areas. Source: Black & Veatch GIS Team, 2011.

15

Additional Considerations: Partnership Funding (Wharfedale, Wooldale and Seadale combined) The Defra Partnership Funding (PF) calculator determines the contribution to flood and coastal erosion scheme costs from Flood Defence Grand in Aid (FDGiA) (funding from government) and partner contributions (stakeholder funding from local authorities, business, community organisations, residents etc.)

This calculation is based on a series of Outcome Measures (OM). OM1 relates to the total benefits achievable by the flood scheme. These include avoidance of damages to housing, business, agriculture, local government, communications, infrastructure, utilities and public health. OM2 relates to households moving from one category of flood risk to a lower category. Households must be at direct risk of flood damage and have been built or converted into housing before January 2012 to be counted.

The greater the level of social deprivation, as measured by the Office for National Statistics (ONS), and the greater the movement of risk before and after a scheme has been implemented the higher the overall FDGiA contributions will be. Any shortfall has to be made up by contributions from partners.

Figure 18 – The Defra FDGiA calculator showing details of the Filey study

In the Filey case, the movement of flood risk to properties from ‘very significant’ to ‘moderate’ or ‘low’ risk is so significant that, when combined with high numbers of these properties within the

KeySummary: prospect of FDGiA funding"FDGIA Contribution":"Raw OM Score": Scheme Benefit to Cost Ratio: 15.84 to 1Cost saving and/or external contribution required: Effective return to taxpayer: 15.84 to 1Less scheme contributions secured: Effective return to area: n/a to 1"Adjusted OM Score":Result:FDGiA required for next phase(s):

1. Scheme detailsWho will maintain asset?PV Whole-Life Costs: millionPV Whole-Life Benefits: millionCash cost of next phase(s): millionDuration of Benefits: yearsAverage flood damages: per householdConstruction phase?

2. Qualifying benefits under Outcome Measure 2: households better protected against flood riskNumber of households in: Before After20% most deprived areas - - - - - - 0 0 021-40% most deprived areas 361 230 119 333 161 - -28 -69 -119 60% least deprived areas 266 264 839 40 85 - -226 -179 -839

At: Moderate Significant Very Moderate Significant Very Moderate Significant Veryrisk risk significant risk risk significant risk risk significant

risk risk riskAnnual damages avoided, compared with a household at low risk 150£ 600£ 1,350£

Change in household damages, in: Per year Over lifetime of scheme Qual. benefits (discounted)20% most deprived areas OM2 (20%)21-40% most deprived areas OM2 (21-40%)60% least deprived areas OM2 (60%)

206,250-£

Change due to scheme

20,625,000-£ 6,159,319£ -£

38,044,432£ 1,273,950-£ 127,395,000-£

-£

2,826,000£

2.82600£ 100

30,000£

62.6890£

LA

Yes - costs for approval include construction

-£

-£ 265.66%

Potential candidate for FDGiA funding dependant upon funding availability265.66%

FDGiA Calculator, based on interim funding arrangements announced 23rd May 2011ePublications Catalogue Product Code - FLHO0511BTXS-E-E

Filey Flood Alleviation Strategy Study - Option 3c (ALL SCHEME)Project Name/ref:

3.9570£

Input cellsCalculated cells

10,511,974£

16

21% to 40% most socially deprived in England (Figure 19), that no partnership funding is required. Within EA investment limits, total funding could be made available through Central Government Flood Defence Grant in Aid.

Figure 19 – Change in household risk in Filey before and after the scheme

Visualising Social Deprivation data

Examples are provided for social deprivation and visualising contributors to support transparent decision for partnership funding. Lower social deprivation ranking mean greater contributions when applying Partnership Funding payment for outcomes methodology. Figure 20 illustrates the levels of social deprivation for Filey.

Figure 20 - Social Deprivation Data from Office of National Statistics Super Output Areas

17

It is important to remember that total flood risk will never be removed completely, but Figure 21 shows how levels of risk can change with the introduction of flood protection measures. For example, the properties coloured orange have all moved from a very significant level or risk to a low level of risk. The more properties with reduced risk following a scheme the better chance for scheme justification under FDGiA.

Figure 21 – Visualising movement between risk zones improves outcome measure score under partnership funding.

©Flood Hazard Research Centre, 2016

Very Significant to Moderate risk Very Significant to Low risk Significant to Moderate risk