The monetary benefit of early flood warningsin Europe

Florian Pappenberger a,f,*, Hannah L. Cloke b,c, Dennis J. Parker d,Fredrik Wetterhall a, David S. Richardson a, Jutta Thielen e

aEuropean Centre for Medium-Range Weather Forecasts (ECMWF), Shinfield Park, Reading, United KingdombDepartment of Geography and Environmental Science, University of Reading, Reading, United KingdomcDepartment of Meteorology, University of Reading, Reading, United Kingdomd Flood Hazard Research Centre, Middlesex University, United KingdomeEuropean Commission, Joint Research Centre (JRC), Institute for Environment and Sustainability (IES), Climate Risk

Management Unit, Ispra, ItalyfSchool of Geographical Sciences, University of Bristol, Bristol, United Kingdom

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1

a r t i c l e i n f o

Article history:

Available online 15 May 2015

Keywords:

Probabilistic flood forecasting

Europe

Monetary benefit

Hydrological Ensemble Prediction

Experiment (HEPEX)

European Flood Awareness System

a b s t r a c t

Effective disaster risk management relies on science-based solutions to close the gap

between prevention and preparedness measures. The consultation on the United Nations

post-2015 framework for disaster risk reduction highlights the need for cross-border early

warning systems to strengthen the preparedness phases of disaster risk management, in

order to save lives and property and reduce the overall impact of severe events. Continental

and global scale flood forecasting systems provide vital early flood warning information to

national and international civil protection authorities, who can use this information to

make decisions on how to prepare for upcoming floods. Here the potential monetary

benefits of early flood warnings are estimated based on the forecasts of the continental-

scale European Flood Awareness System (EFAS) using existing flood damage cost informa-

tion and calculations of potential avoided flood damages. The benefits are of the order of 400

Euro for every 1 Euro invested. A sensitivity analysis is performed in order to test the

uncertainty in the method and develop an envelope of potential monetary benefits of EFAS

warnings. The results provide clear evidence that there is likely a substantial monetary

benefit in this cross-border continental-scale flood early warning system. This supports the

wider drive to implement early warning systems at the continental or global scale to

improve our resilience to natural hazards.

# 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY

license (http://creativecommons.org/licenses/by/4.0/).

Available online at www.sciencedirect.com

ScienceDirect

journal homepage: www.elsevier.com/locate/envsci

1. Introduction

Flood forecasting systems have become an essential part of

flood risk management, across all spatial scales, from local to

* Corresponding author at: European Centre for Medium-Range WeathTel.: +44 118 9499830; fax: +44 118 9869450.

E-mail address: florian.pappenberger@ecmwf.int (F. Pappenberger

http://dx.doi.org/10.1016/j.envsci.2015.04.0161462-9011/# 2015 The Authors. Published by Elsevier Ltd. This

creativecommons.org/licenses/by/4.0/).

continental (Meyer et al., 2012; Pagano et al., 2014; Stephens

and Cloke, 2014). Such systems require substantial investment

for system development and considerable resources to run

operationally (Cloke and Pappenberger, 2009; Thiemig et al.,

2014). The European Flood Awareness System (EFAS) provides

er Forecasts (ECMWF), Shinfield Park, Reading, United Kingdom.

).

is an open access article under the CC BY license (http://

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1 279

probabilistic flood forecasting information to national authori-

ties within Europe, as well as to the Emergency Response

Coordination Centre of the European Commission as early as 10

days before a flood event (Bartholmes et al., 2009; Thielen et al.,

2009a). Development of the system began in 2003 with results

disseminated to the users as ‘research products’. EFAS has been

fully operational since 2012, currently running 138 pan-

European forecasts twice a day, every day, which requires

extensive computer resources. EFAS forecasts and warnings are

continuously improving (ECWMF, 2014; Haiden et al., 2014;

Pappenberger et al., 2011), and the system has demonstrated

valuable early warning capabilities in several recent events

including the Balkan floods in 2014 (Thielen et al., 2014) and the

Central European floods in 2013 (Haiden et al., 2014; Thielen,

2014).

Flood forecasts provide essential information for local and

national authorities who must take decisions on actions (such

as flood gate closures or evacuations) to protect citizens,

property and infrastructure, particularly in urban areas and

industrial zones. Flood forecasts are important for those

authorities making decisions on the availability of disaster risk

finance (Jongman et al., 2014a, 2014b). Floods also represent a

threat to the environment and agriculture as was observed

during the 2014 January floods in the UK (Stephens and Cloke,

2014).

In order for early flood warnings to be translated into

decisions, clear mandates and responsibilities along the early

warning chain from forecast to decision maker must exist.

This is particularly important when assessing continental and

global cross-border early warning systems, such as EFAS, as

they can serve both as the main source of information in

countries which do not have their own early warning system

established, and also as an alternative source of information

which can provide ‘added value’ where there is already

national capability for monitoring and forecasting. In the latter

case, civil protection actions are taken based on all the

information available, and thus the benefit of this alternative

information is not straightforward to determine. In addition,

at the European level, the EFAS information is used directly for

planning of aid and support before and after major flood

events (EC, 2014a); again the monetary benefit is not

straightforward to determine.

Merz et al. (2010) provide a review of flood damage

assessment and highlight two key challenges, absence of

data and uncertainty. Other studies, such as Sampson et al.

(2014) highlight the large impact uncertain precipitation data

have on flood damage calculations, in this case for insurance

loss estimates. However, most studies in this area are usually

set within the context of estimating economic damage based

on flood risk in general (Carrera et al., 2015; Jongman et al.,

2012; Meyer et al., 2013; Molinari et al., 2014; Pfurtscheller,

2014; Saint-Geours et al., 2014; Vilier et al., 2014). Such analysis

is static in time and is only part of the picture for flood

forecasting, which also requires consideration of flood

response pathways and forecast performance.

Flood forecasting is one of the most effective flood risk

management measures (UNISDR, 2004), and studies that have

attempted to quantify avoided damages and forecast benefits

include Parker (1991), Carsell et al. (2004), Priest et al. (2011),

Molinari and Handmer (2011) and Verkade and Werner (2011).

For example, Priest et al. (2011) analyse questionnaires sent

after flood events at the national-level (England and Wales) and

the local-level (Grimma, south-eastern Germany) to establish

avoided costs of flood management with particular reference to

flood forecasting. National and regional flood forecasts have

been shown to provide benefit in the US (EASPE, 2002) and in

Scotland (SNIFFER, 2006–2009), as have upgrades to hydro-

meteorological early warning services in developing countries

(Hallegatte, 2012). Case studies from individual flood events

outside Europe have estimated flood forecasting system cost–

benefit ratios of 1:500 for Bangladesh (Bangladesh 2007 floods,

Teisberg and Weiher, 2008) and 1:176 for Thailand (Thailand

2007 floods, Subbiah et al., 2008). It is notable that in regions with

a low frequency of floods such as Sri Lanka (2003 event) this ratio

can drop substantially 1:0.93 (Subbiah et al., 2008). So in general

the cost–benefit of flood forecasting systems compares ex-

tremely favourably to the cost–benefit of weather and climate

services, which range from 1:2 to 1:20 (Frei, 2010; Perrels et al.,

2013) or other early warning systems in general (Klafft and

Meissen, 2011; Rogers and Tsirkunov, 2010).

Estimating the benefits of flood forecasting systems is

limited not only by the underlying data (e.g. uncertainties of

the vulnerability and exposure data, see Jongman et al., 2012)

but also by many other uncertainties including the methods

employed to estimate damages and avoided damages (Merz

et al., 2010). The analysis presented in this paper uses avoided

damages, and does not address the wider question of

economic value. Estimating the economic value of a forecast-

ing system as a whole is far more complicated (Benson and

Clay, 2004; Bockstael et al., 2000; Merz et al., 2010; Parker, 2003)

as it depends on:

� the starting point (e.g. what type of forecasting system

already exists);

� the spatial and temporal dimensions (e.g. recent flood

history; the lower economic value when compared with

monetary value, associated with the European scale when

compared with national or regional scales, when economic

transfers are taken into account; and the higher value

attained if a flood occurred just a few weeks ago);

� the scalability (response pathways cannot simply be

multiplied across entire river regions as for example

temporary defences are limited resources and be deployed

everywhere).

In addition, in flood situations many decisions do not

necessarily achieve the best possible outcome measured in

monetary terms, as decisions can be made under duress

(Choo, 2009) or influenced by other external limitations (e.g.

availability of temporary flood barriers). Therefore, a flood

forecasting system shares common properties with ecosystem

services to humans, in that complex interactions lead to

benefits which are often difficult to determine uniquely

(Farber et al., 2006).

This study estimates the monetary benefits of a probabi-

listic continental scale flood early warning system, the

European Flood Awareness System (EFAS). The study is based

on EFAS early warnings and flood damage potential calculated

from Barredo (2009), the EM-DAT (EM-DAT, 2014) emergency

events database and complementary information from the

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1280

European Solidarity fund application (EC, 2014a). In the next

section we describe the data and methods used to calculated

avoided flood damages of EFAS early warnings, including

details of EFAS flood forecasts, the EU and national forecasting

context of EFAS, the flood alert decision rules, damage data

sets and the calculation of potential avoided flood damages,

and the sensitivity analysis used to provide an envelope of

potential benefits to address the uncertainties and assump-

tions in directly assessing monetary benefit, and to identify

the most important contributing factor. Results are presented

and discussed in Section 3 in terms of flood occurrence and

associated damages, the calculated potential benefits of EFAS

early flood warnings and the sensitivity analysis. Conclusions

are drawn in Section 4 as to the potential benefits of

continental scale flood early warning systems.

2. Data and methods

In this section we describe the forecast data, the damage data

and the methods used to calculate the potential avoided flood

damages of the EFAS early warnings and the methods used to

estimate the monetary benefit.

2.1. The EFAS flood forecasts

EFAS uses an ensemble of weather forecasts and a hydrologi-

cal model to provide twice daily forecasts of river flow and

flood warnings (e.g. Bartholmes et al., 2009; Pappenberger

et al., 2005, 2011; Ramos et al., 2007, 2013; Thielen et al., 2009a,

2009b). Ensemble forecasts sample the uncertainty inherent in

weather prediction and make many forecasts, known as

ensemble members, by making alterations to the forecasting

model or to the starting conditions (Buizza, 2003, 2015; Cloke

and Pappenberger, 2009; Hagedorn et al., 2012; Vitart et al.,

2008). EFAS uses numerical weather prediction data from the

Deutscher Wetterdienst (German Weather Service, determin-

istic model COSMO-EU and global model), COSMO (high-

resolution limited area model ensemble forecast with 16

ensemble members) and the European Centre for Medium-

range Weather Forecasts (ECMWF, high resolution determin-

istic forecast and ensemble forecast with 51 ensemble

members). The weather forecasts are used to drive the

hydrological model which is set up on a 5 � 5 km2 grid. At

locations where real-time observations are available, the

forecasts are bias corrected and post-processed (Bogner and

Pappenberger, 2011; Bogner et al., 2012).

The EFAS forecasting system entered fully operational

status in 2012 as part of the COPERNICUS Emergency

Management Service (REGULATION (EU) No 377/2014). The

estimated costs of the four EFAS operational centres, based on

contracts awarded, is 21.8 M Euros. In addition, the develop-

ment costs over 10 years are estimated to be on the order of

20 M Euros based on institutional and external support for

EFAS (Thielen, pers. Commun.). The history of the EFAS

development has 3 distinct phases: 2000–2007, where EFAS

was in development, but national services already had access;

2007–2011, where EFAS was pre-operational; and 2012 on-

wards, where EFAS was fully operational. It is difficult to

analyse these distinct phases as there is no information on

how often EFAS forecasts were used in the first two phases,

and so the analysis in this paper focuses on the last

‘operational’ phase; 2012–2013.

Performance of EFAS: Performance is evaluated against

observed river flow and proxy observations (river discharge

generated through running the model using observed meteo-

rological variables) using a large range of probabilistic and

deterministic scores. Performance is then assessed on both a

continuous and an event basis, which includes a systematic

analysis of EFAS warning ‘hits’ (both forecast and observation

show a flood), ‘false alarms’ (forecast shows a flood but

observation does not) and ‘misses’ (observations shows flood

but forecast does not) since 2006. Pappenberger et al. (2011)

showed that the system performance improves by 10–30%

every decade. Performance is reported bi-monthly, in publicly

available bulletins at www.efas.eu (Alfieri et al., 2014; ECWMF,

2014). It is also discussed annually with the EFAS stakeholders.

EFAS reforecast warnings: Each time the EFAS system

undergoes a major update, a ‘reforecast’ is produced, where

the new system is used to reproduce forecasts of past dates

(this can be thought of as ‘forecasts of the past’ and is

sometimes termed a ‘hindcast’ or a ‘retrospective forecast’).

The latest EFAS reforecast was in January 2014 following

hydrological model improvements and new calibration

(ECWMF, 2014; Salamon, 2014) and was used to evaluate the

new changes to the system. This reforecast was computed for

a continuous series of forecasts, with forecasts issued once a

day looking 10 days ahead for every 5 km grid cell over the

whole European area. The reforecast extends for 2 years from

January 2012 to December 2013.

EFAS catchment-based flood warnings: The EFAS area is sub

divided into 786 river catchments across Europe, which are

used in the operational EFAS for monitoring and further post-

processing. In this paper the reforecast is analysed to establish

the hit, miss and false alarm rates for each catchment for the 2

year period 2012–2013 (catchment reforecasts). Also, for the

reforecast data set the European average of the hit, miss and

false alarm rates is also calculated (European average

reforecasts).

2.2. EFAS within the EU and national flood forecastingcontext

Since inception, EFAS has established a partner network of

more than 40 national and regional hydrological services.

EFAS partners sign a Conditions of Access (CoA) agreement

which defines the roles of EFAS and the receiving parties as

well as the rules for communicating EFAS results. As per the

CoA, EFAS real-time information and flood alerts are only

distributed to established EFAS partners who can use the

results for internal and external communications, planning

and actions. Partners receive training on EFAS products and

participate in the EFAS annual meetings where the latest

developments and feedback on the system are presented (De

Roo et al., 2011; Demeritt et al., 2013). Through these training

and knowledge exchange mechanisms it is ensured that the

national authorities have some ownership of the EFAS system

and are in the position to be able to use the EFAS information in

addition to their own warning systems. The advantage of EFAS

is that national authorities can use the early-warning and

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1 281

probabilistic information to modify their own national

warnings, particularly when these capabilities need supporting.

Furthermore, EFAS provides unique information for the

European Civil Protection Mechanism to plan the deployment

of aid ahead of a flood event, thereby reducing post-event

response times. Here it is assumed that EFAS benefits are

unique across Europe; this assumption is addressed in the

sensitivity analysis and should be borne in mind when

considering the results.

2.3. Decision rules for probabilistic flood forecasts

Probabilistic forecasting is more skilful than deterministic

forecasting (Cloke and Pappenberger, 2009; Cloke et al., 2013;

Pagano et al., 2014). Verkade and Werner (2011) have shown

that probabilistic forecasts lead to higher benefits at all lead

times in comparison to single valued forecasts. In determin-

istic forecasts the hit, miss and false alarm rates are fixed for a

certain flood threshold, such as the 5 year return period level,

or the 100 year return period level. An advantage of

probabilistic forecasting is that the false alarm rates can be

set to levels acceptable to the end user or stakeholders and

therefore lead to better decisions that are tailored to specific

circumstances (Cloke et al., 2013; Pagano et al., 2014; Ramos

et al., 2010, 2013).

Probabilistic forecasting systems require sets of rules to

convert forecasts into warnings (Dale et al., 2013), whereby the

act of deciding to issue a warning is dichotomous – a warning

is sent or not sent – and therefore deterministic. The optimal

lead-time at which warnings are provided is therefore not

always equivalent to the longest lead time available in the

forecasts, as false alarms have to be balanced with successful

warnings (for a discussion on this see Verkade and Werner,

2011). EFAS forecasters issue flood watches and alerts

according to river catchment properties and forecast char-

acteristics (Bartholmes et al., 2009) and also based on

agreements with the national hydrological services.

A flood alert is issued when:

1. the catchment is part of the EFAS partner network (with

signed agreements and training on the system);

2. the catchment area is larger than 4000 km2;

3. the forecast is persistent, meaning that 3 consecutive

ECMWF ensemble forecasts exceed the EFAS 5 year return

period threshold with a probability of greater than 30%;

4. at least one deterministic forecast also exceeds this

threshold;

5. the event is more than 48 h ahead with respect to the

forecast date.

A flood watch may be issued by EFAS forecasters for EFAS

partners when any of criteria 2–5 are not met, but the forecast

situation warrants that the authorities should be informed. This

leaves flexibility for interpretation and a flood watch can always

be upgraded to a flood alert if the formal requirements are met.

A flood alert or watch is deactivated if:

1. Observations reported by the national/regional hydrologi-

cal service clearly indicate that the EFAS Flood Alert/Watch

is a false alarm.

2. Observations reported by the national/regional hydrologi-

cal service clearly indicate that discharges/water levels

have decreased to normal values although EFAS simula-

tions still show that simulated discharge exceeds the EFAS

high threshold.

3. The simulated EFAS discharge at the reporting point(s) for

which the EFAS Alert/Watch was issued falls below the

EFAS 5 year return period.

The above rules are used in this study with following

modifications:

(i) Only 2 consecutive forecasts have been used to issue an

alert, as the reforecast system only has 1 forecast per day

(the operational system has 2 forecasts per day).

(ii) Only the last deactivation rule (number 3) has been used to

deactivate flood alerts.

(iii) The reforecasts have been post-processed and combined

at the outlet of each of the 786 catchments. The flood alert

status is calculated only at these outlets under the

assumption that this location is representative of the

flood status of the upstream catchment.

The consequence of providing flood alerts only at catch-

ment outlets, is that the number of flood alerts counted in the

reforecasts will be less than for the operational EFAS, as under

operational conditions a number of EFAS alerts are sent

depending on the size of the flood event and the number of

administrative authorities involved (several warnings for

different rivers and authorities can be sent for one flood

event).

2.4. Damage data sets

The collection of flood damage data is extremely challenging

and it is therefore not possible to base this study on robust,

detailed data (Merz et al., 2004, 2010). Such data are most often

confidentially held by national authorities and are often only

published externally for major flood events with a substantial

time delay. Therefore, instead, three independent data sets

and estimation methods have been used in order to

accommodate some of the uncertainties involved in this

exercise. All figures have been adjusted to 2012 prices

(corresponding to when EFAS entered the operational phase)

using the average inflation published by Eurostat, and in

addition a 5% discount rate has been applied to discount future

payoffs (European Commission, 2008). The damage data used

in this study report flood events in a different way to both the

operational EFAS and the reforecasts; floods are reported by

country or by individual flood event (even if they spread across

several catchments).

2.4.1. Barredo’s flood damage map of EuropeThe Joint Research Centre of the European Commission has

produced a flood damage map of a 100 year return period (1%

annual probability) for Europe assuming that all flood

defences have been removed (Barredo, 2009). In this paper

the EFAS catchment reforecasts ‘hits’ and ‘misses’ are

combined with this flood damage map, with the original

100 m grid cells aggregated to river catchment scale. In order

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1282

to calculate the benefit from issuing flood alerts it is necessary

to estimate the annual average damages.

To calculate annual average damages, the Standard

Weighted Annual Average Damage values (Penning-Rowsell

et al., 2013, pp. 127–130) have been used to rescale the potential

damage as identified by Barredo (2009). The annual avoided

damages of different thresholds of standard protection, i.e.

flood defences removed, flood defences at 20, 50 and 100 year

flood return period protection levels, were also estimated

(Penning-Rowsell et al., 2013, pp. 127–130). Fig. 1 shows the

results of potential flood damage for catchments across

Europe assuming no flood protection, i.e. all current protection

is removed.

It should be noted that this approach has inherent

uncertainties (Ward et al., 2011). The rescaling data are based

on UK case studies and may not be entirely representative of

pan-European conditions. In addition, the standardisation

data are mainly based on older data and are likely to result in

an overestimation of benefits. There is also some underesti-

mation due to the use of the 100-year return period data as

reference; the rescaling can only be applied where the 100 year

values are above 0, and therefore some locations where there

is no data for 100-year events, but less extreme events do

occur, will be missed. It is not possible to exactly quantify

these uncertainties.

2.4.2. EM-DATEM-DAT is an emergency events database which contains

data and effects of many hazards (EM-DAT). From this

database the number of flood events, and the monetary costs

in USD, have been extracted for the European Continent. The

costs have been adjusted using average inflation in Europe

Fig. 1 – Potential flood damage (in Euro, 2012

(Harmonized Index of Consumer Prices) to 2012 costs and

converted into Euros (1 USD = 0.72 Euro). 2012 is used as the

baseline year because this is the year in which EFAS became

an operational system. EM-DAT contains no detailed infor-

mation on return period, it is therefore assumed that any

entry in EM-DAT would have warranted an EFAS flood alert.

EM-DAT data also contain no detailed catchment information

and cannot therefore exploit the detailed simulations

provided by the reforecasts. Therefore the EM-DAT data is

combined with the EFAS European average reforecasts for

2012–2013. It is possible to make two different extreme

interpretations of the EM-DAT data depending on whether or

not flood warnings are incorporated in the data. First, they

can be seen as the damage resulting from having no warnings

(termed ‘‘EM-DAT excl. warnings’’). Second, they can be seen

as the residual damages after warnings have been effective

(termed ‘‘EM-DAT incl. warnings’’). In practice, there is likely

to be a mix of both of these on the European continent. For

Europe, it can be assumed that most countries have short-

term flood forecasting systems in place. It can further be

assumed that with a growing EFAS partner network and the

system becoming increasingly operational, early flood warn-

ings have been taken into consideration.

2.4.3. EU Solidarity Fund informationThe European Union Solidarity Fund (EUSF) was created in

2002 in response to the 2002 Elbe and Danube floods in Central

Europe. It seeks to provide funds to help nations recover from

natural disasters and express European solidarity with

disaster-stricken regions. European nations can make

requests to EUSF for support in relation to different cata-

strophic events including floods, forest fires, earthquakes,

) aggregated on EFAS river catchments.

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1 283

volcanic eruptions, storms and droughts. From 2002 to 2012

EUSF has provided funds to 15 countries recovering from flood

events (36 out of 56 applications). The total reported flood

damages amounted to 54 450 M Euro for which 5027 M Euros

were paid as financial support to the applicant countries

(Table 2). The highest damages were reported for the Elbe and

Danube in 2002 (15 135 M Euros) and 2013 (10 309 M Euros). For

the years 2007 and 2010 about half of the damages of 2013 were

reported. Similarly to EM-DAT two interpretations regarding

the damages can be made, and analysis must be based on

European averages, so the EUSF data is combined with the

EFAS European average reforecast for the period 2012–2013.

2.5. Calculating potential avoided flood damages

In this study, it is assumed that all EFAS warnings have been

treated as operational messages by national authorities and at

the EU level. Therefore, the response to warnings is assumed

to lead, at least to some degree, to flood preparedness actions

which is supported by evidence from the annual EFAS user

workshop. Under this assumption, the avoided flood damages

can be estimated. These are then compared with the system

installation and running costs. The difference is considered to

be the relative benefit of EFAS, and expressed in terms of the

return on 1 Euro investment in the EFAS system.

Flood damage can be avoided through early warning

leading to mitigation measures being taken by the warning

recipients. The maximum potential flood damage (Lp) is

related to the actual damage (La) by:LaðtÞ ¼ nðtÞ � L p

where n(t) is the avoided damage reduction factor due to early

warning.

There is a significant uncertainty in estimates of avoided

flood damages, including different estimates from different

sources and for different time horizons. The International

Commission For The Protection Of the Rhine (2002) has

estimated that flood warnings can help businesses avoid 50–

75% of flood losses. Other estimates of potential avoided flood

damages for flood warnings 48 h ahead, range from 4 to 40%

(Carsell et al., 2004; Chatterton and Farrell, 1977; Day, 1970;

Parker, 1991), although these figures also incorporate other

Table 1 – Avoided damages for various pathways in respondiParker et al. (2007), Scott and Wicks (2012) and Thurston et alconsecutive actions that can be employed.

Pathway De

Flood Defence Operations (FDO) Avoided damages by war

Watercourse Capacity Maintenance

(WCM)

Damages avoided by Wa

Community Based Operations (CBO) Damages avoided by com

Warning Dependent Resistance (WDR) Residual damage avoided

(temporary resistance m

Contents Moved & Evacuated (CME) Residual damages avoide

property contents

Early Warning measures FDO, WCM, CBO

Total FDO, WCM, CBO, WDR, C

factors such as coverage of flood warning service, service

effectiveness and availability (Parker et al., 2005). Specifically

considering domestic properties, SNIFFER (2006–2009) esti-

mates that flood warnings result in avoided flood damages of

7.3%. However, even the estimates of this component of

avoided damages have considerable uncertainty with other

estimates of between 4.54 and 6% (Penning-Rowsell et al.,

2013; Priest et al., 2011; Parker et al., 2007).

Table 1 lists the avoided damage factors for various

pathways in responding to flood warnings (Parker et al.,

2007, 2008; Scott and Wicks, 2012; Thurston et al., 2008).

Although often employed together or in sequence, these

pathways can be seen as different management options

(Farber et al., 2006). All early warning pathways together result

in a percentage avoided damage of 32.85% as the percentages

always apply to the sum which was previously not saved.

Using this figure for EFAS implies the assumption that the

response to continental scale EFAS warnings is the same as the

response to national and local flood warnings. Although

necessary, this assumption does not take into account the

difference in the context of the system, i.e. that EFAS warnings

are used as an additional source of information by national

flood warning authorities, and for response planning by

European level civil protection.

There are several intangible and indirect costs or benefits

of EFAS warnings that have not been considered in this study

because of the difficulties in their estimation (for a classifica-

tion of flood losses see Merz et al., 2010; Parker et al., 2005). It

should be noted that for large scale disasters indirect losses

may be of the same magnitude as direct losses (Hallegatte and

Dumas, 2008). EFAS information is used in particular for

evaluating aid scenarios, moving equipment to the right

places if necessary, preparing transport routes or coordinat-

ing European aid. The costs of implementing such actions are

low and hence assumed to be negligible in this study (similar

to Hallegatte, 2012). The intangible benefits, such as the

earlier provision of aid, which have been reported for

example in the Central European floods of 2013 and the

Balkan floods of 2014, are also not taken into account.

Reputational damage can be significant (Subbiah et al., 2008)

but is very challenging to cost accurately and hence has also

been omitted from this study.

ng to flood warnings (adapted from Parker et al. (2008),. (2008)). The percentages reflect avoided damages due to

scription Avoided damagesdue to early warning (%)

ning dependent flood defences 32%

ter Course maintenance 0.9%

munity-level defences 0.36%

by warning-dependent

easures)

0.0036%

d by moving and evacuation 5.7%

32.85%

ME 36.68%

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1284

2.6. Sensitivity analysis to generate an envelope ofpotential EFAS benefits

Sensitivity analyses are recommended to assess the influence

of uncertainties on model results (Cloke et al., 2008; Dobler and

Pappenberger, 2013; Merz et al., 2008). Such an analysis, set

within a cost–benefit analysis, can be used to prioritise

development efforts and lead to a more efficient system

improvement (Buganova et al., 2013). Here, a sensitivity

analysis is undertaken to indicate the envelope of potential

benefits of EFAS early flood warnings. The analysis tests 19

scenarios which take into account the main assumptions in

the estimation of benefit: the avoided damage factors, the

performance of the forecast system, the discount factors and

the uncertainties in the damage data.

Avoided damage factors: The uncertainty in the avoided

damage factors is discussed in Section 2.5. The values used in

the sensitivity analysis are those presented in Table 1.

Forecast performance: Although issuing EFAS warnings is

governed by underlying decision rules, in practice human

judgement comes into play (Danhelka, 2015) and will impact

upon the calculations of how good the forecast is (how much

numerical skill the forecast has usually in comparison to a

benchmark (Pappenberger et al., 2015)). As more data become

available and the forecast system improves technically, the

system skill would be expected to improve. Another assump-

tion is that the forecast skill is stationary over a period of 20

years and this leads to an underestimation of the cost–benefit

ratio. Earlier studies by Pappenberger et al. (2011) indicate that

a performance improvement of 10% per decade is achievable.

The sensitivity analysis tests warning performance improve-

ments of 10%, 20% and 30%.

Discount factors: A discount factor represents the percentage

rate used to calculate the present value of a future saving or

income. It takes account of the lower value at present of future

savings in comparison to present ones. In this study a discount

factor of 5% across the EU has been employed (EC, 2014b).

There are variations within Europe, with the UK using a factor

of 3.5% and France using 4%. The sensitivity analysis tests the

influence of using a lower discount factor of 3.5%.

Uncertainty of damage estimates: The uncertainties in the

underlying damage datasets are also included in the

sensitivity analysis. First the differences between using EUSF

Table 2 – Flood occurrence and associated damages for variou

Data source Descriptions

EM-DAT Flood occurrence

Damage (M Euros – 2012 prices)

EUSF Applications

Damage (M Euros – 2012 prices)

EFASa Alerts

Hits

False alarms

Reanalysis study Alerts

Hits

Misses

False alarms

a Note that EFAS transitioned into an operational service in 2012 and ther

this year. Not all EFAS alerts have been verified to be hits or false alarm

and EM-DAT are tested. It is not clear whether the

Barredo (2009) data includes indirect damages; although

Barredo (2009) reports that his analysis is based on direct

costs, the EM-DAT documentation states that EM-DAT

damage data includes indirect damages (although it is likely

it only includes those indirect damages which are immedi-

ately apparent). ‘Reported damages’ at the time of the event

often include elements of both direct and indirect damages

because the disruptive effects of floods are apparent almost

immediately (Merz et al., 2010). Uplift factors can be used to

account for often unknown indirect damages, and are used as

a multiplier on the direct costs. Such indirect costs include

disruption to transport networks or mental health impacts or

any factor which cannot be directly attributed to the floods

but is indirectly related. Paccagnan (2012) suggests an average

factor of 2.05 and a maximum and minimum of 2.54 and 1.75.

We use these values to estimate our uncertainty bounds in

our sensitivity analysis. In addition an extreme ‘low bound-

ary’ scenario is used in the sensitivity analysis where the true

damages are set at only 10% of EM-DAT excluding warnings.

3. Results

3.1. Flood occurrence and associated damages

The efficiency of a forecasting system can be defined as the

number of hits divided by the total number of hits and misses.

For the EFAS study, the efficiency (hit rate) amounts to 55%

which is considerably less than the 70% which is seen as

desirable for this type of system (Subbiah et al., 2008). However,

the hit rate should not be used as a single performance measure

(Armistead, 2013; Hogan and Mason, 2012).

In a perfect forecasting system the number of misses would

be zero and the monetary savings would be the damage

multiplied by 32.85% (see above), in which case over the period

2000–2013 on average every year 1092 M Euro or 1627 M Euro

would be saved (using the EM-DAT data excluding and

including warnings respectively). However, forecasting will

also lead to misses and false alarms which will lead to a

ineffectiveness in the warning chain in the case of misses and

negative effects in the case of false alarms (Parker and Priest,

2012). Using the hit rate shown in Table 2 (on average 55% of all

s data sources.

Years covered Average

2000–2013 18.2

3325

2002–2013 2.8

4513

2007–2013 23.4

12.1

3.6

2012–2013 14.2

4.7

3.8

9.5

efore no resources were available to follow up hits and false alarms in

s and their status is unknown.

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1 285

hits and misses are hits), the average saving per year in the

period 2000–2013 will be 607 M Euros or 858 M Euros (using the

EM-DAT data excluding and including warnings respectively).

The latter would rise to 782 M Euros or 1164 M Euros using the

EUSF data excluding and including warnings respectively.

Table 2 shows the number of flood events recorded in EM-

DAT (from 2000), reported flood damages in the EUSF

applications from 2002 to 2013, as well as those reported by

EFAS (from 2007; EFAS Dissemination Centre, 2014) and

computed in this study (2012/2013). As expected from the

discussion in Section 2.1, the number of EFAS alerts is greater

than the number of events reported in the damage data sets. In

addition, the number of events reported in the EM-DAT

database is lower than the number of applications to the EUSF.

This is because only events which exceed a certain threshold

of a country’s GDP qualify for the application and several

events within a country may be regrouped into a single

application. On average the EUSF damages from 2002 to 2013

are higher than the EM-DAT estimates. This indicates that (a)

the majority of the costs arise from the major events that meet

the criteria for EUSF applications and (b) the calculation of

costs in EUSF is more comprehensive that the estimates in EM-

DAT. Flood occurrence and damage data are not always

correlated, for example the years 2012 and 2013 have the same

number of floods but vastly different damages (not shown).

This is because flood damage costs are related not only to flood

occurrence but also to flood severity, duration and location.

This illustrates the uncertainty in estimating flood-related

costs.

3.2. Benefit of EFAS

Benefits of EFAS early warnings have been estimated using

two different methods: (i) forecasts at catchment level have

been used to calculate hits and misses which are then

Fig. 2 – Net benefit of the Europea

combined with Barredo’s (2009) modified flood damage map

for Europe; and (ii) the EM-DAT and EUSF damages for the

period 2000–2013 have been used and combined with the

average warning performance from an EFAS forecast study.

These estimates have been modified using different standards

of protection (for example, a 100 year flood return period

standard of protection will be defences that protect against a

flood level with a 1% annual probability of occurrence). A

scaling between different protection standards can be found in

Penning-Rowsell et al. (2013). Both analyses have been

compared against the installation and running costs of the

EFAS system, with the difference being the estimated net

benefit of the EFAS system. Fig. 2 shows the net benefits of the

current EFAS system and also a ‘perfect’ EFAS system (with no

event misses). The figure shows the return on 1 Euro after 5

and 20 years, and the net benefits for no flood protection, 20, 50

and 100 year flood return period protections. The EM-DAT and

EUSF results are also shown and are independent in terms of

economic costs from the other data, but rely on the same

model simulations.

Fig. 2 shows the results of the calculations of the benefits of

the EFAS system. Assuming lower standards of protection the

net benefits are higher. Fig. 2 shows that for a no protection

scenario using the current system there would be a return of

1:495 Euro on EFAS investments after 5 years, which would

increase to over 1:988 Euro after 20 years. In case of a perfect

system the no protection scenario would give a return of 1:883

Euro after 5 years and 1:1760 after 20 years. The upper

bounding threshold here is flood protection to the 100 year

return period standard (which is unrealistic as average

standards of flood protection across Europe are well below

this level, see Jongman et al., 2014b). Even at this higher bound

the benefit return decreases to only 1:7 after 5 years and to 1:13

after 20 years. The figure thus provides evidence that a net

benefit of the EFAS system is very likely.

n Flood Awareness System.

Table 3 – Sensitivity analysis of estimated cost-benefit ratio with percentage savings due to early warnings.

Pathway Avoided damagesdue to early warning (%)

Ratio of monetarycosts to benefits(after 20 years)

Scenario

Flood Defence Operations (FDO) 32% 1:155 1

Watercourse Capacity Maintenance (WCM) 0.9% 1:4 2

Community Based Operations (CBO) 0.36% 1:2 3

Warning Dependent Resistance (WDR) 0.0036% 1:0.02 4

Contents Moved & Evacuated (CME) 5.7% 1:28 5

Early Warning measures 32.85% 1:159 Base

Total 36.68% 1:178 6

Target Future 70% 1:339 7

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1286

The EM-DAT database facilitates independent assessment

of this net benefit, as the data are independent in terms of

economic costs from the other data (but rely on the same

model simulations). The results for EM-DAT (excluding

warnings) provide a return of 1:159. This is equivalent to

returns of the 20 and 50 year return period values noted in the

analysis above. The results for EM-DAT (including warnings)

are equivalent to the 50 and 100 year return period results. The

EM-DAT (excluding warnings) is used as a base scenario for the

sensitivity analysis in the next section. Calculated as the net

benefit considering the investment and the operating costs of

EFAS this provides a return of 20 M Euros. The EUSF data

provides further independent evidence that EFAS has net

benefit. Here EUSF values (including warning) are equivalent

to between 20 year protection and no protection.

3.3. Sensitivity analysis

The sensitivity analysis of the relative benefit of EFAS early

flood warnings takes into account the main assumptions in

the estimation of benefit, which includes the avoided damage

factors, the performance of the forecast system, the impact of

discount factors and the uncertainty of the damage estimates.

The analysis is displayed in relation to the EM-DAT (excl.

warnings) data although the sensitivity to other damage

estimates is also shown.

As shown in Table 3 a base scenario is used as a reference

for the sensitivity analysis. This scenario has avoided

damages due to early warning of 32.85%, forecast performance

equivalent to the current EFAS (55% efficiency), a discount rate

of 5%, uses the EM-DAT data (excluding warning) and a

monetary cost/benefit ratio, after 20 years of 1:159.

Table 4 – Sensitivity Analysis of estimated cost-benefit ratio w(excl stands for excluding warning and incl stands for includi

Warningperformance

Current 10%better

20%better

30%better

Perfect Current

Discount rate 5% 5% 5% 5% 5% 3.5%

Damage data EM-DAT

(excl)

EM-DAT

(excl)

EM-DAT

(excl)

EM-DAT

(excl)

EM-DAT

(excl)

EM-DAT

(excl)

Monetary

cost/benefit

ratio

(after 20 years)

1:159 1:173 1:187 1:200 1:286 1:176

Scenario Base 8 9 10 11 12

3.3.1. Avoided damages factorIn Table 3 the impact of a range of different avoided damages is

shown, indicative of the wide range of responses to flood

warnings. It can be seen that the system would not be worth

the investment if only Warning Dependent Resistance

measures were used (cost/benefit ratio of 1:0.02). All early

warning measures together lead to a cost/benefit ratio of 1:159

(base scenario) whilst a fully inclusive warning chain would be

1:178 (scenario 6).

3.3.2. Forecast performanceTable 4 shows that with a base cost:benefit ratio of 1:159, a

10% improvement in forecast performance over 20 years

would lead to an increase in the cost:benefit ratio to 1:173

(scenario 8), rising to 1:187 for 20% (scenario 9) and 1:200 for

30% (scenario 10). The estimated benefit–cost ratios of

scenarios 8–10 are all less than the theoretical limit of

1:286 which could be achieved with a perfect warning system

(scenario 11).

3.3.3. Discount factorsTable 4 illustrates the sensitivity towards these factors

showing that a discount rate of 3.5% (scenario 12) leads to a

ratio of benefits to costs of 1:176 (compared to 1:159 for the

base scenario using 5%). This is the same level of sensitivity as

to the 10% increase in skill.

3.3.4. Damage estimationIn Table 4 the difference that results from using the EUSF is

apparent instead of the EM-DAT data, with EUSF data leading

to a relative benefit of 1:205 (excluding warning – scenario 13)

or 1:308 (including warning – scenario 14). EM-DAT in contrast

ith percentage savings due to early warnings set at 32.85%ng warning).

Current Current Current Current Current Current Current

5% 5% 5% 5% 5% 5% 5%

EUSF

(excl)

EUSF

(incl)

EM-DAT

(incl)

175%

EM-DAT

(excl)

205%

EM-DAT

(excl)

254%

EM-DAT

(excl)

10%

EM-DAT

(excl)

1:205 1:308 1:226 1:278 1:326 1:403 1:16

13 14 15 16 17 18 19

Fig. 3 – Sensitivity analysis of the relative monetary benefit of EFAS presented as the percentage difference of 19 scenarios as

compared to the base scenario of all early warning measures (Tables 3 and 4).

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1 287

leads to values of 1:159 (excluding warnings – base scenario)

and 1:226 (including warnings – scenario 15).

The inclusion of uplift factors increases the benefit–cost

ratio from our base case of 1:159, to ratios of between 1:278

(scenario 16) and 1:403 (scenario 18). In addition the ‘low

boundary’ scenario (scenario 19), in which the true cost is only

10% of EM-DAT (excl warning), leads to a ratio of 1:16.

3.3.5. SummaryFig. 3 summarises the results of the sensitivity analysis, with

the bars showing the percentage difference of the results of

the 19 scenarios to the base scenario results. The figure shows

that there is a considerable range in the monetary benefit

observed, with the avoided damages factor introducing the

largest uncertainty. The estimation of the actual damage also

introduces a large variation in the results followed by the

potential system improvement in the future. The lowest

impact on the results is given by the discount rate. This is

summarised in Fig. 3 where the sensitivity analysis of the

relative monetary benefit of EFAS is presented as the

percentage difference of the scenarios as compared to the

base scenario of all early warning measures (Tables 3 and 4). In

only one case (scenario 4) is there a relative cost, and the

envelope of values indicates confidence in the relative

monetary benefits of EFAS.

4. Conclusions

In this paper, the benefits of a continental scale early flood

warning system, the European Flood Awareness System

(EFAS), were analysed in monetary terms. Three different

monetary data sets on flood damages were combined with

flood forecasts to provide evidence that there is substantial net

benefit provided by this pan-European system. This supports

the wider drive to implement early warning systems at the

continental or global scale to improve our resilience to natural

hazards in a changing climate (Alfieri et al., 2013; De Groeve

et al., 2014; Merz et al., 2014; Pappenberger et al., 2012, 2013;

Ward et al., 2013, 2014; Winsemius et al., 2013).

The uncertainty in the estimates of potential avoided flood

damages was tested with a detailed sensitivity analysis of the

avoided damages factor, the forecast performance, the impact

of discount factors and the uncertainty of the damage

datasets. The envelope of estimates of benefit provided robust

evidence of system benefit. The base scenario in this analysis,

considered to be conservative, demonstrates that for every

Euro invested a return of 159 Euros is created after 20 years of

operating EFAS (return of 20 trillion Euros). This value

compares extremely favourably to the cost benefit of weather

and climate services which range from 1:2 to 1:20 (Perrels et al.,

2013) or other early warning systems in general (Klafft and

Meissen, 2011).

Varying the avoided damages factor due to early warning

has a large impact on the results and for example if the

pathway of action due to an early warning comprises only

water course maintenance, then the cost benefit ratio would

reduce to 1:4. In contrast, improved forecast performance

could lead to an increase of the cost benefit ratio to 1:202.

Ratios of up to 1:409 were possible.

The sensitivity analysis highlights that the largest uncer-

tainty in these estimates comes from the avoided damages

through early warning percentages which reflect the wide

range of possible responses to flood warnings. Another large

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1288

source of uncertainty is the damage data used in the derivation

of the monetary benefit. This highlights the importance of

forecast responses in making the most out of a flood forecasting

system. This could also be actively improved and would result

in a greater overall benefit, although it may have a high cost.

Although not as great as the damage data and the avoided

damage percentages, there is still scope to improve the system

benefit by improving forecasting system performance. One

clear conclusion of this study is that investment in medium

range probabilistic flood forecasting systems is always valuable

assuming multiple pathways of actions are taken, because the

cost benefit ratios are always positive. This suggests that these

flood forecasting systems should be high priority for long term

investment and support, because they are effective and save

money as well as lives.

As well as providing evidence of the benefits in continental

scale early flood warning systems, the study has also shown

that improving our resilience to floods and realising the full

benefits of such early warning systems requires a focus on

more than just improving forecast skill. The response to

warnings, including visualisation, better training and re-

sponse procedures are extremely important in making the

most of these early warnings and should be priorities for

future investment. The example of EFAS also demonstrates

the value of regional cooperation, knowledge exchange and

interdisciplinary research teams in developing continental

scale early warning systems. Early warning systems, however

valuable, are only one part of our flood management portfolio,

and should be employed alongside other measures to make

our populations more resilient to flood events, for example by

considering urban design features and green infrastructure to

mitigate floods and reduce flood damage. In conclusion, this

study has provided evidence that a continental scale early

flood warning system can provide valuable information that

prevents substantial flood damage and aids disaster recovery.

This evidence can be used to support development in other

continents to improve resilience, particularly in vulnerable

areas, where early warning systems could provide the

difference between stability and economic collapse.

Acknowledgement

Hannah Cloke is funded by NERC project SINATRA (NE/

K00896X/1).

r e f e r e n c e s

Alfieri, L., Burek, P., Dutra, E., Krzeminski, B., Muraro, D.,Thielen, J., Pappenberger, F., 2013. GloFAS – global ensemblestreamflow forecasting and flood early warning. Hydrol.Earth Syst. Sci. 17, 1161–1171, http://dx.doi.org/10.5194/hess-1117-1161-2013.

Alfieri, L., Pappenberger, F., Wetterhall, F., Haiden, T.,Richardson, D., Salamon, P., 2014. Evaluation of ensemblestreamflow predictions in Europe. J. Hydrol. 517, 913–922.

Armistead, T.W., 2013. H. L. Wagner’s Unbiased Hit Rate and theAssessment of Categorical Forecasting Accuracy. WeatherForecast 28, 802–814.

Barredo, J.I., 2009. Normalised flood losses in Europe: 1970–2006.Nat. Hazards Earth Syst. Sci. 9, 97–104.

Bartholmes, J.C., Thielen, J., Ramos, M.H., Gentilini, S., 2009. TheEuropean Flood Alert System EFAS – Part 2: Statistical skillassessment of probabilistic and deterministic operationalforecasts. Hydrol. Earth Syst. Sci. 13, 141–153.

Benson, C., Clay, E.J., 2004. Understanding the Economic andFinancial Impacts of Natural Disasters. World Bank,Washington, DC# World Bank. https://openknowledge.worldbank.org/handle/10986/15025.

Bockstael, N.E., Freeman, A.M., Kopp, R.J., Portney, P.R., Smith,V.K., 2000. On measuring economic values for nature.Environ. Sci. Technol. 34, 1384–1389.

Bogner, K., Pappenberger, F., 2011. Multiscale error analysis,correction, and predictive uncertainty estimation in a floodforecasting system. Water Resour. Res. 47 .

Bogner, K., Pappenberger, F., Cloke, H.L., 2012. Technical Note:The normal quantile transformation and its application in aflood forecasting system. Hydrol. Earth Syst. Sci. 16, 1085–1094.

Buganova, K., Luskova, M., Hudakova, M., 2013. Early WarningSystems in Crisis Management. In: 2013 InternationalConference on Management Innovation and BusinessInnovation (Icmibi 2013), Pt I 15. pp. 218–223.

Buizza, R., 2003. Weather predictionjensemble prediction. In:Holton, J.R. (Ed.), Encyclopedia of Atmospheric Sciences.Academic Press, Oxford, pp. 2546–2557.

Buizza, R., 2015. Data assimilation and predictabilityjensembleprediction. In: Zhang, G.R.N.P. (Ed.), Encyclopedia ofAtmospheric Sciences. second edition. Academic Press,Oxford, pp. 248–257.

Carrera, L., Standardi, G., Bosello, F., Mysiak, J., 2015. Assessingdirect and indirect economic impacts of a flood eventthrough the integration of spatial and computable generalequilibrium modelling. Environ. Model. Softw. 63, 109–122.

Carsell, K.M., Pingel, N.D., Ford, P.E., 2004. Quantifying thebenefit of a flood warning system. Nat. Hazard Rev. 131–140.

Chatterton, J.B., Farrell, S., 1977. Nottingham Flood WarningSystem: Benefit Assessment. FHRC Enfield, UK.

Choo, C.W., 2009. Information use and early warningeffectiveness: perspectives and prospects. J. Am. Soc.Inform. Sci. Technol. 60, 1071–1082.

Cloke, H.L., Pappenberger, F., 2009. Ensemble flood forecasting:a review. J. Hydrol. 375, 613–626.

Cloke, H.L., Pappenberger, F., Renaud, J.-P., 2008. Multi-methodglobal sensitivity analysis (MMGSA) for modelling floodplainhydrological processes. Hydrol. Process. 22, 1660–1670.

Cloke, H.L., Pappenberger, F., van Andel, S.J., Schaake, J.,Thielen, J., Ramos, M.-H., 2013. Hydrological ensembleprediction systems. Hydrol. Process. 27, 1–4 (Preface).

Dale, M., Ji, Y., Wicks, J., Mylne, J., Pappenberger, F., Cloke, H.L.,2013. Applying probabilistic flood forecasting in floodincident management. In: Technical Report – RefinedDecision-Support Framework and Models, Project: SC090032.Environment Agency, Bristol, UK.

Danhelka, J., 2015. The model, the Forecast and the Forecaster.http://hepex.irstea.fr/the-model-the-forecast-and-the-forecaster/ (last accessed16.03.15).

Day, H.J., 1970. Flood Warning benefit evaluation-SusquehannaRiver Basin (urban residences). National Weather Service,Silver Spring, MD.

De Groeve, T., Thielen, J., Brakenridge, R., Adler, R., Alfieri, L.,Kull, D., Lindsay, F., Imperiali, O., Pappenberger, F., Rudari,R., Salamon, P., Villars, N., Wyjad, K., 2014. Joining forces in aglobal flood partnership. Bull. Am. Meterol. Soc..

De Roo, A., Thielen, J., Salamon, P., Bogner, K., Nobert, S., Cloke,H.L., Demeritt, D., Younis, J., Kalas, M., Bodis, K.D.M.,Pappenberger, F., 2011. Quality control, validation and user

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1 289

feedback of the European Flood Alert System (EFAS). Int. J.Digital Earth 4, 77–90.

Demeritt, D., Nobert, S., Cloke, H.L., Pappenberger, F., 2013. TheEuropean Flood Alert System and the communication,perception, and use of ensemble predictions for operationalflood risk management. Hydrol. Process. 27, 147–157, http://dx.doi.org/10.1002/hyp.9419.

Dobler, C., Pappenberger, F., 2013. Global sensitivity analyses fora complex hydrological model applied in an Alpinewatershed. Hydrol. Process. 27, 3922–3940, http://dx.doi.org/10.1002/hyp.9520.

EASPE, 2002. Use and benefits of the national weather serviceand flood forecast. Report by the National HydrologicWarning Council, Denver, USA, 33http://www.nws.noaa.gov/oh/ahps/AHPS%20Benefits.pdf (accessed 10.05.15).

EC, 2014a. In: Union, T.E.P.a.t.C.o.t.E. (Ed.), Amendment toCouncil Regulation (EC) NO 2012/2002 establishing theEuropean Solidarity Fund. Official Journal of the EuropeanUnion, Brussels http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX: 32014R0661&from=EN (accessed27.6.14).

EC, 2014b. Guide to Cost-Benefit Analysis. Final Report.European Commission, Directorate-General for Regional andUrban Policy, Brussels, http://dx.doi.org/10.2776/97516ISBN:978-92-79-34796-2, http://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/cba_guide.pdf (accessed16.06.08).

ECWMF, 2014. EFAS Bulletin December 2013–January 2014.https://www.efas.eu/download/efasBulletins/2014/bulletin_dec-jan_14.pdf.

EFAS Dissemination Centre, 2014. EFAS Alert Skill. In: 9th EFASAnnual Meeting, 8–9 April 2014, Rijkswaterstaat, Lelystad,The Netherlands.

EM-DAT, 2014. The OFDA/CRED International DisasterDatabase. Universite Catholique de Louvain, Brussels,Belgiumwww.emdat.be.

European Commission, 2008. Guide to Cost-Benefit Analysis ofInvestment Projects. http://ec.europa.eu/regional_policy/sources/docgener/guides/cost/guide2008_en.pdf.

Farber, S., Costanza, R., Childers, D.L., Erickson, J., Gross, K.,Grove, M., Hopkinson, C.S., Kahn, J., Pincetl, S., Troy, A.,Warren, P., Wilson, M., 2006. Linking ecology and economicsfor ecosystem management. Bioscience 56, 121–133.

Frei, T., 2010. Economic and social benefits of meteorology andclimatology in Switzerland. Meterol. Appl. 17, 39–44.

Hagedorn, R., Buizza, R., Hamill, T.M., Leutbecher, M., Palmer,T.N., 2012. Comparing TIGGE multimodel forecasts withreforecast-calibrated ECMWF ensemble forecasts. Q. J. R.Meterol. Soc. 138, 1814–1827.

Haiden, T., Magnusson, L., Tsonevsky, I., Wetterhall, F., Alfieri,L., Pappenberger, F., de Rosnay, P., Munoz-Sabater, J.,Balsamo, G., Albergel, C., Forbes, R., Hewson, T., Malardel, S.,Richardson, D., 2014. ECMWF forecast performance duringthe June 2013 flood in Central Europe. In: ECMWF TechnicalMemorandum No 723. .

Hallegatte, S., 2012. A cost effective solution to reduce disasterlosses in developing countries – hydro-meterologicalservices, early warning an evacuation – Policy ResearchWorking Paper 6058. World Bank, 1–20.

Hallegatte, S., Dumas, P., 2008. Can natural disasters havepositive consequences? Investigating the role of embodiedtechnical change. Ecol. Econ. 68, 777–786.

Hogan, R.J., Mason, I.B., 2012. Deterministic forecasts of binaryevents. In: Jolliffe, I.T., Stephenson, D.B. (Eds.), ForecastVerification: A Practitioner’s Guide in Atmospheric Science.Wiley-Blackwell, New York, pp. 31–59.

International Commission For The Protection Of the Rhine,2002. Non Structural Flood Plain Management – Measuresand their effectiveness. IPR, Koblenz.

Jongman, B., Hochrainer-Stigler, S., Feyen, L., Aerts, J.C.J.H.,Mechler, R., Botzen, W.J.W., Bouwer, L.M., Pflug, G., Rojas, R.,Ward, P.J., 2014a. Increasing stress on disaster-risk financedue to large floods. Nat. Clim. Change 4, 264–268.

Jongman, B., Hochrainer-Stigler, S., Feyen, L., Aerts, J.C.J.H.,Mechler, R., Botzen, W.J.W., Bouwer, L.M., Pflug, G., Rojas, R.,Ward, P.J., 2014b. Reply to ‘Statistics of flood risk’. Nat. Clim.Change 4, 844–845.

Jongman, B., Kreibich, H., Apel, H., Barredo, J.I., Bates, P.D.,Feyen, L., Gericke, A., Neal, J., Aerts, J.C.J.H., Ward, P.J., 2012.Comparative flood damage model assessment: towards aEuropean approach. Nat. Hazards Earth Syst. Sci. 12, 3733–3752.

Klafft, M., Meissen, I., 2011. Assessing the economic value ofearly warning systems. In: Santos, M.A., Souse, L., Portela, E.(Eds.), 8th International Conference on Information Systemsfor Crisis Response and Management, 8–11 May 2011,Lisbon, Portugal.

Merz, B., Aerts, J.C.J.H., Arnbjerg-Nielsen, K., Baldi, M., Becker,A., Bichet, A., Bloschl, G., Bouwer, L.M., Brauer, A., Cioffi, F.,Delgado, J.M., Gocht, M., Guzetti, F., Harrigan, S.,Hirschboeck, K., Kilsby, C., Kron, W., Kwon, H.-H., Lall, U.,Merz, R., Nissen, K., Salvatti, P., Swierczynski, T., Ulbrich, U.,Viglione, A., Ward, P.J., Weiler, M., Wilhelm, B., Nied, M.,2014. Floods and climate: emerging perspectives for floodrisk assessment and management. Nat. Hazards Earth Syst.Sci. 14, 1921–1942 10.5194/nhess-1914-1921-2014.

Merz, B., Kreibich, H., Apel, H., 2008. Flood Risk Analysis:Uncertainties and Validation. Osterreichische Wasser- undAbfallwirtschaft 05–06. .

Merz, B., Kreibich, H., Schwarze, R., Thieken, A., 2010. Reviewarticle ‘‘Assessment of economic flood damage’’. Nat.Hazards Earth Syst. Sci. 10, 1697–1724, http://dx.doi.org/10.5194/nhess-1610-1697-2010.

Merz, B., Kreibich, H., Thieken, A., Schmidtke, R., 2004.Estimation uncertainty of direct monetary flood damage tobuildings. Nat. Hazards Earth Syst. Sci. 4, 153–163.

Meyer, V., Becker, N., Markantonis, V., Schwarze, R., van denBergh, J.C.J.M., Bouwer, L.M., Bubeck, P., Ciavola, P.,Genovese, E., Green, C., Hallegatte, S., Kreibich, H., Lequeux,Q., Logar, I., Papyrakis, E., Pfurtscheller, C., Poussin, J.,Przyluski, V., Thieken, A.H., Viavattene, C., 2013. Reviewarticle: Assessing the costs of natural hazards – state of theart and knowledge gaps. Nat. Hazards Earth Syst. Sci. 13,1351–1373.

Meyer, V., Priest, S., Kuhlicke, C., 2012. Economic evaluation ofstructural and non-structural flood risk managementmeasures: examples from the Mulde River. Nat. Hazards 62,301–324.

Molinari, D., Handmer, J., 2011. A behavioural model forquantifying flood warning effectiveness. J. Flood RiskManage. 4, 23–32.

Molinari, D., Menoni, S., Aronica, G.T., Ballio, F., Berni, N.,Pandolfo, C., Stelluti, M., Minucci, G., 2014. Ex post damageassessment: an Italian experience. Nat. Hazards Earth Syst.Sci. 14, 901–916.

Paccagnan, V., 2012. Updating Uplift Factors for BenefitAssessment. Economic and Social Science Evidence Team,Environment Agency, Bristol.

Pagano, T.C., Wood, A.W., Ramos, M.-H., Cloke, H.L.,Pappenberger, F., Clark, M.P., Cranston, M., Kavetski, D.,Mathevet, T., Sorooshian, S., Verkade, J.S., 2014. Challenges ofoperational river forecasting. J. Hydrometerol. 15, 1692–1707.

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1290

Pappenberger, F., Beven, K.J., Hunter, N.M., Bates, P.D.,Gouweleeuw, B.T., Thielen, J., de Roo, A.P.J., 2005. Cascadingmodel uncertainty from medium range weather forecasts(10 days) through a rainfall-runoff model to flood inundationpredictions within the European Flood Forecasting System(EFFS). Hydrol. Earth Syst. Sci. 9, 381–393.

Pappenberger, F., Dutra, E., Wetterhall, F., Cloke, H.L., 2012.Deriving global flood hazard maps of fluvial floods through aphysical model cascade. Hydrol. Earth Syst. Sci. 16, 4143–4156, http://dx.doi.org/10.5194/hess-4116-4143-2012.

Pappenberger, F., Ramos, M.H., Cloke, H.L., Wetterhall, F.,Alfieri, L., Bogner, K., Mueller, A., Salamon, P., 2015. How do Iknow if my forecasts are better? Using benchmarks inhydrological ensemble prediction. J. Hydrol. 522, 697–713.

Pappenberger, F., Thielen, J., Del Medico, M., 2011. The impact ofweather forecast improvements on large scale hydrology:analysing a decade of forecasts of the European Flood AlertSystem. Hydrol. Process. 25, 1091–1113.

Pappenberger, F., Wetterhall, F., Dutra, E., Di Giuseppe, F.,Bogner, K., Alfieri, L., Cloke, H.L., 2013. Seamless forecastingof extreme events on a global scale. In: Climate and LandSurface Changes in Hydrology. Proceedings of H01, IAHS-IAPSO-IASPEI Assembly. July 2013, Gothenburg, Sweden(IAHS Publ. 359, 2013).

Parker, D., 2003. Designing flood forecasting, warning andresponse systems from a societal perspective. In:International Conference on Alpine Meteorology and Meso-Alpine Programme. Brig, Switzerland, pp. 1–20, http://www.map.meteoswiss.ch/map-doc/icam2003/Presentation/10.1/Brig_document.pdf.

Parker, D., Tunstall, S., Wilson, T., 2005. Socio-economic benefitof flood forecasting and warning. In: InternationalConference on Innovation Advances and Implementation ofFlood Forecasting Technology, Tromso, Norway.

Parker, D.J., 1991. The damage-reducing effects of floodwarnings’. In: Report prepared for Halcrow. National RiversAuthority (Anglian Regional) Regional Telemetry SchemeAppraisal. .

Parker, D.J., Priest, S.J., 2012. The fallibility of flood warningchains: can Europe’s flood warnings be effective? WaterResour. Manag. 26, 2927–2950.

Parker, D.J., Priest, S.J., Schildt, A., Handmer, J.W., 2008.Modelling the damage reducing effects of flood warnings. In:FLOODsite Report No. T10-07-12. HR Wallingford,Wallingford, UK.

Parker, D.J., Tunstall, S.M., McCarthy, S., 2007. New insights intothe benefits of flood warnings: results from a householdsurvey in England and Wales. Environ. Hazards 7, 193–210.

Penning-Rowsell, E., Priest, S., Parker, D., Morris, J., Tunstall, S.,Viavattene, C., Chatterton, J., Owen, D., 2013. Flood andCoastal Erosion Risk Management – A Manual for EconomicAppraisal. Routeldge, Oxon.

Perrels, A., Frei, T., Espejo, F., Jamin, L., Thomalla, A., 2013.Socio-economic benefits of weather and climate services inEurope. Adv. Sci. Res. 10, 65–70.

Pfurtscheller, C., 2014. Regional economic impacts of naturalhazards – the case of the 2005 Alpine flood event in Tyrol(Austria). Nat. Hazards Earth Syst. Sci. 14, 359–378.

Priest, S.J., Parker, D.J., Tapsell, S.M., 2011. Modelling thepotential damage-reducing benefits of flood warnings usingEuropean cases. Environ. Hazards Hum. Policy Dimens. 10,101–120.

Ramos, M.-H., Bartholmes, J., Thielen-del Pozo, J., 2007.Development of decision support products based onensemble forecasts in the European flood alert system.Atomos. Sci. Lett. 8, 113–119.

Ramos, M.-H., Mathevet, T., Thielen, J., Pappenberger, F., 2010.Communicating uncertainty in hydro-meteorologicalforecasts: mission impossible? Meterol. Appl. 17, 223–235.

Ramos, M.H., van Andel, S.J., Pappenberger, F., 2013. Doprobabilistic forecasts lead to better decisions? Hydrol.Earth Syst. Sci. 17, 2219–2232.

Rogers, D., Tsirkunov, V., 2010. Costs and Benefits of EarlyWarning Systems. Global Assessment Report on DisasterRisk Reduction. United Nations International Strategy forDisaster Reduction and World Bank, Geneva, Switzerland/Washington, DCwww.preventionweb.net/english/hyogo/gar/2011/en/bgdocs/Rogers_&_Tsirkunov_2011.pdf.

Saint-Geours, N., Bailly, J.-S., Grelot, F., Lavergne, C., 2014. Multi-scale spatial sensitivity analysis of a model for economicappraisal of flood risk management policies. Environ. Model.Softw. 60, 153–166.

Salamon, P., 2014. Major Update of the European FloodAwareness System–Executive Summary. https://www.efas.eu/download/home/major_update_01-14.pdf (last accessed22.08.14).

Sampson, C.C., Fewtrell, T.J., O’Loughlin, F., Pappenberger, F.,Bates, P.B., Freer, J.E., Cloke, H.L., 2014. The impact ofuncertain precipitation data on insurance loss estimatesusing a flood catastrophe model. Hydrol. Earth Syst. Sci. 18,2305–2324.

Scott, M., Wicks, J., 2012. Supporting the Revision of FIMInvestment Strategy. Initial Review and Recommendations,Halcrow, Swindon.

SNIFFER, 2006–2009. Assessing the Benefits of Flood Warning(UKCC10, UKCC10A, UKCC10B). Download from http://www.sniffer.org.uk.

Stephens, E., Cloke, H., 2014. Improving flood forecasts for betterflood preparedness in the UK (and beyond). Geograph. J..

Subbiah, A.R., Bildan, L., Narasimhan, R., 2008. BackgroundPaper on Assessment of the Economics of Early WarningSystems for Disaster Risk Reduction. World Bank and GlobalFacility for Disaster Reduction and Recovery, Washington,DChttp://gfdrr.org/gfdrr/sites/gfdrr.org/files/New%20Folder/Subbiah_EWS.pdf.

Teisberg, T.J., Weiher, R.F., 2008. Background Paper onAssessment of the Economics of Early Warning Systems forDisaster Risk Reduction. World Bank and Global Facility forDisaster Reduction and Recovery, Washington, DC.

Thielen, J., 2014. Current floods in central Europe: awarenessand monitoring. In: HEPEX Blog. http://hepex.irstea.fr/current-floods-in-central-europe-awareness-and-monitoring/ (last accessed 29.08.14).

Thielen, J., Annunziato, A., Andredakis, I., McCormick, N., Kalas,M., Kechagioglou, X., Kucera, J., Muraro, D., Probst, P.,Salamon, P., Thiemig, V., 2014. Balkans’ worst floods formore than 100 years. In: HEPEX Blog. http://hepex.irstea.fr/balkans-worst-floods-for-more-than-100-years/ (lastaccessed 29.08.14).

Thielen, J., Bartholmes, J., Ramos, M.-H., de Roo, A., 2009a. TheEuropean Flood Alert System – Part 1: Concept anddevelopment. Hydrol. Earth Syst. Sci. 13, 125–140.

Thielen, J., Bogner, K., Pappenberger, F., Kalas, M., del Medico,M., de Roo, A., 2009b. Monthly-, medium-, and short-rangeflood warning: testing the limits of predictability. Meterol.Appl. 16, 77–90.

Thiemig, V., Bisselink, B., Pappenberger, F., Thielen, J., 2014. Apan-African flood forecasting system. Hydrol. Earth Syst. Sci.Discuss. 11, 5559–5597.

Thurston, N., Finlinson, B., Breakspear, R., Williams, N., Shaw, J.,Chatterton, J., 2008. Developing the Evidence Base for FloodResistance and Resilience. Defra R&D Summary Report. .

UNISDR, 2004. Guidelines for Reducing Flood Losses, UnitedNations International Strategy for Disaster Reduction,DRR7639. UNISDR. http://www.unisdr.org/we/inform/publications/558.

Verkade, J.S., Werner, M.G.F., 2011. Estimating the benefits ofsingle value and probability forecasting for flood warning.

e n v i r o n m e n t a l s c i e n c e & p o l i c y 5 1 ( 2 0 1 5 ) 2 7 8 – 2 9 1 291

Hydrol. Earth Syst. Sci. 15, 3751–3765, http://dx.doi.org/10.5194/hess-3715-3751-2011.

Vilier, J., Kok, M., Nicolai, R.P., 2014. In: Steenbergen, R.D.J.M.,van Gelder, P.H.A.J.M., Miraglia, S., Vrouwenvelder, A.C.W.M.(Eds.), Safety, Reliability and Risk Analysis: Beyond theHorizon. Taylor & Francis Group, London, pp. 2415–2423,ISBN 978-1-138-00123-7, http://www.hkv.nl/site/hkv/upload/publication/Assessment_of_the_losses_due_to_business_MK_RN(1).pdf (accessed 10.05.15).

Vitart, F., Buizza, R., Alonso Balmaseda, M., Balsamo, G., Bidlot,J.-R., Bonet, A., Fuentes, M., Hofstadler, A., Molteni, F.,Palmer, T.N., 2008. The new VarEPS-monthly forecastingsystem: a first step towards seamless prediction. Q. J. R.Meterol. Soc. 134, 1789–1799.

Ward, P.J., de Moel, H., Aerts, J.C.J.H., 2011. How are flood riskestimates affected by the choice of return-periods? Nat.Hazards Earth Syst. Sci. 11, 3181–3195.

Ward, P.J., Eisner, S., Florke, M., Dettinger, M.D., Kummu, M., 2014.Annual flood sensitivities to El Nino Southern Oscillation atthe global scale. Hydrol. Earth Syst. Sci. 18, 47–66.

Ward, P.J., Jongman, B., Sperna-Weiland, F., Bouwman, A., VanBeek, R., Bierkens, M.F.P., Ligtvoet, W., Winsemius, H.C.,2013. Assessing flood risk at the global scale: model setup,results, and sensitivity. Environ. Res. Lett. 8, 044019, http://dx.doi.org/10.041088/041748-049326/044018/044014/044019.

Winsemius, H.C., Van Beek, L.P.H., Jongman, B., Ward, P.J.,Bouwman, A., 2013. A framework for global river flood riskassessments. Hydrol. Earth Syst. Sci. 17, 1871–1892.