ZHAO Linna

48
1 ZHAO Linna Zhao Linna, Zhao Linna, WU Hao, Liang Li, DI Jingyue, WANG Zhi Public Weather Service Center, China Meteorological Administration Public Weather Service Center, China Meteorological Administration Qi Dan Qi Dan , TIAN Fuyou National Meteorological Center, China Meteorological Administration National Meteorological Center, China Meteorological Administration Probabilistic flood prediction Probabilistic flood prediction by TIGGE on upriver of HUAIhe by TIGGE on upriver of HUAIhe catchment catchment e 4th THORPEX-Asia Science Workshop & 9th ARC Meeting ARC Meeting ARC Meetin e 4th THORPEX-Asia Science Workshop & 9th ARC Meeting ARC Meeting ARC Meetin 30 October – 3 November 2012, Kunming, China 30 October – 3 November 2012, Kunming, China Zhao Linna Zhao Linna Public Weather Service Center, China Meteorological Administration Public Weather Service Center, China Meteorological Administration

description

The 4th THORPEX-Asia Science Workshop & 9th ARC Meeting ARC Meeting ARC Meeting. Probabilistic flood prediction by TIGGE on upriver of HUAIhe catchment. ZHAO Linna. Zhao Linna, WU Hao, Liang Li, DI Jingyue, WANG Zhi Public Weather Service Center, China Meteorological Administration - PowerPoint PPT Presentation

Transcript of ZHAO Linna

Page 1: ZHAO Linna

1

ZHAO Linna

Zhao Linna, Zhao Linna, WU Hao, Liang Li, DI Jingyue, WANG Zhi Public Weather Service Center, China Meteorological AdministrationPublic Weather Service Center, China Meteorological AdministrationQi Dan Qi Dan , TIAN FuyouNational Meteorological Center, China Meteorological AdministrationNational Meteorological Center, China Meteorological Administration

Probabilistic flood prediction by TIGGE Probabilistic flood prediction by TIGGE on upriver of HUAIhe catchmenton upriver of HUAIhe catchment

The 4th THORPEX-Asia Science Workshop & 9th ARC Meeting ARC Meeting ARC MeetingThe 4th THORPEX-Asia Science Workshop & 9th ARC Meeting ARC Meeting ARC Meeting

30 October – 3 November 2012, Kunming, China30 October – 3 November 2012, Kunming, China

Zhao LinnaZhao LinnaPublic Weather Service Center, China Meteorological AdministrationPublic Weather Service Center, China Meteorological Administration

Page 2: ZHAO Linna

2

ContentsContents

1. Introduction 2. The data and the test catchment3. The Experiment of Probabilistic flood

prediction 4. The Calibration of Precipitation CMA-

EPS and Hydrological Experiments5. Outlook

Page 3: ZHAO Linna

3

1. Introduction • Numerical weather forecasts without uncertainties specified are

hard to be incorporated into operations and decision making of other downstream applications such as early warning of floods and rainfall induced geological hazards.

• The ensemble forecasting technique provides an e ective way to ffquantify those uncertainties. The uncertainty of ensemble forecasting can be expressed in terms of forecast probability density function (PDF), based upon which probabilistic forecast is generated.

• To make full use of all the information available in an ensemble forecast, Bayesian model averaging (BMA) was introduced by Raftery et al.(2005) as a statistical post-processing method for producing probabilistic forecast from an ensemble.

Page 4: ZHAO Linna

4

1. Introduction • By BMA, the overall forecast PDF of any variable of interest is a

weighted average of forecast PDFs based on each of the individual forecasts, where the weights are estimated by posterior probabilities of the models generating the forecasts and reflect the relative forecasting skills of the individual models in the training period.

• The BMA weights can be used to assess the usefulness of ensemble members, and this can be used as a basis for selecting ensemble members, this can be useful given the cost of running large ensembles.

• BMA offers the added advantage, by giving a full predictive PDF, of being able to give probabilities of exceeding arbitrary precipitation amounts, rather than having to create a new logistic regression model for each threshold of interest.

Page 5: ZHAO Linna

5

1. Introduction• The main objective of this work is thus to give a

demonstration for users who need to build or to give probabilistic hydro-meteorological forecasts with TIGGE database.

• In this respect, the results are felt to extend beyond the Dapoling-Wangjiaba basin of China itself and to apply to more similar test catchment.

Page 6: ZHAO Linna

6

Contents1. Introduction 2. The data and the test catchment3. The Experiment of Probabilistic

flood prediction 4. The Calibration of Precipitation

CMA-EPS and Hydrological Experiments

5. Outlook

Page 7: ZHAO Linna

7

The ensemble systems used in this work

ECMWF NCEP CMA

Country/Domain

Ensemble members

Forecast length

Perturbation method

Horizontal resolution

Vertical resolution

Europe

51

15 days

SingularVectors

TL399 (0-10 d)TL255 (11-15 d)

62

United States

21

16 days

ET (Ensemble Transform)

T126

28

China

15

10 days

BV (Bred Vectors)

T213

31

Totally 87 membersTotally 87 members

Page 8: ZHAO Linna

888

Test catchment of Dapoling-WangjiabaTest catchment of Dapoling-Wangjiaba• The Dapoling-Wangjiaba catchment is the origination

of the Huaihe River Basin• Sub-catchment locates at the upper reach of Huaihe

River Basin• Coverage of about 30,630 km2 (is 16% of Huaihe basin)• Altitude ranging from 200 to 500 meters

22 Data and test areaData and test area

• Daily Precipitation observations from June 1, 2008 — August 31, 2008

• 19 rain gauges in the test catchment• The hourly accumulative rain gauge data • Hydrologic data is the daily flow of

Xixian and Wangjiaba hydrological site in upper stream of Huai river

Page 9: ZHAO Linna

9

ContentsContents

1. Introduction 2. The data and the test catchment3. The Experiment of Probabilistic flood

prediction 4. The Calibration of Precipitation CMA-

EPS and Hydrological Experiments5. Outlook

Page 10: ZHAO Linna

10

3. The Experiment of Probabilistic flood 3. The Experiment of Probabilistic flood prediction by TIGGE databaseprediction by TIGGE database

• Flood event: 23 Jul.-3 Aug. 2008 flood over the upriver of the Huaihe basin

• The hydrological characteristics of the main hydrological control stations on the upriver have important demonstrative significance.

• we selected the Huaihe basin as a research area, especially the upriver of the basin. The flood forecasting at Xixian and Wangjiaba hydrological stations with runoff records are investigated.

• The areal precipitation is obtained by averaging the records of 19 observations or simulated precipitation values.

Page 11: ZHAO Linna

11

• Nash-Sutcliffe Nash-Sutcliffe coefficientcoefficient is used to is used to assess the predictive assess the predictive power of hydrological power of hydrological models. It is defined as:models. It is defined as:

200

20

)()(

0.1QQQQ

E s

3. The Experiment of Probabilistic flood prediction by TIGGE 3. The Experiment of Probabilistic flood prediction by TIGGE database : Methodologydatabase : Methodology

• Essentially, the closer the model efficiency is to 1, the more accurate the model is.

• where Qo is observed discharge, and Qm is modeled discharge. Qot is observed discharge at time t.

• Nash–Sutcliffe efficiencies can range from −∞ to 1. An efficiency of 1 (E = 1) corresponds to a perfect match of modeled discharge to the observed data. An efficiency of 0 (E = 0) indicates that the model predictions are as accurate as the mean of the observed data, whereas an efficiency less than zero (E < 0) occurs when the observed mean is a better predictor than the model or, in other words, when the residual variance (described by the numerator in the expression above), is larger than the data variance (described by the denominator).

from Nash, J. E. and J. V. Sutcliffe (1970), Journal of Hydrology

Page 12: ZHAO Linna

12

3. The Experiment of Probabilistic flood prediction by TIGGE 3. The Experiment of Probabilistic flood prediction by TIGGE database : Methodologydatabase : Methodology

VIC hydrological model VIC hydrological model

0

100

200

300

04-aug 05-aug 06-aug 07-aug 08-aug 09-aug 10-aug 11-aug 12-aug 13-aug 14-aug

Q (m

3/s)

ESP-median ESP-quartile ESP-min/maxOPERATIONAL OBS

98%75%50%25%2%

Hydrological probabilistic forecastHydrological probabilistic forecast

5%5%

95%95%

Confluence ModelRunoffExport section flow

Precipitation probabilisticforecast

p(Ut) U(t)

Precipitation probabilisticforecast

p(Ut)p(Ut) U(t)TIGGE-NCEPTIGGE-NCEP

TIGGE-CMATIGGE-CMA

TIGGE-ECMWFTIGGE-ECMWF

Page 13: ZHAO Linna

1313 13

day-3 Xi XianResults Results __ River discharge predictionRiver discharge prediction

• The NCEP-EPS and CMA-EPS can bracket half of the observed discharges.

• The 5th–99th percentile distribution of the EC-EPS is large and nearly brackets all discharge observations during the period 23 July–3 August 2008 except for the flood ascending period on 24 and 25 July

• The performance of the EC-EPS is the best among the three systems, which is consistent with the precipitation forecast results of the EPSs.

• The performance of the Grand-EPS is equal or better than that of EC-EPS.

• Both the single EC-EPS and the Grand-EPS can bracket most of the observations between 5th and 99th quantile.

Page 14: ZHAO Linna

141414

Results Results __ Comparison of the three EPSs and the grand ensemble between

5th,25th,50th,75th,95th and 99th percentile for the 3-day lead time runoff forecasts day-3 Xi Xian

• All of the EPSs did not obviously provide indicative significance for the Xixian flooding process, especially in the first rising limb and the next recession limb, because most of the predictions occurred in the extreme area between 95th and 99th quantile.

• Generally, the main referenced inter-zone between 25th and 75th quantile is usually considered as a credible range in the EPS.

• However, it is clear that the forecast performance in the second rising limb on 1 August 2008 is better than the first rising limb for all of the EPSs.

Page 15: ZHAO Linna

1515

day-3 Wang JiabaResults Results __ River discharge predictionRiver discharge prediction

• Wangjiaba station is the outlet of the upper Huaihe River catchment.

• All of the EPSs predict the flood in good agreement with the observed discharge, which falls in the 5th–99th quantile except the NCEP-EPSexcept the NCEP-EPS

Page 16: ZHAO Linna

16

Results Results __ Comparison of the three EPSs and the grand ensemble between

5th,25th,50th,75th,95th and 99th percentile for the 3-day lead time runoff forecasts day-3 Wang Jiaba

• The result is different from the discharge prediction at Xixian station in that the twice rising discharge thresholds at Wangjiaba station are captured nicely and the predictions of discharge are all nearly in the 25th–75th quantile

• It is a valuable reference for decision-making, but one deficiency is that the prediction in the recession limb is generally quicker than the observationin the recession limb is generally quicker than the observation.

• For the three EPSs, the NCEP-EPS predicted discharge is generally smaller than the observation, while the EC-EPS and the CMA-EPS produce better predictions of discharge in the study period at Wangjiaba station, which provides helpful probabilistic hydrological forecasts for decision-making.

Page 17: ZHAO Linna

17

Characteristics of Nash-Sutcliffe efficiency coefficient of Wangjiaba and XixianCharacteristics of Nash-Sutcliffe efficiency coefficient of Wangjiaba and Xixian

Wang JiabaWang Jiaba Xi XianXi Xian

Max Max MinMin AveAve Max Max MinMin AveAve

ECMWFECMWF 0.970.97 0.54 0.54 0.680.68 0.960.96 -0.15-0.15 0.260.26

NCEPNCEP -1.22-1.22 -1.47-1.47 -1.36-1.36 0.410.41 -0.08-0.08 0.210.21

CMACMA 0.740.74 0.280.28 0.510.51 0.760.76 -0.45-0.45 -0.13-0.13

• The greatest NS value appears in the prediction of the EC-EPS, which is The greatest NS value appears in the prediction of the EC-EPS, which is 0.970.97

for Wangiiaba station and 0.96 for Xixian station. for Wangiiaba station and 0.96 for Xixian station.

• The lowest NS value The lowest NS value -1.47-1.47 appears in the prediction of the NCEP-EPS. For appears in the prediction of the NCEP-EPS. For

Wangjiaba stationWangjiaba station

• The average value of the Nash-Sutcliffe index in the EC-EPS isThe average value of the Nash-Sutcliffe index in the EC-EPS is 0.68,0.68, which is which is

greater than that of other EPSs.greater than that of other EPSs.

Evaluation of the EPSs at Xixian and Wangjiaba stations

Page 18: ZHAO Linna

HuaibinHuaibin CMA CMA Xi XianXi Xian CMA CMA

Xi XianXi Xian NCEP NCEPHuaibinHuaibin NCEP NCEP

Xi XianXi Xian ECMWFECMWFHuaibinHuaibin ECMWF ECMWF

Xi XianXi Xian GrandGrandHuaibin Huaibin GrandGrand

Ensemble forecasts of precipitation

• The CMA-EPS largely underestimated the precipitation amount while the NCEP-EPS missed many precipitation events especially at Xixian.

• For most of the forecast days, the EC-EPS produced precipitation forecasts within the range of 5th–95th percentile, which are closest to the observation, so it outperformed the other two EPSs.

• The Grand-EPS production is the best since almost half of the forecasts are within the range of 25th–75th percentile.

• These results are in accordance with the results of the river discharge predictions.

• The performance of the ensemble precipitation forecasts plays an important role in the forecasts of the river discharge.

Page 19: ZHAO Linna

19

Another reason is …• The different performances of the EPSs at the two

stations may be caused by the differences in the geographic location and topographical distribution.

• Xixian station is located in the upriver of the entire basin, where the topography is fairly complex

• so there will be no enough time for this station to respond to the heavy rainfall.

• But Wangjiaba station, located at the exit cross-section of the catchment, has sufficient time to respond to the flood process

Page 20: ZHAO Linna

20

ContentsContents

1. Introduction 2. The data and the test catchment3. The Experiment of Probabilistic flood

prediction 4. The Calibration of Precipitation CMA-

EPS and Hydrological Experiments5. Outlook

Page 21: ZHAO Linna

21

4. The Calibration of Precipitation Ensemble 4. The Calibration of Precipitation Ensemble Forecast Using Bayesian Model Averaging and Forecast Using Bayesian Model Averaging and Hydrological Experiments Hydrological Experiments

VIC hydrological model VIC hydrological model

0

100

200

300

04-aug 05-aug 06-aug 07-aug 08-aug 09-aug 10-aug 11-aug 12-aug 13-aug 14-aug

Q (m

3/s)

ESP-median ESP-quartile ESP-min/maxOPERATIONAL OBS

98%75%50%25%2%

Hydrological probabilistic forecastHydrological probabilistic forecast

5%5%

95%95%

Confluence ModelRunoffRunoffExport section flow

Precipitation probabilisticforecast

p(Ut) U(t)

Precipitation probabilisticforecast

p(Ut)p(Ut) U(t)TIGGE-NCEPTIGGE-NCEP

TIGGE-CMATIGGE-CMA

TIGGE-ECMWFTIGGE-ECMWF

Page 22: ZHAO Linna

22

• The Calibration of Precipitation Ensemble Forecast on TIGGE-CMA is investigated• VIC (Variable Infiltration Capacity) model is a hydrological model based on spatial

distribution grid (Wood, 1992; Liang Xu, etc., 2003)• For research purpose, conveniently• All date are interpolated in 15km×15km spatial resolution

4. The Calibration of Precipitation Ensemble 4. The Calibration of Precipitation Ensemble Forecast Using Bayesian Model Averaging and Forecast Using Bayesian Model Averaging and Hydrological Experiments Hydrological Experiments

Page 23: ZHAO Linna

23

• BMABMA (( Bayesian Model AveragingBayesian Model Averaging )) is a statistical way of post-processing is a statistical way of post-processing forecast ensembles to create predictive probability density functions (PDFs) forecast ensembles to create predictive probability density functions (PDFs) for weather quantities.for weather quantities.– It represents the predictive PDF as a weighted average of PDFs

centered on the individual bias-corrected forecasts, where the weights are posterior probabilities of the models generating the forecasts and reflect the forecasts’ relative contributions to predictive skill over a training period.

– BMA model not only can provide the biggest forecast possibility, but also described the weather forecast uncertainty.

• Daily Precipitation observations from June 1 — August 31, 2008• Verification:

– CRPS (continuous ranked probability score )& MAE(mean absolute error, The smaller the score is, the better the calibration result is

– The 25th, 75 percentile of hydrological probabilistic forecast are verified by three index, namely certainty coefficient, relative error of flood peak, difference of peak time

4. The Calibration of Precipitation Ensemble 4. The Calibration of Precipitation Ensemble Forecast Using Bayesian Model Averaging and Forecast Using Bayesian Model Averaging and Hydrological Experiments Hydrological Experiments

Page 24: ZHAO Linna

2424

The contrast of MAE and CRPS score of 24h, 48h and 72h forecast between the original ensemble forecast (CMA) and bayesian average model ensemble forecast (BMA)

Date Score24h 48h 72h

CMA BMA CMA BMA CMA BMA

7 月 21日 CRPS 2.05 0.62 0.71 1.21 0.86 1.52

MAE 2.52 0.43 0.88 0.70 1.02 1.16

7 月 22日 CRPS 9.36 9.13 9.82 8.73 8.99 8.73

MAE 10.84 10.02 11.02 10.96 10.74 10.71

7 月 23日 CRPS 31.45 45.59 37.79 47.12 40.26 48.42

MAE 38.62 56.36 45.28 56.64 49.28 57.47

7 月 24日 CRPS 12.01 11.52 12.32 13.31 14.72 12.07

MAE 14.80 14.55 14.97 14.84 18.34 17.05

7 月 25日 CRPS 5.11 3.16 4.25 3.31 3.52 2.96

MAE 6.81 3.41 5.31 3.71 4.58 3.39

7 月 26日 CRPS 8.63 4.23 4.38 3.56 4.50 3.86

MAE 10.78 4.96 5.38 4.28 5.74 4.70

7 月 27日 CRPS 6.97 4.74 5.75 4.79 5.33 4.82

MAE 8.91 5.96 7.25 6.32 6.65 6.42

7 月 28日 CRPS 7.14 4.14 6.36 3.60 3.52 3.04

MAE 9.53 4.66 8.63 3.93 4.89 3.31

7 月 29日 CRPS 2.82 1.18 1.56 1.08 3.20 1.63

MAE 3.64 0.86 2.17 0.49 4.62 0.85

7 月 30日 CRPS 8.91 6.39 8.15 6.97 7.85 6.59

MAE 11.22 8.85 8.78 8.64 8.94 8.35

• Known from the CRPS and MAE, the 24h forecast of BMA model obviously has a better deviation correction effect compared to the CMA ensemble forecast.

• The calibration of 48h or 72h forecast also performs well.

Page 25: ZHAO Linna

2525

(a) BMA before BMA

The box plot of BMA model and that of before for stations at Xixian 4 Results4 Results

Ensemble mean is the mean value of 15 ensemble members, so that it can be taken as deterministic forecast. Although its forecast value is probably more close to the true value compared with some individual member, the deviation between them is still obvious.

The BMA forecast, the majority of the The BMA forecast, the majority of the dates are in the valid intervaldates are in the valid interval

8/10 obs. are captured 5/10 obs. are captured

Page 26: ZHAO Linna

2626

(a)BMA

4 Results4 Results

(b) Before BMA

The valid interval given by the BMA model contains the observations. However, the valid interval given by CMA ensemble forecast contains few observations relatively.The valid interval forecast given by the BMA model is more likely to contain the true value of observations. Its forecast capacity is better than the deterministic forecast.

The box plot of BMA model and that of before for stations at Wangjiaba

1/10 obs. are captured7/10 obs. are captured

Page 27: ZHAO Linna

2727

Xixian 24h Xixian 48h

Xixian 72h

4 Results4 Results• The comparison chart between simulated discharges by BMA precipitation forecast and observed flow of Xixian hydrologic station

The results of discharge simulation of 24h, 48h and 72h precipitation forecast from 25th to 75th percentile by the BMA model include the possibility of the flood peak occurs and recedes.

Page 28: ZHAO Linna

2828

4 Results4 Results• The comparison chart between simulated discharges by BMA precipitation forecast and observed flow of Wangjiaba hydrologic station

Wangjiaba 24h Wangjiaba 48h

Wangjiaba 72hProbability forecast which forced the hydrological model has contained uncertainty component, so it is very difficult to simulate ideal deterministic runoff by using probabilistic results. But the runoff got by probabilistic forecast is relatively effective to capture the trend of the runoff.

Page 29: ZHAO Linna

29

4 Results4 Results• Statistics of each verify index of Hydrological

probabilistic forecast

Station Lead hour

Certainty coefficient Relative error of peak Difference of

peak time ( da

y)25th percentile

75th

percentile25th

percentile75th

percentile

Xianxi24h 0.56 0.70 -0.41 -0.36 048h -1.13 -1.85 -0.60 -0.50 172h -0.56 -2.65 -0.81 -0.70 2

Wangjiaba24h 0.58 0.60 0.16 0.26 -148h 0.71 0.82 0.04 0.17 -172h 0.34 0.55 0.19 0.03 0

Xixian•Certainty coefficient: -2.65~0.7 at 24,48 & 72hr•Relative error of peak: -0.41~-0.36

Wangjiaba•Certainty coefficient: 0.34~0.82 at 24,48 & 72hr•Relative error of peak: 0.03~0.26

Page 30: ZHAO Linna

30

5. Outlook5. Outlook• For the predictions of flood discharge by the three single

EPSs, the performance of the EC-EPS is the best, while the NCEP-EPS is the worst, and the Grand-EPS is better than any of the single EPS.

• The result of hydrological simulation which forced by the calibrated precipitation is almost consistent with the runoff tendency of observations.

• Grand-EPS produces more reliable predictions of a flooding event and therefore brings significantly valuable results for the operational flood forecasting and warning service.

Page 31: ZHAO Linna

31

5. Outlook5. Outlook• This work gives an encouraging indication that a multi-

model ensemble can provide more valuable probability forecasts than a deterministic prediction for extreme flood events.

• As the probabilistic hydrological forecasting is a hot topic in the hydro-meteorological science, this study provides a good example for how to use the TIGGE achieve data to drive the hydrological model for probabilistic flood forecasts.

• Probabilistic hydrological forecasting is foreseen as inevitable in the development of hydrological forecasting in future.

Page 32: ZHAO Linna

32 : [email protected]

Ref:Zhao LinnaZhao Linna, QI Dan, Tian Fuyou, et al., 2012: Probabilistic flood prediction in the upper Huaihe catchment using TIGGE data. Acta meteor. SinicaActa meteor. Sinica, 26(1), 62-71, doi: 10.1007/s13351-012-0106-3.

Page 33: ZHAO Linna

discussions• The 24h forecast of BMA model obviously has a better

performance compared to the CMA ensemble forecast. The calibration of 48h or 72h forecast also performs well.

• The robustness of Bayesian method needs more comprehensive research. Such as BMA model parameter estimation process has over-estimated risks.

• How to determine the deformation index of fk in the logistic regression model, and how to check the correlation between the index and fk . All of these need continued study.

• Due to the limitation of the time and space resolution of meteorological data, the verify index of hydrological forecast results can't be meticulously analyzed which need collect high-resolution data for further study.

Page 34: ZHAO Linna

• This may explain why it performs well for forecasts at high thresholds where the amount of training data is small. This suggests that BMA may be useful both for routine precipitation forecasting and for forecasting the risk of extreme precipitation events.

• In addition, the application of hydrological model includes many uncertainty factors, such as input, model parameters, structures and so on. In this paper, only the uncertainty of precipitation forecast input is discussed.

34

Page 35: ZHAO Linna

Weight of BMA member on 22 July

35

Page 36: ZHAO Linna

363636

3. Methodology3. Methodology

 

• percentilepercentile • The time lasts from 1 July

to 6 August, 2008, for totally 37 days.

• All precipitation intensities were taken into consideration with the percentile precipitation evaluation for examining the ability of forecasting the extreme rainfall events

5%

25%

50%

(median)

75%

95%

a1

an

ak

small

large

Page 37: ZHAO Linna

37

4 Results4 Results • The forecast of 15 members of CMA model are integrated as probabilistic forecast to contrast with that of BMA model

• The box plot for the CMA model take the maximum of 15 members for the maximum, the minimum of 15 members for the minimum

• the 25th and 75th percentile for box plot

• The box plot for the BMA model take five statistics , namely 95 percentile for the maximum, 5th percentile for the minimum , the 25th and 75th percentile

5%

25%

75%

95% Maximum forecast of original model (CMA)

25%

75%

37

BMA

Minimum forecast of original model (CMA)

valid interval

Page 38: ZHAO Linna

383838

• Areal percentile precipitationAreal percentile precipitation • An established percentile method presented by

Hyndman and Fan (1996) is adopted for the areal percentile precipitation. The equation is given as

3. Methodology3. Methodology

)1()()1()( jji AApQ

• where j = int (p × n+(1+p)/3) and γ = p × n + (1 + p)/3 − j, • p is the percentile, Qi(p) is the percentile areal precipitation, A is the array of the forecasted areal precipitation in ascending order, and n is the number of ensemble members. • The areal precipitation is obtained by averaging the records of 19 observations or simulated precipitation values.

Page 39: ZHAO Linna

393939

• Nash-Sutcliffe Nash-Sutcliffe coefficientcoefficient is used to is used to assess the predictive assess the predictive power of hydrological power of hydrological models. It is defined as:models. It is defined as:

200

20

)()(

0.1QQQQ

E s

3. Methodology3. Methodology

• Essentially, the closer the model efficiency is to 1, the more accurate the model is.

• where Qo is observed discharge, and Qm is modeled discharge. Qot is observed discharge at time t.

• Nash–Sutcliffe efficiencies can range from −∞ to 1. An efficiency of 1 (E = 1) corresponds to a perfect match of modeled discharge to the observed data. An efficiency of 0 (E = 0) indicates that the model predictions are as accurate as the mean of the observed data, whereas an efficiency less than zero (E < 0) occurs when the observed mean is a better predictor than the model or, in other words, when the residual variance (described by the numerator in the expression above), is larger than the data variance (described by the denominator).

from Nash, J. E. and J. V. Sutcliffe (1970), Journal of Hydrology

Page 40: ZHAO Linna

4040

2 Methodology2 Methodology•BMA methods

• BMABMA (( Bayesian Model AveragingBayesian Model Averaging )) is a is a statistical way of post-processing forecast statistical way of post-processing forecast ensembles to create predictive probability ensembles to create predictive probability density functions (PDFs) for weather quantities.density functions (PDFs) for weather quantities.– It represents the predictive PDF as a weighted average of PDFs

centered on the individual bias-corrected forecasts, where the weights are posterior probabilities of the models generating the forecasts and reflect the forecasts’ relative contributions to predictive skill over a training period.

– BMA model not only can provide the biggest forecast possibility, but also described the weather forecast uncertainty.

40

Page 41: ZHAO Linna

4141

2 Methodology2 Methodology

• In BMA for ensemble forecasting, each ensemble member forecast fk is associated with a conditional PDF k(y| fk ), which can be thought of as the PDF of the weather quantity y given fk , conditional on fk being the best forecast in the ensemble. and

PDF: 11

( | , ) ( | )K

K k k kk

p y f f h y f

…,1

1Kkk

41

•BMA methods

Page 42: ZHAO Linna

4242

2 Methodology2 Methodology• BMA methods • The model for hk(y| fk) is in two parts. The first part specifies PoP as a function of the

forecast fk. using logistic regression with a power transformation of the forecast as a predictor variable using the cube root of the forecast as a predictor variable.

• The second part of the model specifies the PDF of the amount of precipitation given

that it is not zero . Here fit gamma distributions to precipitation amounts.

11

( | , ) ( 0 | ) [ 0] ( 0 | ) ( | ) [ 0]K

K k k k k kk

p y f f P y f I y P y f g y f I y

…,

11( | ) exp( / )( )

k

kk k kk k

g y f y y

42

kf 1/3kf y 1/3y

1/30 1 2

( 0 | )0 | ) log( 0 | )

kk k k

k

P y fP y f a a f aP y f

Page 43: ZHAO Linna

4343

2 Methodology2 Methodology

43

Parameter estimation of BMA model

1/30 1 2

( 0 | )0 | ) log( 0 | )

kk k k

k

P y fP y f a a f aP y f

(0ka 1ka 2ka

0kb 1kb

0 1 2k k k k k kb b f b 2

0 1k k k kc c f

k k k 2 2

k k k

Page 44: ZHAO Linna

4444

2 Methodology2 Methodology

• Parameter estimation is based on data from a training period, here take 30 days of forecast and verifying observation data preceding initialization.

• The training period is a sliding window, and the parameters are re-estimated for each new initialization period.

44

• Parameter estimation of BMA model

Page 45: ZHAO Linna

4545

2 Methodology2 MethodologySelection of hydrological model and Verification VIC (Variable Infiltration

Capacity) model is a hydrological model based on spatial distribution grid (Wood, 1992; Liang Xu, etc., 2003)

Its output are evaporation, soil moisture content, runoff depth and so on.

The spatial resolution of VIC is 15km×15km

1220 grids15km×15km

Page 46: ZHAO Linna

4646

2 Methodology2 MethodologyVerification of precipitation probabilistic forecast 

• CRPS (continuous ranked probability score) :观测和预报的累积分布函数( CDF)的差别

• MAE(mean absolute error):是能反映预报误差的指标

The smaller the score is, the better the calibration result is

12 2

1 1

( ) ( )K K

m m m mm m

CRPS X X X X

1

1 ( )n

k kk

MAE x xn

Page 47: ZHAO Linna

4747

2 Methodology2 MethodologyVerification of precipitation probabilistic forecast 

• CRPS (continuous ranked probability score) :观测和预报的累积分布函数( CDF)的差别

• MAE(mean absolute error):是能反映预报误差的指标

The smaller the score is, the better the calibration result is

12 2

1 1

( ) ( )K K

m m m mm m

CRPS X X X X

1

1 ( )n

k kk

MAE x xn

Page 48: ZHAO Linna

4848

2 Methodology2 MethodologySelection of hydrological model and Verification The 25th, 75 percentile of hydrological probabilistic

forecast are verified by three index, namely certainty coefficient relative error of flood peak difference of peak time

Certainty coefficient