Riesgo Hidrológico de Eventos Lecture Outline Extremos en ...

1

Riesgo Hidrológico de EventosExtremos en Condiciones

No-Estacionarias

Jose D. SalasColorado State University, USA

J. ObeysekeraSouth Florida Water Management District, USA

Laboratorio Nacional de Hidráulica, Universidad Nacional de Ingeniería, Lima Perú

Lecture Outline• Introduction. Causes of change in extreme events• Review if basic concepts of probability and statistics• Models of extreme events for stationary and non-

stationary conditions• Parameter estimation and model selection for

stationary conditions• Other model alternatives for extreme events• Introduction to software R for modeling extreme

events

2

Lecture Outline (cont.)• Introduction to software R for modeling extreme

events• Parameter estimation and model selection for non-

stationary conditions• Return period and risk for stationary and non-

stationary conditions• Examples of analysis of extreme events based on

stationary and non-stationary conditions• Hands on experience in using software R for

analyzing extreme events under stationary and non-stationary conditions

3

Resources

4

2


No-Estacionarias




Introduction: Causes of changes in extreme events

• Course objectives

• What is expected to be learned from the course

• Summary of the course content

• Format

• Materials and references

• Others

Introduction

Outline• Extreme hydrologic events

• Types of extreme value data

• Types of changes (non‐stationarity) in hydrologic data

• Causes of non‐stationarity

Introduction Extreme Hydrologic Events • Floods• Low flows and droughts • Max. and min. precipitation• Max. and min. temperature (heat waves)• Maximum wind• Max. and min. storages (e.g. reservoir, groundwater

levels, soil moisture, snow pack, glaciers)• Max. sea levels• Max. erosion rates, sediment transport, and sediment

deposition• Maximum water quality pollutant concentrations• Many others

3

Extreme Hydrologic Events

Drastic damages and changes produced by the extraordinary flood originated from Hurricane Mitch of October 1998. (a) a river and a bridge in Guatemala, Source: Time Magazine, Nov. 1998, and (b) the Choluteca River in Tegucigalpa, Honduras, Source: National Geographic Vol. 196(5), p. 111, Nov. 1999.

Extreme Floods

(a) (b)


Tous Dam at Rio Jucar, Spain breached and caused a flood of about 15,200 m3/s along River Jucar in October of 1982. Source: Dr. J. Marco‐Segura, U. Valencia.

Extreme Floods

Fort Collins flood, July 1997


Extreme Floods


Extreme Floods

Washed out road over Milford Dam near Milford, Kansas in July 1993(source: cover of ASCE Civil Engineering Magazine, Vol. 64(1), January.)

Mississippi flood of 1993, USA(source unknown)

4


Extreme Floods

Damage of extreme flood occurred in Venezuela, Dec. 15, 1999(source: Central University in Caracas, Prof. Marco P. Rivero)

Venezuela Venezuela


Extreme Floods

Queensland, Australia, 2011

Pakistan, 2010 India, 2010 China


Extreme Floods

ʺThe frequency (of extreme weather situations) is way up,ʺ Andrew Cuomo, Governor of New York, 10/31/2012


Extreme Droughts

Dust bowl, Midwest US 1930’s(source unknown)

Hayman fire near Cheesman Reservoir(source: Denver Water)

5


Extreme Droughts

A photo of a wildfire in the west in 2000 due to drought conditions, Aug.

6, 2000 in the Bitterroot Valley National Forest, MT.

Drought in the Amazon RiverSource: G. Chamorro, SENAMHI


Extreme Droughts

Cheesman Reservoir, ColoradoDrought 2002

(source: Denver Water)

Dillon Reservoir, ColoradoDrought 2002

(source: Denver Water)

19

Types of Extreme Value Data : Block Maxima

Blocks1 2 3 4 5

Block length could be:One YearSeason (wet and dry)

Example: Daily Precipitation

Types of Extreme Value Data : Peaks Over Threshold (POT)

• In hydrology, this is known as Partial Duration Series

20

Threshold


6

Definitions of Low Flows

Time series of daily flows, 1951‐2000

Definition of Low Flows (1)

Unit time period Tu

dd

d

v1 vj vm


The d‐day low flow for a given unit time period Tu is:Q’ = min(v1, v2, …, vm)where m = number of d‐day flows in a unit time period, e.g. m=356 for d=10 and Tu=1 year

For N unit time periods (e.g. for N years of record) we will get the sequenceQ’1 , Q’2 , . . . , Q’NThen, frequency analysis must be performed to get theT‐year d‐day low flow. Note that for low flowsT = 1/q (q=non‐exceedance probability)


Low flow for a variable duration

In this case the duration of low flows is a random variableA threshold flow Qo is selected and the following low flow variables can be defined (refer to the following figure)d* = max(d1, d2, . . . , dm) max. duration of low flows

v* = max(v1, v2, . . . , vm) max. deficit

I* = max(I1, I2, . . . , Im) max. intensity of low flowsOther quantities can be defined such as averages or maximums, etc.Series: d*1 , d*2 , . . . , d*N to do frequency analysis.

7


d1 d2 d3

v1 v2 v3

Example of low flow duration frequency analysis

The daily flows of Edisco River, SC for the period 1951‐2000

Example of low flow duration frequency analysis Extreme multiyear droughts

Poudre River annual flows at the Mouth of the Canyon (1884‐2002)

8

Drought definition and properties

Critical droughts can be extracted from the time series

Types of Changes (non‐stationarity)in Hydrologic Data

• Trends (increasing and decreasing)• Shifts (upward or downward)• Mixed

• Seasonality (periodicity, cycles)

• Pseudo‐ cycles

• Clustering

• Others

Trends (increasing floods)

(a) Abjerjona Basin (b) Little Sugar Creek Basin.

μt = μ0 ‐ ‐ ‐ μt = μ0 + a t

Trend in the mean and in the standard deviation

Trends (increasing and decreasing)in sea levels

(a) Key West, Florida (b) Adak, Alaska

Example: c = ‐ 1.34 mm/yr, b = 2.71 x 10‐5, e = 1079 mm,σ = 111.3 mm, and ε = ‐ 0.26 for the Adak gage.

9

Shifts in Hydrologic Data

Niger River annual flows. Example of upwards and downwards shifts St. Johns River (Florida) Annual FloodsShift related to the AMO

Shifts in Flood Data

Annual Cycle in Hydrologic Data

Monthly streamflow data

Daily Cycle in Hydrologic Data

Hourly rainfall for the month of July for Denver Airport

10

Causes of Non‐stationarity• Human intervention in river basins (watersheds)

Construction of hydraulic structuresDamsDiversionsTunnelsGroundwater pumpingOthers

Land use changesUrbanizationIrrigation and farming activitiesDeforestationBuilding transportation systemsOthers

Industrial development and operationsMining activitiesOthers

Causes of Non‐stationarity

• Human intervention in river basins (watersheds)

• Natural climate variabilityEffects of low frequency (large scale) phenomena such as ENSO (years) PDO (decades), and AMO (multi‐decades)

• Climate change due to increase of green‐house gasesincrease in air temperatureincrease in moisture of the airhydrologic effects uncertain and still debatable


Effect of construction of hydraulic structures along de Han River in Korea (source Dr. D.R. Lee)

Human intervention: Construction of hydraulic structures


Effects of construction of hydraulic structures along de Han River in Korea (source Dr. D.R. Lee)

Human intervention: Construction of hydraulic structures

11


(a) Aberjona River, WinchesterMassachusetts

(b) Little Sugar Creek, Charlotte North Carolina

Human intervention: effects of urbanization


Natural climate variability: Effects of low frequency

33 years

40 years

26 years 32 yearswarm

coldcold


Effect of natural climate variability, PDO(source: Akintug and Rasmussen, 2005)

Effect of natural climate variability, AMOsource: Dr. J. Obeysekera, SFWMD

Natural climate variability: Effects of low frequency

Final Remarks

12


No-Estacionarias




Review of Basic Concepts of Probability, Statistics, and Time Series

Outline• Random events• Probability of random events• Random variable• Probability laws• Independent and dependent random variables• Population moments• Moments of linear functions of random variables• Moments of non‐linear functions of random variables• Central limit theorem

Review of Basic Concepts of Probability

Review of Basic Concepts of Probability Review of Basic Concepts of Probability

13



14

Review of Basic Concepts of Probability Review of Basic Concepts of Statistics

Outline• Random sample• Sample moments• Moments of sample moments• Estimation (estimates and estimators) • Methods of estimation • Properties of estimators• Confidence limits on population parameters• Confidence limits on population quantiles

Review of Basic Concepts of Statistics Review of Basic Concepts of Statistics

15



16


Review of Basic Concepts of Time Series Review of Basic Concepts of Time Series

17

Final Remarks Riesgo Hidrólogico de EventosExtremos en Condiciones

No-Estacionarias




Models of Extreme Events for Stationary andNon‐Stationary Conditions

Outline

• Typical models for extreme events (stationary conditions)

• General extreme value (GEV) model (stationary conditions)

• General extreme value (GEV) model (non‐stationary conditions)

Models of Extreme Events for Stationary and Non‐Stationary Conditions

• Typical models for extreme events (stationary conditions)

Stationary data

18

Typical models for extreme events (stationary conditions)


Lognormal


Gamma (Pearson)


Log‐Pearson Type III

19


General extreme value

If the model is called Logistic


Generalized logistic

‐ Exponential if

‐ Uniform if


Generalized Pareto

Vilfredo Pareto1848‐1923


Kappa

Dr. Paul MielkeProfessor Emeritus, CSU

20


Kappa


Mixtures and products

Example: Mixture of three distrib. (Waylen & Caviedes, 1986)



21


General Extreme Value (GEV) Model(stationary conditions)

• Assume block maxima coming from a series of independent and identically distributed (i.i.d.) observations:

• Distribution of Mn (assuming iids):

• Then for large n, and some an and bn:


G(z) belongs to one of these:


Emil Gumbel1891‐1966

ʺIt seems that the rivers know the theory. It only remains to convince the engineers of the validity of this analysis.ʺ

Maurice Frechet1878‐1973

Waloddi Weibull1887‐1979

22




Type III(ξ<0)Type I(ξ=0) Type II(ξ>0)


23

Non‐stationary data

Annual maximum floods

Built‐out?

Non‐stationary data

Mean and maximum sea levels for some sites in the USA

General Extreme Value (GEV) Model(non‐stationary conditions)

General Extreme Value (GEV) Model(non‐stationary conditions)

24

Final Remarks Riesgo Hidrólogico de EventosExtremos en Condiciones

No-Estacionarias




Parameter Estimation and Model Selection forStationary GEV Models

Outline• Parameter estimation and quantile estimation for GEV

modelsMOM, PWM, ML, BayesianProfile Likelihood

• Uncertainty of parameters and quantilesApproximate standard errors (delta method)Better approximation (profile likelihood)

• Model selectionLikelihood ratio (deviance statistic)AICDiagnostic plots

Parameter Estimation & Model Selectionfor Stationary GEV Models

Parameter estimation and quantileestimation for GEV models

25





26





Example: Umpqua flood data

27


PWM estimates

Likelihood ratio test

ML estimates and standard errors

Calculations using R software




28





Standard errors of parameters

29


Variance‐covariance matrix Estimation of quantiles and confidence limits




30



Estimation of quantiles for T=5, 10, 25, 50, 100, 250, 500 years and theirconfidence limits



31



*Keep θi constant and maximizeℓ(θ) with respect to all otherparameters, θ‐I

*Repeat for different θi

θi

ℓp(θi)

Profile likelihood for θi


Chi‐square distribution

θi

Confidence intervalfor θi

3.8415/2for k=1

Profile likelihood confidence limits for θi


32


Confidence Interval

Confidence Interval

Shape Parameter 25‐yr quantile

Inference using profile likelihood


218 390

267.6

‐0.11 0.24

0.0514

Delta method gives:(‐0.24,0.73)

Delta method givesZq:(193.7 341.57)

Results based on profile likelihood



33


Model selection based on Deviance statistic and AIC




Results for the UmpquaFlood data based on GEV

34

Final Remarks


No-EstacionariasOther model alternatives for extreme events

Credit:Jana Sillmann


J. Obeysekera (‘Obey’)South Florida Water Management District, USA


Types of Extreme Value Data : Block Maxima (review)

136

Blocks1 2 3 4 5

Block length could be:One YearSeason (wet and dry)


35

Models based on r-largest statistics Daily Rainfall (Fort

Lauderdale, Florida) 5 largest values of

Rainfall every year

137

Models based on r-th largest statistics

, … . ,

!

1/

, … ,

138

Modeling the r-th Largest Order values

, … , 1,2, . . , Likelihood

, , 1/

x 1

Example: 10 largest sea-levels in VeniceYear r1 r2 r3 r4 r5 r6 r7 r8 r9 r101931 103 99 98 96 94 89 86 85 84 791932 78 78 74 73 73 72 71 70 70 691933 121 113 106 105 102 89 89 88 86 85

139

Example

r l

1 -222.7 111.1(2.6) 17.2 (1.8) -0.077 (0.074)

5 -732.0 118.6(1.6) 13.7(0.8) -0.088 (0.033)

7 916.5 119.1(1.47) 13.25(0.7) -0.09(0.029)

10 1139.1 120.5 (1.36) 12.8 (0.55) -0.113 (0.019)

r-largest order statistics for Venice sea levels

140

(*ignore trend for the moment)

36

141

Global Warming ProtestersPeaks Over Threshold (POT) Models

Motivation: Can weuse all “extreme”values above a giventhreshold?

142

Threshold Example: Daily Precipitation

Peaks Over Threshold - Theory

Stochastic model:

It can be shown that (Coles,2001):

then

143

y>0

Generalized Pareto Distribution (GPD)

For large enough u and Y=X-u,

PDF

Case: 0 Threshold Selection is important

144

0

37

Can we get the equivalent GEV parameters from GPD?

Need , , from , is the same for both GEV and GPD Need to estimate , Need some new formulation to estimate the

third parameter

145

Poisson-GPD Model

The number of N exceedances of the threshold level, u, has a Poisson distribution with mean

Assuming N ≥ 1, the excess values Y1 ,..,Ynare IIDs from GPD

For z > u, the probability that the annual maximum of Y is,

146

Poisson-GPD model (cont.)

147

This will be identical to the form of the GEV if,

Knowing (=# above u/total #) we can now compute the equivalent GEV parameters

A more elegant approach: Point Process Model (PP)

Interpretation of extreme value behavior that unifies all models (GEV, GPD, Poisson-GPD)

Leads to a “likelihood” (to be discussed) that enables a more natural formulation of non-stationarity in threshold excesses than from GPD alone

148

38

Poisson Process (PP) Model (preview) Domain, D=[0,T] x u,∞ X is viewed as two-

dimensional, non-homogenous Poisson process (t, x u

149

0 T

u

A

t1 t2

x

The rate is, The number of points in A, N(A) is a Poisson

Process with mean:, also

Poisson Process (PP) Model (Cont.)

Advantages: Direct estimation of GEV parameters is not a function of u (unlike in GPD) Easier to implement in the “non-stationary”

setting (to be discussed later)

150

Summary of Models

Block Maxima GEV Gumbel Others

Peaks Over Threshold (POT) GPD Poisson-GPD PP

151

Generalize Pareto Distribution (GPD) –Maximum Likelihood Estimation

Recall:

Two parameters to estimate:

152

39

MLE for GPD - Spreadsheet

153

=-$K$5*LN($Q$5)-(1+1/$Q$6)*SUM(M7:M364)

File:Fortgpd.xlsx

Poisson Process (PP) Model (review) Domain, D=[0,T] x u,∞ X is viewed as two-

dimensional, non-homogenous Poisson process (t, x u

154

0 T

u

A

t1 t2

x

The rate is, The number of points in A, N(A) is a Poisson

Process with mean:, also

Poisson Process (PP) Model –Inference

Advantages: Direct estimation of GEV parameters is not a function of u (unlike in GPD) Easier to implement in the “non-stationary”

setting (to be discussed later) Fort Collins Precipitation data again

155

Point Process (PP) Model - MLE

Likelihood function (Coles 2001, chapter 7)

Ii is a small interval around xi , I is the full range

GEV parameters directly:

156

40

MLE for PP - Spreadsheet

157

=-$O$6*(1+$O$10*($O$7-$O$8)/$O$9)^(-1/$O$10)-$K$8*LN($O$9)-(1+1/$O$10)*SUM(L10:L367)

File:Fortpp.xlsx

Dependent Sequences – Declustering

158

Average Daily Temperature

U

U

Clusters for different r

r=1,2r=3

Declustering Procedure

Use an empirical rule to define clusters of exceedences

Identify maximum excess within each cluster Assume cluster maxima are independent,

and that the conditional excess is distributed as a generalized Pareto

Fit the GPD to the cluster maxima

159

Change to Return Level Formula Rate at which clusters must be taken into

account. m observation return level:

is the extremal index Let nu = number of exceedances above u nc = number of clusters above u

160

41

Declustering of Fort Collins Rainfall> load("Fort.RData")> attach(a)> dcRain1<-dclust(Rain,u=0.4,r=1)> names(dcRain1)[1] "xdat.dc" "ncluster" "clust" >gpd<-gpd.fit(Rain,0.4,npy=214)>gdc=gpd.fit(dcRain1$xdat.dc,0.4,npy=214)gpd$nexc=358 dcRain1$ncluster=288gpd$mle: 0.394(0.031) 0.164(0.058)gdc$mle: 0.446(0.037) 0.130(0.060)Quantile(100-yr)gpd: 5.05 (3.84,7.89)gdc: 4.79 (3.71,7.16)

161

Non-stationary GEV models Parameters are function of time (or any

other variable-”covariates”)

Examples:

162

Non-stationary GEV: Examples (cont.)

163

Change Point Models

Math Classic..

164

42

Final Remarks

165


No-EstacionariasIntroduction to modules of the software R for

modeling extreme events





What is R

Most cost effective statistical package (why?) R is a system for statistical computation and

graphics. Also a programming language, high level

graphics, interfaces to other languages The R language is a dialect of S which was

designed in the 1980s

167

What is R (Cont.) an effective data handling and storage facility, a suite of operators for calculations on arrays, in

particular matrices, a large, coherent, integrated collection of intermediate

tools for data analysis, graphical facilities for data analysis and display either

on-screen or on hardcopy, and a well-developed, simple and effective programming

language which includes conditionals, loops, user-defined recursive functions and input and output facilities

168

43

Installation of R http://www.r-project.org/ Select Comprehensive R Archive Network (CRAN) mirror

site Download R for Windows (or for the particular operating

system) Install base (install R for the first time) Provides basic functions which let R function as a

language Base package and contributed packages For a complete list of functions, use library(help = "base")

169

Help for R

http://www.r-project.org/ http://www.rseek.org/ http://archive.org/details/TheRBook

170

Downloading R

171

Downloading R

172

44

Downloading R

173

Downloading R

174

Doing more.. Need to get “contributed packages” http://cran.r-

project.org/web/packages/available_packages_by_name.html

> help("install.packages") Eg. Installing package “extRemes”

> install.packages("extRemes") Will ask for CRAN mirror site

>getwd() to find current working directory >setwd(“path”) to set the working directory - or use

R.gui()

175

Data structures 1/6 Vector A list of numbers, such as (1,2,3,4,5) R: a<-c(1,2,3,4,5) or a=c(1,2,3,4,5) Command c creates a vector that is assigned to object a

Factor a = c(1,2,2,4,4,5) A list of levels, either numeric or string R: b<-as.factor(a) Vector a is converted into a factor b:1 2 2 4 4 5

with Levels: 1 2 4 5

176

45

Data structures 2/6 Data frame A table where columns can contain numeric and

string values R: d<-data.frame(a, b)

Matrix All columns must contain either numeric or string

values, but these can not be combined R: e<-as.matrix(d) Data frame d is converted into a matrix e

R: f<-as.data.frame(e) Matrix e is converted into a dataframe f

177

Data structures 3/6 List Contains a list of objects of possibly different types. R: g<-as.list(d) Converts a data frame d into a list g

>g=gev.fit(x) g is now a list object >names(g) check names of the objects in g "trans" "model" "link" "conv" "nllh"

"data" "mle" "cov" "se" "vals"

g$mle gives the values of mle object:301.8081211 169.9874853 0.3544377

178

Data structures 4/6 Some command need to get, for example, a matrix, and do

not accept a data frame. Data frame would give an error message.

To check the object type: R: class(d)

To check what fields there are in the object: R: d R: str(d)

To check the size of the table/matrix: R: dim(d)

To check the length of a factor of vector: R: length(a)

179

Data structures 5/6 Some data frame related commands: R: names(d) Reports column names

R: row.names(d) Reports row names

These can also be used for giving the names for the data frame. For example: R: row.names(d)<-c("a","b","c","d","e") Letters from a to e are used as the row names for

data frame d Note the quotes around the string values!

R: row.names(d)

180

46

Data structures 5/6 Naming objects: Never use command names as object names! If your unsure whether something is a command name,

type to the comman line first. If it gives an error message, you’re safe to use it.

Object names can’t start with a number Never use special characters, such as å, ä, or ö in object

names. Underscore (_) is not usable, use dot (.) instead: Not acceptable: good_data Better way: good.data

Object names are case sensitive, just like commands

181

Reading data 1/2

Command for reading in text files is:read.table(”suomi.txt”, header=T, sep=”\t”)

This examples has one command with three arguments: file name (in quotes), header that tells whether columns have titles, and sep that tells that the file is tab-delimited.

182

Reading data 2/2 It is customary to save the data in an object in

R. dat<-read.table(”suomi.txt”, header=T, sep=”\t”)

Here, the data read from file suomi.txt is saved in an object dat in R memory.

The name of the object is on the left and what is assigned to the object is on the right.

Command read.table( ) creates a data frame.

183

Using data frames Individual columns in the data frame can be accessed using

one of the following ways: Use its name: dat$year

dat is the data frame, and year is the header of one of its columns. Dollar sign ($) is an opertaor that accesses that column.

Split the data frame into variables, and use the names directly: attach(dat)

Use subscripts

184

47

Subscripts 1/2 Subscripts are given inside square brackets

after the object’s name: dat[,1] Gets the first column from the object dat

dat[,1] Gets the first row from the object dat

dat[1,1] Gets the first row and it’s first column from the

object dat Note that dat is now an object, not a

command!

185

Subscripts 2/2 Subscripts can be used for, e.g., extracting a subset of the data:

dat[which(dat$year>1900),]

Now, this takes a bit of pondering to work out… First we have the object dat, and we are accessing a part of it,

because it’s name is followed by the square brackets Then we have one command (which) that makes an

evaluation whether the column year in the object dat has a value higher than 1900.

Last the subscript ends with a comma, that tells us that we are accessing rows.

So this command takes all the rows that have a year higher 1900 from the object dat that is a data frame.

186

Writing tables To write a table: write.table(dat, ”dat.txt”, sep=”\t”)

Here an object dat is written to a file called dat.txt. This file should be tab-delimited (argument sep).

Write.csv(dat, ”dat.csv”, row.names=FALSE)

This file can be opened in Excel

187

Commonly used R commands in this course

Command What it does>R start R>?plot Get help on 'plot' command>ls() Check the object space>class(a) Check the data type of object a>head(a) What the data object looks like (few rows printed)>names(a) names of object a>a=c(1,2,3,4,5) Create a vector object a with values 1:5>c=cbind(a,b) Combine vector objects a and b into the object c>plot(x,y) plot y versus x

>is.na(a) A logical vector showing which values in a are missing>sum(is.na(a)) Number of missing values in object a>b=a[‐20,] Remove 20th row of a and place it in b

>a=read.csv("file.csv",header=T)Read data in "file.csv" which has a header and place in to object a

>write.csv(a,"file.csv") Write the object a into csv file called "file.csv">a=seq(5,100,5) Create a sequence of numbers, 5 to 100 with increments of 5

188

48

Creating a fancy plotfout= “myplot.png”#set the device for plottingpng(fout,pointsize=12,units="in",width=8,height=6,res=350) par(mar=c(5.1,6.1,3.1,3.1))par(font=2,font.lab=2,font.axis=2,cex=1.5,cex.axis=1.1,cex.lab=1.1)#now plotplot(yrs,x,pch=19,xlab="Year",ylab=“Discharge”, col="red")title(“myplot”)#set the device offdev.off()

189

Reading and Manipulating Data>dat=read.csv(“sample.csv”,header=TRUE)#notice missing values denoted as “NA”>class(dat)>head(dat)>attach(dat)>amean=tapply(Rain,Year,mean,na.rm=T)>momean = tapply(Rain,Month,sum,na.rm=T)>sum(is.na(dat))#setting missing values to zero>dat[is.na(dat[,3]),3]=0>plot(Rain)>boxplot(Rain~Month)

190

Compiling & Running a functionmyplot <- function(Year,Rain,outflag=0) {

if(outflag != 0 ) png("rainplt.png")

amean = tapply(Rain,Year,sum,na.rm=T)

yrs = as.numeric(names(amean))

plot(yrs,amean,pch=19,ylab="Rain",col="red",cex=1.5)if(outflag !=0 ) dev.off()return(list(yrs=yrs,amean=amean))}

>v = myplot(Year,Rain)

191

> v$yrs[1] 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905

> v$amean1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905

52.35 62.47 73.28 46.77 40.95 55.96 34.54 37.02 25.51 42.34 52.11

Quitting R Use command q() or menu choise File->Exit. R asks whether to save workspace image. If you

do, all the object currently in R memory are written to a file .Rdata, and all command will be written a file .Rhistory.

These can be loaded later, and you can continue your work from where you left it.

Loading can be done after starting R using the manu choises File->Load Workspace and File-> Load History.

192

49

193

A powerful IDE for R (www.rstudio.com) Extremes Package Demo

library(extRemes) extremes.gui()

194

Task ismev/extRemes evd evir fExtremes POT VGAMData generation

gev gen.gev rgev rgev gevsim, gumbelsim rgevgpd gen.gpd rgpd rgpd gpdSim rgpd rgpdGEV fitting

MLE gev.fit fgev gev, gumbel gevFit, gumbelFit gev

Profile Likelihoodgev.prof,gev.profxi,gev.parameterCI profile.evd

Diagnostics gpd.giag

GPD fitting

Threshold mrl.plot mrlplotmeplot,

findthreshmrplot, findThreshold,

mxfPlot mrlplot,lmomplotgpd.fitrange tcplot tcplot

MLE gpd.fit fpot gpd gpdFit fitgpd gpd

Profile Likelihoodgpd.prof, gpd.profxi, gpd.parameterCI profile.evd gevrlevelPlot

gpd.pfrl,gpd.pfscale, gpd.pfshape

Fisherbasedgpd.firl,gpd.fiscale,

gpd.fishape

bootstrappingboot.matrix, boot.sequence

PP fittingMLE pp.fit fpot pot pointProcess fitpp

decluster declust, decluster deClusterCovariatesGEV gev.fit fgevGPD gpd.fit ?PP pp.fit 195

Final Remarks

196

50


No-EstacionariasParameter estimation and model selection for

non‐stationary conditions





Climate change

198

Extreme Data Types

Stationary Stochastic properties do not change with time

Non-Stationary Stochastic properties vary with time Trends (mean, frequency and intensity) Cycles (diurnal, annual) Because of co-variation with other variables (e.g.

Flood = f(El Nino phenomenon)

199

Non-stationary GEV models Parameters are function of time (or any

other variable-”covariates”)

Examples:

200

51

Nonstationary GEV Models (cont.)

201

Change Point Model “Seasons” model

Parameter Estimation (Non-stationary case)

Non-stationary GEV model: Zt = GEV( (t), (t), (t)) Shape parameter is difficult to estimate and it is

unrealistic to model it as a smooth a function of time

Log-likelihood:

202

t , t , t T

Parameter Estimation (Cont.)

Non-stationary Gumbel

Two parameters, but both functions of time

203

Parameter Estimation (Cont.) Example 1

Parameters, = ( 0 , 1 , , )

Example 2

Parameters, = ( 0 , 1 , 2 , , ) Approximate Standard errors derived using the

same techniques that we discussed for stationary case

204

52

205

Math Classic..Example: Aberjona River, Winchester, Massachusettes Typical of

basins where land use changes have caused increasing floods

Nonstationarymean, variability?

206

R and Excel Solutions a=read.csv("Aberjona_1102

500.csv",header=T) cov=yrs-mean(yrs) cov=as.matrix(cov,ncol=1) gev.fit(x,ydat=cov,mul=1)

Output:$nllh 410.931

$mle 319.3587323 2.8814896 163.3669736 0.3040809$se 25.3638274 1.0190117 21.1793573 0.1337115

207

Confidence Limits for parameters: Approximate Normality of MLEs

For large n:

208

ExpectedInformationMatrix

53

209

Response variable: Discharge [1] "cov.selected = Cov"

Convergence successfull![1] "Convergence successfull!"[1] "Maximum Likelihood Estimates:“ MLE Stand. Err.MU: (identity) 319.35873 25.36373Cov: (identity) 2.88149 1.01901SIGMA: (identity) 163.36697 21.17922Xi: (identity) 0.30408 0.13371

[1] "Negative log-likelihood: 410.931027606394"Parameter covariance:

[,1] [,2] [,3] [,4][1,] 643.318930 7.19927261 358.945815 -1.33843641[2,] 7.199273 1.03838421 4.686030 -0.05198826[3,] 358.945815 4.68603046 448.559343 -0.37964699[4,] -1.338436 -0.05198826 -0.379647 0.01787876

extRemes packageConfidence Limits using Bootstrapping Recall that

Z ~ GEV( t , t ,Then e = [Z- t / tis GEV~(0,1, The basic approach

is to bootstrap the residuals, e and fit multiple models

Fit Z ~ GEV( (t), (t), ) Compute residuals, e For k = 1, Nsamples

Resample residuals with replacement = enew

Recreate z = Znew using ( (t),(t),

Fit Znew ~ GEV( (t), (t), ) Compute ZT for the desired T

Compute 5th and 95th interval using Nsamples of ZT

210

Model Selection (Likelihood Ratio Test) Let a model, M1=f(θ(1), θ(2)) and M0=f(θ(1)=0, θ(2)). M0 is a subset of M1. The

question is, Is model M1 any better than M0? Deviance Statistic:

Reject M0 in favor of M1 if

where cα is the (1-α) quantile of the χ2k

distribution dimension of θ(1)

Alternatively:211

α

cα

Model Selection based on AIC & BIC

Akaike’s Information Criteria (AIC) AIC(k) = -2llh (k)+2 k, k=number of parameters Select the model which has the minimum AIC

Bayesian Information Criterion (BIC) BIC(k) = -2llh(k) + k ln T k = number of parameter T = sample size Select the model which has the minimum BIC

212

54

Model Diagnostics (Non-stationary) case

~ , , Standardized variable

11

Note: follows a Gumbel distribution

213

Model Diagnostics (Cont.)

Propbability plot: , exp exp ̃ 1,2, … ,

Quantile plot: ̃ , log log ; 1,2, … ,

214

Modeling Non-stationarity -Summary

Fit various modeling using MLE Select an appropriate model using AIC, BIC

and Likelihood Ratio Test as criteria Compute Return Level, and Risk for a given

Return Period Compute confidence intervals for

parameters and return level

215

Example: Aberjona River, Winchester, Massachusettes Typical of

basins where land use changes have caused increasing floods

Nonstationarymean, variability?

216

55

Aberjona River – First Stationary Case

>dat=read.csv("Aberjona_1102500.csv",header=TRUE)

>head(dat) (yrs x)>attach(dat)>gum<- gum.fit(x) #fit Gumbel$gum$mle: {337.4(28.1) 209.3(23.0)}$gum$nllh= 419.7192> gum.diag(gum)>AICgum=2*(gum$nllh+2) =843.4384

>gev<- gev.fit(x) #fit GEVgev$mle {301.8(25.5) 170.0(22.3)

0.354(0.13)} gev$nllh=414.9693>gev.diag(gev)

217

Modeling Non-staionarity using covariates

g = gev.fit(x, ydat = cov, mul=c(1,..), sigl=c(1,..), shl=c(1,..), siglink = exp, muinit=c(…),…)

Covariate matrix, covYear SOI AMO1945 …. ….1946 …. …. R-Demo

218

Aberjona River - Model Checking & Non-stationary Modeling>D= -2*(gev$nllh-gum$nllh)> chi=qchisq(0.95,1)> p=pchisq(D,1,lower.tail=FALSE)AICgev=2*(gev$nllh+3)

D = 9.499 ; chi =3.84; p=0.002; AICgev= 835.9386 D > Chi, and AICgev < AICgum GEV is chosen over Gumbel#Probability Weighted Moments> lmr<-lmom.ub(x)> pwmgev<-pargev(lmr)

#nonstationarity in location parameter>cov <- as.matrix(yrs - mean(yrs),ncol=1)>gevmu <- gev.fit(x,ydat=cov,mul=1)gevmu$nllh=410.931gevmu$mle {319.4(25.4) 2.88(1.02)

163.4(21.2) 0.304 (0.134)}> D= -2*(gevmu$nllh-gev$nllh)> chi=qchisq(0.95,1)> p=pchisq(D,1,lower.tail=FALSE)> AICgevmu=2*(gevmu$nllh+4)D=8.076 chi=3.841459 p=0.004

AICgevmu=829.8621> Select GEV-MU(t) over GEV!

219

Aberjona River – Model Comparison

IndexModel Name Parameters nllh Comparison D chi p AIC

1 gum , 419.72 843.42 gummu (t), 414.01 2 vs. 1 11.42 3.84 0.001 834.03 gumsc , (t) 412.21 3 vs. 2 15.01 3.84 0.000 830.44 gummusc (t), (t) 406.23 4 vs. 2 15.56 3.84 0.000 820.55 gev , , 414.97 5 vs. 1 9.50 3.84 0.002 835.96 gevmu (t), , 410.93 6 vs. 5 8.08 3.84 0.004 829.97 gevsc , (t), 411.99 7 vs. 6 5.96 3.84 0.015 832.08 gevmusc (t), (t), 405.79 8 vs. 7 10.29 3.84 0.001 821.6

220

56

Non-stationary Model –GEV{ t t ,

221

Confidence Limits using Bootstrapping Recall that

Z ~ GEV( t , t ,Then e = [Z- t / tis GEV~(0,1, The basic approach

is to bootstrap the residuals, e and fit multiple models

Fit Z ~ GEV( (t), (t), ) Compute residuals, e For k = 1, Nsamples

Resample residuals with replacement = enew

Recreate z = Znew using ( (t),(t),

Fit Znew ~ GEV( (t), (t), ) Compute ZT for the desired T

Compute 5th and 95th interval using Nsamples of ZT

222

Snippets of code#initial guess R0library(BB)p0 = 1/T0yp0 = -log(1-p0)R0 = mu0 - (sig/xi)*(1-yp0^(-xi))#compute Return level for the desired value of T – finding root using dfsaned = dfsane(par=R0,fn=myfun,T=T,Nmax=Nmax,mut=mut,sig=sig,xi=xi,quiet=TRUE,control=list(trace=FALSE))Rm = d$parconv <- d$convergence

myfun <- function(x,T,Nmax,mut,sig,xi) {pt = 1- exp(-((1+(xi/sig)*(x-mut))^(-1/xi)))pt[is.nan(pt)]<-1ptc = cumprod(1-pt)px =matrix(1,Nmax,1)for(i in 1:Nmax) {if(i == 1) px[i] = pt[1] else px[i] = ptc[i-1]*pt[i]}t=1:Nmaxex = sum(t*px)y <- T - ex#print(ex)y}

223

Rest of the code (for boostrapping)for(k in 1:Nsample) {enew <- sample(e,replace=TRUE)znew <- enew * g$vals[,2]+g$vals[,1]gnew<-gev.fit(znew,ydat=cov,mul=1)muslope = gnew$mle[2]mu0=gnew$vals[yrs==lastyr,1]+(consyr-lastyr)*muslopesig = gnew$mle[3]xi = gnew$mle[4]mut = mu0 + (t-1)*muslopeR0new = mu0 - (sig/xi)*(1-yp0^(-xi))

.. Continued to the right

d = dfsane(par=R0new,fn=myfun,T=T0,Nmax=Nmax,mut=mut,sig=sig,xi=xi,quiet=TRUE,control=list(trace=FALSE))Rmnew = d$parcvals[k] <- d$convergenceRms[k]<-Rmnew….}

#5th and 95th confidence limits?r <- quantile(Rms,probs=c(0.05,0.95))

224

57

Results - Parameters

225

Results – Return Level (Design Quantile)

226

Trend with a change point Little Sugar Creek at Archdale

Drive in Charlotte, North Carolina (Gumbel Distribtion)

>cov = yrs>cov[yrs <= 1945] <- 1945cov:1945 1945 1945 1945 1945 1945 1945 1945 1945

1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975………………

>cov=matrix(cov-mean(cov),ncol=1)>gum=gum.fit(x)>gumu=gum.fit(x,ydat=cov,mul=1)

227

Two Change points Mercer Creek,

Washington State>yr1 = 1970>yr2 = 1986>cov=yrs>cov[yrs <= yr1] <- yr1>cov[yrs >= yr2] <- yr2cov: 1970 1970 1970 1970 1970 1970

1970 1970 1970 1970 1970 1971 1972 1973 1975 1977 1978 1980 1981 1982 1983 1985 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986

>gev = gev.fit(x)>gevmu=gev.fit(x,ydat,mul=1)

228

58

Sea Level Rise – Modeling extremesKey West tide gauge>a=read.csv("KeyWest.csv",header=TRUE)> head(a)Year Mean Max1913 1495.4 20421914 1481.2 2070>attach(a)>gevmu=gev.fit(Max,mul=1,ydat=as.matrix(Year-mean(Year)))>g=lm(Mean~Year) #regression

Non-linear change possible in the future Relation shop mean and max! Can we use that property? – later

229

Offset=552 mm

Sea Level Rise Projections (in US)

230

Modeling with Covariatesdata(fremantle)Year SeaLevel SOI1897 1.58 -0.671898 1.71 0.57…attach(fremantle)ydat=cbind(Year-mean(Year),SOI-mean(SOI))gev=gev.fit(SeaLevel)gevmu=gev.fit(SeaLevel,ydat=ydat,mul=1)gevmusoi =gev.fit(SeaLevel,ydat=ydat, mul=c(1,2)) So which model is better?

D<- -2*(gevmu$nllh-gev$nllh)chi<-qchisq(0.95,1)p<-pchisq(D,1,lower.tail=FALSE)AIC=2*(gevmu$nllh+4)

231

Fremental Extreme Sea Levels –Model Comparison

IndexModel Name Parameters nllh Compare D chi p AIC

1 gev , , -43.567 NA NA NA NA -81.132 gevmu (t), , -49.914 2 vs. 1 12.69 3.84 0.0004 -91.833 gevmusoi (t,soi), , -53.898 3 vs. 2 7.97 3.84 0.0047 -97.793 gevmusoi (t,soi), , -53.898 3 vs. 1 20.66 5.99 ~0 -97.79

gevmu is better than gev Adding SOI appears to improve the model

further (compare 3 vs. 2)

232

59

Atlantic Multi-Decadal Oscillation (AMO) index as a covariate St. Johns River, Florida. Floods

influenced by the phase of AMO>dat=read.csv(“Stjohn.csv”,header=T)>attach(dat)Year StjQ1944 33001945 9230>ydat=rep(1,1,length(Year))>ydat[yrs <= 1969] = -1>ydat<-matrix(ydat,ncol=1)>gev=gev.fit(StjQ)>gevmu=gev.fit(StjQ,ydat=ydat,mul=1)

gev$nllh=400.7574gevmu$nllh=396.4719D=8.571; Chi=3.841; p=0.003AICgev=403.8; AICgevmu=400.5

233

Math Classic..

234

Final Remarks

235

Riesgo Hidrólogico de EventosExtremos en Condiciones

No-Estacionarias




Return Period and Risk for Stationary andNon‐stationary Extreme Conditions

60



Return Period and Risk Under Stationary Conditions

Return Period Under StationaryConditions

61


Design flood and constant values of exceeding (p) and non‐exceeding (q = 1‐p) probabilities throughout years 1 to t,

i.e. stationary condition (Salas and Obeysekera, 2013)

Distribution of the “Waiting Time”for Stationary Conditions

Distribution of the Waiting Time(also known as “First Arrival Time”)

Flood occurrenceFlood occurrence


62



Hydrologic Risk Under StationaryConditions

Hydrologic Risk Under StationaryConditions

63

Return Period and Risk Under Non‐Stationary Conditions

Fig.5 Example of non‐stationary annual flood data

Developments to deal with Non‐Stationarity



64

Distribution of the “Waiting Time”for Non‐Stationary Conditions

Distribution of the “Waiting Time”for Non‐Stationary Conditions

Return Period for Non‐Stationary Conditions

Return Period for Non‐Stationary Conditions

65

Hydrologic Risk for Non‐Stationary Conditions

Hydrologic Risk for Non‐Stationary Conditions

Hypothetical Example for DeterminingT and R for Non‐stationary Conditions

Final Remarks

66

Riesgo Hidrólogico de EventosExtremos en Condiciones

No-Estacionarias




Examples of Analysis of Extreme Events forNon‐stationary Conditions

Examples of Analysis of Extreme Events for Non‐stationary Conditions

Outline

• Examples based on the exponential distribution

• Examples of increasing floods

• Examples of increasing and decreasing sea levels

• Examples of shifting flood regimes

Example Using an Exponential Distribution

Example Using an Exponential Distribution

67



Examples Using the GEV forIncreasing Flood Events


68





69



Aberjona River Basin, Winchester, Massachusetts (Vogel et al, 2011)


Aberjona River Basin, Winchester, Massachusetts


Aberjona River Basin, Winchester, Massachusetts

70





71





72

Examples Using the GEV forIncreasing and Decreasing Sea Levels

(a) (b)


(a) (b)


(a) (b)


73

Further Remarks

Riesgo Hidrológico de Eventos Lecture Outline Extremos en ...

Documents

Transcript of Riesgo Hidrológico de Eventos Lecture Outline Extremos en ...