For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 1

Preliminary Analysis of ABFM Data WSR 11 x 11-km Average

Harry Koons

30 October 2003

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 2

Scatter Plot of dBZ vs EmagWSR 11 x 11-km Average

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 3

Approach

Objective is to determine the probability of an extreme electric field intensity for a given radar return

Use the statistics of extreme values to estimate the extreme electric field intensities

– Reference: Statistical Analysis of Extreme Values, Second Edition, R. -D. Reiss and M. Thomas, Birkhäuser Verlag, Boston, 2001

As an example analyze WSR 11x11- km Average data Use both Peaks over Threshold (POT) and Maximum out of

Blocks (MAX) methods Determine extreme value distribution functions for two-dBZ

wide bins– For example the 0-dBZ bin is defined to be the range:

1 < dBZ < +1

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 4

Extreme Value Methods

Parametric models n, n, n, n

Parameters strictly hold only for the sample set analyzed Basic assumption is that the samples, xi , come from

independent, identically distributed (iid) random variables– In our experience the techniques are very robust, as verified

by the Q-Q plots, when this assumption is violated Analysis Methods

– Peaks Over Threshold (POT)• Take all samples that exceed a predetermined, high

threshold, u.• Exceedances over u fit by Generalized Pareto (GP)

distribution functions– Maxima Out of Blocks (MAX)

• Take the maximum value within a pass, Anvil, etc. • Tail of distribution is fit by extreme value (EV)

distribution functions

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 5

Generalized Pareto (GP) Distribution Functions(Peaks-over-Threshold Method)

Exponential (GP0, = 0):

Pareto(GP1, > 0):

Beta (GP2, < 0):

The – Parameterization:

xexW 1)(0

xxW 1)(,1

)(1)(,2

xxW

/1)1(1)( xxW

0),()( 0 xWxW

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 6

Standard EV Distribution Functions(Maximum out of Blocks Method)

Gumbel (EV0, = 0):

Fréchet (EV1, > 0):

Weibull (EV2, < 0):

The – Parameterization:

)exp()(0xexG

)exp()(,1

xxG

))(exp()(,2

xxG

))1(exp()( /1 xxG

0),exp()(0 xexG

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 7

Tutorial Example Based on a ManufacturedGaussian (Normal) Distribution Function

Select 100 random samples uniformly from a normal distribution

– Mean value, = 5– Standard deviation, = 1

Maximum Likelihood Estimates for a normal model are the sample mean and sample standard deviation

= 4.81789 = 1.05631

In general you maximize the likelihood function by taking appropriate partial derivates

ini

xL ,,

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 8

Distribution function, F, of a real-valued random variable X is given by

F (X) = P{ X x } Histogram

The density function, f, is the derivative of the distribution function

– Sample Density

– Model Density

– Kernel Density

Distribution and Density Functions

njnjpn /

2175.0 xxk

b

xxk

nbxxg iib

1,

01 banddyyk

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 9

Quantile Functions and the Q-Q Plot

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 10

T-Year Values

T-year level is a higher quantile of the distribution function

T-year threshold, u(T), is the threshold such that the mean first exceedance time is T years

– u(T) = F-1(1-1/T)

– 1-1/T quantile of the df The T-year threshold also is

exceeded by the observation in a given year with the probability of 1/T

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 11

Scatter Plot of dBZ vs EmagWSR 11 x 11-km Average; 0 dBZ bin

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 12

Cumulative Frequency Curve for Emag–1 < dBZ < +1

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 13

Sample Statistics-1 < dBZ < +1

Sample Size, N = 271 Minimum = 0.02 kV/m Maximum = 2.41 kV/m Median = 0.6 kV/m Mean = 0.67 kV/m

Choose Peaks Over Threshold (POT) Methodfor Extreme Value Analysis

u = 1.0 kV/m (high threshold) n = 57 (samples over threshold) k = 32 (mean value of the stability zone)

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 14

Gamma Diagram

Point of stability is onThe plateau between30 and 40

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 15

Kernel Density Functionand Model Density Function

Peaks Over Threshold Method (POT)

– N = 271 samples between –1 and +1 dBZ

– u = 1.0 kV/m (high threshold)

– n = 57 (samples over threshold)

MLE (GP0)

– k = 32 (mean zone of stability)

– = 0.0

– = 1.00457 (~left end point)

– = 0.30387

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 16

Sample and Model Distribution Functions

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 17

Sample and Model Quantile Functions

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 18

Q – Q Plot

Sam

ple

Model

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 19

T-Sample Electric Field Intensity-1 < dBZ < +1

T, Samples Probability, p Quantile, pkV/m

100 0.01 1.74

1,000 0.001 2.43

10,000 0.0001 3.12

100,000 0.00001 3.81

1,000,000 0.000001 4.50

10,000,000 0.0000001 5.20

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 20

Extend Analysis to Other Bins

Use bins centered at –2, 0, +2, and +4 dBZ

Kernel Density Plot

Sample Statistics

Model Parameters

T-Sample Electric Field Intensity

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 21

Sample Kernel Densities for Exceedances

Blk: -2 dBZRed: 0 dBZGrn: +2 dBZBlu: +4 dBZ

Emag kV/m

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 22

Sample Statistics and Model Parameters

-2 dBZ 0 dBZ +2 dBZ +4 dBZ

u, kV/m 0.75 1.0 1.0 1.0

N 173 271 369 418

k 24 32 57 84

0.0 0.0 0.0 0.0

0.81 1.00 1.06 1.09

0.15 0.30 0.28 0.37

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 23

T-Sample Electric Field Intensity, kV/m

T-Sample -2 dBZ 0 dBZ +2 dBZ +4 dBZ

100 1.21 1.74 1.82 2.20

1,000 1.56 2.43 2.47 3.05

10,000 1.91 3.12 3.11 3.90

100,000 2.26 3.81 3.75 4.76

1,000,000 2.61 4.50 4.39 5.61

10,000,000 2.96 5.20 5.05 6.46

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 24

Maximum Out of Blocks Method #1One Block is One Pass

Analyze the 0 dBZ bin 38 Blocks (unique pass numbers) Maximum Emag for each block shows a slight dependence

on sample-count per block

– We will ignore this MLE (EV) Model Parameters

= 0.138

= 0.634

= 0.322

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 25

Sample Count vs. Pass NumberMax for Passes

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 26

Number of Occurrences vs. Sample CountMax for Passes

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 27

Plot of Maximum Emag vs. Sample CountMAX for Passes

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 28

Density FunctionsMAX for Passes

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 29

Q-Q PlotMAX for Passes

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 30

T-Pass Electric Field Intensity-1 < dBZ < +1

T Emag, kV/m

50 2.30

100 2.70

200 3.14

500 3.80

1,000 4.35

2,000 4.96

5,000 5.85

10,000 6.61

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 31

Maximum Out of Blocks Method #2One Block is One Anvil

Analyze the 0 dBZ bin 21 Blocks (unique anvil numbers) MLE (EV) Model Parameters

= -0.044

= 0.830

= 0.431

Right End Point = 10.5 kV/m

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 32

Density FunctionsMAX for Anvils

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 33

Distribution FunctionsMAX for Anvils

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 34

Q-Q PlotMAX for Anvils

For information contact H. C. Koons

E-Mail: hkoons@aero.org

30 October 2003 35

T-Anvil Electric Field Intensity-1 < dBZ < +1

T Emag, kV/m

50 2.37

100 2.62

200 2.86

500 3.17

1,000 3.39

2,000 3.61

5,000 3.89

10,000 4.09