Post on 28-Jan-2016
description
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 1
Preliminary Analysis of ABFM Data WSR 11 x 11-km Average
Harry Koons
30 October 2003
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 2
Scatter Plot of dBZ vs EmagWSR 11 x 11-km Average
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 3
Approach
Objective is to determine the probability of an extreme electric field intensity for a given radar return
Use the statistics of extreme values to estimate the extreme electric field intensities
– Reference: Statistical Analysis of Extreme Values, Second Edition, R. -D. Reiss and M. Thomas, Birkhäuser Verlag, Boston, 2001
As an example analyze WSR 11x11- km Average data Use both Peaks over Threshold (POT) and Maximum out of
Blocks (MAX) methods Determine extreme value distribution functions for two-dBZ
wide bins– For example the 0-dBZ bin is defined to be the range:
1 < dBZ < +1
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 4
Extreme Value Methods
Parametric models n, n, n, n
Parameters strictly hold only for the sample set analyzed Basic assumption is that the samples, xi , come from
independent, identically distributed (iid) random variables– In our experience the techniques are very robust, as verified
by the Q-Q plots, when this assumption is violated Analysis Methods
– Peaks Over Threshold (POT)• Take all samples that exceed a predetermined, high
threshold, u.• Exceedances over u fit by Generalized Pareto (GP)
distribution functions– Maxima Out of Blocks (MAX)
• Take the maximum value within a pass, Anvil, etc. • Tail of distribution is fit by extreme value (EV)
distribution functions
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 5
Generalized Pareto (GP) Distribution Functions(Peaks-over-Threshold Method)
Exponential (GP0, = 0):
Pareto(GP1, > 0):
Beta (GP2, < 0):
The – Parameterization:
xexW 1)(0
xxW 1)(,1
)(1)(,2
xxW
/1)1(1)( xxW
0),()( 0 xWxW
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 6
Standard EV Distribution Functions(Maximum out of Blocks Method)
Gumbel (EV0, = 0):
Fréchet (EV1, > 0):
Weibull (EV2, < 0):
The – Parameterization:
)exp()(0xexG
)exp()(,1
xxG
))(exp()(,2
xxG
))1(exp()( /1 xxG
0),exp()(0 xexG
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 7
Tutorial Example Based on a ManufacturedGaussian (Normal) Distribution Function
Select 100 random samples uniformly from a normal distribution
– Mean value, = 5– Standard deviation, = 1
Maximum Likelihood Estimates for a normal model are the sample mean and sample standard deviation
= 4.81789 = 1.05631
In general you maximize the likelihood function by taking appropriate partial derivates
ini
xL ,,
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 8
Distribution function, F, of a real-valued random variable X is given by
F (X) = P{ X x } Histogram
The density function, f, is the derivative of the distribution function
– Sample Density
– Model Density
– Kernel Density
Distribution and Density Functions
njnjpn /
2175.0 xxk
b
xxk
nbxxg iib
1,
01 banddyyk
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 9
Quantile Functions and the Q-Q Plot
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 10
T-Year Values
T-year level is a higher quantile of the distribution function
T-year threshold, u(T), is the threshold such that the mean first exceedance time is T years
– u(T) = F-1(1-1/T)
– 1-1/T quantile of the df The T-year threshold also is
exceeded by the observation in a given year with the probability of 1/T
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 11
Scatter Plot of dBZ vs EmagWSR 11 x 11-km Average; 0 dBZ bin
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 12
Cumulative Frequency Curve for Emag–1 < dBZ < +1
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 13
Sample Statistics-1 < dBZ < +1
Sample Size, N = 271 Minimum = 0.02 kV/m Maximum = 2.41 kV/m Median = 0.6 kV/m Mean = 0.67 kV/m
Choose Peaks Over Threshold (POT) Methodfor Extreme Value Analysis
u = 1.0 kV/m (high threshold) n = 57 (samples over threshold) k = 32 (mean value of the stability zone)
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 14
Gamma Diagram
Point of stability is onThe plateau between30 and 40
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 15
Kernel Density Functionand Model Density Function
Peaks Over Threshold Method (POT)
– N = 271 samples between –1 and +1 dBZ
– u = 1.0 kV/m (high threshold)
– n = 57 (samples over threshold)
MLE (GP0)
– k = 32 (mean zone of stability)
– = 0.0
– = 1.00457 (~left end point)
– = 0.30387
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 16
Sample and Model Distribution Functions
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 17
Sample and Model Quantile Functions
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 18
Q – Q Plot
Sam
ple
Model
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 19
T-Sample Electric Field Intensity-1 < dBZ < +1
T, Samples Probability, p Quantile, pkV/m
100 0.01 1.74
1,000 0.001 2.43
10,000 0.0001 3.12
100,000 0.00001 3.81
1,000,000 0.000001 4.50
10,000,000 0.0000001 5.20
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 20
Extend Analysis to Other Bins
Use bins centered at –2, 0, +2, and +4 dBZ
Kernel Density Plot
Sample Statistics
Model Parameters
T-Sample Electric Field Intensity
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 21
Sample Kernel Densities for Exceedances
Blk: -2 dBZRed: 0 dBZGrn: +2 dBZBlu: +4 dBZ
Emag kV/m
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 22
Sample Statistics and Model Parameters
-2 dBZ 0 dBZ +2 dBZ +4 dBZ
u, kV/m 0.75 1.0 1.0 1.0
N 173 271 369 418
k 24 32 57 84
0.0 0.0 0.0 0.0
0.81 1.00 1.06 1.09
0.15 0.30 0.28 0.37
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 23
T-Sample Electric Field Intensity, kV/m
T-Sample -2 dBZ 0 dBZ +2 dBZ +4 dBZ
100 1.21 1.74 1.82 2.20
1,000 1.56 2.43 2.47 3.05
10,000 1.91 3.12 3.11 3.90
100,000 2.26 3.81 3.75 4.76
1,000,000 2.61 4.50 4.39 5.61
10,000,000 2.96 5.20 5.05 6.46
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 24
Maximum Out of Blocks Method #1One Block is One Pass
Analyze the 0 dBZ bin 38 Blocks (unique pass numbers) Maximum Emag for each block shows a slight dependence
on sample-count per block
– We will ignore this MLE (EV) Model Parameters
= 0.138
= 0.634
= 0.322
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 25
Sample Count vs. Pass NumberMax for Passes
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 26
Number of Occurrences vs. Sample CountMax for Passes
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 27
Plot of Maximum Emag vs. Sample CountMAX for Passes
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 28
Density FunctionsMAX for Passes
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 29
Q-Q PlotMAX for Passes
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 30
T-Pass Electric Field Intensity-1 < dBZ < +1
T Emag, kV/m
50 2.30
100 2.70
200 3.14
500 3.80
1,000 4.35
2,000 4.96
5,000 5.85
10,000 6.61
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 31
Maximum Out of Blocks Method #2One Block is One Anvil
Analyze the 0 dBZ bin 21 Blocks (unique anvil numbers) MLE (EV) Model Parameters
= -0.044
= 0.830
= 0.431
Right End Point = 10.5 kV/m
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 32
Density FunctionsMAX for Anvils
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 33
Distribution FunctionsMAX for Anvils
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 34
Q-Q PlotMAX for Anvils
For information contact H. C. Koons
E-Mail: hkoons@aero.org
30 October 2003 35
T-Anvil Electric Field Intensity-1 < dBZ < +1
T Emag, kV/m
50 2.37
100 2.62
200 2.86
500 3.17
1,000 3.39
2,000 3.61
5,000 3.89
10,000 4.09