Taming Massive Distributed Datasets: Data Sampling Using Bitmap Indices
Indices versus Data Indices are information derived from data Indices are information derived from...
-
Upload
gyles-robinson -
Category
Documents
-
view
225 -
download
0
Transcript of Indices versus Data Indices are information derived from data Indices are information derived from...
![Page 1: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/1.jpg)
Indices versus DataIndices versus Data Indices are information derived from Indices are information derived from
datadata Proxy for dataProxy for data More readily released than dataMore readily released than data
• Indices generally not of economic value Indices generally not of economic value – only research value and for – only research value and for applicationsapplications
• More on this laterMore on this later Are not reproducible without the dataAre not reproducible without the data
• A key component of scienceA key component of science
![Page 2: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/2.jpg)
Many different ways to Many different ways to calculate indices for calculate indices for
extremesextremes
![Page 3: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/3.jpg)
![Page 4: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/4.jpg)
What types of extremes?What types of extremes? Trends in extreme events characterised by the Trends in extreme events characterised by the
size of their societal or economic impactssize of their societal or economic impacts Trends in “very rare” extreme events analysed Trends in “very rare” extreme events analysed
by the parameters of extreme value by the parameters of extreme value distributionsdistributions
Trends in observational series of phenomena Trends in observational series of phenomena
with a daily time scale and typical return with a daily time scale and typical return period < 1 yearperiod < 1 year(as indicators of extremes)(as indicators of extremes)
NO
NO
YES
![Page 5: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/5.jpg)
Motivation for choice of Motivation for choice of “extremes”“extremes”
The detection probability of trends The detection probability of trends depends on the return period of the depends on the return period of the extreme event and the length of the extreme event and the length of the observational seriesobservational series
For extremes in daily series with For extremes in daily series with typical length ~50 yrs, the optimal typical length ~50 yrs, the optimal return period is 10-30 return period is 10-30 daysdays rather rather than 10-30 than 10-30 yearsyears
![Page 6: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/6.jpg)
ApproachApproach Focus on counts of days crossing a Focus on counts of days crossing a
threshold; either absolute/fixed threshold; either absolute/fixed thresholds or percentile/variable thresholds or percentile/variable thresholds relative to local climatethresholds relative to local climate
Standardisation enables comparisons Standardisation enables comparisons between results obtained in different between results obtained in different countries, and even different parts of countries, and even different parts of the worldthe world
![Page 7: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/7.jpg)
![Page 8: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/8.jpg)
Expert Team on Climate Change Detection and Indices (ETCCDI)
started in 1999
jointly sponsored by CCl, CLIVAR and JCOMM
![Page 9: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/9.jpg)
the ETCCDI developed an internationally coordinated set of 27 climate indicesfocus on counts of days crossing a threshold; either absolute/fixed thresholds or percentile/variable thresholds relative to local climateused for both observations and models, globally as well as regionallycan be coupled with
– simple trend analysis techniques– standard detection and attribution methods
complements the analysis of more rare extremes using EVT
![Page 10: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/10.jpg)
Klein Tank, Zwiers and Zhang, 2009, WCDMP-No. 72,WMO-TD No. 1500, 56pp.
![Page 11: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/11.jpg)
Example: Russian heat wave, July 2010
In-situNOAA
![Page 12: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/12.jpg)
ERA-InterimECMWF
MSUUAH
Courtesy: John Christy (top), Adrian Simmons (bottom)
In-situNOAA
Example: Russian heat wave, July 2010
![Page 13: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/13.jpg)
ETCCDI indices add relevant information
In-situNOAA
Example: Russian heat wave, July 2010
![Page 14: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/14.jpg)
31 days withT-max > 25°C against 9.5 days in a normal July
Example: Russian heat wave, July 2010
![Page 15: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/15.jpg)
16 nights with T-min > 20°C against 0.5 night in a normal July
Example: Russian heat wave, July 2010
![Page 16: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/16.jpg)
![Page 17: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/17.jpg)
Extremes Indices
![Page 18: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/18.jpg)
upper 10-ptile 1961-1990
the year 1996
lower 10-ptile1961-1990
Indices example
![Page 19: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/19.jpg)
upper 10-ptile 1961-1990
the year 1996
lower 10-ptile1961-1990
“cold nights”
Indices example
![Page 20: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/20.jpg)
upper 10-ptile 1961-1990
the year 1996
lower 10-ptile1961-1990
“warm nights”
“cold nights”
Indices example
![Page 21: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/21.jpg)
De Bilt, the Netherlands
Indices example
![Page 22: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/22.jpg)
![Page 23: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/23.jpg)
1) Identify heavy falls using a site specific threshold = 95th percentile at wet days in the 1961-90 period
Changes in heavy falls
![Page 24: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/24.jpg)
1) Identify heavy falls using a site specific threshold = 95th percentile at wet days in the 1961-90 period
2) Determine fraction of total precipitation in each year that is due to these days
Changes in heavy falls
![Page 25: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/25.jpg)
1) Identify heavy falls using a site specific threshold = 95th percentile at wet days in the 1961-90 period
2) Determine fraction of total precipitation in each year that is due to these days
3) Trend analysis in series of fractions
Changes in heavy falls
![Page 26: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/26.jpg)
Alexander et al.,2006; in IPCC-AR4
Changes in heavy falls
![Page 27: Indices versus Data Indices are information derived from data Indices are information derived from data Proxy for data Proxy for data More readily released.](https://reader034.fdocuments.in/reader034/viewer/2022051203/5a4d1b8a7f8b9ab0599be718/html5/thumbnails/27.jpg)
Extremes table IPCC-AR4, WG1 report (IPCC, 2007)