Post on 30-Dec-2015
IPWG Validationcurrent status and future directions
Chris Kidd
Beth Ebert
John Janowiak
The University of Birmingham, Birmingham, UK
Bureau of Meteorology, Melbourne, Australia
CPC/NCEP/NWS/NOAA, USA
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
IPWG validation sites
Addresses IPWG objectives:
#2) Establish standards for validation and independent verification of precipitation measurements
#3) Foster the exchange of data on intercomparisons of operational precipitation measurements from satellites
Main regions (initiated 2002) are:
Australia, United States, Europe and South America,
with Japan, South Africa and others for selected periods
Common graphical, descriptive and statistical analysis of daily, 0.25° resolution products.
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
GV-IPWG synergies
Criteria GV program IPWG
Type of validation Priority on physical, also statistical
Has focused on descriptive and statistical
Source of validation data Arranged for and collected by principle investigators
Doesn't request. IPWG participants free to contribute
Source of observational data Specific satellite-based products
participants provide products directly to validation groups
Types of Validation data Gauge, radars and specialist instrumentation, diverse in specific locations
Conventional gauge and/or radar networks, usually part of a national network
Types of observational data single-sensor, instantaneous, full-resolution datasets
Blended satellite sensor products, time/area averaged.
After Turk & Arkin, BAMS 2008
Both approaches are complementary
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
GV=Ground Validation
Global precipitation data sets
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
IPWG Intercomparison regionsNear real-time intercomparison of model & satellite estimates vs radar/gauge
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
Processing systemInitial setup:Setting of datesCleaning out old/decayed data
Acquiring data:Searching existing dataListing missing dataCreation of .netrc fileftp data sources
Remapping of data:… to regional grid or5 km PSG projection…
Results generation:Statistical analysisGraphical output
Web pages:Generate HTML filesCopying to server
Processing checksforeach day (d0-d0-31)
dn=dn+1
set d0=today
foreach day (d0-d0-31)
foreach product (p1-pn)
if (product for day) !exist
add to .netrc file
foreach datasource (s0-sn)
ftp datasource (4k)
N
Y
foreach product & day
remap to PSG using LUTs & standardise format
standardise filename
foreach product & day
Generate statistics
Generate plots
foreach product & day
generate HTML files
Processing checksforeach day (d0-d0-31)
dn=dn+1
set d0=today
foreach day (d0-d0-31)
foreach product (p1-pn)
if (product for day) !exist
add to .netrc file
foreach datasource (s0-sn)
ftp datasource (4k)
YN
foreach product & day
remap to PSG using LUTs & standardise format
standardise filename
foreach product & day
Generate statistics
Generate plots
foreach product & day
generate HTML files
Set up list of past dates/days
Usually okay: sometimes needs
tweaking
Checks for a products results:
Okay if no results, but not if bad data
Prepares products into
common formatUsually okay…
Generates outputs:
Okay if there is rain…
Generates raw HTML:
Occasional issues with server
FTP runs several times:
4K buffer limit on macros
Automated systems they are NOT!
Validation productsBoth gauge and radar are used – depending upon the
real-time accessibility to these data sets
Australia: gauge analysis
United States: gauge and radar analysis
Europe: radar analysis
South America: Gauge analysis
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
European radar data: European composite with different radar and calibration and QC strategies; nominal 5km 15-minute product; available within ~1 hour; errors include range effects and anaprop (shipping, etc)
Gauge data: 09-09Z data (dated at end of period); ~6000 daily gauges with mean 10km spacing; available 1 month in arrears
Radar vs gauge data
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
Rad
ar (
daily
inte
grat
ed)
Gau
ge d
ata
Overall results
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
• There is a clear seasonal dependence of satellite product performance (poorer in winter)
• Models overestimate rain area/extent
• Satellite products underestimate rain area/extent
• Significant day-to-day variations
• Surface contamination of satellite products noticeable in winter
Performance dependent upon precipitation extent and intensity, spatial and temporal resolution of product – implications for high-resolution precipitation products
The future of IPWG validation regions
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
What are the key requirements in the future to make best of our own limited resources?
What are the requirements of the user community?
What are the requirements of the algorithm/product developers?
What new sources of data are available?
Should we go beyond daily regional comparisons (local-global, instantaneous-seasonal)?
The future of IPWG validation regions
New sites:• BALTEX region – limited number of algorithms (and
need to read HDF5!); also Canada (limited coverage)• High-latitude oceanic location (Iceland; Shetlands?)• Broaden existing regions (e.g. Europe)
New comparison techniques:• high-resolution products will need to rely upon
descriptive statistics• implementation of fuzzy-logic statistical techniques
New performance results:• Seasonal and global comparisons
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
Internationalframework
Current European radarnetwork – used in IPWG
EUMETNET radarcoverage
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
The future of IPWG validation regions
Monthly and seasonal validationMonthly and seasonal diagnostic validation summaries
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
The future of IPWG validation regions
Suggested analysis levels:
• real time analysis: 'correctness' of near real-time products: should these be easily available (cics?)
• 'historical' analysis: 'authorised' release of quality-controlled products
• high-resolution analysis: similar set-up to daily/0.25 degree products (including instantaneous)
• 'training' data sets: example data sets from a comprehensive range of precipitation products – common naming convention and format
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
Personal recommendations
Evaluation of combined techniques:• individual component products for combined algorithm• actual combination techniques
Standardisation of precipitation product format• simple 2D arrays with accompanying text files• common format –variable size• quantitative resolution – 0.1 mm/day (WMO standard)• error fields can be accompanying files• Naming conventions standardised
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
The IPWG gauntlet: …. to generate global 1km, 1 minute rainfall estimates….
Anagnostou & Hossain: Satellite error propagation in flood predictionNorthern Italy: Basin scales from 116 to 1200 km2, storm induced flash flood event (Nov 1996)Satellite products: Kidd, 1h/4km, and 3B42, 3h/25km: reference rainfall: radar at 1h/1kmEnsembles generated based on SREM2D; Hydrological model simulations based on tRIBS distributed rainfall-runoff model
0 20 40 60 80 100 120 140 1600
100
200
300
400
500
600
700
Time (hrs)
Dis
char
ge (m
3/s
)
radar 1km
SREM2D KIDD 4km
0 20 40 60 80 100 120 140 1600
100
200
300
400
500
600
700
Time (hrs)
Dis
char
ge (m
3/s
)
radar 1km
SREM2D 3B42
Bac
chig
lio
ne
(1
200
km2)
0 20 40 60 80 100 120 140 1600
50
100
150
200
250
Time (hrs)
Dis
char
ge (m
3/s
)
radar 1km
SREM2D KIDD 4km
0 20 40 60 80 100 120 140 1600
50
100
150
200
250
Time (hrs)
Dis
char
ge (m
3/s
)
radar 1km
SREM2D 3B42
Po
sin
a (1
16
km
2)
PMIR: 4km/30min 3B42RT: 1deg/3hr
High resolution products do work!
Recommendations
To establish guidelines for satellite precipitation product data: establish common criteria that can and will be used to promote long-term data sets
Investigate statistical tests that are more relevant to the data sets we are dealing with, and that be applied over a range of spatial and temporal scales
To promote near real-time high resolution (sub-daily, <0.25 degree) precipitation inter-comparisons
4th International Precipitation Working Group (IPWG) Workshop13-17 October 2008 Beijing, China
To widen our validation regions to provide globally representative (inter)comparisons: at daily scales, but also monthly and seasonal scales.