Joint Ensemble Forecast System (JEFS) NCAR Sep 2005.
-
date post
20-Jan-2016 -
Category
Documents
-
view
222 -
download
0
Transcript of Joint Ensemble Forecast System (JEFS) NCAR Sep 2005.
Joint Ensemble Forecast System(JEFS)
NCAR
Sep 2005
Overview
Motivation/Goal
Requirements
Resources
System Design
Roadmap
Products/Applications
Prove the value, utility, and operational feasibility of ensemble forecasting to DoD operations.
Deterministic Forecasting
?• Ignores forecast uncertainty• Potentially very misleading• Oversells forecast capability
• Reveals forecast uncertainty• Yields probabilistic information• Enables optimal decision making
EnsembleForecasting
…etc
JEFS’ Goal
AFW Strategic Plan and Vision, FY2008-2032 Issue #3/4-3: Use of multi-scale (kilometer to meter resolution), ensemble, and consensus model forecasts, combined with automation of local techniques, to support planning and execution of military operations.
“Ensembles have the potential to help quantify the certainty of a prediction, which is something that users have beeninterested in for years. The military applications of ensemble forecasting are only at their beginnings; there are years’ worth of research waiting to be done.”
Operational Requirements Document, USAF 003-94-I/II/III-D, Centralized Aerospace WeatherCapability (CAWC ORD)
…will support ensemble forecasting with the following capabilities: 1) The creation of sets of perturbed initial conditions of the fine-scale model initialized fields in selected regional windows.2) Assembly of ensemble forecasts either from model output sets derived from the multiple sets of perturbed initial conditions or from sets assembled from the output from different models.3) Evaluation of forecasting skill of ensemble forecasts compared to single forecast model outputs.
Air Force Weather, FY 06-30, Mission Area Plan (AFW MAP)Deficiency: Mesoscale Ensemble Forecasting
“The key to successful ensemble forecasting is many different realizations of the same forecast events. Studies usingdifferent models - or the same model with different configurations - consistently yield better overall forecasts.This demonstrates a definite need for multiple model runs.”
R&D PortfolioMSA Shortfall D-08-07K: Insufficient ensemble forecasting capability for AFWA’s theater scale model
Ensemble Forecast RequirementsAir Force (and Army)
No documented requirement or supporting Fleet request for ensemble prediction.
Navy ‘requirements’ are written in terms of warfighting capabilities. The current (draft) METOC ICD (old MNS) only specifies parameters required for support. However, ensembles present a solution for the following specified warfighter requirements:
• Long-range prediction for mission planning, optimum track ship routing, severe weather avoidance
• Tropical cyclone prediction for safety of operations, personnel safety
• Winds, turbulence, boundary layer structure for chem/bio/nuclear dispersion (WMD support)
• Cloud base, fog, aerosol for slant range visibility (aerial recon, flight operations, targeting)
• Boundary layer structure/atmospheric refractivity (T, q) for EM propagation (detection, tracking, communications)
• Surface winds (ASW, mine drift, SAR, flight operations in enclosed/narrow waterways)
• Surf and sea heights (SOF, small boat ops, logistics)
• Turbulence, cloud base/tops (OPARS, safety of flight)
Whenever the uncertainty of the wx phenomena exceeds operational sensitivity, either a reliable probabilistic or a range-of-variability prediction is required.
Ensemble Forecast RequirementsNavy
JEFS
TEAM
Organization Contribution Players AFWA - JEFS integration
- FY05-FY07 Funding Maj Tony Eckel Dr. Jerry Wegiel Mr. Norm Mandy
FNMOC - JEFS integration - NOGAPS members for JGE
Dr. Mike Sestack
HPCMP - Primary Hardware Funding - Programming Environment and Training (PET) onsite at AFWA
Mr. John Boisseau Dr. Steve Klotz (at AFWA)
NRL - JGE and JME initial conditions - COAMPS model perturbations
Dr. Craig Bishop Dr. Jim Doyle Dr. Carolyn Reynolds Ms. Sue Chen Mr. Justin McLay
ARL
- Uncertainty visualization tool: Weather Risk Analysis and Portrayal (WRAP)
Mr. Dave Knapp Ms. Barb Sauter Mr. Hyam Singer (Next Century) Mr. Allen Hill (Next Century)
DTRA - FY05-FY09 Funding CDR Stephanie Hamilton Mr. Pat Hayes
NCAR - WRF model perturbations Dr. Jordan Powers Dr. Chris Snyder
UW - Calibration (bias correction and BMA) - Product Design/Development
Dr. Cliff Mass Dr. Eric Grimit
20 OWS - JEFS operational testing and evaluation Lt Col Mike Farrar Maj David Andrus
17 OWS - JEFS operational testing and evaluation Maj Christopher Finta 1Lt Perry Sweat
Yokosuka NPMOC
- JEFS operational testing and evaluation ?
NPS - Research project(s) Dr. Russ Elsberry Maj Bob Stenger
ONR - Consultation Dr. Steve Tracton
& AFIT
• Apr 03: FNMOC and AFWA proposed a split distributed center to the DoD High Apr 03: FNMOC and AFWA proposed a split distributed center to the DoD High Performance Computing Modernization Program (HPCMP) as a Performance Computing Modernization Program (HPCMP) as a DoD Joint Operational DoD Joint Operational Test Bed for the Weather Research and Forecast (WRF) modeling frameworkTest Bed for the Weather Research and Forecast (WRF) modeling framework
• Apr 04: Installation began of $4.2M in IBM HPC hardware, Apr 04: Installation began of $4.2M in IBM HPC hardware, split equally between FNMOC and AFWAsplit equally between FNMOC and AFWA (two 96 processor IBM Cluster 1600 p655+ systems)(two 96 processor IBM Cluster 1600 p655+ systems)
• Fosters significant Navy/Air Force collaboration in NWP forFosters significant Navy/Air Force collaboration in NWP for
1) Testing and optimizing of WRF configurations to meet1) Testing and optimizing of WRF configurations to meet unique Navy and Air Force NWP requirementsunique Navy and Air Force NWP requirements
2) Developing and testing mesoscale ensembles based on 2) Developing and testing mesoscale ensembles based on multiple WRF configurations to meet DoD needsmultiple WRF configurations to meet DoD needs
3) Testing of Grid Computing concepts and tools for NWP3) Testing of Grid Computing concepts and tools for NWP
• Apr 08: Project CompletionApr 08: Project Completion
FY04 HPCMP Distributed Center (DC) Award
• Description: Combination of current GFS and NOGAPS global, medium-range ensemble data. Possible expansion to include ensembles from CMC, UKMET, JMA, etc.
• Initial Conditions: Breeding of Growing Modes 1
• Model Variations/Perturbations: Two unique models, but no model perturbations
• Model Window: Global
• Grid Spacing: 1.0 1.0 (~80 km)
• Number of Members: 40 at 00Z 30 at 12Z
• Forecast Length/Interval: 10 days/12 hours • Timing
• Cycle Times: 00Z and 12Z• Products by: 07Z and 19Z
1 Toth, Zoltan, and Eugenia Kalnay, 1997: Ensemble Forecasting at NCEP and the Breeding Method. Monthly Weather
Review: Vol. 125, No. 12, pp. 3297–3319.
Joint Global Ensemble (JGE)
5 km
15 km
• Description: Multiple high resolution, mesoscale model runs generated at FNMOC and AFWA
• Initial Conditions: Ensemble Transform Filter 2 run on short-range (6-h),
mesoscale data assimilation cycle driven by GFS and NOGAPS ensemble members
• Model variations/perturbations: • Multimodel: WRF-ARW, COAMPS • Varied-model: various configurations of physics packages• Perturbed-model: randomly perturbed sfc boundary conditions (e.g., SST)
• Model Window: East Asia (COPC directive, Apr ’04)
• Grid Spacing: 15 km for baseline JME (summer ’06) 5 km nest later in project
• Number of Members: 30 (15 run at each DC site)
• Forecast Length/Interval: 60 hours/3 hours
• Timing• Cycle Times: 06Z and 18Z• Products by: 14Z and 02Z
~7 h production /cycle
2 Wang, Xuguang, and Craig H. Bishop, 2003: A Comparison of Breeding and Ensemble Transform Kalman Filter Ensemble Forecast Schemes. Journal of the Atmospheric Sciences: Vol. 60, No. 9, pp. 1140–1158.
Joint Mesoscale Ensemble (JME)
Storage of principal fields
NCEP Medium Range Ensemble 44 staggered GFS runs, T126, 15 d Analysis perturbations: Bred Modes Model Perturbations: in design
Joint Ensemble Forecast System
lateral boundaryconditions
multiplefirst guesses
Joint Mesoscale Ensemble (JME) 30 members, 15/5km, 60 h, 2/day One “demonstration” theater Multi model (WRF, COAMPS) Perturbed model: varied physics and surface boundary conditions
FNMOC
JME Products Apply postprocessing calibration Short-range products tailored to support warfighter operations
AFWA
Observations
“warm start”
Data Assimilation3DVAR / NAVDAS
FNMOC Medium Range Ensemble 18 00Z, 8 12Z NOGAPS, T119, 10 d Analysis Perturbations: Bred Modes Model Perturbations: None
Storage of principal fields
Calibrate
Joint Global Ensemble (JGE) Products Apply postprocessing calibration Long-range products tailored to support warfighter planningEnsemble Transform
Generate initial condition perturbations
Calibrate
Observations and Analyses
GFS ensemble Grids to AFWA and FNMOC
NOGAPS ens. grids to AFWA
Interpolate and calibrate JGE
Make/Distribute JGE products
Obtain global analysis
Update JGE Calibration
Data Assimilation
Run 6-h forecasts and do ET
Run JME models
Exchange output
Make/Distribute JME Products
Update JME Calibration
00 03 06 09 12 15 18 21 24(Z)
00Z cycle data 06Z cycle data 12Z cycle data 18Z cycle data
06Z production cycle 18Z production cycle
JEFS Production Schedule
Notional Roadmapfor JEFS and Beyond
1. AFWA/FNMOC Awarded HPCMPO DC Nov 03
2. AFWA Awarded PET-CWO On-Site
3. NRL Awarded mesoscale ensemble research
4. DTRA-AFWA Ensemble Investment
5. ARL SIBR Phase I & II and AFWA UFR
6. NCAR & UW Contract, funded by AFWA Wx Fcst 3600
JEFS Design
1
2
4
5. ARL SIBR Phase II w/ AFWA UFR
JGE RDT&E
JME RDT&E
3 3. Probabilistic Pred. of High Impact Wx
5. Phase I
2. Programming Environment and Training - Climate Weather Ocean On-Site
JGE IOC
1st Meso. EPS H/W Procurement*
2nd Meso. EPS H/W Procurement*3rd Meso. EPS H/W Procurement*
Mesoscale EPS IOC
Mesoscale EPS FOC
1. HPCMPO DC H/W
* Note: Funded via PEC 35111F Weather Forecasting (3080M)
FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11
Phase I
6. NCAR & UW Contract
Phase II
4. DTRA-AFWA Support
Tailor products to customers’ needs and weather sensitivities
Forecaster Products/Applications Design to help transition from deterministic to stochastic thinking
Warfighter Products/Applications Design to aid critical decision making (Operational Risk Management)
Product Strategy
PACIFIC AIR FORCES Forecasters20th Operational Weather Squadron17th Operational Weather Squadron607 Weather Squadron
WarfightersPACAF5th Air Force
Naval Pacific Meteorological and Oceanographic Center ForecastersYokosuka Navy Base
Warfighters7th Fleet
FIFTHAir Force
SEVENTHFleet
Operational Testing & Evaluation
Forecaster Products/Applications
• Consensus (isopleths): shows “best guess” forecast (ensemble mean or median)
• Model Confidence (shaded)
Increase Spread in Less Decreased confidence the multiple forecasts Predictability in forecast
MaximumPotential Error
(mb, +/-)
6
5
4
3
2
1
<1
Consensus & Confidence Plot
• Probability of occurrence of any weather phenomenon/threshold (i.e., sfc wnds > 25 kt )
• Clearly shows where uncertainty can be exploited in decision making
• Can be tailored to critical sensitivities, or interactive (as in IGRADS on JAAWIN)
%Probability Plot
Current
Deterministic
Meteogram
• Show the range of possibilities for all meteogram-type variables
• Box & whisker, or confidence interval plot is more appropriate for large ensembles
• Excellent tool for point forecasting (deterministic or stochastic)
1000/500 Hpa Geopotential Thickness [m] at YokosukaInitial DTG 00Z 28 JAN 1999
0 1 2 3 4 5 6 7 8 9 10Forecast Day
5520
5460
5400
5340
5280
5220
5160
5100
5040
4980
Multimeteogram
Probability of Warning Criteria at McGuire AFB Based on 15/06Z MM5 Ensemble
010
20304050
607080
90100
Date/Time
T Storm
Winds>35kt
Winds>50kt
Snow>.5"/hr
Fzg Rain
15/06 12 18 16/00 06 12 18 17/00 06
Probability of Warning Criteria at Osan AB
What is the potential
risk to the mission?When is a warning required?
0
5
10
15
20
25
30
35
40
45
50
Valid Time
Wind Speed (kt) .
0
5
10
15
20
25
30
35
40
45
50
11/18 12/00 06 12 18 13/00 06 12 18 14/00 06 Valid Time (Z)
90%CI
ExtremeMin
ExtremeMax
Surface Wind Speed at Misawa AB
Mean
Valid Time (Z)
Requires paradigm shift into
“stochastic thinking”
Sample JME Products
Probability of Severe Turbulence @FL300
70%
50%
10%
10%
10%
50%
30%
90%
30%
70%
Sample JGE Product (Forecaster)
Upper Level Turbulence
280
350
Sample JGE Product? (Warfighter)
Chance of Upper Level Turbulence Intensity: Severe
Low
Med
High
250/370
280/370
300/330
Base/Top
LEGENDNegligible Chance
Sample JGE Product (Warfighter)
Warfighter Products/Applications
Integrated Weather Effects Decision Aid (IWEDA)Deterministic
Forecast
> 13kt
10-13kt
0-9kt
Weapon SystemWeather Thresholds*
Drop ZoneSurface Winds
6kt
*AFI 13-217
?
Stochastic Forecast Binary Decisions/Actions
Bombs
on
Target
Go / No Go AR RouteClear & 7
CrosswindsIn / Outof Limits
T-StormWithin 5
Flight Hazards
IFR / VFR
GPSScintillation
Bridging the Gap
10%
20%
70%
Stochastic Forecast
Drop ZoneSurface Winds
6kt3 6 9 12 15 18kt0 10 20 30 40 50 60 70
0
0.01
0.02
0.03
0.04
0.05
Probabilistic IWEDA
-- for Operational
Risk Management
(ORM)
Event: Damage to parked aircraft Threshold: sfc wind > 50kt
Cost (of protecting): $150K
Loss (if damaged): $1M
Hit
FalseAlarm
Miss
CorrectRejection
YES NO
YES
NO
Forecast?
Observed?
$150K $1000K
$150K $0K
Method #1: Decision Theory Minimize operating cost (or maximize effectiveness) in the long run by taking action based on an optimal threshold of probability, rather than an event threshold.
What is the cost of taking action? What is the loss if…
the event occurs and without protection? opportunity was missed since action was not taken?
Good for well defined, commonly occurring events
Optimal Threshold = 15%
Example (Hypothetical)
Forecast Value
+Median
ForecastValue
70% confidence80% confidence90% confidence
Forecast Value
+Median
ForecastValue
70% confidence80% confidence90% confidence
The greater the confidence required
(i.e., less acceptable risk), the less
certain we can be of the desired
outcome.
90% Confidence
Army Research Lab’s stochastic decision aid, in development by Next Century Corporation
Stoplight color based on 1) Ensemble forecast probability distribution2) Weapon systems’ operating thresholds3) Warfighter-determined level of acceptable risk
Drop Zone Surface Winds (kt)
80%70%
Method #2:Weather Risk Analysis and Portrayal (WRAP)
5 10 15
Cu
mu
lati
ve P
rob
abili
ty
9ktThreshold
13ktThreshold
Surface Winds (kt)
5 10 15
5 10 15
5 10 15
Acceptable Risk Decision Input
Low
Med
High
Low
Med
High
Low
Med
High
99% 1% 0%
1% 31% 68%
37% 52% 11%
(90th Percentile)
(60th Percentile)
(30th Percentile)
9ktThreshold
13ktThreshold
Drop Zone #1
Drop Zone #2
Drop Zone #3
18kt ?Threshold
Method #2:Weather Risk Analysis and Portrayal (WRAP)
Method #2:Weather Risk Analysis and Portrayal (WRAP)
ENSEMBLESAHEAD
Backup Slides
Sensitive to Initial Conditions: nearby solutions diverge Describable State: system specified by set of variables that evolve in “phase space”
Deterministic: system appearsrandom but process is governedby rules
Solution Attractor: Limited regionin phase space where solutionsoccur
Aperiodic: Solutions neverrepeat exactly, but may appear similar
The Atmosphere is a Chaotic, Dynamic System
AnalogyTwo adjacent drops in a waterfall end up very far apart.
Predictability is primarily limited by errors in the analysis
To account for this effect, we can make an ensemble of predictions (each forecast being a likely outcome) to encompass the truth.
T
The true state of the atmosphere exists as a single point in phase space that we never know exactly.
A point in phase space completely describes an instantaneous state of the atmosphere. (pres, temp, etc. at all points at one time.)
Nonlinearities drive apart the forecast trajectory and true trajectory (i.e., Chaos Theory)
PHA
SE
SPACE
Encompassing Forecast Uncertainty
12hforecast 36h
forecast
24hforecast
48hforecast
T
48hverification
T
T
T
12hverification
36hverification
24hverification
An analysis produced to run a model is somewhere in a cloud of likely states.
Any point in the cloud is equally likelyto be the truth.
T
Ensemble Forecasting: -- Encompasses truth -- Reveals uncertainty -- Yields probabilistic information
T
PHA
SE
SPACE
48h forecast Region
Analysis Region
An ensemble of likely analyses leads to an ensemble of likely forecasts
Encompassing Forecast Uncertainty
The Wind Storm That Wasn’t(Thanksgiving Day 2001)
Mean Sea Level Pressure (mb)and shaded Surface Wind Speed (m s-1)
Eta-MM5 Forecast Verification
avn-MM5 Forecast
ngps-MM5 Forecast
cmcg-MM5 Forecast
tcwb-MM5 Forecastukmo-MM5 Forecast
eta-MM5 Forecastcent-MM5 Forecast
avn-MM5 Forecast ngps-MM5 Forecast
cmcg-MM5 Forecast
tcwb-MM5 Forecast ukmo-MM5 Forecast
The Wind Storm That Wasn’t(Thanksgiving Day 2001)
eta-MM5 Forecast Verification
Deterministic Forecasting
Single solution
Variable and unknown risk
Attempt to minimize uncertainty
Utility reliant on:
1) Accuracy of analysis
2) Accuracy of model
3) Flow of the day
4) Forecaster experience
5) Random chance
Cost / Return: Mod / Mod
Deterministic vs. Ensemble Forecasting
Ensemble Forecasting
Multiple solutions
Variable and known risk
Attempt to define uncertainty
Utility reliant on:
1) Accounting of analysis error
2) Accounting of model error
3) Flow of the day
4) Machine-to-Machine
5) Random sampling (# of model runs)
Cost / Return: High / High+
The Deterministic Pitfall
The deterministic atmosphere should be modeled deterministically.
A high resolution forecast is better.
A single solution is easier for interpretation and forecasting.
The customer needs a single forecast to make a decision.
A single solution is more affordable to process.
NWP was designed deterministically.
There are many spectacular success stories of deterministic forecasting
Notion Reality
A better looking simulation is not necessarily a better forecast. (precision ≠ accuracy)
Misleading and incomplete view of the future state of the atmosphere.
Poor support to the customer since in many cases, a reliable Y/N forecast is not possible.
Good argument in the past, but not anymore.How can you afford not to do ensembles?
Yes and no. NWP founders designed models for deterministic use, but knew the limitation.
Result of forecast situation with low uncertainty, or dumb luck of random sampling.
Need for stochastic forecasting is a result of the sensitivity to initial conditions.
Event: Satellite drag alters LEO orbits Threshold: Ap > 100
Cost (of preparing): $4.5K
Loss (of reacting): $10K
Hit
FalseAlarm
Miss
CorrectRejection
YES NO
YES
NO
Forecast?
Observed?
$150K $1000K
$150K $0K
Method #1: Decision Theory Minimize operating cost (or maximize effectiveness) in the long run by taking action based on an optimal threshold of probability, rather than an event threshold.
What is the cost of taking action? What is the loss if…
the event occurs and without protection? opportunity was missed since action was not taken?
Good for well defined, commonly occurring events
Example (Hypothetical)
Optimal Threshold = 45%
EF Vision 2020
United Global Mesoscale Ensemble
Runs/Cycle: O(100) Resolution: O(10km) Length: 10 days
Global Mesoscale Ensemble
Runs/Cycle: O(10) Resolution: O(10km) Length: 15 days
Microscale Ensemble
Runs/Cycle: O(10) Resolution: O(100m) Length: 2 days
Global Mesoscale Ensemble
Runs/Cycle: O(10) Resolution: O(10km) Length: 10 days
Global Mesoscale Ensemble
Runs/Cycle: O(10) Resolution: O(10km) Length: 10 daysFNMOC
Microscale Ensembles
Runs/Cycle: O(10) Resolution: O(100m) Length: 24 hours
Microscale Ensembles
Runs/Cycle: O(10) Resolution: O(100m) Length: 24 hours
Coalition Weather CentersGlobal Mesoscale Ensembles
AFWA
JMA ABM
MSC…etc.