Error Bars in Normal Distributions. Error Bars in Column / Bar Graphs .
Modelling error distributions for NWP Data...
Transcript of Modelling error distributions for NWP Data...
© Crown copyright Met Office
Modelling error distributions for NWP Data AssimilationMPE CDT Workshop: “Stochastic Modelling in GFD, data assimilation, and non-equilibrium phenomena”
Andrew Lorenc, Imperial College 2-6 November 2015.
Abstract
Nearly all operational Numerical Weather Prediction (NWP) systems base their data assimilation systems in principle on BayesTheorem, however the prior comes from past data using a forecast model that is very large, complex, chaotic and erroneous.We can only afford one “best estimate” (or a small ensemble of forecasts) with such a model so the error distribution of thebest estimate forecast is unknowable. We still need to perform data assimilation and issue an operational forecast, so weproceed with the Bayesian methodology anyway, making modelling assumptions about the error PDF. The Gaussianapproximation is practically essential if we are to model the PDF for a large forecast with about a billion degrees of freedom;without it we have no hope of even sampling its billion-dimensional space.
Bayes Theorem and a Gaussian assumption for PDFs leads us to the extended Kalman filter equations. Even these areunaffordable for an NWP system since they require the propagation and inversion of non-sparse matrices of order a billion.
Further assumptions about balance relationships, and stationarity in time space and direction, are often invoked. A recent
© Crown copyright Met Office Andrew Lorenc 2
Further assumptions about balance relationships, and stationarity in time space and direction, are often invoked. A recentfocus has been the derivation of PDF information from an ensemble of forecasts. Although the basic Kalman filter can beapplied sequentially, these additional approximations can introduce inaccuracies in the assimilation of observations which arenear in space but not processed together - in this case the resulting approximate analsis need not fit accurate observations asit should. For this and more practical reasons operational NWP processes observations in batches each spanning a time-window (typically 6 hours for global NWP). Within each window, we get benefit from considering the observation times, i.e. wedo an approximate Kalman smoother. Different approaches are presented for making this computationally feasible - hybrid-4DVar and hybrid-4DEnVar. The former uses a linear forecast model and its adjoint to construct the time-dimension of the PDFwhereas the latter uses the time-evolution of the ensemble trajectories. Between batches we use the same methods as anensemble Kalman filter to generate an ensemble of forecasts sampling the PDF. I show some recent results demonstrating thebenefit of using improved ensembles in these methods.
In order to estimate covariances from a small (40~200) ensemble our hybrid-4DVar and hybrid-4DEnVar employ localisationsimilar to that in ensemble Kalman filter (EnKF) methods. The ensemble-Var methods allow a much more generalinterpretation of “localisation”, for example in spectral-space and time, as well as hybridisation with more stationary“climatological” estimates (these are much harder in the EnKF). Some examples of localised covariances from an NWP modelare shown.
© Crown copyright Met Office Andrew Lorenc 3
© Crown copyright Met Office Andrew Lorenc 4
Schlatter’s (1975) multivariate covariances
Specified as multivariate 2-point functions.
Not easy to ensure that specified functions are
© Crown copyright Met Office Andrew Lorenc 5
functions are actually valid covariances.
Used in OI and related observation-space methods.
© Crown copyright Met Office Andrew Lorenc 6
© Crown copyright Met Office Andrew Lorenc 7
3D-Var + FGAT (mean 0.26%)
synoptic 4D-Var (mean 0.62%)
basic 4D-Var (mean 0.85%)
Improvement in RMS verification v obscompared to Basic 3D-Var
synoptic
basic
4D-Var
Evolved
Obs -
increment time
Cumulative mean improvements:
Page 8Andrew Lorenc, 4th WMO DA Symposium, Prague, April 2005, Lorenc and Rawlins(2005). 4D-Var was implemented Oct 2004 (Rawlins et al. 2007).
-5% -4% -3% -2% -1% 0% 1% 2% 3% 4% 5%
3D-Var
+ FGAT
synoptic
4D-Var
Obs –guess time
Evolved
covariances
Met Office 4DVar v 3DVar results Lorenc et al. (2014)
Our first trial copied settings from the hybrid-4DVar:
• C with localisation scale 1200km, • hybrid weights βc
2=0.8, βe2=0.5
Results were disappointing:
hybrid-4DVar
The reason was the large weight given to the climatological covariance, which is treated like 3DVar in 4DEnVar
© Crown copyright Met Office Andrew Lorenc 9
hybrid-3DVar = hybrid-3DEnVar
hybrid-4DVar 3.6% better
hybrid-4DEnVar 0.5% better
© Crown copyright Met Office Andrew Lorenc 10
© Crown copyright Met Office Andrew Lorenc 11
© Crown copyright Met Office Andrew Lorenc 12
Hybrid-4DVar
• Adding ensemble covariances from the MOGREPS system to 4DVar gave ~1% improvement (Clayton et al. 2013)
• But we do not want to rely on 4DVar :
• Increasingly expensive on MPP computers;
• We are already developing a- New dynamical core (GungHo)- In a new software system (LFRic).
• Nevertheless current hybrid-4DVar + ENDGAME model form a world-class system which will be good for ~5 years.
© Crown copyright Met Office Andrew Lorenc 13
u increments fitting a single u ob at 500hPa, at different times.
4D-Var
Unfilled contours show T field.
Clayton et al. 2013at end of 6-hour
windowat start of window
Hybrid 4D-Var
© Crown copyright Met Office Andrew Lorenc 2015 14
Hybrid-4DVar(Operational at the Met Office since July 2011)
• Background error covariance at beginning of windowimplicitly propagated by a linear “Perturbation Forecast” (PF) model:
f
ee PCB o=
Spatial localisationcovariance
Raw ensemblecovariance
(MOGREPS-G, based on localised ETKF. Currently 44 members.)
eecc BBB22 ββ +=
Climatologicalcovariance
Ensemblecovariance
© Crown copyright Met Office Andrew Lorenc 2015 15
© Crown copyright Met Office Andrew Lorenc 16
An new form of linear model
© Crown copyright Met Office Andrew Lorenc 18
0
Test with a single wind ob, in a jet, at the start of the window
3 6
ob
100% ensemble1200km localization scale
4DEnVar
© Crown copyright Met Office Andrew Lorenc 19
En-4DVarerror
© Crown copyright Met Office Andrew Lorenc 20
50-50% hybrid1200km localization scale
4DEnVar
© Crown copyright Met Office Andrew Lorenc 21
4D-Var
100% climatological B
4DEnVar≡3DVar
© Crown copyright Met Office Andrew Lorenc 22
4D-Var
100% ensemble500km localization scale
4DEnVar
© Crown copyright Met Office Andrew Lorenc 23
4D-Var
Hybrid-4DEnVar(coded for testing, based on hybrid-4DVar)
• No PF model, but much more IO required to read ensemble data:
• 11 times faster with 22 N216 members and 384 PEs. (IO around 30% of cost)
• Analysis consists of two parts:
• A 3DVar-like analysis based on the climatological covariance Bc
• A 4D analysis consisting of a linear combination of the ensemble perturbations.
• Localisation is currently in space only: same linear combination of ensemble perturbations at all times.
Met Office trial of 4DEnVarLorenc et al. (2014)
Our first trial copied settings from the hybrid-4DVar:
• C with localisation scale 1200km, • hybrid weights βc
2=0.8, βe2=0.5
Results were disappointing:
hybrid-4DVar
The reason was the large weight given to the climatological covariance, which is treated like 3DVar in 4DEnVar
© Crown copyright Met Office Andrew Lorenc 25
hybrid-3DVar = hybrid-3DEnVar
hybrid-4DVar 3.6% better
hybrid-4DEnVar 0.5% better
© Crown copyright Met Office Andrew Lorenc 26
© Crown copyright Met Office Andrew Lorenc 27
Bowler et al. (2015b) trials of hybrid covariances in DA
© Crown copyright Met Office Andrew Lorenc 28
© Crown copyright Met Office Andrew Lorenc 29
Exploring the implicit covariances
© Crown copyright Met Office Andrew Lorenc 30© Crown copyright Met Office Andrew Lorenc 2015 30
Climatological covariances of ualong a line from N to S poles.
© Crown copyright Met Office Andrew Lorenc 31
Ensemble (K=44) cov. of ualong a line from N to S poles.
© Crown copyright Met Office Andrew Lorenc 32
Ensemble (K=308) cov. of ualong a line from N to S poles.
© Crown copyright Met Office Andrew Lorenc 33
Localised ensemble (K=44) cov. of ualong a line from N to S poles.
© Crown copyright Met Office Andrew Lorenc 34
Scale-dependent localised ensemble (K=44) cov. of ualong a line from N to S poles.
© Crown copyright Met Office Andrew Lorenc 35
Scale-dependent and spectrally localised ensemble (K=44) cov. of ualong a line from N to S poles.
© Crown copyright Met Office Andrew Lorenc 36
Wavebands used for spectral and scale-dependent localisation
0.8
1
wb1: L=80063km wb2: L=2859km wb3: L=690km wb4: L=600km
© Crown copyright Met Office Andrew Lorenc 37
0
0.2
0.4
0.6
0 20 40 60 80 100 120 140 160 180 200 220
GLOBAL WAVENUMBER
Ensemble (K=44) variance of u
© Crown copyright Met Office Andrew Lorenc 38
Ensemble (K=308) variance of u
© Crown copyright Met Office Andrew Lorenc 39
Localised ensemble (K=44) variance of u
© Crown copyright Met Office Andrew Lorenc 40
Scale-dependent localised ensemble (K=44) variance of u
© Crown copyright Met Office Andrew Lorenc 41
Scale-dependent and spectrally localised ensemble (K=44) variance of u
© Crown copyright Met Office Andrew Lorenc 42
Summary
• The error distribution for a large complex NWP model is unknowable – nevertheless we try to model it so that we can use Bayesian DA methods.
• Physical understanding and a long history of operational forecasts enables us to build plausible models of the structure of errors, especially for their time-evolution.
• Increasingly important is the information from ensembles of forecasts. With appropriate localisation, a useful estimate of error covariances can be estimated from ensembles much smaller than the degrees of freedom being analysed.
© Crown copyright Met Office Andrew Lorenc 43
© Crown copyright Met Office
Questions and answers
© Crown copyright Met Office Andrew Lorenc 45
© Crown copyright Met Office Andrew Lorenc 46