Data assimilation in mesoscale modeling and numerical weather

predictionNils Gustafsson

Croatian – USA Workshop on Mesometeorology – June 2012

Perspective:What are the important issues for

development of operational data assimilation when we are in the move:

From hydrostatic models with a grid resolution of 5-20 km (HIRLAM)

To non-hydrostatic models with a grid resolution of 1-3 km (HARMONIE)

Taking also the needs for probabilistic forecasting into account ??

+ work in a wider community (HIRLAM, ALADIN, Meteo-France + use of the IFS-code (ECMWF)

Outline of talk

Relative importance of different model variables in the initial conditions

Candidate assimilation methods Host model coupling issues Moisture assimilation Radar data assimilation Initialization Surface assimilation

OUTCRY: ‘‘There are too many types of data, too many new ones appearing every year, and there are too few people to work on their assimilation!’’

Could there be a way to prioritize data assimilation efforts?

Importance of different model variables in the initial conditions – convection permitting scales

Questions (for mesoscale foreasting of

convection in this study): What is the relative importance of different model

variables in the initial conditions? What is the time-scale for how perturbations in one model

variable influence other model variables? (Tadjust

)

What is the time-scale for non-linearities to have important effects for different model variables? (T

nl)

What is the “signal strength” of different instruments measuring different variables?

Important: Tadjust

< assimilation window < Tnl

Fabry & Sun:

Approach and tools Identical twin model simulation

experiments Perturb initial data of different variables,

check how perturbations influence other variables

WRF, 4 km grid resolution Use different amplitudes of initial

perturbations, check for how long time linearity holds

Fabry & Sun:

Linear, non-linear and contradictory regimes

Fabry & Sun:

Rainfall accumulation from 16 control cases

Fabry & Sun:

Horizontal spectra and variances of simulated initial errors

Fabry & Sun:

Experimentation strategy

Fabry & Sun

RESULTS – error growthFabry & Sun:

Error spread to different model variables 3 hours

15 minutes

Fabry & Sun:

Non-linearity index Contradictory index

Fabry & Sun:

What can we learn from Fabry & Sun?

Exept for low-level moisture and soil properties, most initial condition errors propagate quickly to other variables. Uncertainties in mid-level moisture caused the greatest uncertainties in the forecasts. For mesoscale forecasts longer than 6h, the uncertainties in all other variables had comparable effects. For forecasts of a few hours, uncertainties in temperatures, low-level moisture and mid-level winds stood out as the greatest cause of forecast uncertainties after mid-level moisture. There are limits on the assimilation interval that are different for different variables, in particular condensates have shorter predictability and can be used with shorter windows only.

Remember: Identical twin experiment !!!

The assimilation method

• Mesoscale balance constraints, in particular involving moisture? Analytical or statistical constraint? Utilization of ensembles?

• How to handle inhomogeneities and an-isotropy? • Flow-dependent structure functions? 4D-Var, ensemble

techniques or synthesis?• How to handle model errors?

Candidate data assimilation techniques for the mesoscale

Variational techniques (3D-Var and 4D-Var) Ensemble Kalman Filters Hybrid variational ensemble methods Nudging (will not be discussed) Warping or alignment to correct for phase errors

Basic idea of 4D-Var: Fit model trajectory to observations over a time window

4D-Var

Possibilities:

• Handle weak non-linearities via outer loops

• Provide flow-dependencies within

assimilation window

Problems:

• Strong non-linearities

• Static influence of observations at the start of the window

• Maintain tangent linear and adjoint codes

• Scalability on parallell computers

Comparison of HIRLAM 3D-Var and 4D-Var for the stormy month of December 1999

(Gustafsson et al. 2012)

The Danish storm, 3 December 1999 18 UTC

3D-Var+30h

4D-Var +30 h

3D-Varanalysis

4D-Varanalysis

Gustafsson et al. 2012)

Why does 4D-Var make a better job than 3D-Var in storm cases?

Improved implicit flow-dependent structure functions. Better quality control decisions due to a better

background and better implicit structure functions Use of more observation? In general, it is difficult to point to a specific reason

for improvements, the improvements generally occur

gradually through the assimilation cycles

Assimilation increments

3D-Var06UTC

3D-Var12UTC

4D-Var06UTC

4D-Var12UTC

Surface pressure increments for the Danish mesoscale storm

(Single simulated observation experiment)

3D-Var

4D-Var, spectral TL prop. of incr

4D-Var; gp model prop. of incr.

Mesoscale data assimilation and the use of ensembles

• Will assumptions on weak non-linearities at 10 km resolution break down at the km scale ? Most likely yes !

• Can ensembles provide information on uncertainties and balances at the mesoscale ? Most likely Yes !

• Is it too risky to put all our efforts on 4D-Var, that works on the 10 km scale but may fail at the km scale ? Yes !

• Should we drop our investments in variational data assimilation and start with ensemble Kalman Filters or even particle filters? No, EnsKF and particle filters still have weaknesses !

• Can we develop hybrid data assimilation schemes that combine the best of the two worlds? We believe so !

Synoptic variation of covariances as calculated from 3 days of 12h minus 36 h forecast differences

20060116-20060118

20060116-20060118

500 hPa geopotential height

Wavelet representation of 500 hPa temperature covariances

From Landelius

Ensemble Kalman Filters

Basic idea: Estimate forecast error covariances from an ensemble of forecasts

Ensemble generation:Perturbed observationsModel physics perturbationsModel tendency perturbations

Advantages:Flow-dependenciesNatural together with EPS

Problems:50-100 members needed; CostlyLow rank of covariance matrix (ensemble size) makes some type of localization necessary

From Whitaker

HIRLAM first approach to use ensembles in 3D-Var and 4D-Var

• Use the ETKF algorithm for re-scaling of a 6h forecast ensemble to an analysis ensemble (estimation of the analysis error covariance).

• Use ensemble of 6h forecasts to estimate the background error covariance and blend it with the static background error covariance.

First version of HIRLAM implementation

Ensemble weights a have horizontal variations only (horizontal localization) and is controlled in spectral space with the assumption of isotropy – this is equivalent to a horizontal localization of covariances with a Schur product based on a horizontal correlation function. The localization is applied for vorticity, divergence, temperature, surface pressure and specific humidity for better balancing.

Forecast verification scores for a first winter case

(12 ensemble members only)

3D-Var (red curve)Hybrid (green curve)Hybrid + tuning of obs. error s.dev. (blue curve)

Difficult to show the same improvement for the hybrid over 4D-Var!

Model level around 800 hPa

Notice increments along a front

Example of assimilation increments (temperature and wind)

(Landelius et al., 2011)

Landelius et al. 2011)

Host model coupling issues

Large scales are poorly assimilated on a small regional domain – one may wish to handle these large scales differently

In case lateral boundary conditions are coming from an earlier host model run, one may wish to update also the inner domain when new LBC conditions are available

One may also wish to control lateral boundary conditions in a mesoscale 4D-Var

Large scale error constraint in 3-4D-Var

(from Dahlgren and Gustafsson, 2012)

Add large-scale constraint to 4D-Var cost function:

Large scales of vorticity are constrained by a short range host model forecast at the start of the 4D-Var assimilation window.

Impact of Jk on surface pressure forecast verification scores

(Dahlgren and Gustafsson, 2012)

WHY?+0h

+6h

In case an object that we want to assimilate is outside the lateral boundaries at the start of the assimilation window, but described by observations inside the lateral boundaries later during the assimilation window, we must control it by lateral boundary conditions. Let it come in!Similarly observations may tell us to let objects go out even if this is not described by the host model LBCs.

Control of lateral boundary conditions

Control of Lateral Boundary Conditions

(1) Introduce the LBCs at the end of the data assimilation window as assimilation control variables (full model state = double size control vector)(2) Introduce the adjoints of the Davies LBC relaxation scheme and the time interpolation of the LBCs(3) Introduce a “smoothing and balancing” constraint for the LBCs into the cost function to be minimized

J = Jb + Jo + Jc + Jlbc

whereJlbc = (Xlbc- (Xlbc)b)T B-1 (Xlbc- (Xlbc)b)

And, as a first approximation, B is taken as identical to B for the background constraint

Single simulated observation impact studyWith control of lBC

Simulated observation SW 9 m/s at 32N 12W3Dec 1999 11 UTC

3 December 1999Assimilation window06UTC – 11UTC

Strong SW inflow in the background field

06 UTC

09 UTC

11 UTC

(Gustafsson, 2012)

Specific problems in assimilation of moisture

Non-Gaussian behavour of water vapour close to zero humidity and saturation

Flow-dependent structures, for example in “rain” and “no-rain” regimes

Sharp vertical gradients (inversions, PBL top)

Handling of model errors (biases)

Effects of a model dry bias (700 hpa RH)

A new moisture control variable (Elias Holm)

Pseudo-relative humidity:

Normaliztion with a standard-deviation depending also on the increment (non-linear transform):

Effect on PDF of new moisture control variable

Standard deviation of relative humidity forecast differences as a function of “background” relative

humidity

Effect on assimilation increments near saturation

Q control variableRH* control variableRH* control variable + relinearization

BUT, no effect on forecast verification scores!

Simulated observation experiments

Shown to be very important for assimilation of radar data in a mesoscale model (AROME)!

From Montmerle & Berre

Smoothing effect of climatological vertical correlations

Single observation experiment with HIRLAM 4D-Var Cloud-affected SEVIRI IR data (water vaour channel) Observation operator includes diagnosis of clouds

(ECMWF simplified physics) and radiative transfer Elias Holm moisture control variable

From Stengel et al. (2012) in preparation

Cloud layer

Jacobians IncrermentsWithout SP With SP

q

T

Impact of clear SEVIRI radiances on forcast scores (relative RMS reduction)

Additional impact of cloud-affected radiances

Assimilation of radar data

As an example, the approach to assimilation of radar reflectivity data within the HIRLAM and ALADIN communities are briefly described.

Meteo-France has been the leading partner (Caumont, Ducrocq, Wattrelot et al.).

HIRLAM and ALADIN have ongoing trials to collect, pre-process and assimilate radar data from an extended European network.

The slides have been “borrowed” from a presentation by Caumont et al. (2006) and Mahfouf (2011)

From Caumont et al. (2006)

From Mahfouf (2011)

From Mafouf (2011)

From Mahfouf (2011)

Weak digital filter constraint

Try to come close to the “slow manifold” at the same time as minimization of Jo helps us to come close to the observations! Note the un-known coefficient γ

df.

Does this help to prevent “noise” in the non-linear model integration?

Horizontal average of the absolute value of the surface pressure tendency.

Scale-selective DFI

(Termonia, 2008)

Also: Scale-selective DFI => less “noise”!

Validation and tuning of HARMONIE surface assimilation

1. Validate the present OI scheme using pre-calculated coeffficients for the soil variables2. Compare the performance of the new EKF scheme for soil variables with the OI scheme – check that everything is correctly implemented3. Tune if needed the EKF scheme (beat the OI scheme !)4. Use of satellite data (ASCATT and SMOS)?

Work started with 1-3.

Example of time-series in SYNOP stations - Gardermoen

Without OI With OI

Reduced wg2

Improveddaily cycle!

Improved RH2

Verification – T2 and RH2

Verification – PMSL !!

Summary – Priority areas for HIRLAM, ALADIN in mesoscale DA

Use of high-resolution observations: - Radar reflectivity and radial winds; - Cloudy radiances; - Ground-based GPS; - Low-level winds (10 met, scattereometer)

Flow-dependencies: - Heterogenious structure functions (rain-no rain, cloud-no cloud); Hybrids; 4D Ens Var; 4D-Var (as reference); – Alignment, warping and dynamical transform;

Balances: - Flow-dependent structure functions; - Mesoscale DF; – Spinup (consistency between DA and physics)

Improve large scales: Jk & Control of LBC in 4D-Var

Surface assimilation: - EKF (=> flow and model dependency); – satellite observations; snow, lakes; - interpolation for heterogenious surfaces

Meteorologisk institutt met.no

powerfulLAM EPS

efficientDA scheme

skillfulcontrol forecast

realisticLAM perturbations

Solution for long term perspective

Transversary issue: interaction of initialisation/dynamics/physics/model error

Greetings from my student Lena Bojarova, 2012