Parameter estimation: To what extent can data assimilation techniques correctly uncover...

Parameter estimation:

To what extent can data assimilation techniques correctly uncover

stochasticity?

Jim HansenMIT, EAPS

[email protected]

(with lots of help from Cecile Penland and Greg Lawson)

mailto:[email protected]

Indistinguishable?

Accounting for vs. reducing model inadequacy

• Accounting for model inadequacy– “If you can show me how I can make better forecasts

using chicken bones and voodoo dolls, then I’m going to use them!”

» Harold Brooks, NSSL

– initial conditions (Q-term)– forecasts (MM, stochastic, MOS, forecast 4d-Var)

• Reducing model inadequacy– Making changes to our model so that it becomes a

better representation of the true system– parametric error– structural error

Reducing model inadequacy

• Reducing model inadequacy is best framed as an off-line, or “reanalysis” activity

– The process of attempting to identify model inadequacy tends to make both initial conditions and forecasts worse

– The aim is to quantify how the model is wrong, fix it, and then worry about data assimilation and forecasting

A proposed approach

• Use data assimilation tools to alter model parameters to better fit observations

• Identify relationships between fit parameters and prognostic variables (a parametric MOS)

• Change model to reflect relationships• Repeat

When all relationships have been uncovered, the history of fit parameter values provides a distributionfrom which to (carefully) draw for the purpose ofstochastic parameterizations

Use data assimilation tools to alter model parameters to better fit observations

• Augment control vector with unknown parameters

• Augmentation removes the nonlinearity from the observation operator and inserts it into the specification of the control vector

xx

α

Txx

Tαα

Txα

Tαx

Txx

Augmented control vector sample covariance

Parametric error example: L63

• System equations • Model equations

( )x x y

y rx xz y

z xy bz

( )

0

x x y

y rx xz y

z xy bz

[ ]Tx y z x

Importance of a state-dependent background error covariance

• 4d-Var, static covariance • Ensemble 4d-Var

time time

para

met

er

para

met

er

Structural error example: Lorenz ‘96

22 1 1 1i i i i i ix x x x x x F


System:

Model:

[ ]Tx x

ix

x(1)

par

amet

er

Regressing parametervs. prognostic variablegives:

Alter model equations with new information



System:

Original model:

2 1 1 1 ( )i i i i i i ix x x x x x x F New model:

Example: Lorenz ’96 Model II

2 1 1 1 ,1

Jx

i i i i i i j ij

h cx x x x x x F y

b

, 1, 2, 1, 1, ,y

j i j i j i j i j i j i i

h cy cby y cby y cy x

b

2 1 1 1 ,1

Jx

i i i i i i j ij

h cx x x x x x F y

b

, 1, 2, 1, 1, ,y



b

System:

Model:

2 1 1 1 ( )i i i i i i ix x x x x x F t

[ ]Tx x F

par

amet

er

x(1)

i ixF G

Regressing parametervs. prognostic variablegives:

2 1 1 1 ,1

Jx

i i i i i i j ij

h cx x x x x x F y

b

, 1, 2, 1, 1, ,y



b

System:

Original model:

2 1 1 1 ( )i i i i i i ix x x x x x F t

New model:

2 1 1 1 ( 1)i i i i i ii ix x x x x x Gx

SDE crash course

• The type of calculus used to integrate the stochastic bits of SDEs matters– Stratonovich calculus

• noise process is continuous (typical assumption for geophysical fluid flows)

– Ito calculus• noise process is discrete (like DA!)

• SDEs can be tricky (and expensive) to integrate– used stochastic RK4 (Hansen and Penland, 2005)

What if the system really is stochastic?

• System is an SDE • Model is an ODE

0 ( )

( )

( )

( )

s

dx x y dt

x y dW

dy rx xz y dt

dz xy bz dt

( )

0

x x y

y rx xz y

z xy bz

[ ]Tx y z x

Can DA uncover the correct form of the stochasticity? - NO

• Ensemble 4d-Var • EnKF

para

met

er

para

met

er

time time

10.08, ( ) 0.36std 10.02, ( ) 0.32std

0 10, 0.1s

Why can’t DA uncover the correct form of the stochasticity?

• Stochasticity operating at different time-scales – SDE has infinitesimal time-scale– ODE with DA has 6-hourly time-scale

• System is using Stratonovich calculus, DA is using Ito calculus

• All leads to a danger of misinterpretation

Model error!

1. Deterministic model using constant, tuned parameter value ( )

2. Stochastic model using mean and standard deviation of tuned parameter value ( )

3. Deterministic, multi-model ensemble with parameters drawn from ( )

4. Deterministic model where parameter varies in the same manner as it was estimated ( )

How should we use this information for forecasting?

, ( )std

, ( )std

( )x x y

y rx xz y

z xy bz

Tuned deterministic

, ( )std






, ( )std

, ( )std

Incorrect SDE

, ( )std

( )

( )( )

( )

( )

dx x y dt

std x y dW

dy rx xz y dt

dz xy bz dt






, ( )std

, ( )std

Multi-model

, ( )std

N( )( ), var( )x x y

y rx xz y

z xy bz

where the draw from is held constant over the entireforecast period.

, va )N( r( )






, ( )std

, ( )std

Hybrid

, ( )std

N( )( ), var( )x x y

y rx xz y

z xy bz

where the draw from is made every 6 model hours.

, va )N( r( )

Median of ensemble mean forecast distributions

Tuned deterministicIncorrect SDEMulti-modelHybridPerfect

Nor

mal

ized

RM

SE

Forecast lead (model days)

std(

err/

ens_

std)

Must assess probabilistically!

Tuned deterministicIncorrect SDEMulti-modelHybridPerfect


rela

tive

entr

opy

Relative (to perfect) entropy

Multi-modelHybrid


What if we use a stochastic model?

• System is an SDE • Model is an SDE

0 ( )

( )

( )

( )

s

dx x y dt

x y dW

dy rx xz y dt

dz xy bz dt

0[ ]Tsx y z x

0 ( )

( )

( )

( )

s

dx x y dt

x y dW

dy rx xz y dt

dz xy bz dt

Now can DA uncover the correct form of the stochasticity? - NO

para

met

er s

tdtime

0 10.11 0.26s

0 10, 0.1s

para

met

er m

ean

time

0 and are not uniques

What’s the problem this time?

• Wrong trajectory of random numbers

Model error!

SDE forecast errors

s

0

What does it all mean?• Deterministic model DA approaches alone are not

enough to uncover the correct form of stochasticity– Implies that we cannot attach physical significance to

tuned parameter values or distributions

• Our efforts to reduce model inadequacy ultimately lead to a sensible way to account for model inadequacy

• Synoptic time-scale, Ito-like stochasticity via parameter estimation does a great job accounting for model inadequacy during forecasting

The future(?) of data assimilation

• Model error issues• Nonlinearity• New disciplines: e.g. paleo, climate• Improved image• DA is part of a larger problem


• Nonlinearity– Implementing nonlinear approaches– Extend minimum error variance approaches a bit

more into the nonlinear regime• Feature-based non-Gaussianity


• Improved image– DA has a bad/boring reputation– Ensemble methods bringing DA to the masses

• University research can be quasi-operational• Reasonable DA now where none before

ATMOSATMOS

COLLEGECOLLEGE


• Improved image– DA has a bad/boring reputation– Ensemble methods bringing DA to the masses

• University research can be quasi-operational• Reasonable DA now where none before


• DA is part of a larger problem– The future of DA is not independent of the future of

observations, ensemble forecasting, verification, calibration, etc..

• Ensemble forecasting• Targeting• Increasing ensemble forecast size at low cost• Ensemble synoptic analysis

Transformed Lag Ensemble Forecasting (TLEF)

• Ensemble size is increased by using ensemble-based data assimilation techniques to transform (scale and rotate) old forecasts using new observations.

Time


• DA is part of a larger problem– The future of DA is not independent of the future of

observations, ensemble forecasting, verification, calibration, etc..

• Ensemble forecasting• Targeting• Increasing ensemble size at low cost• Ensemble synoptic analysis

HakimandTorn

WRF,100

ensemble members,

surface pressure

obs

HakimandTorn

Ertelaa

Ertel pvAxAXPV 1

Ensembles make PV inversion fun and easy!

Note, no worries about balance assumptions or boundary conditions

Approach defined by

HakimandTorn


• Model error issues• Nonlinearity• New disciplines: e.g. paleo, climate• Sales/Marketing• DA is part of a larger problem

Parameter estimation: To what extent can data assimilation techniques correctly uncover...

Documents

Transcript of Parameter estimation: To what extent can data assimilation techniques correctly uncover...