1

Kalman filter, analog and wavelet postprocessing in the NCAR-Xcel

operational wind-energy forecasting system

Luca Delle Monachelucadm@ucar.edu

Research Applications LaboratoryNational Center for Atmospheric Research

WORKSHOP ON ENVIRONMETRICS – 15 October 2010, NCAR

2

ACKNOWLEDGMENTS

NCAR: Aime Fournier, Yubao Liu, Gregory RouxUBC: Thomas Nipen, Roland Stull

3

OUTLINE

• Ensemble and Kalman filtering (KF) to improve Numerical

Weather and Air Quality Predictions

• New methods based on KF and an analog approach (ANKF, AN)

• Tests of KF, ANKF, and AN to correct 10-m wind speed

• Wavelet filtering for hub-height winds

• Summary

4

A Kalman filter bias correction for deterministic and probabilistic ozone predictions

• Summer of 2004 (56 days)• 8 photochemical models• 360 ozone surface stations

Sources: Delle Monache et al. (JGR, 2006b)Delle Monache et al. (Tellus B, 2008)Djalalova et al. (Atmospheric Environment, 2010)

5

Timet = 0

day-1day-2day-6 day-5 day-4 day-3day-7

OBS

PRED

KF-weight

KF

6

ii

n

1i

2ii

n

1i

2ii

22

bOaP

)PP(n

1u

RMSE,)OP(n

1s

RMSE,u

RMSEs

RMSERMSE

ˆ

ˆˆ

Ensemble averaging and Kalman Filtering effects on systematic and unsystematic RMSE components

RMSE decomposition (Willmott, Physical Geography 1981):

, a and b least-squares regression coefficients of P i and Oi

7

Kalman Filtering effects on probabilistic prediction reliability

8

Kalman Filtering effects on probabilistic prediction resolution

(6 %) (0.1 %)

9

Timet = 0

day-1day-2day-6 day-5 day-4 day-3day-7

OBS

PRED

KF-weight

KF

ANKFAN

“Analog” Spaceday-4day-7day-5 day-3 day-2 day-1day-6

PRED

OBS

farthest analog

closest analog

NOTEThis procedure is applied independently at each

observation location and for a given forecast time

10

How to find analogs? (1)

We can define a metric:

Where,

Nvar is the number of variable to compute the “distance” between and

wvar is the weight given to each variable while computing the metric

is the standard deviation of the set with 0 ≤ t ≤ t0

is the half-width of the time window over which differences are computedt

var

var

var var 2var

var 1

1( )

i n i n i

N t

tt tt k t kk tf

d f A w f A

varf var{ }tf

ntf itA

11

How to find analogs? (2)

We can define a metric:

tn

tn +1

ti-1

t0 = 0 Timetn -1

ti

ti+1

var

var

var var 2var

var 1

1( )

i n i n i

N t

tt tt k t kk tf

d f A w f A

varf

12

How to find analogs? (3)

We can define a metric:

tn

tn +1

tn -1

ti

ti+1

var

var

var var 2var

var 1

1( )

i n i n i

N t

tt tt k t kk tf

d f A w f A

13

How to find analogs?!? (4)

We can define a metric:

tn

tn +1

tn -1

ti

ti+1

var

var

var var 2var

var 1

1( )

i n i n i

N t

tt tt k t kk tf

d f A w f A

14

WRF Model physics:• Lin et al. microphysics• YSU for PBL• Monin-Oboukov for SL• Kain-Fritsch CUP (Domain 1/2)• Noah Land Surface Model

Modeling settings for wind predictionsProject: Wind energy predictions (Sponsor: Xcel Energy)

D1: 30 km, 128x114 D2: 10 km, 253x232D3: 3.3 km, 541x571NOTE: 37 vertical levels, with 12 levels in the lowest 1-km

D2D1

D3

Source: Delle Monache et al. (Submitted to MWR, 2010)

15

Analog-based methods:improvements relative to KF

16

Wind speed: statistics as function of time

• 6 months period• 500 surface stations• • • Variables: u, v, T, P, Q @ surface

var,1var w1~ t

17

Wind speed: Bias (m s-1) as function of space

18

Sensitivity to data set length • NWP model: MM5• 1 year period• 22 surface stations in NM• • • Variables: u, v, T @ surface

Courtesy of Josh Hacker (of NPS) and Daran Rife (of NCAR)

var,1var w1~ t

19

Wavelet filteringW

avel

et C

om

po

nen

ts

Time SeriesRaw Pred OBS Corrected Pred

Sum

SmoothestComponent

15 min.

30 min.

1 h

2 h

4 h

Wavelet filtering: RMSE and Correlation at Cedar Creek Wind Farm

• 230 days• 4 turbines• 1~ t

• • Variables to search analogs:

spd, dir, T, P, Q @ surface

var,1var w

21

SUMMARY

• Ensemble and Kalman filtering (KF) to improve Numerical Predictions (AQ: ozone, PM; MET: wind, T, Td, sfc pressure, etc.)

• New methods based on KF and an analog approaches (ANKF, AN)

• Test KF, ANKF, and AN to correct 10-m wind speed– ANKF and AN beat KF over a range of metrics– ANKF and AN gain vs KF grows with length of data set

• The combination of ANKF and AN with a wavelet decomposition

improve predictions when dealing with noisy data (@ a wind farm)

22

Thanks!(lucadm@ucar.edu)

23

Source: Djalalova et al. (Atmospheric Environment, 2010)

A Kalman filter bias correction for deterministic and probabilistic PM2.5 predictions

24

• fn is a forecast at time tn and at a given location, with tn > t0

• is a metric to measure the “distance” between fn and Ai

• {Ai} is a set of “analog” forecasts at a time ti, with ti < t0

– {Ai} are ordered with respect to di : di -1 > di , and

We can now introduce the Kalman filter bias correction procedure as follows:

• The true unknown forecast bias at time tn can me modeled by

• And the actual forecast error can be expressed as

KF in analog space (1)

ini Afd

}}{2:,{ iANiNi N

)(,11211 ||

2ttttt ntnnnnn

0,N xx

)(,11

2ttttttttt itiiiiiiiii

0,N xxOAy

~

~

25

The optimal recursive predictor of xt can be written as

Where K, the “Kalman gain” is

And p, the expected mean square error is

NOTE: The system of equations is closed by:

•first running the filter for (with constant)

•

KF in analog space (2)

)ˆ(ˆˆ21121211 ||||

nnnnnnnnn ttttttttt xyKxx

)( 22|

2|

|

1121

121

1

ntntnn

ntnn

nntt

tttt p

pK

)1)((211211 |

2||

nnntnnnn tttttt Kpp

22 r

2

2 2

2 2and

26

“Global” statistics

27

Wind speed: CRMSE (m s-1) as function of space

28

Wind speed: Rank correlation as function of space

29

ANKF…how does it work?

30