ECMWF | Advancing global NWP through international ... · The ensemble transform Kalman filter...

��

��

��

S.J. Majumdar1, S.D. Aberson2, C.H. Bishop3, R. Buizza4, M.S. Peng3

and C.A. Reynolds3

Research Department

1. RSMAS Division of Meteorology and Physical Oceanography, University of Miami. 2. NOAA/AOML Hurricane Research Division.

3. Naval Research Laboratory, Monterey. 4. European Centre for Medium-Range Weather Forecasts.

December 2005

To be published in Mon.Wea.Review

Series: ECMWF Technical Memoranda A full list of ECMWF Publications can be found on our web site under: http://www.ecmwf.int/publications.html [email protected] © Copyright 2005 European Centre for Medium Range Weather Forecasts Shinfield Park, Reading, Berkshire RG2 9AX, England Literary and scientific copyrights belong to ECMWF and are reserved in all countries. This publication is not to be reprinted or translated in whole or in part without the written permission of the Director. Appropriate non-commercial use will normally be granted under the condition that reference is made to ECMWF. The information within this publication is given in good faith and considered to be true, but ECMWF accepts no liability for error, omission and for loss or damage arising from its use.

A comparison of adaptive observing guidance for Atlantic tropical cyclones

Technical Memorandum No.482 1

Abstract Airborne adaptive observations have been collected for more than two decades in the neighborhood of tropical cyclones, to attempt to improve short-range forecasts of cyclone track. However, only simple subjective strategies for adaptive observations have been used, and the utility of objective strategies to improve tropical cyclone forecasts remains unexplored. Two objective techniques that have been used extensively for mid-latitude adaptive observing programs, and the current strategy based on the ensemble deep-layer mean (DLM) wind variance, are compared quantitatively using two metrics. The ensemble transform Kalman filter (ETKF) uses ensembles from NCEP and ECMWF. Total-energy singular vectors (TESVs) are computed by ECMWF and the Naval Research Laboratory, using their respective global models. Comparisons of 78 guidance products for two-day forecasts during the 2004 Atlantic hurricane season are made, on both continental and localized scales relevant to synoptic surveillance missions.

The ECMWF and NRL TESV guidance identifies similar large-scale target regions in 90% of the cases, but are less similar to each other in the local tropical cyclone environment (56% of the cases) with a more stringent criterion for similarity. For major hurricanes, all techniques usually indicate targets close to the storm center. For weaker tropical cyclones, the TESV guidance selects similar targets to those from the ETKF (DLM wind Variance) in only 30% (20%) of the cases. ETKF guidance using the ECMWF ensemble is more like that provided by the NCEP ensemble (and DLM wind variance) for major hurricanes than for weaker tropical cyclones. Minor differences in these results occur when a different metric based on the ranking of fixed storm-relative regions is used.

1. Introduction

Tropical cyclones (TCs) spend the majority of their lifetimes over the ocean, in regions of relatively sparse data coverage. Given the high human and economic losses commonly incurred from TCs and the fact that poor initial conditions in numerical models may lead to large forecast errors, it is useful to supplement the routine observational network with adaptive in-situ and remotely-sensed observations in ‘sensitive’ areas to improve numerical forecasts. At the time of writing, the only adaptively deployed observations are Global Positioning System (GPS) dropwindsondes, released primarily in the Atlantic basin during synoptic surveillance missions conducted by the National Oceanic and Atmospheric Administration’s (NOAA) Gulfstream-IV aircraft (see Aberson 2003 for a history of adaptive observing in TCs), and in the north-west Pacific basin (Wu et al. 2005). The value of adaptive observations has been demonstrated for TC forecasts (e.g. Franklin and DeMaria 1992, Burpee et al. 1996, Aberson 2002, 2003) and for winter weather forecasts (e.g., Montani et al. 1999, Langland et al. 1999, Szunyogh et al. 2000).

NOAA’s currently operational aircraft deployment strategy is based subjectively on a combination of uniform sampling around the TC, and sampling of areas of large variance of the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) mass-weighted deep-layer mean (DLM, 850-200hPa) wind ensemble forecast, valid at the observing time (Aberson 2003). The dropwindsondes are typically released within 333-1500km of the storm center. This strategy does not account for the propagation and growth of errors between the adaptive observing time and the verification time for which the forecast is to be made (Fig. 1). Hence, while Aberson (2003) and previous authors have demonstrated that adaptive observations have reduced TC track errors of 12-60h forecasts by up to 30%, one might expect further improvements if the adaptive observing strategy specifically aims to curtail errors in these forecasts.

Objective adaptive observing techniques have been developed, based upon this premise. Two methods, the ensemble transform Kalman filter (ETKF, Bishop et al. 2001) and Singular Vectors (SVs, Palmer et al. 1998) have been utilized during field programs in the mid-latitudes, including the Fronts and Atlantic Storm-Tracks Experiment (FASTEX, Joly et al. 1999), the North Pacific Experiment (NORPEX, Langland et al. 1999), and the Atlantic THORPEX Observing Systems Test (ATOST, Langland 2005). The ETKF has also been used during annual operational National Weather Service Winter Storm Reconnaissance (WSR) Programs since


2 Technical memorandum No.482

2001 (Szunyogh et al. 2002, Majumdar et al. 2002a). Guidance provided by the ETKF and total-energy Singular Vectors (TESVs) was compared by Majumdar et al. (2002b) for 10 NORPEX winter weather forecasts. The ETKF utilized the global 51-member ensemble from the European Centre for Medium-Range Weather Forecasts (ECMWF), whereas two sets of TESV guidance based on the ECMWF and Navy Operational Global Atmospheric Prediction System (NOGAPS) models were used. Based on two metrics, the ETKF and TESV guidance were found to exhibit similar characteristics on large scales, but small-scale aspects often differed considerably. The target regions often corresponded to baroclinic zones in which upper-level wave amplification occurred, or regions of large low-level vorticity. Further insights into the dynamical meaning of SVs in the mid-latitudes are given by Buizza & Palmer (1995), Hoskins et al (2000), Reynolds et al. (2001) and Morgan (2001). Insights into the evolution of ETKF reduction in forecast error variance and its relation to ensemble perturbation dynamics are given in Majumdar et al. (2002a).

Figure 1 Timeline for adaptive sampling. In order to improve a 48h forecast of the hurricane track between the future analysis (observing) time ta and a future verification time tv, trajectories and ensembles initialized at time ti are used in the ETKF and TESV techniques respectively. The forecast location of Hurricane Ivan valid at (i) ti = 00 UTC 08 September 2004, (ii) ta = 00 UTC 10 September 2004 and (iii) tv = 00 UTC 12 September 2004 is shown by the respective hurricane symbols, based on the ECMWF model initialized at ti. The forecast verification region valid at time tv is denoted by the rectangular box. The relevant times for all cases are given in Table 1 and the Appendix.

The current versions of the ETKF and TESV techniques have recently been tested for use with TCs. They are based on dry energy metrics and the assumption that the error dynamics are linear, and have focused on the environmental flow around the TC. The assumptions in both techniques may be more severely compromised in the tropics, compared with the mid-latitudes, since non-linear processes may dominate. Peng and Reynolds (2005b) found that dry NOGAPS TESVs applied to typhoons highlighted both local and remote influences on the storm evolution. The ETKF has been tested during the 2004 hurricane season as an alternative operational method for deploying the NOAA G-IV aircraft, with support from the NOAA Joint Hurricane Testbed (JHT). Preliminary evaluations of forecast improvements based on data collected in regions deemed sensitive by the ETKF during 2004 are encouraging, and will be reported in the near future.

Presently, little is understood about the consistency or physical meaning of guidance provided by objective adaptive sampling techniques for TCs. The purpose of this paper is to compare 5 guidance products based upon three different techniques – DLM Wind Variance (NCEP), ETKF (NCEP and ECMWF) and TESV (NOGAPS and ECMWF) – for 78 cases of 2-day TC forecasts in the Atlantic Basin during the 2004



hurricane season.. Although this paper is philosophically similar to Majumdar et al. (2002b), it is broader in scope. No such comparison has previously been attempted for TCs, and the much higher number of cases should lead to statistically significant conclusions. Comparisons are made on scales covering the Atlantic basin and continental North America, to address the issues of where extra satellite-borne (or other) data may be important over the oceans, and whether additional observations over land may be useful. Comparisons are also made on scales local to the TC to identify the consistency between the respective guidance products from a synoptic surveillance perspective. Detailed insights into the respective targets will be presented in a future paper.

The theoretical basis of the DLM wind variance, ETKF and SV techniques, and their similarities and differences are described in Section 2. Guidance maps for two hurricane forecasts are presented in Section 3. A quantitative comparison of guidance from the respective techniques on large and local scales is performed in Section 4. Concluding remarks follow in Section 5.

2. Adaptive Observing Techniques

The 5 types of adaptive observing guidance for TCs are summarized in Table 1. The ensemble DLM wind variance only considers the targeted observing (analysis) time, denoted by ta, whereas the ETKF and TESV techniques consider error propagation from time ta into a given forecast verification region at a verification time tv, 2 days after ta (Fig. 1). All sets of guidance use ensembles (ETKF/Variance) or non-linear

trajectories (TESVs) initialized at time ti, at least 48h prior to ta. This value of ta-ti is selected to be consistent with the logistics of planning synoptic surveillance missions, in which a decision on aircraft deployment is

required at least 36h prior to ta.

Technique VARIANCE TESV TESV ETKF ETKF

Model NCEP GFS ECMWF NOGAPS ECMWF NCEP GFS

# ens / SVs 10 ens 10 SVs 3 SVs 50 ens 20 ens

NLM resolution T126 L28 TL95 L60 T239 L30 TL255 L40 T126 L28

Resolution of output 1° TL95 L60 T79 L30 1o 1o

tv-ta N/A 48h 48h 48h 48h

ta-ti 48h 48h 48h 60h 48/60h

Perturbations Computed about

48h NCEP Control

48h ECMWF Control

48h NOGAPS Control

60h ECMWF Ens Mean

48h NCEP Ens Mean

Table 1: Summary of 5 guidance products used in comparison. The abbreviation “ens” refers to the number of members in the ensemble. For the SV techniques, resolution of output is also the resolution of the singular vector calculation. “NLM” refers to “non-linear model”.

2.1 Ensemble Variance of Deep-Layer Mean Wind

Given that operational data assimilation schemes use (i) a synthetic or relocated vortex if necessary and (ii) localized, quasi-isotropic error covariance information, the priority for synoptic surveillance missions has been to distribute observations uniformly around the TC. In recent years, extensions beyond uniform sampling have been tested. Aberson (2003) demonstrated that the subset of observations collected in

locations of high NCEP GFS ensemble DLM Wind Variance valid at ta improved TC track forecasts more than uniformly sampled observations. Areas of large variance imply that the uncertainty in the DLM wind

analysis at ta will likely be high, unless observations are assimilated in these areas. The DLM ensemble variance technique is based on this premise; however, it may or may not necessitate a reduction in forecast

uncertainty within a selected verification region at future time tv. On the one hand, large DLM ensemble variance may amplify and propagate into the verification region, e.g. if the TC track forecasts diverge. On



the other hand, ensemble variance may diminish or propagate to locations outside the verification region. In such instances, removing the error variance associated with the ensemble variance may have little effect on forecast accuracy within the verification region. Errors and ensemble perturbations decay when their structures lose energy to the background flow. Perturbation deformation by the large-scale flow and/or non-linear saturation are typical mechanisms by which ensemble perturbations and errors decay (Bishop 1993a,b). Hence, an adaptive observing strategy to improve a given forecast ought to select structures at time

ta that most significantly reduce the forecast error variance within the verification region at time tv. The techniques described in the next two sub-sections attempt to account for this.

2.2 Ensemble Transform Kalman Filter (ETKF)

The current version of the Ensemble Transform Kalman Filter (ETKF, Bishop et al. 2001) is similar to that run operationally at NCEP (Majumdar et al. 2002a) during Winter Storm Reconnaissance Programs1. The

ETKF uses operational ensemble forecast perturbations (initialized at time ti and listed in matrix Zi) to predict the reduction in forecast error variance produced by targeted observations. An ensemble perturbation

is defined here as the difference between an ensemble member and the mean over all KENS+1 ensemble

members. First, the analysis error covariance matrix Pr(ta) pertaining to the routine observational network comprising rawinsondes and satellite-based temperature fields is found by solving the Kalman Filter error statistics equations

Pr(ta) = Pi(ta) - Pi(ta) H

rT (Hr Pi(ta) HrT + Rr)-1 Hr Pi(ta), (1)

where Hr and Rr are the observation operator and error covariance matrices, respectively, for the routine

observational network and Pi(ta) = Zi(ta) ZiT(ta) gives the ensemble-based estimate of forecast error

covariance valid at the targeted observing time. This new update of Pr(ta) has been used during WSR since 2003, and is different to that of Majumdar et al. (2002a, b). It gives a theoretically superior estimate by reducing (increasing) the ensemble variance in well-observed (data-sparse) locations, as demonstrated by the theoretically identical ensemble initialization technique of Wang and Bishop (2003). However, it does not yet account for routine observations collected at intervening times between ti and ta. Note also that the DLM

wind variance can be written as Tr(Pi(ta)) if the Zi(ta) matrix lists ensemble perturbations of the DLM wind.

The analysis error covariance matrix Pq(ta) for the observational network augmented by the qth hypothetical

“test-probe'' of adaptive wind and temperature observations with operator Hq and error covariance matrix Rq is then expressed as

Pq(ta) = Pr(ta) - Pr(ta) H

qT (Hq Pr(ta) HqT + Rq)-1 Hq Pr(ta) (2)

The associated “signal covariance” matrix valid at the verification time tv is then given by

Sq (tv) = Pr(tv) – Pq(tv) = M Pr(ta) HqT (Hq Pr(ta) H

qT + Rq)-1 Hq Pr(ta) MT (3)

where M propagates perturbations from ta to tv. The ensemble forecast (96-h for NCEP, 108-h for ECMWF)

perturbations at tv are used to rapidly compute the trace of Sq(tv) localized within the verification region. The ETKF ‘summary map’ represents this signal variance in the verification region at the verification time as a function of the central location of adjacent 3 x 3 test-probes at 1o resolution. The test-probe location that produces the highest signal variance is deemed optimal for targeting. The ETKF signal variance yields 1 The ETKF used in this study uses 1o resolution (u,v,T) at 250, 500 and 850hPa, archived on NCEP’s IBM SP supercomputers. The ETKF used in WSR up to and including 2005 used (u,v,T) at 2.5 o resolution.



accurate estimates of forecast error variance reduction provided that (a) the forecast error projects onto the ensemble, (b) error covariance matrices specified by the ETKF and the operational data assimilation scheme are accurate and consistent, and (c) the ensemble perturbations are sufficiently small for linear dynamics to be obeyed2. While these conditions do not hold in practice, the ETKF has demonstrated the ability to predict the reduction in forecast error variance produced by targeted observations in the mid-latitudes (Majumdar et al. 2001, 2002a).

In this study, the ETKF uses operational ECMWF and NCEP ensembles to estimate a vertically averaged “kinetic energy signal variance” valid at tv, based on observations of horizontal wind and temperature at ta. The ECMWF ensemble is run at TL255 L40 resolution. Initial perturbations are generated using T42 L40 singular vectors optimized over 48h (Buizza et al. 2003). Fifty ECMWF ensemble perturbations of horizontal wind and temperature at 200, 500 and 850hPa initialized at 12 UTC are used here. Ten NCEP GFS ensemble members at T126 L28 resolution are generated every 6h, using masked breeding (Toth and Kalnay 1997). This study uses 20 NCEP horizontal wind and temperature perturbations at 250, 500 and 850hPa, initialized at 00/12 UTC3. For both ECMWF and NCEP ensembles, horizontal wind and temperature perturbations are included at time ta, whereas the signal variance valid at time tv is only based on horizontal wind perturbations. Further details of the ensembles are given in Table 1. Due to computational constraints, it is currently impractical to produce significantly more operational ensemble forecasts. The performance of the ETKF is therefore compromised by spurious correlations in the Pr and Pq matrices, which can lead to erroneous targets at large distances from the TC. Substantial efforts have been made to alleviate these spurious correlations in data assimilation via covariance localization (Houtekamer et al. 2001, Hamill et al. 2001). However, it is presently unclear as to how these localized covariance structures may be maintained as they are propagated from the assimilation time to a future verification time.

Method Common Grid points C

METS

TESV ECMWF vs TESV NOGAPS 7 0.06

TESV ECMWF vs ETKF ECMWF 0 -0.02

TESV ECMWF vs ETKF NCEP 0 -0.02

TESV ECMWF vs NCEP VARIANCE 0 -0.02

TESV NOGAPS vs ETKF ECMWF 19 0.22

TESV NOGAPS vs ETKF NCEP 6 0.05

TESV NOGAPS vs NCEP VARIANCE 0 -0.02

ETKF ECMWF vs ETKF NCEP 21 0.25

ETKF ECMWF vs NCEP VARIANCE 5 0.04

ETKF NCEP vs NCEP VARIANCE 8 0.07

Table 2 Number of common grid points C (out of X=50) for each pair of guidance shown in Fig. 4. The corresponding METS statistic, computed using (13) and E(C)=2.87, is listed.

2.3 Total-Energy Singular Vectors (TESV)

Targeted singular vectors (SVs), introduced by Palmer et al. (1998) and Buizza and Montani (1999), are a particular type of the general class of targeted analysis error covariance (AEC) optimals. The targeted AEC

optimals vi of the tangent forward propagator L(ta;tv) (with corresponding Hermitian transpose or adjoint L*)

sample the phase space directions of maximum growth during a finite optimization time interval (ta,tv),

2 Note that the signal variance is still theoretically equal to the reduction in forecast error variance in an imperfect model setting, as proven on p.1360 of Majumdar et al. (2002a). 3 The NCEP GFS ensembles initialized at 06Z and 18Z were not archived, and are not used in this study.



evolving into the leading eigenvectors of the forecast error covariance matrix P(tv). Given an analysis error

covariance metric ||v||r2 = <v(ta); (Pa)-1 v(ta)> and verification-time metric ||v||v

2 = <v(tv); (Pv)-1 v(tv)>, the

analysis-time optimals vi(ta) are computed via the eigenvalue equation

v a a a* ( ) (t ) (t ) ( ) (t ) (t )a�− −=v 1 2 1i i i(GL) P (GL)v P v (4)

where the operator G which localizes perturbations within the verification region. The square roots of the

eigenvalues �i are the singular values and the eigenvectors vi(ta) the (right) singular vectors of GL with respect to the metrics (Noble and Daniel 1977). By definition, the targeted AEC optimals depend on (i) the optimization time interval tv-ta, (ii) the linear forward and adjoint propagator (which is a function of the non-

linear forecast trajectory), (iii) the analysis and verification norms r and v, and (iv) the verification region.

The class of targeted AEC optimals in which Pa and Pv are identical, diagonal, and set equal to the fixed total

energy weights such that (Pa)-1 = (Pv)-1 = E and ||v||E2 = <v; Ev>, are known as Total Energy Singular Vectors

(TESVs). A quadratic “perturbation total energy” is computed, and each “TESV summary map” presented in this paper is then given by the weighted average of the leading TESVs:

2

21 1

( ) ( , )SVN

jj

j

tσσ=

= ∑F � � � (5)

where ej(x,t) is the vertically integrated total energy of the jth TESV at latitude/longitude grid location x. Further details of the ECMWF and NOGAPS TESVs are summarized in Table 1. Both sets of TESVs have been computed using tangent linear and adjoint models that include surface drag, horizontal and vertical diffusion, but not moist processes. The influence of resolution, metric and moist processes on SV structures is presented in Buizza (1998), Palmer et al. (1998) and Coutinho et al (2004) respectively. The improvement of ECMWF ensemble performance in the tropics due to the inclusion of moist SVs is demonstrated in Puri et al. (2001) and Barkmeijer et al. (2001).

2.4 A proposed marriage between ETKF and optimal perturbation techniques

Upon a first examination of sub-sections 2.2 and 2.3, one might conclude that the ETKF and TESVs have little to do with each other. Although this is true for the current implementations of the respective techniques, the ETKF and targeted AEC optimals (including TESVs) can in fact be joined together in a union that may be stronger than the sum of its parts. This marriage was first made evident by Leutbecher (2003), who showed that by applying the ETKF to an ensemble of AEC optimals (in his case, Hessian SVs) associated with the routine observational network, one could compute the reduction in forecast error variance associated with a large number of feasible deployments of the adaptive observational network. Leutbecher (2003) produced AEC optimals based on an approximation to the 4D-Var routine analysis error covariance matrix

SV

rKP . First, he let the leading KSV targeted AEC optimals pertaining to the routine observational network be

listed in an N x KSV matrix . This enabled equation (4) to be rewritten to produce the sum of the squares of

the leading KSV singular values as follows:

2

1

( * ( ) )SVK

ii

trσ −

=

=∑ K KSV SV

r T v 1 ra aV (t )(GL) P GLV (t ) (6)

where we have used the fact that AEC optimals are orthonormal under the analysis error covariance metric at the analysis time. To formulate the forecast error variance within the verification region for the v-norm,



Leutbecher (2003) combined (6) with the reduced-rank truncation of the routine analysis error covariance

matrix ( ) ( )K K KSV SV SV

r r ra at t= TP V V to yield

2

1/ 2 1/ 2

1/ 2 1/ 2

1/ 2 1/ 2

1

2

1

(|| || )

( ( )( )* )

( ( )* )

( ( ) ( ) ( )* )

( ( ) ( )*( ) ( ))

V

KSV

K KSV SV

K KSV SV

SV

r

r ra a

r ra a

K

ii

E

tr E

tr

tr t t

tr t t

σ

− −

− −

− −

−

=

=≈

=

=

= ∑

v T v

v v

v T v

T v

GLx

P GL xx GL P

P GLP GL P

P GLV V GL P

V GL P GLV

(7)

where the penultimate line of this equation comes from the fact that the trace of AAT is equal to the trace of

ATA. In principle, one could also compute the forecast error variance from singular values associated with

the KSV leading optimals ( )q tKSV

V for any augmented observational network, provided that an estimate of the

appropriate analysis error covariance matrix ( )qatP was available. However, it is unfeasible in practice to

compute hundreds of sets of AEC optimals to produce a summary map of reduction in forecast error variance due to hundreds of possible configurations of adaptive observations. To overcome this difficulty, Leutbecher devised an approximation of the forecast error variance produced by any augmentation of the routine observational network. As noted by Leutbecher, it can be shown that this approximation is equivalent to

applying the ETKF to the targeted AEC optimals ( )KSV

r tV .

2.5 Why the present techniques differ

As shown in subsection 2.4, the ETKF can in principle be applied to targeted AEC optimals to fuse information about adaptive observations with information contained within targeted AEC optimals on error growth and the effect of routine observations on error. In practice, because of the high computational cost of

producing AEC optimals based on a sophisticated estimate of Pr (e.g. Hessian SVs), TESVs are typically used for adaptive sampling. While the ETKF could be applied to TESVs, this has not been done in field programs to date. In this paper, our primary focus is simply to compare the adaptive sampling guidance products that are produced by the currently-used versions of the ETKF and TESV techniques. In doing so, we ignore the potential of the ETKF to add adaptive observation network information to the information contained within SVs, and the summary maps produced by the ETKF and TESVs in this paper will differ for the following reasons:

i) The TESV summary map based on (5) gives a vertically integrated, weighted sum of TESVs pertaining to the routine observational network, whereas the ETKF produces a map of reduction in forecast error variance associated with augmentations to the routine observational network.

ii) The ETKF accounts for the flow and the routine observational network in its ensemble-based

estimate of Pr, whereas TESV computations presently use a diagonal estimate of Pr that does not account for initial condition error.



iii) The ETKF uses evolved ensemble perturbations. These perturbations describe the evolution of a

low-rank ensemble based estimate of Pr. Typically, these perturbations will grow more slowly than

the high-rank energy norm approximation to Pr used by the TESVs.

Sections 3 and 4 explore the consequences of these differences between the ETKF and TESV techniques via qualitative and quantitative comparisons of their respective guidance products.

3. Qualitative Comparison

During the 2004 Atlantic hurricane season, four hurricanes (Charley, Frances, Ivan, Jeanne) made landfall in Florida and several other TCs affected North America (Alex, Bonnie, Gaston, Hermine, Matthew). NOAA conducted an unprecedented number (31) of synoptic surveillance missions in 2004. All 78 cases in which the NOAA National Hurricane Center (NHC) issued forecasts at 00 UTC that existed at 48h (ta) and 96h (tv) were selected for this study. In a few of these cases (e.g. Tropical Storm Earl), the storm dissipated within 48h; however, that was not to be known at the time that a decision on surveillance needed to be made. The verification region is centered on the NHC 96-h forecast position of the TC valid at tv. Our sample includes multiple cases from the same storm on consecutive days; and we treat all TC forecasts separated by at least 24-h as independent (Aberson and DeMaria 1994). The 78 cases are summarized in the Appendix, and corresponding guidance maps are presented on http://orca.rsmas.miami.edu/~majumdar/tc_comparisons/. Before proceeding to a general discussion of the respective guidance products, we present two cases to highlight some similarities and differences, and illustrate issues involved in comparing them.

3.1 Two examples

A case in which the guidance generally agrees is shown in Fig. 2. On 10 September 2004, Hurricane Ivan (maximum sustained wind speed 130 kt or 67 m/s, Category 4 on the Saffir-Simpson Scale) was moving west-northwestward through the central Caribbean Sea toward Jamaica, south of the subtropical ridge. As this ridge was forecast to remain strong for at least the next 48h, no mid-latitude influences on the track of Ivan were expected. All 5 maps show primary targets in the region of Ivan. The ETKF and DLM Wind Variance both indicate the TC and its vicinity, whereas the TESV maxima occur not at the center of the storm, but in an annulus of radius about 5 degrees from the storm center (consistent with Peng and Reynolds 2005b). ETKF NCEP shows a modulation of the NCEP DLM wind variance (Majumdar et al. 2002a). Some minor differences between the 5 maps do exist, such as two secondary maxima in ETKF ECMWF to the west of Costa Rica associated with the quasi-stationary monsoon gyre, and maxima in ETKF ECMWF and NCEP DLM Wind Variance associated with the mid-latitude jet stream. These two regions are unlikely to affect the track of Ivan and are possibly spurious. A secondary maximum exists in TESV ECMWF in the northeastern Gulf of Mexico, associated with a weakness in the subtropical ridge. Observations in this region will likely help decide how far west Ivan will move before turning substantially northward, though this feature will have greater importance at longer forecast lead times than 48h.

In contrast, an example of the complicated nature of contrasting guidance is shown in Fig. 3. Hurricane Jeanne completed an anticyclonic loop after moving northeastward and before moving westward to strike eastern Florida. On 21 September 2004, Jeanne (80 kt, 41 m/s, Category-1) meandered between the Bahamas and Bermuda in a cull region between a strong mid-latitude trough to its northeast and a strong mid-latitude area of high pressure over the Great Lakes (Fig. 3e). Some models forecasted Jeanne to continue slowly northeastward and eastward, whereas others amplified the subtropical high eastward causing Jeanne to turn toward the southwest or west. The NCEP DLM Wind Variance, ETKF NCEP, and TESV NOGAPS all



Figure 2 Comparison of 5 adaptive sampling guidance maps together with NCEP DLM wind analysis for Hurricane Ivan valid at the analysis time ta = 0000 UTC, 10 September 2004. Verification time tv = 0000 UTC, 12 September 2004. The forecast location of Ivan for each model valid at ta is denoted by the hurricane symbol. The relevant model for each map is listed in the final column of Table 1. The verification region, which is identical for all ETKF and TESV computations, is denoted by the red rectangle. The shading in each guidance map in this paper represents values that are normalized with respect to the highest value on that map.

Figure 3 As in Fig. 2, for Hurricane Jeanne valid at the analysis time ta = 0000 UTC, 21 September 2004. Verification time tv = 0000 UTC, 23 September 2004.



indicated that the local feature would be important for the first 48 h. Both TESVs show complicated asymmetric patterns near Jeanne (Figs 3a and b), unlike the annular rings surrounding Ivan (Figs 2a and b), and secondary maxima in New England related to the flow along the eastern side of the trough (Peng and Reynolds 2005b). The primary maximum according to ETKF ECMWF is far from the TC, situated over Newfoundland associated with the center of the deep trough. On the other hand, ETKF NCEP shows only a very small influence associated with the major trough. Another distant maximum in ETKF NCEP and the NCEP DLM Wind Variance is associated with Hurricane Karl, suggesting that Karl may influence the future of Jeanne, possibly by affecting the ridge that developed between the two cyclones. This case also indicates that the guidance issued by the same technique but based on different numerical models can differ significantly.

3.2 General Remarks

The ECMWF and NOGAPS TESVs exhibit a target in an annulus around the TC in 75-80% of the 78 cases. This initial sensitivity occurs in the region where the radial vorticity gradient changes sign, indicative of instability associated with the vortex (Peng and Reynolds 2005b). For TCs that are expected to recurve, both sets of TESV guidance usually identify an area in the mid-latitude flow upstream of the recurvature location, corresponding to a trough. The general shape of both sets of TESV guidance is often similar, although the relative emphasis on particular regions may differ (Figs 2 and 3). In the cases where one TESV summary map identifies an area close to the TC and the other does not, the TC is usually relatively weak or small. The local target likely depends on the ability of the TESVs to resolve the TC.

The TESVs and ETKF generally produce similar targets local to the TC for major hurricanes (maximum 1-minute sustained winds of >96kt or 49m/s). For re-curving TCs, the TESVs place more emphasis on the upstream mid-latitude trough than the ETKF. Unlike the TESVs, the ETKF guidance sometimes emphasizes areas over the Atlantic Ocean associated with the periphery of the sub-tropical ridge, or even the mid-latitude storm track over the ocean. The ensemble variance is often large in such an area, and the ETKF finds that this area is statistically correlated with the vicinity of the TC.

The ETKF ECMWF, ETKF NCEP and NCEP DLM Wind Variance guidance identify the TC for adaptive sampling in >85% of the 78 cases, since the ensemble variance is usually large near the TC, and errors associated with the TC itself are likely to propagate into the verification region. Differences between ETKF ECMWF and ETKF NCEP likely arise due to differences in the respective ensembles’ representations of large (co)variance in various features, such as mid-latitude troughs and the TC itself. ETKF ECMWF identifies local targets in 3 cases in which ETKF NCEP does not, all weakening storms moving towards the northeast. ETKF NCEP identifies local targets in 7 cases in which no local ETKF ECMWF target exists, all cases of weak storms except for case 53 (Fig. 2).

ETKF NCEP often represents a modulated version of the wind variance, with reduced (but non-negligible) emphasis at large distances from the TC. This observation is in contrast to the mid-latitudes, in which the ETKF and ensemble variance may differ significantly (Majumdar et al. 2002a). In each of the 6 cases in which a local target is identified by the NCEP DLM Wind Variance but not ETKF NCEP, the TC is weak. In the 2 cases in which ETKF NCEP identifies a local target but the NCEP DLM Wind Variance does not, the TC is dissipating in high vertical wind shear.

In general, a correlation appears to exist between the existence of a local target and TC intensity, for both ETKF and TESV techniques. A local target often does not appear early or late in a TC life cycle. Once a technique identifies a local target, it remains until the weakening phase.



4. Quantitative Comparison

The overall degree of similarity between the respective guidance products is analyzed objectively, on large and local (synoptic surveillance) scales for all 78 cases. First, all 5 guidance products are interpolated onto a

latitude-longitude grid of highest common resolution (1.5°). All data within 333km of the respective models’

TC center location4 at time ta are ignored, since observations to improve global model track forecasts are currently not intended to be collected within 333km of the center. Operational data assimilation schemes and vortex bogus/relocation methods are currently unable to exploit observations very close to the storm, and moreover it can be argued that observations would obviously be required within the storm if the data assimilation scheme is able to make use of these observations. Hence, the comparison focuses on similarities between the respective targets outside the 333km radius, relevant to global models.

4.1 Common target locations

The first test of commonality between any two maps is identical to that introduced in Section 4 of Majumdar et al. (2002b) and focuses on the primary target areas chosen by the respective maps. The following procedure is performed for each of the 78 cases: For each of the 5 guidance products, the grid point locations corresponding to a fixed number X of highest values are first stored. The number C of common grid points between each of the 10 pairs of guidance are then found, and the similarity metric for each pair is computed using a Modified Equitable Threat Score (Majumdar et al. 2002b):

)(

)(

CECX

CECMETS

−−−=

2 (13)

The quantity E(C) is the expected number of common grid points between all feasible realizations of

guidance. The value of E(C) is estimated empirically, assuming that the guidance from all 78 cases is independent and equally likely. To compute the expected number of common grid points between TESV NOGAPS versus ETKF NCEP guidance, the 77 values of C between TESV NOGAPS Case 1 and each of ETKF NCEP Cases 2-78 are first computed (for a fixed X). The values of C between TESV NOGAPS Case 2 and each of ETKF NCEP Cases 1-78 (not including Case 2) are then computed. The procedure is repeated until C has been computed for all 78 x 77 independent TESV NOGAPS and ETKF NCEP cases. Similar

computations are performed for the other nine pairs of guidance, to produce values of C for

N=78x77x10=60060 independent pairs of guidance. These 60060 values of C are then averaged to give E(C).

Further details on the computation of E(C) are given in Appendix B of Majumdar et al. (2002b).

The value of METS is calculated for all 78 cases using (13). A METS value of 1 corresponds to identical targets (C=X). A METS value greater than (less than) zero implies that a larger (smaller) number of common grid points exist than by pure chance. The percentage of cases out of 78 that are subjectively deemed similar (METS>0) is tabulated for each pair of targets.

4.1.1 Large scale

To compare the 5 sets of guidance on the basin-wide / continental scale, a fixed domain is chosen (120-20W, 0-60N: 2788 grid points). Figure 4 shows the guidance for a 48-h forecast of Hurricane Ivan as it approaches the Gulf of Mexico coast. The X=50 leading grid points corresponding to the primary target regions are

4 The TC center in each model is found using the NCEP tracking algorithm (Marchok 2004) based on the appropriate forecast listed in the final row of Table 1. When the tracking algorithm was unable to find the TC center, the coordinates were estimated using equivalent forecasts of mean sea-level pressure, and ensemble-mean and low-resolution control forecasts.



shown. (The value of X can be raised to include secondary target regions). The number of common grid

points C is then found for each of the 10 pairs of guidance (Table 2). In this case, the two ETKF maps are

most similar, with C=21 common locations corresponding to the vicinity of Hurricane Ivan and a mid-

latitude trough to the north-west of the Great Lakes. In contrast, four pairs of guidance exhibit C=0 common

grid points. The METS is then computed with E(C)=2.87. Note that the distribution of METS from (13) is

asymmetric about zero, with a minimum of METS=-0.02 when C=0 (negative METS occurs in 48% of all 78x10=870 comparisons). The two sets of TESV guidance are minimally similar (Table 2). The shaded regions in Figs 4(a) and 4(b) look similar qualitatively. However, the TESV ECMWF selects a trough over the continental United States as optimal for adaptive sampling, whereas the TESV NOGAPS indicates both the trough and especially the southeastern quadrant of Ivan.

Figure 4 Large-scale comparison of respective guidance maps (with NCEP DLM Wind Analysis at the analysis time for reference). Black dots represent X=50 grid points with highest values. Case 45: Hurricane Ivan. Analysis: 0000 UTC 16 Sep 2004. Verification: 0000 UTC 18 Sep 2004. The forecast location of Ivan for each model valid at ta is denoted by the hurricane symbol.

Table 3a lists the percentage of the 78 cases for which METS>0. The large-scale NOGAPS and ECMWF TESV guidance are similar in 90% of the cases. Hence, while their leading maxima may lie in different locations, there is usually some overlap when X=50. The ETKF ECMWF and ETKF NCEP are also similar to each other in the majority of cases. In contrast, the TESVs often do not exhibit similar targets to the ETKF or especially the ensemble variance. Hence, on large scales, the same targeting methodology usually gives similar guidance irrespective of the model, whereas different methodologies are less similar.

To assess the relationship between the similarity of optimal target locations and TC intensity, the 78 cases are divided into 22 major hurricane (maximum sustained winds >96 kt), 19 Category 1 and 2 hurricane (64-96 kt), and 37 non-hurricane cases (<64 kt). The percentages of the three subsets for which METS>0 are listed in Table 3b. Though the two TESV maps are often similar for all TC strengths, the results are more revealing in the ETKF versus TESV comparisons. In particular, TESV NOGAPS (and to a lesser extent TESV ECMWF) is often in agreement with both ETKF maps for major hurricanes, but less so for other TCs. Both sets of ETKF guidance and the NCEP ensemble variance are also more similar to each other for major hurricanes than for weaker TCs.



Method TESV NOGAPS ETKF ECMWF ETKF NCEP NCEP VARIANCE

TESV ECMWF 90% 32% 31% 17%

TESV NOGAPS 41% 38% 22%

ETKF ECMWF 76% 77%

ETKF NCEP 96%

Table 3a. Percentage of the 78 targeting cases in which METS > 0 for X=50 on the large scale, for each combination of the 5 guidance products.


TESV ECMWF 91/95/86% 55/37/16% 59/16/22% 36/21/3%

TESV NOGAPS 82/32/22% 91/26/14% 55/16/5%

ETKF ECMWF 91/74/68% 82/74/76%

ETKF NCEP 100/95/95%

Table 3b. As for Table 3a, but divided into 22 major hurricanes / 19 Cat-1 & Cat-2 hurricanes / 37 non-hurricanes.

Figure 5 Local-scale comparison of (a) ETKF NCEP and (b) NCEP DLM Wind Variance using X=30 grid points corresponding to highest interpolated values at 150km resolution. White disk represents zero values within 333km of TC center. Case 25: Hurricane Frances. Analysis: 0000 UTC 02 Sep 2004. Verification: 0000 UTC 04 Sep 2004.

4.1.2 Local scale (synoptic surveillance)

The METS (with different E(C)) is tested at scales on which synoptic surveillance flights are deployed. To emphasize a comparison between the adaptive sampling techniques instead of the respective models, a storm-relative coordinate system is chosen. In order to compare identical grid points relative to the TC, a new equal-area 21x21 grid of 150km resolution is created for every case, and values from the guidance products are interpolated linearly onto this grid. The extent of the grid corresponds to the largest distances from the TC that surveillance aircraft usually release dropwindsondes (1500km). The X=30 grid points corresponding to the largest guidance values outside a region of radius 333km from the TC center are found. For example, in Fig. 5, the NCEP Variance emphasizes a fairly symmetric sampling distribution around Hurricane Frances, whereas NCEP ETKF guidance indicates that sampling to the east of Frances is required.

If two maps were drawn at random, the expected number of common grid points out of the top 30 is E(C)=17.6, computed over all N=60060 independent cases. A large E(C) is expected since most targets are



situated near the TC. The criterion for two maps to be similar is therefore strict, and METS is interpreted on the local scale as: “Given that we know most targets are close to the TC, how similar are the targets?” In contrast to the large-scale, TESV ECMWF and TESV NOGAPS targets are similar in only 56% of the 78 cases (Table 4). The TESV and ETKF optimal targets possess similar agreement as on large scales, whereas ETKF ECMWF, ETKF NCEP and NCEP DLM Wind Variance disagree more often on local scales than on large scales. These 3 sets of guidance again agree on optimal targets more often for major hurricanes than non-major hurricanes.


TESV ECMWF 56% 33% 26% 26%


ETKF ECMWF 64% 56%

ETKF NCEP 87%

Table 4 Percentage of the 78 targeting cases in which METS > 0 for X=30 on the synoptic surveillance scale, for each combination of the 5 guidance products.

4.2 Ranking of neighboring regions

A second test of commonality between the 5 guidance products is based on their respective rankings of Y adjacent geographical regions in their suitability for targeting. Each region represents a square of equal area, comprising G x G grid points at 150km resolution. For example, Fig. 6 shows Y=48 adjacent square regions of 5 x 5 grid points in a storm-relative coordinate system. The average of the guidance values within each region is computed, and these regions are ranked between 1 and Y. The degree of similarity between any two

maps is then computed via the Spearman Rank Correlation Coefficient Rs (Wilks 1995). The null hypothesis tested here states that two guidance products are unrelated, whereas the alternative hypothesis is that a positive correlation exists between the rankings of two guidance products. A 1-tailed test at the 95% significance level with Y-2 degrees of freedom is performed (Conover 1980).

Figure 6 Regions to be ranked for ECMWF TESV: each of the Y=48 areas consists of 5 x 5 grid points, distributed around Hurricane Charley at 150km resolution. The region in the northern Great Plains is ranked first, whereas the easternmost regions are ranked last. Case 05. Analysis time: 0000 UTC 13 Aug 2004. Verification time: 0000 UTC 15 Aug 2004.



The size of the square regions is chosen to be sufficiently large such that the values in adjacent regions are not strongly correlated, thereby giving quasi-independent data for the rankings. To select the size of the square regions, the correlation between guidance values at all combinations of grid points is computed for each of the 5 types of guidance, for all 78 cases. The average correlation over all cases is plotted as a function of distance in Fig. 7. The average correlation over all types of guidance approaches 0.5 when G=5,

so this value is chosen for our study. Though it may be desirable to include larger values of G corresponding to lower correlations between adjacent regions, the total number of regions in the domain containing targets becomes undesirably small, particularly for the local-scale comparison. The TESV correlation length scale is smaller than that of the ETKF or Variance. ETKF ECMWF guidance possesses the highest correlations between points separated by <2000 km, whereas ETKF NCEP guidance is significantly more localized. At distances >2000km, ETKF NCEP is less correlated than NCEP DLM Wind Variance, suggesting that ETKF NCEP reduces the emphasis on targets far from the storm compared with the NCEP DLM Wind Variance.

Figure 7 Correlation between points on guidance maps averaged over all 78 cases, as a function of distance between points. For example, for all pairs of points separated by 750km, the average correlation over all 78 TESV ECMWF maps is 0.5

4.2.1 Large scale

Table 5 lists the percentage of cases in which a pair of maps is deemed similar (i.e. the null hypothesis is rejected) for Y=48 and G=5. The TESV ECMWF versus TESV NOGAPS comparison shows similar rankings in all cases, as does ETKF NCEP versus NCEP Variance. Results for other pairs are consistent with the “common grid points” results in Table 3a, except for the ETKF NCEP comparison against both TESVs. ETKF NCEP identifies similar primary target areas as TESVs in 30-40% of cases, whereas the 48 fixed regions are ranked similarly in only 10-15%. Hence, the secondary maxima of the respective guidance products are likely unrelated. This is in contrast to TESV ECMWF versus TESV NOGAPS (and ETKF NCEP versus NCEP Variance) in which similar structures are exhibited on large scale maps (e.g. Figs 4a and 4b) with some disagreement on the order of preference of the primary targets. The similarity of regional rankings is again prominent for major hurricanes, although not as pronounced as for the “common grid points” metric. Tests have also been performed for different specifications of Y and G, and though there is sensitivity to both parameters and the level of statistical significance, the general conclusions remain.




TESV ECMWF 100% 31% 12% 8%


ETKF ECMWF 72% 69%

ETKF NCEP 100%

Table 5 Percentage of cases in which the rankings between any set of 2 targets are deemed similar at the 95% significance level, for Y=48 and G=5

4.2.2 Local (surveillance) scale

The TC-centered grid is reduced to (3000km)2 with Y=16 and G=5. For example, in Fig. 8 the locations of highest TESV ECMWF sensitivity local to Hurricane Frances are distributed in an annulus around Frances, in contrast to ETKF ECMWF which emphasizes the south-eastern quadrant of Frances. The percentage of cases in which the pairs of guidance are deemed similar at the 95% level (with 14 degrees of freedom) is listed in Table 6. In contrast to Table 5, TESV ECMWF and TESV NOGAPS are sometimes dissimilar. Additionally, the NCEP-based ETKF and wind variance resemble the other guidance more often on local scales than on large scales.

Figure 8 Local-scale comparison between (a) TESV ECMWF and (b)ETKF ECMWF. Each of the 16 areas consists of G=25 grid points, spaced symmetrically around Hurricane Frances. All values are set to 0 within a radius of 333km of the storm center. Case 25: Analysis: 0000 UTC 02 Sep 2004. Verification: 0000 UTC 04 Sep 2004.

4.3 Latitude and storm motion

Comparisons were also performed on large and local scales, dividing the 78 cases by (i) TC latitude and (ii) TC forward motion between the observing and verification times. In the deep tropics, the TC is typically steered by deep-layer easterlies to the south of the subtropical ridge (e.g. Frances, Ivan at peak intensity). At higher latitudes, the steering is often associated with a trough (Charley, Frances near landfall) or the western

edge of the subtropical ridge (Danielle, Karl). However, the METS and Rs statistics (not shown) did not exhibit significant differences between TCs north and south of 23.5N (or similar latitudes). The second test was motivated by Peng and Reynolds (2005b), who found that TESV NOGAPS guidance often indicated the rear right quadrant within 500km of straight-moving TCs for adaptive observing. In contrast, they found no preferred quadrant for recurving and irregularly moving TCs, and the maxima of leading TESVs were often situated in the large-scale environment of the TC. In our study, both similarity statistics between TESV



ECMWF and TESV NOGAPS did not change based on TC motion (selected with little ambiguity). The same is mostly true for the other 9 comparisons.

5. Conclusions and discussion

Two objective metrics have been used to evaluate the similarity between 5 types of adaptive sampling guidance for TCs, for 78 two-day forecasts during the Atlantic hurricane season of 2004. Comparisons were performed on the basin-wide scale, and in a local, storm-relative area in which synoptic surveillance missions are conducted. The first metric used a Modified Equitable Threat Score based on the overlap of the

leading 50 (30) of 2788 (441) grid points at 1.5° resolution on large (local) scales. The second metric assessed the similarity of the respective guidance products’ rankings of 48 (large) or 16 (local) fixed regions around the TC, based on their suitability for adaptive sampling. Based on the two metrics, the key conclusion is that guidance using the same adaptive sampling technique with different numerical models was often similar, whereas guidance using the two main techniques usually differed significantly. The main findings are as follows:

(i) Optimal target regions selected by TESV guidance produced using the ECMWF and NOGAPS models were deemed similar on large (local) scales in 90% (56%) of the 78 cases. The flight rankings were found to be similar in all cases, indicating structural similarities in the respective TESVs.

(ii) For major hurricanes, both the ETKF and TESV guidance recommended the vicinity of the storm for adaptive sampling. Optimal ETKF ECMWF and ETKF NCEP targets were found to be similar to TESV NOGAPS (ECMWF) in 86% (57%) of cases on large scales, although secondary characteristics usually disagreed.

(iii) For TCs below major hurricane intensity, both TESV maps possessed similar common optimal locations and rankings to ETKF guidance in only 25% of cases.

(iv) The cases in which guidance from the same technique with different models disagreed most strongly usually correspond to weak TCs.

(v) The optimal target locations selected using TESVs were collocated with those of the ETKF more often than those of the NCEP DLM wind variance.

(vi) ETKF ECMWF guidance resembled ETKF NCEP guidance more often for major hurricanes (75-90%) than for weaker TCs (55-75%). ETKF NCEP guidance often resembled the NCEP DLM wind variance, although the average correlations of the respective products differ on scales of >2000km.

While the percentages quoted above vary depending on the subjectively-chosen criteria for similarity in each metric, the general conclusions remain consistent.

Consistent with the results of Peng and Reynolds (2005b), both sets of TESVs were often found to exhibit maxima (i) within an annulus around 500km from the TC center, and/or (ii) at long distances from the TC, in locations associated with features such as mid-latitude troughs that were expected to influence the track of the TC. The regions for adaptive sampling suggested by ETKF guidance were either (i) localized around the TC, in particular for hurricanes in which large ensemble variance propagated into the verification region, or (ii) in upstream or downstream locations where wind or temperature errors were correlated with those near the verification region. The locations close to the TC generally corresponded to uncertainty in its position and/or the environmental wind or temperature field associated with the flow near the storm (such as the



subtropical ridge). Differences between ETKF ECMWF and ETKF NCEP guidance exist due to the respective model trajectories, initialization methods, and horizontal resolution of the initial perturbations.

The TESV NOGAPS guidance is often more similar to both sets of ETKF guidance than TESV ECMWF is to ETKF, particularly for major hurricanes. One reason may be that the NOGAPS TESVs are computed about a higher-resolution non-linear trajectory than the ECMWF TESVs (Table 1), which may lead to a stronger representation of the TC in the NOGAPS TESVs and more localized targets near the TC. Moreover, NOGAPS uses a bogus vortex in its initialization, whereas ECMWF does not. Also, NOGAPS TESV summaries are computed using the leading 3 TESVs, whereas ECMWF uses the leading 10 TESVs.

As was described in Subsection 2.4, the ETKF could be applied to any set of targeted analysis error covariance (AEC) optimals, including TESVs, to produce estimates of reduction in forecast error variance due to targeted observations. However, as noted by Buizza (1994), there is little similarity between the evolved ensemble perturbations used by the ETKF in this paper and initial time SVs used for targeted observations. On a related note, since the ETKF attempts to account for initial condition error, it usually indicates that observations to reduce initial uncertainty in the depth, size, intensity and position of the storm are likely to be more useful than other observations. Areas of high initial uncertainty in the storm also coincide often with areas of high DLM wind variance. In contrast, since the TESVs do not estimate initial condition error, the uncertainty associated with the storm is not accounted for. Hence, it is not surprising that the ETKF and TESV maps are usually not similar. We use similar arguments to speculate that the ETKF and TESV guidance would look less similar as the forecast lead-time extends beyond 2 days.

This paper has extended the use of the most common versions of the SV and ETKF adaptive sampling techniques to tropical cyclone forecasts. The conclusions presented here may change in the future due to continuous upgrades of the techniques and their associated models, and new applications of the ETKF to targeted AEC optimals. In particular, the specification of the routine analysis error covariance matrix Pr(ta) ought to resemble that of the operational data assimilation scheme. Two examples that attempt to do this are Hessian SVs (Barkmeijer et al. 1999, Leutbecher 2003) which specify Pr using the Hessian of the cost function in a variational data assimilation scheme, and Variance SVs (Gelaro et al. 2002, Reynolds et al. 2005) which use a Pr that is consistent with operational estimates of analysis error variance. The inclusion of moist processes in the tangent forward and adjoint models (Ehrendorfer et al. 1999, Coutinho et al. 2004) may improve the structure of SVs and change the similarity between their guidance and those of other methods based on non-linear, full-model forecasts. As for the ETKF, while it is impossible to specify a Pr that resembles the present operational specification, future data assimilation schemes will likely incorporate an anisotropic, flow-dependent, ensemble-based Pr (Lorenc 2003, Houtekamer et al. 2005). As the number of operational ensemble members continues to increase and ensemble initialization improves, the erroneous long-distance correlations in the ETKF estimate of Pr are expected to be reduced. The introduction of new verification norms of TC track, intensity and precipitation may be considered.

Insights into the respective targets will be reported in the future. Other necessary studies include the evaluation of the efficacy of observations collected within the respective target areas, and research into the influence of specific observations on TC forecasts and their relation to AEC optimals and ETKF signal variance. The extent to which non-linearity, limited resolution, moist physics, and small ensembles compromise the accuracy of the techniques requires investigation. Furthermore, new adaptive sampling techniques that focus on TC structure and intensity change require development, given the ability of new numerical models and data assimilation schemes to resolve the TC inner core. These outstanding issues can be addressed during field programs aimed at improving TC track and intensity prediction, with the ultimate



goal being to provide accurate quantitative predictions of forecast error variance reduction due to any feasible deployment of in-situ and remotely-sensed observations (Majumdar et al. 2001, Leutbecher 2003).

Acknowledgements

Sharanya Majumdar and Sim Aberson acknowledge financial support from the NOAA Joint Hurricane Testbed, and Zoltan Toth and the Environmental Modeling Center at NCEP for making the paper possible via the provision of an account on the IBM SP supercomputer. Craig Bishop, Melinda Peng, and Carolyn Reynolds acknowledge the support of the Naval Research Laboratory and the Office of Naval Research under program element 0601153N, project number BE-033-03-4M. The extension to the ETKF formulation described in Subsection 2.2 was performed by Jonathan Moskaitis under funding from a NWS/UCAR COMET Grant. The authors are grateful to James Franklin, Brian Etherton, Chun-Chieh Wu and David Kofron for valuable discussions. Comments by Istvan Szunyogh and 2 anonymous reviewers improved the manuscript.



APPENDIX

Table A1. Summary of all 78 cases used in this study. The TESV control trajectories (ensembles) are

initialized 48h (48-60h) prior to the analysis time ta (see Table 1). All analysis and verification times (tv) are at 00 UTC. Hence, ‘0803’ = 00 UTC, August 3rd 2004. The verification region is centered on the official NOAA National Hurricane Center 96-h forecast position of the TC valid at the verification time. The western and eastern (in degrees W) and northern and southern (in degrees N) periphery of the rectangular verification region is listed. The column headed “Int” refers to the official estimate of the storm intensity (in knots) valid

at time ta.

# Name ta tv W Lon E Lon S Lat N Lat Int

1 Alex 0803 0805 72.0 52.0 34.0 54.0 60

2 Alex 0804 0806 59.0 39.0 38.5 58.5 80

3 Alex 0806 0808 30.5 10.5 34.0 54.0 75

4 Charley 0812 0814 94.0 74.0 12.0 32.0 65

5 Charley 0813 0815 92.0 72.0 21.0 41.0 90

6 Charley 0814 0816 78.0 58.0 36.5 56.5 75

7 Charley 0815 0817 76.5 56.5 38.5 58.5 40

8 Danielle 0816 0818 52.0 32.0 9.0 29.0 90

9 Danielle 0817 0819 52.0 32.0 15.0 35.0 90

10 Danielle 0818 0820 49.0 29.0 22.5 42.5 80

11 Danielle 0819 0821 45.5 25.5 25.5 45.5 40

12 Danielle 0820 0822 37.0 17.0 29.0 49.0 35

13 Danielle 0821 0823 38.0 18.0 26.0 46.0 30

14 Danielle 0822 0824 47.0 27.0 23.0 43.0 20

15 Danielle 0823 0825 51.8 31.8 31.0 51.0 20

16 Earl 0816 0818 83.0 63.0 7.0 27.0 40

17 Earl 0817 0819 91.3 71.3 9.3 29.3 20

18 Earl 0818 0820 99.0 79.0 9.0 29.0 20

19 Frances 0827 0829 61.0 41.0 8.0 28.0 75

20 Frances 0828 0830 64.0 44.0 9.0 29.0 100

21 Frances 0829 0831 69.0 49.0 11.5 31.5 115

22 Frances 0830 0901 75.4 55.4 12.1 32.1 110

23 Frances 0831 0902 80.6 60.6 12.4 32.4 110

24 Frances 0901 0903 84.5 64.5 14.5 34.5 120

25 Frances 0902 0904 88.0 68.0 16.5 36.5 120

26 Frances 0903 0905 91.0 71.0 18.0 38.0 115

27 Frances 0904 0906 93.0 73.0 20.0 40.0 90

28 Frances 0905 0907 95.5 75.5 21.5 41.5 90

29 Frances 0906 0908 97.0 77.0 25.0 45.0 55

30 Frances 0907 0909 97.0 77.0 26.0 46.0 35

31 Frances 0908 0910 89.0 69.0 31.5 51.5 25

32 Gaston 0830 0901 82.0 62.0 29.0 49.0 30

33 Gaston 0831 0902 69.0 49.0 35.0 55.0 35

34 Ivan 0905 0907 63.5 43.5 2.5 22.5 60

35 Ivan 0906 0908 69.5 49.5 4.5 24.5 115

36 Ivan 0907 0909 77.0 57.0 6.5 26.5 90

37 Ivan 0908 0910 81.7 61.7 9.7 29.7 115



38 Ivan 0909 0911 88.7 68.7 9.5 29.5 125

39 Ivan 0910 0912 91.0 71.0 11.0 31.0 130

40 Ivan 0911 0913 92.0 72.0 13.5 33.5 130

41 Ivan 0912 0914 92.5 72.5 15.5 35.5 145

42 Ivan 0913 0915 93.5 73.5 20.0 40.0 140

43 Ivan 0914 0916 95.5 75.5 21.0 41.0 140

44 Ivan 0915 0917 94.5 74.5 24.0 44.0 120

45 Ivan 0916 0918 96.0 76.0 24.0 44.0 115

46 Ivan 0917 0919 95.0 75.0 25.5 45.5 30

47 Ivan 0918 0920 94.0 74.0 26.0 46.0 25

48 Jeanne 0916 0918 81.3 61.3 12.7 32.7 55

49 Jeanne 0917 0919 84.0 64.0 13.0 33.0 60

50 Jeanne 0918 0920 84.0 64.0 16.0 36.0 40

51 Jeanne 0919 0921 89.0 69.0 19.5 39.5 40

52 Jeanne 0920 0922 84.0 64.0 19.0 39.0 50

53 Jeanne 0921 0923 80.5 60.5 17.5 37.5 75

54 Jeanne 0922 0924 78.0 58.0 16.5 36.5 80

55 Jeanne 0923 0925 80.1 60.1 17.3 37.3 85

56 Jeanne 0924 0926 85.5 65.5 19.5 39.5 90

57 Jeanne 0925 0927 90.5 70.5 20.0 40.0 85

58 Jeanne 0926 0928 90.5 70.5 23.0 43.0 100

59 Jeanne 0927 0929 85.0 65.0 27.5 47.5 45

60 Jeanne 0928 0930 75.0 55.0 32.0 52.0 30

61 Jeanne 0929 1001 67.0 47.0 33.0 53.0 25

62 Karl 0919 0921 55.5 35.5 8.5 28.5 100

63 Karl 0920 0922 57.0 37.0 15.5 35.5 115

64 Karl 0921 0923 58.0 38.0 18.0 38.0 120

65 Karl 0922 0924 56.0 36.0 25.5 45.5 105

66 Karl 0923 0925 54.5 34.5 34.0 54.0 95

67 Karl 0924 0926 44.0 24.0 44.0 64.0 80

68 Karl 0925 0927 24.0 4.0 53.0 73.0 60

69 Lisa 0922 0924 55.6 35.6 5.3 25.3 60

70 Lisa 0923 0925 58.1 38.1 5.7 25.7 45

71 Lisa 0924 0926 58.5 38.5 8.0 28.0 30

72 Lisa 0925 0927 57.0 37.0 8.5 28.5 30

73 Lisa 0926 0928 58.0 38.0 14.0 34.0 45

74 Lisa 0927 0929 58.0 38.0 17.0 37.0 45

75 Lisa 0928 0930 60.5 40.5 19.5 39.5 35

76 Lisa 0929 1001 60.5 40.5 19.0 39.0 55

77 Lisa 0930 1002 57.0 37.0 25.5 45.5 60

78 Lisa 1001 1003 52.0 32.0 31.0 51.0 60



References

Aberson, S. D., and DeMaria M., 1994: Verification of a nested barotropic hurricane track forecast model

(VICBAR). Mon. Wea. Rev., 122, 2804–2815.

Aberson, S. D., 2002: Two years of operational hurricane synoptic surveillance. Wea. Forecasting., 17, 1101–1110.

Aberson, S. D., 2003: Targeted observations to improve operational tropical cyclone track forecast guidance.

Mon. Wea. Rev., 131, 1613-1628.

Barkmeijer, J., Buizza R., and Palmer T. N., 1999: 3D-Var Hessian singular vectors and their potential use in

the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 125, 2333–2351.

Barkmeijer, J., Buizza R., Palmer T. N., Puri K., and Mahfouf J.-F., 2001: Tropical singular vectors

computed with linearized diabatic physics. Quart. J. Roy.Meteor.Soc., 127,685–708.

Bishop, C. H., 1993a: On the behaviour of baroclinic waves undergoing horizontal deformation. I: The ‘RT’

phase diagram. Quart. J. Roy. Meteor. Soc., 119, 221-240.

Bishop, C. H., 1993b: On the behaviour of baroclinic waves undergoing horizontal deformation. II: Error

bound amplification and Rossby wave diagnostics. Quart. J. Roy. Meteor. Soc., 119, 241-267.

Bishop, C. H. and Toth, Z. 1999: Ensemble transformation and adaptive observations. J. Atmos. Sci., 56, 1748-1765.

Bishop, C.H., Etherton B.J. and Majumdar S.J.,2001:Adaptive Sampling with the Ensemble Transform

Kalman Filter. Part I: Theoretical Aspects. Mon. Wea. Rev., 129, 420-436.

Buizza, R., 1994: Localization of optimal perturbations using a projection operator. Quart. J. Roy. Meteor.

Soc., 120, 1647-1681.

Buizza, R., 1998: Impact of horizontal diffusion on T21, T42 and T63 singular vectors. J. Atmos. Sci., 55, 1069-1083.

Buizza, R. and Palmer, T. N., 1995: The singular-vector structure of the atmospheric general circulation. J.

Atmos. Sci., 52, 1434-1456.

Buizza, R. and Montani, A. 1999: Targeted observations using singular vectors. J. Atmos. Sci., 56, 2965-2985.

Buizza, R., Richardson, D. S., & Palmer, T. N., 2003: Benefits of increased resolution in the ECMWF

ensemble system and comparison with poor-man's ensembles. Quart. J. Roy. Meteor. Soc., 129, 1269-1288 .

Burpee, R. W., Franklin J. L., Lord S. J., Tuleya R. E., and Aberson S. D., 1996: The impact of Omega

dropwindsondes on operational hurricane track forecast models. Bull. Amer. Meteor. Soc., 77, 925–933.

Conover, W. J., 1980: Practical Nonparametric Statistics. 2nd Ed. Wiley and Sons.

Coutinho, M. M., Hoskins, B. J., & Buizza, R., 2004: The influence of physical processes on extratropical

singular vectors. J. Atmos. Sci., 61, 195-209.

Ehrendorfer, M., Errico, R. M. and Raeder, K. D., 1999: Singular-Vector Perturbation Growth in a Primitive

Equation Model with Moist Physics. J. Atmos. Sci, 156, 1627-1648.



Franklin, J. L., and DeMaria M., 1992: The impact of Omega dropwindsonde observations on barotropic

hurricane track forecasts. Mon. Wea. Rev., 120, 381–391.

Gelaro, R., Rosmond, T. and Daley, R., 2002: Singular vector calculations with an analysis error variance

metric. Quart. J. Roy. Meteor. Soc., 128, 205-228.

Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background-error

covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776-2790.

Hoskins, B. J., Buizza, R., & Badger, J., 2000: The nature of singular vector growth and structure. Q. J. R.

Meteorol. Soc., 126, 1565-1580.

Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data

assimilation. Mon. Wea. Rev., 129, 123-137.

Houtekamer, P. L., Mitchell, H. L., Pellerin, G., Buehner, M., Charron, M., Spacek, L. and Hansen, B., 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea.

Rev., 133, 604–620.

Joly, A. and Coauthors, 1999: Overview of the field phase of the Fronts and Atlantic Storm-Track

Experiment (FASTEX) project. Quart. J. Roy. Meteor. Soc., 125, 3131-3163.

Langland, R. H., 2005: Observation Impact during the North Atlantic TReC-2003. Mon. Wea. Rev., in press.

Langland, R. H., and Coauthors, 1999. The North Pacific Experiment (NORPEX-98). Targeted observations

for improved North American Weather forecasts. Bull. Amer. Meteor. Soc., 80, 1363-1384.

Leutbecher, M., 2003: A reduced rank estimate of forecast error variance changes due to intermittent

modifications of the observing network. J. Atmos. Sci., 60, 729-742.

Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP—A comparison with 4D-Var.

Quart. J. Roy. Meteor. Soc., 129, 3183–3203.

Majumdar, S. J., Bishop, C. H., Etherton, B. J., Szunyogh, I. and Toth, Z. 2001: Can an ensemble transform Kalman filter predict the reduction in forecast error variance produced by targeted observations? Quart. J.

Roy. Meteor. Soc., 127, 2803-2820.

Majumdar, S. J., Bishop, C. H., Etherton, B. J. and Toth, Z. 2002a: Adaptive Sampling with the Ensemble

Transform Kalman Filter. Part II: Field Program Implementation. Mon. Wea. Rev., 130, 1356-1369.

Majumdar, S. J., Bishop, C. H., Buizza, R. and Gelaro, R. 2002b: A comparison of ensemble transform Kalman filter targeting guidance with ECMWF and NRL total energy singular vector guidance. Quart. J.

Roy. Meteor. Soc., 128, 2527-2549.

Montani, A., Thorpe, A. J., Buizza, R. and Unden, P., Forecast skill of the ECMWF model using targeted

observations during FASTEX. Quart. J. Roy. Meteor. Soc., 125, 3219-3240.

Morgan, M. C., 2001: A potential vorticity and wave activity diagnosis of optimal perturbation evolution. J.

Atmos. Sci., 58, 2518–2544.

Noble, B. and Daniel, J. W., 1997: Applied Linear Algebra, 2nd Ed., Prentice-Hall.



Palmer, T. N., Gelaro, R., Barkmeijer, J. and Buizza, R. 1998: Singular vectors, metrics, and adaptive

observations. J. Atmos. Sci., 58, 210-234.

Peng, M. S., and Reynolds, C. A., 2005a: Double trouble for typhoon forecasters. Geophys. Res. Lett., 32, L02810, doi:10.1029/2004GL021680.

Peng, M. S., and Reynolds, C. A., 2005b: Sensitivity of tropical cyclone forecasts. Submitted to J. Atmos.

Sci.

Puri, K., Barkmeijer J., and Palmer T. N., 2001: Ensemble prediction of tropical cyclones using targeted

diabatic singular vectors. Quart. J. Roy. Meteor. Soc., 127, 709–732.

Reynolds, C.A., Gelaro, R. and Doyle, J. D., 2001: Relationship between singular vectors and transient

features in the background flow. Quart. J. Roy. Meteor. Soc., 127, 1731-1760.

Reynolds, C. A., R. Gelaro, and T. E. Rosmond, 2005: A comparison of variance and total-energy singular

vectors. Quart. J. Roy. Meteor. Soc., 131, 1955-1973.

Szunyogh, I., Toth, Z., Zimin, A., Majumdar, S. J. and Persson, A. 2002: On the propagation of the effect of

targeted observations: The 2000 Winter Storm Reconnaissance Program. Mon. Wea. Rev., 130, 1144-1165.

Szunyogh, I., Toth, Z., Morss, R. E., Majumdar, S. J., Etherton, B. J. and Bishop, C. H. 2000: The effect of targeted dropsonde observations during the 1999 Winter Storm Reconnaissance program. Mon. Wea. Rev.,

128, 3520-3537.

Toth, Z. and Kalnay, E. 1997: Ensemble forecasting at NMC and the breeding method. Mon. Wea. Rev., 125, 3297-3319.

Wang, X. and Bishop, C. H., 2003: A comparison of breeding and ensemble transform Kalman filter

ensemble forecast schemes. J. Atmos. Sci., 60, 1140-1158.

Wilks, D.S.,1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467pp

Wu, C.-C., Lin, P.-H., Aberson, S. D., Yeh, T.-C., Huang, W.-P., Lu, G.-C., Hsu, K.-C., Lin, I.-I., Chou, K.-H., Lin, P.-L. and Liu, C.-H., 2005a: Dropwindsonde observations for typhoon surveillance near the Taiwan

region (DOTSTAR): An Overview. Bull. Amer. Meteor. Soc., 86, 787-790.

ECMWF | Advancing global NWP through international ... · The ensemble transform Kalman filter...

Documents

Transcript of ECMWF | Advancing global NWP through international ... · The ensemble transform Kalman filter...