PREDICTABILITY AND CHAOSIonospheric Physics. London: Academic Press. Roble RG (1986) Chemistry in...

Crowley G (1991) Dynamics of the Earth’s thermosphere.US National Report to International Union of Geodosyand Geophysics 1987–1990, Review of Geophysics,Supplement, pp. 1143–1165.

Hedin AE and Carignan GR (1985)Morphology of thermo-spheric composition variations in the quiet polar meso-sphere from Dynamics Explorer measurements. Journalof Geophysical Research 90: 5269–5277.

Killeen TL (1987) Energetics of the Earth’s thermosphere.Reviews of Geophysics 25: 433–454.

Killeen TL and Roble RG (1988) Thermospheric dynamics:contributions from the first 5 years of the DynamicsExplorer program. Reviews of Geophysics 26: 329–367.

Rees MH (1989) Physics and Chemistry of the UpperAtmosphere. Cambridge: Cambridge University Press.

RishbethH (2001) The centenary of solar-terrestrial physics.Journal of Atmospheric and Solar–Terrestrial Physics 63:1883–1890.

Rishbeth H and Garriott OK (1969) Introduction toIonospheric Physics. London: Academic Press.

Roble RG (1986) Chemistry in the thermosphere andionosphere.Chemical EngineeringNews. 64(24): 23–38.

Roble RG (1992) The polar lower thermosphere. PlanetarySpace Science 40: 271–297.

Rocket P (1952) Pressure, densities and temperaturesin the upper atmosphere. Physics Review 88:1027–1032.

Schunk RW and Nagy AF (2000) Ionospheres, Physics,Plasma Physics and Chemistry. Cambridge: CambridgeUniversity Press.

PREDICTABILITY AND CHAOS

L A Smith, London School of Economics, London, UK

Copyright 2003 Elsevier Science Ltd. All Rights Reserved.

Introduction

Chaos is difficult to quantify. The nonlinear dynamicthat gives rise to chaos links forecasting on the shortesttime scales of interest with behavior over the longesttime scales. In addition, the statistics which quantifychaos forbid an appeal to most of the traditionalsimplifications employed in statistical estimation.Nevertheless, both the concept of chaos in physicalsystems and its technical relative in mathematics havehad a significant impact on meteorological aims andmethods. This is particularly true in forecasting.Chaos implies sensitivity to initial conditions: smalluncertainties in the current state of the system willgrow exponentially-on-average. Yet, as discussedbelow, neither this exponential-on-average growthnor the Lyapunov exponents that quantify it reflectmacroscopic predictability. The limits chaos places onpredictability are much less severe than generallyimagined. Predictability is more clearly quantifiedthrough traditional statistics, like uncertainty dou-bling times. These statistics will vary from day to day,depending on the current state of the atmosphere.Maintaining the uncertainty in the initial state withinthe forecast is a central goal of ensemble forecasting(see Weather Prediction: Ensemble Prediction).Achieving the ultimate goal of meaningful probabilityforecasts for meteorological variables would be ofgreat societal and economic value. Fundamentallimitations in the realism of models of the atmosphere

will limit our ability to make probability (PDF)forecasts, just as uncertainty in the initial conditionlimits the utility of single forecasts even if the model isperfect.

Initially, it appears that chaotic systems will beunpredictable, and this is true in that it is not possibleto make extremely accurate forecasts in the verydistant future. Yet chaos per se does not imply onecannot sometimes make accurate forecasts well intothe medium range. And perhaps just as importantly,with a perfect model one can determine which of theseforecasts will be informative and which will not at thetime they are made. As we shall see below, both theAmerican and the European weather forecast centershave adopted this strategy operationally, with the aimof quantifying day-to-day variations in the likely rangeof future meteorological variables. Quantifying thisrange can be of significant value evenwithout a perfectmodel. Although accurate probability forecasts arelikely to require a perfect model, current operationalensemble prediction systems already provide econom-ically valuable information on the uncertainty ofnumerical weather prediction (NWP) well into ‘weektwo’, and research programs on seasonal time scalesare underway.

Why is perfect foresight of the future state of theatmosphere impossible? First, it should be no surprisethat if our knowledge of the present is uncertain thenour knowledge of the future will also be uncertain; thequestion of prediction then turns to how to bestquantify the dynamics of uncertainty. Here the fullimplications of chaos, or more properly of nonlinear-ity, mix what are often operationally distinct tasks:observing strategy, data assimilation, state estimation,

PREDICTABILITY AND CHAOS 1777

ensemble formation, forecast evaluation, and modelimprovement. This complicates the forecasting prob-lem beyond the limits in which classical predictiontheory was developed. First the traditional goal of asingle best first guess (BFG) forecast with a minimalleast-squares error is not a viable aim in this scenario,nor is a least squares solution desired. Indeed (asdiscussed below) this approach would reject theperfect model which generated the observations!Second, current models are not perfect. The term‘model inadequacy’ is used to summarize imperfec-tions in a given model and the entire model class fromwhich that model is a member. Model improvementand the search for a superior model class, along withthe investigation of more relevant measures of modelskill, are areas of active research.

A Mathematical Framework forModelling Dynamical Systems

Chaos is a phenomenon found in many nonlinearmathematical models. While one should never forgetthe distinction between the model and the systembeing modeled, precise mathematical definitions aremore easily made within the perfect model scenario(PMS).Within this useful fiction, one assumes that themodel in-hand is itself the physical system of interest.Before moving back to forecasts of the real world, ofcourse, one must recover from this self-deception.

Given a model, an initial condition is simply anassignment of values to all model variables at aparticular starting time. Thus the initial conditionreflects the current state of the model: it is a vector xðtÞwhich specifies the value of every variable in themodelat time t. For the classical model of a pendulum, thestate consists of two variables (the angle and theangular velocity). These two numbers completelydefine the current state of the model, and so the modelis called two-dimensional, or, equivalently, said tohave a two-dimensional state space. The famousLorenz model of 1963 is three-dimensional, as thereare three variables: x ¼ ðx; y; zÞ. These low-dimen-sional systems shouldbe contrastedwith the state of anoperationalNWPmodel,whichmay consist of over 10million variables.

The sequence of states a dynamical system passesthrough defines the history of the system; this sequenceis called a trajectory. For deterministic systems, anysingle state x along a trajectory defines all future statesof the system. For the classical pendulum, thesesolutions can be written down analytically; but forall but the simplest nonlinear systems only numericalsolutions exist. Thus it is difficult to prove that a given

realization of the Lorenz model is chaotic, and evenmore so modern NWP models.

Chaos

Mathematically, a chaotic system is a deterministicsystem inwhich (infinitesimally) small uncertainties inthe initial condition will grow, on average, exponenti-ally fast. The average is taken over (infinitely) longperiods of time. Of course, finite uncertainties oftengrow rapidly as well, in which case any uncertainty inthe initial conditionwill limit predictability in terms ofa single BFG of the future state, even with a perfectmodel. But inasmuch as it is defined by the behaviourof (infinitesimally) small uncertainties of (infinitely)long periods of time, chaosper seplaces no limits of thegrowth of finite uncertainty over a finite period oftime. Chaotic systems are often said to displaysensitive dependence on initial condition (SDIC), atechnical term for systems inwhich states initially veryclose together tend to end up very far apart, eventually.

Suppose the true state of the system is ~xx: what is thebehaviour of a near-by solution xxwhere xx ¼ ~xxþ e? Inthe pendulum, a small initial e grows slowly, if at all. Ina chaotic system, the magnitude of e will growexponentially-on-average; yet this does not implythat the actual magnitude of e ever grows exponenti-ally in time. Indeed, since e is a distance and jej >0 forany given value of t, one can always define a value

l ¼ 1

tlog ½jeðtÞj=jeð0Þj�

for any system, chaotic or not! In this case, observing avalue of l >0 for finite t does not even suggestexponential growth. Statistics like l become interest-ing only when they approach a constant as t ! 1; bydefinition, chaos reflects properties only in this limit.

Clearly, chaos includes special cases where magni-tude of e is growing uniformly, say doubling everysecond; but it also allows the more common casewhere the growth of e is a function of the state x andhence changes with time. In general, the growth willnot be uniform in time. In fact in some chaotic systems,including the Lorenz 1963 model, there are regions ofthe state space in which every e will decrease! Suchregions are said to represent ‘return of skill’ asforecasts become more accurate in the least-squaressense as time passes.

For instance, consider the case where, on average,half the time e is constant and the other half of the timeit grows by a factor of four. This will yield in the sameexponential-on-average growth as doubling everytime step, yet there will be times when prediction iseasy. Or for a more extreme case, consider where e

1778 PREDICTABILITY AND CHAOS

shrinks by a factor of two 9 times out of 10, but once in10growsby a factor of 219 (about half amillion); againthis is exponential-on-average growth equivalent todoubling every time step. The question, then, iswhether or not these variations of predictability canbe identified in advance. As discussed below, ensembleforecasts aim to do just that.

The Statistics of Chaos: LyapunovExponents and Doubling Times

Given a deterministic system which remains in abounded region of state space, chaos is defined by astatistic called the Lyapunov exponent. In a one-dimensional system, the Lyapunov exponent reflectsthe logarithm of the (geometric) average growth ofinfinitesimal uncertainties. In the limits e ! 0 andt ! 1, the geometric average of jeðt þ 1Þj=jeðtÞjdefines the Lyapunov exponent, usually called l.Note that nowhere is there the suggestion thatjeðtÞj � jeð0Þjelt. A system will have as many Lyapu-nov exponents as the dimension of the state space. Thelargest Lyapunov exponent is often called the ‘leading’Lyapunov exponent, and if the leading Lyapunovexponent of a bounded deterministic system is posi-tive, then the system is ‘chaotic’. Hence the threesystems noted above – one in which e doubles everytime step, and the other two in which e sometimesgrows and sometimes does not – each have the sameLyapunov exponent. Typically, the logarithm is takenwith base two, so that if, for example, the uncertaintydoubles every second, the Lyapunov exponent is onebit per second, thus the Lyapunov exponent is anaverage rate.

Also note that for every statexon the attractor, therecorresponds a unique direction in state space associ-ated with the leading Lyapunov exponent. If the statespace has a dimension greater than one, then estimat-ing Lyapunov exponents involves matrix multiplica-tion along a trajectory.Matrixmultiplication does notcommute, thus when dealing with statistics likeLyapunov exponents one has to apply multiplicativeergodic theorems; this makes many of their propertiesappear counterintuitive. Many intuitive methods ofstatistical estimation fail when applied to chaoticsystems. None of this is surprising, since most statis-tical intuition is developed in the context of morefamiliar ergodic theorems.

If the sumof all the Lyapunov exponents is negative,then the trajectories will evolve towards a set whosedimension is less than the dimension of the state space;this set is called an ‘attractor’. An attractor may besomething as common as a fixed point, a periodicorbit, or a torus; in such cases the attractor has simple

geometry. Alternatively, an attractor may have astrange geometry: it may consist of a fractal set ofpoints in state space, inwhich case it is called a ‘strangeattractor’. Note that being chaotic reflects a propertyof the dynamics of the system, while strangenessreflects the geometry of the set on which the systemevolves, not the dynamics of the evolution itself. Givena chaotic systemwith a strange attractor, the choice ofinitial conditions should reflect the local structure ofthe attractor, yet this structure is determined by thelong-time behavior for the system. In this way, chaoslinks the longest time scales of the system to theshortest time scales of interest.

The attractor of the Lorenz 1963model with typicalparameter values is shown in Figure 1. It is believedthat there are parameter values for which the Lorenz1963 model has chaotic dynamics on a strangeattractor, but, as noted above, such properties aredifficult to prove even in this fairly simple system ofequations.

In meteorology, the doubling time, t2, provides amore traditional measure of predictability than the

Figure 1 The distribution of uncertainty doubling times on the

Lorenz attractor. Points colored red double in less than one Lorenz

second. Points colored red have a t2o1 Lorenz time step, orange

points t2o2, and so on through yellow, light green, dark green,

blue, and purple. Themauve points on the inner and outer edges of

the attractor for which t2o7. The density of points with t2o5 has

been reduced for clarity. The visible line in the foreground which

separates red points from each of the other colors shows the

locationof pointswhichdouble just as theyenter the region inwhich

all uncertainties decrease, referred to in the text. (Adapted with

permission from Figure 1 of Smith LA (1994) Local optimal

prediction. Philosophical Transactions of the Royal Society of

London Series A 348: 371–381.)


Lyapunov exponents. It is also a more relevantmeasure, although if the growth is not uniform thenthe time required for an initial uncertainty at x toincrease by a factor of four will not be twice thedoubling time at x (or, more generally, tq2 6¼ 2tq). Inpractice, one just has to look at the statistics. Anaverage time is a more relevant measure of predicta-bility than an average rate. Computing an average raterequires stating the duration over which to average apriori, while the relevant time scale is itself thequantity of interest. In fact, one can generate a familyof chaotic systems each of which has a leadingLyapunov exponent greater than one, yet containingmembers with an average t2 as large as desired!Indeed, estimating a time-like statistic with the inverseof an average rate is a dubious endeavor. To convinceyourself of this, consider estimating the average valueof a by one over the average of 1=a, when a isuniformly distributed between zero and one.

In Figure 1, points on the Lorenz attractor arecolored by the doubling time of an infinitesimaluncertainty aligned initially in the local orientationcorresponding to the leading Lyapunov exponent. Thecoloration is neither uniform nor random. Note theline separating the red points on one side from theband of each color on the other. The origin of thisdemarcationwill be explained in the next section. Redpoints have a t2 of less than one second, for orangepoints it is less than two, and so on through yellow,light green, dark green, blue, and purple. The mauvepoints on the inner and outer edges of the attractor forwhich t2 > 7. This is a clear illustration that predict-ability will vary with initial condition in an organizedway! Which, in turn, suggests that predictability willvary in a predictable way: quantifying this in practiceis the goal of ensemble forecasting. Yet evenwithin thePMS, one is interested in finite initial uncertainties andforecasts over a finite duration. The accuracy of suchforecasts need not reflect the Lyapunov exponents ofthe system in any way. Thus chaos per se places fewrestrictions on predictability.

True Limits of Predictability

So what are the limits to predictability of a chaoticsystem? The answer depends on the use to which theforecast is to be put.

Linear prediction theory aims to identify the opti-mal least-squares predictor: the model which, onaverage, yields a BFG future state with the smallest(squared) prediction error. This is a coherent approachto Gaussian uncertainties evolved under linear mod-els, but not when applied to nonlinear systems withuncertain initial conditions.

If the initial condition is uncertain, then thisuncertainty will evolve nonlinearly. It can be proventhat given a series of uncertain observations of achaotic system, there will always be uncertainty in thecurrent state. This is the case even if a perfect model isin hand and the observations extend into the infinitepast. Even then, there will be a set of indistinguishablestates, each consistent with the series of observationsand with the long-time dynamics of the system. Theideal forecast is then an ensemble forecast, where themembers of the ensemble are drawn from the set ofindistinguishable states, and each member weightedwith its likelihood given the available observations. Inthe limit of infinitely large ensembles, this forecast canaccurately quantify the relative probability of differentevents and the decay of predictability, correctly reflectthe variations in each from day to day.

In practice, ensemble forecasting is a Monte Carloapproach to estimating the probability density func-tion (PDF) of future model states given uncertaininitial conditions. An ensemble forecast for the Lorenz1963 system is shown in Figure 2. The vertical axis istime, the horizontal axis is the variable x from theLorenz system, and each line at constant time repre-sents the probability density function of x at that time.

At t ¼ 0 the system is near x � 0 and the initialensemble consists of 512 initial states, each of which isboth indistinguishable from the true state given theobservations and also consistent with the long-termdynamics of the system (that is, ‘on the attractor’).This constitutes a perfect ensemble. While only thevalue of the x component is shown, each member ofthe ensemble is a complete state of the system, andcorresponding figures could be drawn for y and z.

Initially the distribution spreads out as might beexpected, while the average value of x increases. Att � 0:4, however, the volume of the convex hull of theensemble shrinks, showing a true ‘return of skill’ as theensemble enters a region where all uncertaintiesdecrease! Here a BFG forecast at t � 0:4 is expectedto bemore accurate than the corresponding forecast att � 0:2. This is the origin of the discontinuity indoubling times noted above in Figure 1: red points toone side of the line double just before entering theregion, while points in the rainbow bands just acrossthe line enter the shrinking region before they double,and must wait a finite time to be advected out of theshrinking region before they might double. Hence thediscontinuity.

Returning to Figure 2, notice that as distributionreturns near x ¼ 0 at t � 0:7, a small fraction of theensemble members switch to the wing of the attractorwith negative values of x, while the majority makeanother circuit with x >0. Owing to the symmetry ofthe attractor, there is a somewhat artificial return of


skill at t � 1:5. After this, however, the ensemblemembers dividemore evenly between the twowings ofthe attractor, and the distribution turns blue. The colorhere reflects the standard deviation of the forecastensemble.

At this point, the standard deviation of the forecastis greater than that of a set of points taken at randomfrom the attractor (that is, the climatology). Manyclassical measures of predictability would not see thesignificant information content that the forecast ob-viously continues to possess; good skill scores shouldreflect the information content of the ensemble. Evenmore worrying, the ‘optimal’ least-squares forecast att ¼ 2:7 would be near x ¼ 0 where there is zeroprobability of observing the system. One importantpoint illustrated here is that knowing the mean valueexactly is often of much less value than knowing the

likely distribution of values even approximately. Asecond point is that tuning nonlinear models with theaim of a ‘better’ average least-squares error will makethe models worse, as it systematically forces modelparameters away from more realistic, but heavilypenalized, behavior. Such models are expected to beunderactive rather than realistic.

The information in the initial ensemble will slowlydiffuse away, and whether or not the information inthe forecast at any given time is useful depends on theaims of the user. Eventually, any finite ensemble willitself become indistinguishable from a random drawfrom the climatology. At this point the forecast isuseless, but this time is unrelated to the Lyapunovexponent, or the doubling time, or any other measureof infinitesimal uncertainties.

Accountable Ensemble Forecasts

Corresponding to each probability forecast there isonly a single verification; thus no single forecast can beevaluated. Rather, the quality of a (long) series ofprobability forecasts must be considered. And inas-much as nonlinearity will mix aspects of data assim-ilation (see Weather Prediction: Data Assimilation),ensemble formation and model inadequacy, the en-semble prediction system (EPS) can be evaluated onlyas awhole. This should not come as a surprise, since ina nonlinear system one expects to lose the benefits oflinear superposition.

While the absolute accuracy of the EPS will varywith the level of initial uncertainty, ensemble forecastsunder a perfect model using perfect initial ensemblesare ‘accountable’: the uncertainty in any forecastvariable computed from this ensemble will reflect thetrue value with an accuracy limited only by the finitenumber of members in the ensemble. Karl Popperintroduced the notion of accountability for BFGforecasts in order to illustrate that a good modelshould indicate how accurately the initial conditionmust be measured in order to guarantee the accuracyof a forecast at any fixed lead time. The notion is easilyextended to ensemble forecasts, in that an accountableensemble forecast system should indicate how large aninitial ensemble should be in order to reflect eventsaccurately with a given level of probability.

Of course, the detailed shape of each forecastdistribution will differ from day to day, Figure 2shows the probability distribution for a particularinitial condition and set of observations. Yet if the EPSis accountable, then as the number of members in theensemble is increased, the probability forecast willgrowmore accurate in a predictableway. For example,every time the ensemble size is doubled, the frequency

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

5.50

6.00

1.00

0.50

0.00

�

0.0

20.0

Figure 2 Evolution of a perfect ensemble under a perfect model,

showing the probability distribution of one component of the model

state, the variable x , in the Lorenz system. Time is denoted along

the ordinate, while the abscissa is centered about zero. Each line

reflects a particular time: the height reflects the probability density

at that value of x , while the color of each line reflects the standard

deviation of the ensemble at that time. (Adapted with permission

fromFigure 23 of Smith LA (1997) Themaintenance of uncertainty.

Proceedings of the International School of Physics ‘‘EnricoFermi ’’,

course CXXXIII, pp. 177–246, Societa Italiana di Fisica, Bologna,

Italy.)


with which any particular variable in the verificationwill fall outside the ensemble will be cut in half. [Infact, the ensemble size must be increased from Nmembers to ð2N þ 1Þ members, since the probabilitythat the next random draw falls outside the currentrange is 2=ðN þ 1Þ.]

In practice, ensembles are not drawn from a set ofindistinguishable states; there are a number of com-peting methods now used operationally, and othermethods are soon to join them. Current formationschemes include sampling directions of forecast errorsof the recent past, or the directions of fastest growth inthe near future. Neither approach attempts to samplethe initial uncertainty accurately, and thus accurateprobability forecasts could not be expected from theraw forecasts, even were the models to be perfect.

Operational ensembles typically consist of between10 and 100 members, evolved over a duration of twoweeks, although seasonal ensembles are a currenttopic of research. Recalling that operational model-state spaces typically have ten million dimensionsgives an indication of just how difficult sampling theinitial uncertainty may prove to be. Despite thetechnical difficulties, the value of operational ensem-bles is reflected in Figure 3, a 42-hour forecast for 26December 1999. The three panels in the top left cornershow the low-resolution (control) and high-resolution(T319) BFG forecasts, and themore colourful analysis(1 0 5)which serves as the verification. The color in theanalysis reflects the intense winter storm that sweptacross Europe. The other 50 panels in the figure eachshow a member of the ensemble forecast at t5142hours. This collection of ‘postage stamp’ maps isanalogous to a single PDF at constant time in Figure 2.Note that about 20% of the ensemble memberscontain storms, and that even though there is noknown way to extract an accountable probabilityestimate from this operational forecast, there issignificantly more information than is provided bythe control forecasts. In its present state, this infor-mation is already of significant societal and economicvalue.

Physical Systems and MathematicalModels

Arguably that, no physical system is ever isolated, andperturbations from outside the system imply that nodynamical system can be perfectly modeled as deter-ministic. What then does one mean by saying that aphysical system is chaotic? Lorenz (1993) suggests aphysical system should be called chaotic if its behaviorwould be chaoticwere it to be isolated. This, of course,assumes there is perfect mathematical description of

the hypothetical isolated system, but it is similar to themanner in which other mathematical terms are inter-preted in physics; for example, the definition ofperiodicity in a physical system. Periodicity is a usefulconcept in physics, although arguably no physicalsystem is truly periodic. Similarly, chaos may be auseful concept within physics, even if no physicalsystem is truly deterministic.

One property that distinguishes periodic and cha-otic systems is that periodic systems eventually returnto exactly the same state x in state space. While thisnever happens in chaotic systems, near returns dooccur for all points on the attractor; the longer is theduration of the observations, the closer are the nearestreturns. Such systems, like the Lorenz 1963model, aresaid to be ‘recurrent’. What does it mean to say aphysical system is recurrent.

At this point one has to leave the perfect-modelscenario behind.Observations of a physical system areat best uncertain measurements of variables in thesystem’s state space (if such a thing exists); in order touse them in the model the observations must be castinto a model-state space. Mathematically, a dataassimilation scheme (see Weather Prediction: DataAssimilation) is simply a projection operator whichaccomplishes this task. Whatever the projection oper-ator may be, the fact that forecasts are made in themodel-state space holds deep consequences for at-tempts to make accountable probability forecasts.Estimates of predictability reflect the limitations of ourmodels, while the underlying physical system is not soconstrained.

Once some method of data assimilation is adoptedso that the observations of the system can be projectedinto the model-state space, one can ask if a physicalsystem is likely to be recurrent within a particularmodel-state space. Will two similar states be observedduring the likely duration of the observations? Overthe lifetime of the system? Often the answer is yes.

Many physical systems are also recurrent within themodel-state space over the time of a typical experi-ment. Near recurrence in the model-state space opensup many modeling possibilities, the simplest being touse (local) linear regression (see Data Analysis: TimeSeries Analysis). It also introduces the possibility thatwe can learn from past mistakes, improving the modelby identifying state-dependent systematic errors. Ofcourse, doing so may increase the dimension of themodel-state space to the extent that given the availableobservations it is no longer recurrent!

Given that the estimated recurrence time of theEarth’s atmosphere is longer than the lifetime of theSolar System (longer, in fact, than the expected lifetimeof the Universe!), this remedy is not availableto meteorologists modeling the global circulation.


L

L

L

L

Forecast 3 Forecast 4 Forecast 5

L


L

Forecast 9 Forecast 10


L

Forecast 16 Forecast 17 Forecast 18 Forecast 19 Forecast 20


L

Forecast 25 Forecast 26 Forecast 27 Forecast 28 Forecast 29 Forecast 30

L

Forecast 33 Forecast 34 Forecast 35 Forecast 36

L


L

L

Forecast 40

Control e1 oper T319

984

Analysis 105

L





L


L

Forecast 44

L981


L


Figure3 Ensemble forecast for the French storm of 1999. Each ‘postage stamp’ is aweathermap of southern England andFrance. The three panels in the top left corner show two best-guess

forecastsmade at differentmodel resolutions and the analysis, which indicates that the verificationwas rather different fromeither of these forecasts. Each of the 50members of the ensemble at

the time of the verification is also shown. (Reproduced with permission of the European Centre for Medium-Range Weather Forecasts.)

PREDIC

TABILITYAND

CHAOS

1783

Of course, one may be able to exploit recurrence inbuilding parameterizations (see Boundary Layers:Modeling and Parameterization) or in applicationson smaller scales over shorter forecast horizons. Butchaos is defined for the system (or model) as a whole.There is a clear anduseful intuition ofwhat ismeant bythe concept of ‘approximately periodic’, which islacking for the phrase ‘approximately chaotic’. Fur-ther, it is not obvious how to interpret physical systemsas chaotic, if they are not expected to exist over thetime scales required for chaos tomanifest itself in theirmathematical analogs.

Loss of Predictability: ModelInadequacy and Shadowing

In addition to uncertainty in the initial condition anduncertainty in parameter values, meteorologists mustcontend with ‘model inadequacy’: there are aspects ofthe real physical system that our model is simplyunable tomimic.When nomodel in the available classof models is structurally adequate to duplicate theobserved phenomena, it is unclear what is meant bythe ‘correct’ initial condition or the ‘true’ parametervalues. While the Bayesian agenda provides a princi-pled scheme for handling uncertainty in initial condi-tion and parameter value, no systematic approach isavailable for handling model inadequacy. Progresshere requires having a good idea.

Recall that in the discussion of ‘uncertainty’ in theinitial condition above, itwas assumed that in additionto abest-guess initial condition, xx, therewas also a trueinitial condition, ~xx. The error in the initial conditionwas defined as the difference between these twopoints.When the model is imperfect, there is no ‘true’ initialcondition (even if the model variables have thesame names as the system variables) and the very con-cept of ‘uncertainty in the initial condition’ has to bereconsidered.

As an example, note that since the current resolutionof NWP models is at best tens of kilometers, manydifferent states of the atmosphere (with differentfutures) will be mapped onto the same state of themodel. This is but one example of the projectioneffects noted above: themodel initial conditions are, atbest, projections of the true system state into themodel-state space. The model cannot, then, be ex-pected to reproduce the evolution of every atmospher-ic state, simply because there are more atmosphericstates than model states! Of course, the model mayhave trajectories which shadow the observed atmos-pheric states, remaining indistinguishable from thetrajectory of the atmosphere given the observationaluncertainty. How might meteorologists distinguish

forecast failures due to limitations in the ensembleformation scheme from those due to model in-adequacy? One approach is to look for ‘shadowingtrajectories’ within the historical observations.

Given an imperfect model, there may or may not bea model trajectory that stays close to the series ofobserved states, no matter which data assimilationmethod is used to translate the observations intomodel states. In this context, ‘close’ must be interpret-ed in relation to the uncertainty in the observations. Amodel trajectory that remains near a set of target statesis said to ‘shadow’ the target states. Each analysis willhave an associated shadowing time, just as it has anassociated value of t2. The distribution of shadowingtimes reflects the relevance of model inadequacy.

If shadowing trajectories exist, then initial condi-tion(s) which shadow may be cast in the role of ‘truth’(that is, the role of ~xx) when computing uncertainty in theinitial condition, at least for forecasts that are shortrelative to the duration over which the model canshadow. This suggests that our very definition of‘observational noise’ will itself depend on the quality ofthe model in hand. Indeed, many data assimilationschemes are based on the assumption that long shadow-ing trajectories exist almost everywhere in state space.

If no shadowing trajectory exists on the time scale ofinterest, then the model mixes ‘uncertainty in theinitial condition’ and ‘model inadequacy’ to the extentthat the former cannot be unambiguously defined. Onthese time scales, all model trajectories differ signifi-cantly from the observations: the set of indistinguish-able states is empty, and there is no optimal method ofensemble formation. Indeed, outside PMS the issue ofmodel improvement is linked to that of forecast usage;there need be no unique best way forward. Neverthe-less, current ensemble forecasts are of great value inidentifying when the forecasts are sensitive to uncer-tainties in the initial condition, since any single BFGforecast can be identified, at the time it is issued, asunlikely to be an accurate anticipation of reality.Hence they can be expected to provide useful identi-fication of when the forecast will be unreliable;empirical studies suggest they are also useful inidentifying forecasts which are likely to have highskill. In addition, when two members of the sameensemble lead to radically different forecasts in themedium range, determining what distinguishes themat short lead times can suggest valuable observationsfor improving the forecast.

In addition to ensembles over initial conditions,research is underway aimed at determining howto better include stochastic effects into nonlinearmodels. Such stochastic effects are commonly referredto as ‘dynamical noise’ to distinguish them fromobservational noise; the latter alters the observations


but not the trajectory. A major difficulty here isformulating a relevant state-dependent dynamicalnoise, as the traditional approaches tend to spreadthe forecast into unphysical directions. Using multiplemodels provides one approach, stochastic parameter-izations is another.

Current research is also exploring the use ofensembles over distinct models, or even ensemblesover trajectories each ofwhich uses a variety of distinctmodels. Ideally, these models should be independent,so that they share as few common inadequacies aspossible. Methods for allocating resources amongmodels, and for the evaluation of the distributions soobtained as forecasts, provide yet other interestingareas of current research. The traditional goal ofidentifying the ‘optimal’ least-squares predictor needno longer be a desirable end for any real forecast user.Modern forecast users, in particular industrial users,are quite capable of exploiting probability forecasts.

Since the introduction of the electronic computer,indeed since L. F. Richardson’s computations early inthe last century, weather prediction has been at theforefront of research into the predictability of nonlin-ear dynamical systems. One safe forecast is that it willremain there for the foreseeable future.

See also

Boundary Layers:Modeling and Parameterization.DataAnalysis: Time Series Analysis. Weather Prediction:Data Assimilation; Ensemble Prediction.

Further Reading

Lorenz (1993) provides a general introduction to boththe history and physics of chaos, while Smith (1998)is an overview of the implications chaos holdsfor philosophy as well as a general introduction.Introductions to both operational and theoreticalensemble forecasts can be found in Palmer (2000) andSmith et al. (1999), respectively. A general discussion ofthe role of model inadequacy in predictablity fromphilosophical, physical, and Bayesian perspectives canbe found in Cartwright (1983), Smith (2001), andKennedy andO’Hagan (2001), respectively and referencethereof.

Cartwright N (1983)How the Laws of Physics Lie. Oxford:Oxford University Press.

Kennedy M and O’Hagan A (2001) Bayesian calibration ofcomputer models. Journal of the Royal Statistical SocietySeries B 63: 425–464.

Lorenz E (1993) The Essence of Chaos. London: UniversityCollege London Press.

Palmer TN (2000) Predicting uncertainty in forecasts ofweather and climate. Reports on Progress in Physics 63:71–116.

Smith LA (2001)Disentangling uncertainty and error: on thepredictability of nonlinear systems. In: Mees AI (ed.)Nonlinear Dynamics and Statistics, pp. 31–64. Boston,MA: Birkhauser.

Smith LA, Ziehmann C and Fraedrich K (1999) Uncertaintydynamics and predictability in chaotic systems.Quarter-ly Journal of the Royal Meteorological Society 125:2855–2886.

Smith P (1998) Explaining Chaos. Cambridge: CambridgeUniversity Press.


PREDICTABILITY AND CHAOSIonospheric Physics. London: Academic Press. Roble RG (1986) Chemistry in...

Documents

Transcript of PREDICTABILITY AND CHAOSIonospheric Physics. London: Academic Press. Roble RG (1986) Chemistry in...