NONLINEAR REGRESSIONS WITH INTEGRATED TIME SERIES BY...

NONLINEAR REGRESSIONS WITH INTEGRATED TIME SERIES

BY

JOON Y. PARK and PETER C.B. PHILLIPS

COWLES FOUNDATION PAPER NO. 1016

COWLES FOUNDATION FOR RESEARCH IN ECONOMICS YALE UNIVERSITY

Box 208281 New Haven, Connecticut 06520-8281

2001

http://cowles.econ.yale.edu/

Ž .Econometrica, Vol. 69, No. 1 January, 2001 , 117�161

NONLINEAR REGRESSIONS WITH INTEGRATEDTIME SERIES

BY JOON Y. PARK AND PETER C. B. PHILLIPS1

An asymptotic theory is developed for nonlinear regression with integrated processes.The models allow for nonlinear effects from unit root time series and therefore deal withthe case of parametric nonlinear cointegration. The theory covers integrable and asymp-totically homogeneous functions. Sufficient conditions for weak consistency are given anda limit distribution theory is provided. The rates of convergence depend on the propertiesof the nonlinear regression function, and are shown to be as slow as n1�4 for integrablefunctions, and to be generally polynomial in n1�2 for homogeneous functions. Forregressions with integrable functions, the limiting distribution theory is mixed normal withmixing variates that depend on the sojourn time of the limiting Brownian motion of theintegrated process.

KEYWORDS: Functionals of Brownian motion, integrated process, local time, mixednormal limit theory, nonlinear regression, occupation density.

1. INTRODUCTION AND HEURISTIC IDEAS

THE ASYMPTOTIC THEORY OF NONLINEAR REGRESSION plays a central role ineconometrics, underlying models as diverse as simultaneous equations systemsand discrete choice. In the context of time series applications, a longstandingrestriction on the range of potential applications has been the availability ofsuitable strong laws or central limit theorems, effectively restricting attention to

Žmodels with stationary or weakly dependent data. While it is well known e.g.,Ž ..Wu 1981 that consistent estimation does not rely on assumptions of stationar-

ity or weak dependence, the development of a limit distribution theory has beenhamstrung by such restrictions for a very long time.

Two examples in econometrics where these restrictions are important areGMM estimation and nonlinear cointegration. GMM limit theory was originally

Ž Ž ..developed for ergodic and strictly stationary time series Hansen 1982 forwhich all measurable functions are stationary and ergodic, so that applications

Ž .of strong laws and central limit theory CLT are straightforward. Althoughsome attempts have been made to extend the theory to models with determinis-

Ž Ž . Ž ..tically trending data e.g., Wooldridge 1994 , Andrews and McDermott 1995 ,traditional CLT approaches have still been used and no significant progress has

1 The authors thank a co-editor and three referees for helpful comments on earlier versions ofthe paper and Yoosoon Chang for many helpful discussions on the subject matter of the paper. Theresearch reported here was begun in 1994 and the first version of the paper was completed inJanuary 1998. Park thanks the Department of Economics at Rice University, where he is an AdjunctProfessor, for its continuing hospitality, and the Cowles Foundation for support during several visitsover the period 1994�1998. Park acknowledges research support from the Korea Research Founda-tion. Phillips thanks the NSF for support under Grant No. SBR 94-22922 and SBR 97-30295. Thepaper was typed by the authors in SW2.5.

117

J. Y. PARK AND P. C. B. PHILLIPS118

been made. Nonlinear cointegrating models also seem important in a range ofŽ Ž ..applications e.g., Granger 1995 and models with nonlinear attractor sets have

been popular in economics for many years. Yet, the statistical analysis of suchmodels with trending data has been effectively restricted to models that are

Ž Ž .linear in variables and nonlinear in parameters Phillips 1991 ; SaikonnenŽ ..1995 . In fact, in such models not only is a limit theory undeveloped, but ratesof convergence are also generally unknown and this inhibits the use of thetraditional machinery of asymptotic analysis. Saikonnen put it this way in the

Ž .conclusion of his recent paper 1995 :Ž .‘‘ . . . the limiting distribution of a consistent nonlinear ML estimator may not be

obtained in the conventional way unless something is known about the order of consis-tency.’’

Indeed, traditional methods limit our understanding somewhat further than thisstatement indicates, because they do not provide a basis even for the asymptoticanalysis of sample moments of nonlinear functions of nonstationary data,quantities that are fundamental to our understanding of the simplest of regres-sions.

The purpose of the present paper is to introduce new machinery for theasymptotic analysis of nonlinear nonstationary systems. The mechanism for theasymptotic analysis of linear systems of integrated time series that was intro-

Ž . Ž .duced in Phillips 1986, 1987 and Phillips and Durlauf 1986 relied on weakconvergence in function spaces, the use of the continuous mapping theorem,and weak convergence of martingales to stochastic integrals. These methodshave been in popular use ever since and play a major role in nonstationary timeseries analysis. However, they are unequal to the task of analyzing even simplenonlinear functionals, as the following example makes clear.

Let x be a standard Gaussian random walk with zero initialization, andtŽ . �1�2 � �define a process X r �n x for 0� r�1 with � denoting the greatestn � nr �

integer part. Then, X � W, where W is standard Brownian motion. Then dŽ . Ž 2 .nonlinear function f x �1� 1�x is everywhere continuous and is well

behaved at the limits of the domain of definition of x . What then is the limittn Ž .behavior of the sample sum Ý f x ? The standard approach outlined in thet�1 t

previous paragraph suggests the following analysis:n n n1 1 1 1 1 dr1Ž .1 � � �Ý Ý Ý H2 2 2 2n n1�x 1 x x Ž .0 X rt t tt�1 t�1 t�1 n� ž / ž /' 'n n n

dr1� .Hd 2Ž .0 W r

However, while this approach looks convincing, it fails to deliver a useful resultbecause the limit is undefined. Indeed, the behavior of the integral is dominated

Ž .by the local behavior of the Brownian motion W r in the vicinity of the originŽ Ž .. Ž .and it is well known e.g., Shorack and Wellner 1986 that W r satisfies a local

NONLINEAR REGRESSIONS 119

law of the iterated logarithm at the origin, so that

Ž .W rlim sup �1,

1r�0�2 r log log( ž /r

and hence

� �1 1dr� dr�� a.s.H H2 1Ž .0 0W r 2 r log log ž /r

1 Ž .2 Ž .for ��0. Thus, H dr�W r �� a.s. and all we have shown in 1 is that0n Ž 2 .Ý 1� 1�x diverges as n��. Thus, as intimated earlier, traditional meth-t�1 t

ods including those of functional weak convergence, which have become verypopular in time series econometrics, fail to reveal even the order of magnitude,not to mention the limiting form, of sample moments of nonlinear functions ofnonstationary series.

How then do we analyze the limit behavior of this apparently simple function?Our new approach is conceptually very easy. The key notion is to transport thesample function into a spatial function that relies on the good behavior of thefunction itself. In essence, we replace the sample sum by a spatial sum and treatit as a location problem in which we use the average time spent by the process in

Ž . Ž � � .the vicinity of spatial point s, i.e., 1�2� � time x s�� , s�� ; t�1, . . . , n .t' 'Ž .Noting that x is of stochastic order O n we set �� n � for some small� n�� p��0. The heuristic development that follows outlines the essential ideas. Theseare made rigorous in the rest of the paper.

We start by writing

Ž .max xn t� n t1 1 1 2� 1Ž .2 � �Ý Ý2 2' ' 2�1�x 1�sn ntt�1 Ž .min xt� n t

Ž � � .� time x s�� , s�� ; t�1, . . . , nt

Ž .max xt� n t 2� 1 1� �Ý 2 n 2�1�sŽ .min xt� n t

x s st� time �� , �� ; t�1, . . . , nž /' ' 'n n n

Ž .max xt� n t 2� 1 1� �Ý 2 n 2�1�sŽ .min xt� n t

n x s st� 1 �� , �� ,Ý ½ 5' ' 'n n nt�1


� 4where here and elsewhere in the paper we denote the indicator function by 1 � .Ž .The essential simplification that is involved in the transition to the form 2 is

that it converts a nonlinear function of x into a formulation that involves xt tlinearly, inside the indicator function, so that it can be readily standardized by'n .

Ž . Ž .Now, as n��, we note that max x ��, min x ��, so that fort � n t t � n tlarge n and small � we have the spatial approximation

Ž .max xt� n t �2� ds� .Ý H2 21�s 1�s��Ž .min xt� n t

Also for all finite s we haven1 1 x s st

� 1 �� , ��Ý ½ 5' ' 'n 2� n n nt�1

n1 1 xt � �� 1 �� , �Ý ½ 5'n 2� nt�1

1 1 � � Ž . � 4� 1 X r �� drH n2� 0

1 1 � � Ž . � 4� 1 W r �� dr .H2� 0

From these heuristics we get the approximate expressionn �1 1 ds 1 1Ž . Ž � Ž . � .3 � 1 W r �� dr ,Ý H H2 2ž / ž /' 2�1�x 1�sn �� 0tt�1

which is given in terms of the product of a spatial integral and a functional ofthe limiting Brownian motion process. Note that the resulting formula is free ofthe sample size, so that the order of the magnitude of the sample function is

Ž .now properly determined, as distinct from 1 .Ž .The final step in these heuristics is to simplify 3 . Noting that � was arbitrary,

Ž .we can let ��0 in 3 . In fact, the final expression has a natural limit as ��0that measures the spatial density of Brownian motion over the time interval� �0, 1 . Specifically, the limit

1 1Ž . Ž . Ž � Ž . � .4 L 1, 0 � lim 1 W r �� drHW 2��0� 0

is well defined and is known as the local time of standard Brownian motion atŽ Ž . .the origin cf. equation 15 below . It is analogous to a probability density, but

is a random process rather than a deterministic function. Local time is a veryuseful process associated with Brownian motion and it will be used extensivelyin the development of our theory, so more exposition and discussion of itsproperties is provided in Section 2 of the paper. For the moment, we are content


Ž . Ž .to note that using 3 and 4 , our heuristics lead us to the following asymptoticbehavior as n��:

n �1 1 dsŽ . Ž .5 � L 1, 0 .Ý Hd W2 2ž /' 1�x 1�sn ��tt�1

Clearly, this limit expression is very different from the usual limit formulae forsample moments of linear functionals of integrated processes, yet it is simple

Ž .and neat. Obviously, the heuristic argument that leads to 5 could have beenused to obtain the limit behavior of the sample mean of an arbitrary integrablefunction f , specifically

n �1Ž . Ž . Ž . Ž .6 f x � f s ds L 1, 0 ,Ý Ht d Wž /'n ��t�1

a formula that we will derive rigorously in an extended form in Theorem 3.2 ofSection 3.

Ž .In addition to studying the limit behavior of sample means like 6 , a theory ofregression also requires that we be able to analyze sample covariances betweennonstationary and stationary random elements. In the simple case of linearregression, this amounts to studying the sample covariance between an inte-

Ž .grated process and a stationary process. It was shown in Phillips 1987 andŽ .Phillips and Durlauf 1986 that this can be accomplished by directly establish-

ing weak convergence of the sample moment to a stochastic integral. Unfortu-nately, the sample covariance of a nonlinear nonstationary random process anda stationary process is as resistant to analysis by these methods as the samplemean function considered above. To illustrate, consider the sample covariance

n Ž . Ž . Ž 2 .Ý f x u between the integrable nonlinear function f x �1� 1�x andt�1 t t t tŽ .an iid 0, 1 process, u . To fix ideas, we take x to be FF -measurable and u tot t t�1 t

� 4 Ž .be FF -measurable with respect to some filtration FF� FF , and suppose Y r �t t n1� � nr �2 Ž .n Ý u � U r , a standard Brownian motion. If we let ��x and allowt�1 t d t t

Ž .� �E u �0, then U and W are correlated Brownian motions withu t t�1covariance � . If the conventional approach were attempted, we would beutempted to write

n n1 1 1 ut' 'Ž .7 n u � nÝ Ýt2 2' '1�x 1 xn nt tt�1 t�1 0� ž /'n n

n Ž . Ž .1 u dY r dU r1 1t n� � � ,Ý H Hd2 2 2'x n Ž . Ž .0 0X r W rtt�1 nž /'n


Ž .�2which is again undefined because the integrand W r is not square integrable.Thus, direct analysis fails, as before, revealing only that the sample covarianceŽ .7 diverges.

n Ž .Taking an alternate approach, we note that Ý f x u is a martingalet�1 t tn Ž .2whose conditional variance is the sample mean function Ý f x , which mayt�1 t

Ž .be treated in the same spatial manner as 6 above, leading to the followinglimit:

n �1 2 2Ž . Ž . Ž . Ž .8 f x � f s ds L 1, 0 .Ý Ht d Wž /'n ��t�1

1� 4This result indicates that n scaling is appropriate for the martingalen Ž .Ý f x u . Further, observe that the conditional covariance between thet�1 t t

n Ž . n n�1 Ž .martingales Ý f x u and x �Ý is Ý f x � , which, for thet�1 t t n t�1 t t�1 t u1 1� n �4 2Ž .scaled quantities n Ý f x u and n x , producest�1 t t n

n�11 1Ž . Ž .9 f x � �O � 0,3 1Ý t u p pž /4 4n nt�1

indicating that the two scaled martingales have uncorrelated limits. A technicalembedding argument then allows us to apply the martingale central limit

1� n4 Ž .theorem to n Ý f x u , showing that this standardized sum converges to at�1 t tscaled normal variate of the form

12� 2Ž . Ž . Ž .f s ds L 1, 0 � , �� N 0, 1 ,H W dž /��

i.e., a mixed normal limiting distribution, with a mixing variate that is given by'Ž .the limit of the conditional variance 8 . It is the n convergence rate of this

14conditional variance that determines the n convergence factor in the sample

1� n4 Ž .covariance n Ý f x u . Interestingly, in the present example, the limitt�1 t tŽ .Brownian motions U and W arising from the two shocks u and aret tŽ . Žcorrelated, but the mixing variate 8 whose randomness depends on W through

.the local time L is independent of � . This independence arises because of theWŽ .zero conditional covariance 9 between the limiting martingales. It is, in fact,

Ž .the presence of the integrable nonlinear function f x in the martingaletn Ž .Ý f x u that secures this independence. In effect, the full sample path of xt�1 t t t

is no longer needed in the limit theory, unlike the linear case which inevitablyŽ . �1�2relies on functionals of the limiting trajectory, W r , of n x , a trajectory� nr �

Ž .that is correlated with U r when � �0. Indeed, the integrable nonlinearuŽ . n Ž .function f x influences the martingale Ý f x u in such a way that onlyt t�1 t t

contributions from x in the neighborhood of the origin are important in thetŽ Ž . .limit as is apparent from 8 , the conditional variance limit . Independence is

then a consequence of the fact that u is orthogonal to x for all t, including oft tn Ž .course, those t that most matter in the sum Ý f x u . This heuristict�1 t t

argument reveals the major role that nonlinearity can play in influencing the


asymptotics of nonstationary series. It also reveals the importance of therequirement that the innovation, u , be a martingale difference.t

With these heuristics behind us, the plan of the rest of the paper is as follows.Section 2 outlines the model we will be using, the assumptions needed, and givessome preliminary discussion of Brownian local time and some of its propertiesthat we utilize in our development. Section 3 provides an asymptotic theory forcertain families of nonlinear functions. This section builds on some earlier work

Ž . 2by the authors in Park and Phillips 1999 hereafter, P and makes rigorousthe heuristic ideas described above. Consistency in nonlinear regression isproved in Section 4 and the limit distribution theory is developed in Section 5.Section 6 concludes. A technical Appendix is provided and is divided into twoparts. Some useful technical lemmas are given in Section 7 and proofs of thetheorems in the paper are given in Section 8.

Ž .A final word in this introduction about notation. For a vector x� x or aiŽ . � �matrix A� a , the modulus � is taken element by element. Therefore,i j

� � Ž � �. � � Ž � �. � �x � x and A � a . The maximum of the moduli is denoted by � , i.e.,i i j� � � � � � � � � �A �max a and x �max x . The notation � is also used to denote thei, j i j i isupremum of a function. For a function f , which can be vector- or matrix-

� �valued, � signifies the supremum norm over a subset K of its domain, so thatK� � � Ž .�f �sup f x . The subset K , over which the supremum is taken, willK x Knot be specified if it is clear from the context. Standard terminologies andnotations in probability and measure theory are used throughout the paper. Inparticular, notations for various notions of convergence such as � , � anda. s. p� frequently appear. The notation � signifies equality in distribution.d dFinally, we denote by R the set of positive real numbers.�

2. THE MODEL AND PRELIMINARY RESULTS

We consider the nonlinear regression model for y given byt

Ž . Ž .10 y � f x , �ut t 0 t

where f : R�Rm �R is known, x and u are the regressors and regressiont terrors, respectively, and is an m-dimensional true parameter vector that lies0

Ž .in the parameter set � . In model 10 , we let x be an integrated process and ut tbe a martingale difference sequence, as will be specified more precisely subse-quently. The model is thus critically different from the standard nonlinearregression with stationary regressors. It can be viewed as a nonlinear cointegrat-ing regression.

Ž .This paper concentrates on the estimation of 10 by nonlinear least squaresŽ .NLS . The approach given here is applicable to other procedures, but to avoidunnecessary complications in this initial development of the theory we will keepthe focus on NLS. If we let

n2Ž . Ž Ž ..Q � y � f x , ,Ýn t t

t�1


ˆ Ž .then the NLS estimator is defined as the minimizer of Q over � , i.e.,n n

ˆŽ . Ž .11 � arg min Q .n n�

2 Ž . n 2Accordingly, an error variance estimate is given by � � 1�n Ý u , whereˆ ˆn t�1 tˆ ˆŽ .u �y � f x , . It is assumed that exists and is unique for all n. Moreover,ˆt t t n n

we assume throughout the paper that � is compact and convex, and is an0interior point of � . This is a standard assumption for stationary nonlinearregression.

Write x more specifically ast

x �x � .t t�1 t

Ž .The initial value x of x may be any O 1 random variable. However, we set0 t px �0 in the paper to avoid unnecessary complications in exposition. In the0

Ž .O 1 case, the initialization does not affect the asymptotic results anyway, as isp2 Ž 1�2 .evident from P . When the initialization is in the distant past and x �O n ,0 p

Žthe initial condition does affect the asymptotic theory e.g., see Phillips and ParkŽ ..1998 and appropriate adjustments to some of our formulae will be required inthis event, but will be fairly obvious. To focus on essentials in our developmentof nonlinear regression, we will retain the simplification x �0.0

For the time series u and , respectively, we define the stochastic processest t� �U and V on 0, 1 byn n

� � � �nr nr1 1Ž . Ž .U r � u and V r � Ý Ýn t n t�1' 'n nt�1 t�0

� �where s denotes the largest integer not exceeding s.

Ž . Ž . Ž . Ž .ASSUMPTION 2.1: a U , V � U, V as n��, where U, V is a ectorn n dŽ .Brownian motion. Moreoer, assume for each n, there exists a filtration FF ,nt

t�0, . . . , n, such that:Ž . Ž . Ž 2 . 2b u , FF is a martingale difference sequence with E u �FF �� a.s. fort nt t n, t�1

Ž � � q .all t�1, . . . , n, and sup E u F �� a.s. for some q�2, and1� t � n t n, t�1Ž .c x is adapted to FF , t�1, . . . , n.t n, t�1

Assumption 2.1 is satisfied for a wide variety of data generating processes.Ž .Condition a is the usual assumption routinely imposed to analyze linear

models with integrated time series, as in our earlier work, Park and PhillipsŽ .1988 . It is known to hold for mildly heterogeneous time series, as well asstationary processes. The martingale difference assumption for the regression

Ž .errors in b is standard in much stationary time series regression and it is usedfor our development of nonstationary regression theory here, although it is notessential in some cases. As is now well known, serial correlation in the errorsand cross correlation between the errors and regressors is allowed in linearcointegrating regression theory. The correlations do not affect, for instance, the


consistency of the least squares estimator, but generally do affect the limitdistribution theory. Since our model includes the linear cointegrating regressionas a special case, it is therefore reasonable to expect that some of our

Ž .subsequent results apply without condition b , perhaps with some modification.It will be pointed out when this is the case. Moreover, as the heuristic discussion

Ž .of the examples in the introduction clarify, condition b does play an importantŽ .role in the limit distribution theory. Under condition c , x becomes predeter-t

mined. The condition can simply be met by choosing the natural filtration ofŽ . Ž . Ž . Ž .u , x for FF . Note that conditions b and c together imply, in particular,t t�1 nt

Ž . Ž .that E y �FF � f x , a.s.t n, t�1 t 0Ž . � �2 � �The stochastic process U , V takes values in D 0, 1 , where D 0, 1 denotesn n

� � � �the space of cadlag functions defined on the unit interval 0, 1 . The space D 0, 1is usually equipped with the Skorohod toplogy. However, it is more convenient

Ž .in our context to topologize it with the uniform topology, and interpret U , Vn nŽ . Ž . � �� U, V in Assumption 2.1 a as weak convergence in D 0, 1 with the supre-d

Ž .mum norm. The reader is referred to Billingsley 1968 for detailed discussionon the subject. It then follows from the so-called Skorohod representation

Ž . Ž 0 0.theorem that there is a common probability space � , FF, P supporting U , Vn nŽ .and U, V such that

Ž . Ž . Ž 0 0 . Ž 0 0 . Ž .12 U , V � U , V and U , V � U, Vn n d n n n n a . s .

� �2in D 0, 1 with the uniform topology. Moreover, we have the following strongapproximation.

Ž . Ž . 0LEMMA 2.1: Let Assumptions 2.1 b and c hold. Then we may represent UnŽ .introduced in 12 by

t nt0U �Un ž /ž /n n

Ž .with an increasing sequence of stopping times in � , FF, P with �0 such thatnt n0

� tntŽ .13 sup � 0a . s .�n1�t�n

Ž .as n�� for any ��max 1�2, 2�q where q is the moment exponent gien inŽ .Assumption 2.1 b .

In the paper, we establish the weak consistency and derive the asymptoticˆ Ž .distribution of the NLS estimator defined in 11 . For our purposes, itn

Ž . Ž 0 0.therefore causes no loss in generality to assume U , V � U , V , instead ofn n n nŽ . Ž 0 0. Ž .U , V � U , V as in 12 . This convention will be made throughout then n d n n

Ž .paper. It allows us to avoid repetitious embedding of U , V in the probabilityn nŽ . Ž 0 0.space � , FF, P , where U , V is defined. Due to the convention introducedn n

above, all the subsequent convergence results of the sample moments with �a. s.and � , as well as those with � , should generally be interpreted as thep d


corresponding ones with � . If, however, the convergence is to a nonrandomd

limit, then we may as well interpret it as � , since � and � are identicalp d p

in such a case.Stronger assumptions on the data generating process for x will often bet

required to fully develop the asymptotics for the nonlinear regressions. We nowintroduce the following assumption.

Ž . Ž . Ž .ASSUMPTION 2.2: Let a , b , and c be gien as in Assumption 2.1. We let:Ž . Ž . � Ž . � � �d �� L � �Ý � � , with � 1 �0 and Ý k � ��, and assumet t k�0 k t�k k�0 k� 4 � � pthat � is a sequence of i.i.d. random ariables with mean zero and E � �� fort t

some p�4, the distribution of which is absolutely continuous with respect toŽ . r Ž .Lebesgue measure and has characteristic function c � satisfying lim � c � �0��

for some r�0.

Assumption 2.2 introduces more restrictive conditions on x , but still permitst

a wide variety of models that are used in practical applications, including allŽ . 0invertible Gaussian ARMA models. Under condition d , it follows that V � V,n d

Ž .as shown in Phillips and Solo 1992 .The asymptotic theory for nonlinear functions of integrated time series

heavily relies on the local time of Brownian motion, or more generally that of acontinuous semimartingale. The background needed is discussed in P2 and

Ž .Phillips and Park 1998 and a full discussion is contained in Revuz and YorŽ .1994 , to which the reader is referred. In brief, for a continuous semimartingale

� �M with quadratic variation process M , the local time of M is defined to be aŽ .two parameter stochastic process L t, s , which satisfies the following impor-M

tant lemma.

Ž .LEMMA 2.2 The Occupation Time Formula : Let T be a nonnegatie transfor-mation on R. Then

�tŽ . Ž Ž .. � � Ž . Ž .14 T M r d M � T s L t , s dsrH H M0 ��

for all t�.

The local time, as a function of its spatial parameter s, has the interpretationŽ .as an occupation density. In formula 14 , the occupation time is defined with

� �respect to the measure d M , which may be regarded as the natural time-scalerŽ .for M in terms of its variation. It is known that the local time L t, s of aM

continuous semimartingale M is a.s. continuous in t and cadlag in s. Due, inŽ . Ž . Ž .particular, to the right continuity of L t, � , we may apply 14 with T x �M


� 41 s�x�s�� to get

tŽ . Ž . � Ž . 4 � �15 L t , s � lim 1 s�M r �s�� d M ,rHM��0 0

Ž .a representation that explains why L �, s is called the local time of M at s.MŽ . � �Roughly speaking, L t, s measures the time, in the time-scale given by M ,M

� �that is spent by M in the vicinity of s over the interval 0, t .Ž .The formula 14 , of course, applies to Brownian motion as a special case. For

Ž .Brownian motion, the result in 14 is known to hold for any locally integrableŽ .transformation T. See, for instance, Chung and Williams 1990, Corollary 7.4 . It

also holds for other diffusion processes such as the Ornstein-Uhlenbeck process,which has been used for the asymptotic analyses of models with near-integrated

Ž .processes as in Phillips 1987 . For the development of our subsequent theory,we will frequently refer to the local time L of the Brownian motion V. ForVnotational simplicity, we will in fact use a scaled local time L of V defined by

Ž . Ž . Ž 2 . Ž .16 L t , s � 1�� L t , s V

where � 2 is the variance of V. It is often much more convenient to present ourŽ .results in terms of L, instead of L . If we apply the formula 14 to V, then weV

have for any locally integrable T

� �t 2Ž Ž .. Ž . Ž . Ž . Ž . Ž .T V r dr� 1�� T s L t , s ds� T s L t , s dsH H H V0 ��

� � 2 Ž .since d V �� dr. The scaled local time L t, s can therefore be regarded asr the actual time spent by V up to time t in the neighborhood of s. It is called

Ž .chronological local time in Phillips and Park 1998 .All our subsequent results are presented in terms of the Brownian motions U

and V introduced in Assumption 2.1, the covariance of which will be denoted by� . The variances of U and V are, as already specified, written as � 2 and � 2,u

Ž .respectively. The scaled local time L of V defined in 16 will also be usedwithout further reference. Finally, some of our theoretical results involveanother vector Brownian motion W. The process W is independent of V, andtherefore of L, and has variance � 2I. Of course, we may, and do, assume that

Ž .W is defined in the common probability space � , FF, P containing the processesU and V. These conventions will be used throughout the paper.

3. ASYMPTOTICS FOR LINEAR FUNCTIONS OF INTEGRATED PROCESSES

3.1. Regular Functions

We now present some preliminary results for nonlinear transformations ofintegrated time series, which are used in our subsequent development of theasymptotic theory of nonlinear regression. These are related to some earlierconcepts introduced in P2. We start with the concept of a regular transforma-tion.


DEFINITION 3.1: A transformation T on R is said to be regular if and only if

Ž .a it is continuous in a neighborhood of infinity, andŽ .b for any compact subset K of R given, there exist for each ��0 continuous

Ž . Ž . Ž . � �functions T , T , and � �0 such that T x �T y �T x for all x�y ��

Ž .Ž .on K , and such that H T �T x dx�0 as ��0.K � �

The regularity conditions in Definition 3.1 are somewhat stronger than thosein P2. However, they are satisfied by most functions used in practical nonlineartime series analyses. The class of regular transformations is closed under theusual operations of addition, subtraction, and multiplication, as we show inLemma A1. Continuous functions are, of course, regular. This can easily be seen

Ž . Ž . Ž .by setting, for any continuous function T on R, T x �T x �� and T x ��

Ž .T x �� with the usual � for the �� formulation of uniform continuity, and�

these functions apply for any compact subset of R. It is also quite clear that anycontinuous function on a compact support is regular. All piecewise continuousfunctions are therefore regular, due to Lemma A1 in the Appendix. Naturally,we call a vector- or matrix-valued function regular when each of its componentsis regular.

Logarithmic functions and reciprocals are not regular. Therefore, our subse-quent theory for regular functions is not directly applicable to these functions.

Ž . Ž . Ž . � � � 4However, for such functions T , say , we may consider T x �T x 1 x ��

Ž . � � � 4T � 1 x �� for some small ��0 instead of T itself. Note that T and T are�

identical over any finite set of nonzero points, if we take ��0 to be smallerthan the minimum of their moduli. Therefore, if x is driven by an error processtwhose distribution is of the continuous type, then T and T are practically�

Ž . Ž .indistinguishable in finite samples. In this case, we have T x �T x for allt � tt�1, . . . , n if ��0 is sufficiently small. Of course, we can make this approachmore rigorous by letting � be n-dependent, say � , such that � �0, andn n

considering the asymptotics of T �T . This is done in P2. We assume through-n � n

out the paper that these conventions are made for logarithmic functions andreciprocal functions.

Extending the theory in P2, we now consider families of functions indexed bysome parameter, rather than individual functions. This extension is needed forthe analysis of nonlinear regressions. In the subsequent development of ourtheory, we are mainly concerned with a family F : R��Rm of functions fromR to Rm with index set � . Below we introduce a regular family of functions,which is fundamental to our analysis. We have already defined the terminologyregular in Definition 3.1 for individual functions, and here it is extended to afamily of functions. The asymptotics for these families then follow. In particular,

�1 n 'Ž .we present limiting results for the sample functions n Ý F x � n , � andt�1 t�1�2 n 'Ž .n Ý F x � n , � u for regular F. Following our terminology in thet�1 t t

Introduction, these will be referred to subsequently as sample mean and samplecoariance asymptotics for F.


DEFINITION 3.2: We say that F is regular on � if

Ž . Ž .a F �, � is regular for all �� , andŽ . Ž .b for all xR, F x, � is equicontinuous in a neighborhood of x.

Ž . Ž .Conditions a and b in Definition 3.2 will be called regularity conditions.Ž .Lemma A2 shows that regularity condition a is sufficient to guarantee that

Ž .both sample mean and sample covariance asymptotics for F �, � are wellŽ . Ž .defined for each �� . Equicontinuity of F x, � in regularity condition b

ensures, as shown in Lemma A3 in the Appendix, the existence of a neighbor-Ž . Ž .hood N of any given � � for which sup F �, � and inf F �, �0 0 � N � N0 0

are regular. This is required for uniform convergence in sample mean asymp-totics. The condition is, of course, automatically satisfied if � is a singleton set.

THEOREM 3.1: Let Assumption 2.1 hold. If F is regular on a compact set � ,then as n��

n1 x 1t Ž Ž . .F , � � F V r , � drÝ Ha . s .ž /'n n 0t�1

Ž .uniformly in �� . Moreoer, if F �, � is regular, thenn1 x 1t Ž Ž . . Ž .F , � u � F V r , � dU rÝ Ht dž /' 'n n 0t�1

as n��.

For regular F, Theorem 3.1 shows that sample mean asymptotics involve arandom mean functional of F, specifically the time average of a nonlinearfunction of the Brownian motion V. The sample covariance asymptotics involvea stochastic integral of F, which will generally have a nonzero mean except forthe special case where � �0, i.e., U and V are independent. This stochasticuintegral also has a non-Gaussian distribution, although in the special case where� �0 it will be a variance mixture of Gaussian distributions. For a family ofu

Ž .homogeneous functions like polynomials , we may also easily apply Theorem3.1 to get asymptotics for moments of unnormalized functions of integrated timeseries. If specialized to the linear or quadratic functions and to a singletonparameter set � , the results in Theorem 3.1 are already well known. Compara-ble results in this case have been obtained earlier by several authors underconditions weaker than Assumption 2.1. The reader is referred to Phillips and

Ž . Ž .Solo 1992 and Hansen 1992 , and the references cited there.

3.2. Function Classes and Asymptotics for Unnormalized IntegratedTime Series

The limit behavior of sample moments of functions of unnormalized inte-grated time series critically depends on the type of function involved, as shown

2 2 Ž .in P . P consider three classes of functionsintegrable functions I , asymptot-


Ž . Ž .ically homogeneous functions H , and explosive functions E . The first classincludes all integrable transformations. The second class comprises functions

Žthat behave asymptotically like homogeneous functions including homogeneous.functions as a special case . This class also includes transformations such as

Ž . � � x Ž x. Ž . � � kT x � log x , e � 1�e , arctan x, and T x � x . The third class is for func-Ž . xtions that increase at an exponential rate, and transformations like T x �e or

� � k xx e belong to this class. The three classes of functions are mutually exclusiveand have no function in common except for the zero function.

Here we consider the first two families of functions studied in P2. IntroducedŽ .below are regularity conditions for the two I- and H- families of functions.

Each family is presented with its asymptotics. Subsequently, we give asymptoticn Ž . n Ž .results for the sample moments Ý F x , � and Ý F x , � u , appropri-t�1 t t�1 t t

ately normalized. As before, we refer to these as sample mean and samplecovariance asymptotics for F.

Ž .3.2 a . Integrable Functions

DEFINITION 3.3: We say that F is I-regular on � if

Ž .a for each � � , there exist a neighborhood N of � and T : R�R0 0 0� Ž . Ž .� � � Ž .bounded integrable such that F x, � �F x, � � �� T x for all �0 0

N , and0Ž . Ž .b for some constants c�0 and k�6� p�2 with p�4 given in Assump-

� Ž . Ž .� � � ktion 2.2, F x, � �F y, � �c x�y for all �� , on each piece S of theiricommon support S�� m S �R.i�1 i

Ž .We call the conditions in Definition 3.3 I-regularity conditions. Condition aŽ .requires that F x, � be continuous on � for all xR, as in standard nonlinear

Ž .regression theory. The condition holds, for instance, if sup � F �, � �� is� �

� Ž . �bounded and integrable, and implies that sup F �, � is bounded and� �

integrable, if � is compact. When � consists only of a single point � , theŽ .boundedness and integrability of F �, � is sufficient for the condition to hold.

Ž .Condition b requires that all functions in the family should be sufficientlysmooth piecewise on their common support, which is independent of � . Thecondition allows for functions that are progressively less smooth as the underly-ing process has higher moments.

THEOREM 3.2: Let Assumption 2.2 hold. If F is I-regular on a compact set � ,then as n��

n �1Ž . Ž . Ž .F x , � � F s, � ds L 1, 0Ý Ht p ž /'n ��t�1


Ž .uniformly in �� . Moreoer, if F �, � is I-regular,1�2n �1

Ž . Ž . Ž . Ž . Ž .F x , � u � L 1, 0 F s, � F s, � � ds W 1Ý Ht t d4 ž /��' t�1n

as n��.

Both in sample mean and sample covariance asymptotics, the convergencerates for functions of integrated time series are an order of magnitude slowerthan they are for stationary time series. Roughly speaking, this reduction inconvergence rate occurs because observations from an integrated time series

'diverge in probabilityat a rate of n for the sample of size n. Any observa-tion, unless it is realized in a neighborhood of the origin, therefore loses itsimpact asymptotically if it is transformed by a function which vanishes at infinityas is the case with an integrable function. The asymptotics for I-regular Finvolve the local time L of the limit Brownian motion V. Note that both thesample mean and sample covariance asymptotics depend upon L only throughits value at the spatial parameter zero. Therefore, only the time that V spends inthe neighborhood of the origin matters for the asymptotics of I-regular func-tions. The sample covariance asymptotics yield a limit distribution that is anormal mixture with a mixing variate given by L. Note that W is independent ofV, and therefore of L.

Ž .3.2 b . Asymptotically Homogeneous Functions

For our asymptotic analysis, the class of locally bounded transformations on Rwill play an important role and we denote this class by TT . Any regularL Btransformation T on R, defined in Definition 3.1, belongs to TT . We often needL Bto consider a subclass TT 0 of TT consisting only of locally bounded transforma-L B L B

Ž .tions that are exponentially bounded, i.e., transformations T such that T x �Ž c � x �. � �O e as x �� for some cR . Also introduced are the class TT of� B

bounded transformations on R, and its subclass TT 0 including all transformationsBthat are bounded and vanish at infinity, i.e., transformations T such thatŽ . � �T x �0 as x ��. As shown in Lemma A3 in the Appendix, these are the

classes of transformations on R for which sample mean and sample covarianceasymptotics can effectively be bounded. Clearly, TT 0 �TT �TT 0 �TT . As forB B L B L Bregular functions, vector- and matrix-valued functions belong to a given class offunctions when all of the individual components belong to the class.

Denote by � the set of parameters, and by Rm2the set of m�m matrices of�

positive numbers. Let z : R ��Rm2be nonsingular, and a, b : R ��Rm2

.� � � �Ž . Ž .We say that a�o z and b�O z on � if, as ��,�1 �1� Ž . Ž . � � Ž . Ž . �z �, � a �, � �0 and z �, � b �, � ��

uniformly in �� . Define Z : R�R ��Rm. The following definition�Ž .determines the asymptotic order of a family of functions Z �, �, � , parameter-

Ž .ized by �� , in terms of z �, � for large �.


DEFINITION 3.4: We say that Z is of order smaller than z on � if

Ž . Ž . Ž . Ž . Ž . Ž .Z x , �, � �a �, � A x , � or b �, � A x , � B � x , �

Ž . Ž . Ž . 0 Ž .where a�o z and b�O z on � , sup A �, � TT and sup B �, �� L B � �

TT 0.B

With the notion introduced in Definition 3.4, we may now be precise aboutthe family of asymptotically homogeneous functions that we will consider.

DEFINITION 3.5: Let

Ž . Ž . Ž . Ž .F � x , � �� , � H x , � �R x , �, �

where � is nonsingular. We say that F is H-regular on � if:

Ž .a H is regular on � , andŽ . Ž . Ž .b R x, �, � is of order smaller than � �, � for all �� .

We call � the asymptotic order and H the limit homogeneous function of F. If� does not depend upon � , then F is said to be H -regular.0

The conditions in Definition 3.5 will be referred to as H-regularity conditionsin our subsequent discussions. Roughly speaking, the class of H-regular func-tions consists of functions that are asymptotically equivalent to some regularhomogeneous functions, which we call their limit homogeneous functions. Con-

Ž .dition b allows us to establish this asymptotic equivalence. The limit homoge-Ž .neous function H introduced in a is defined uniquely due to the negligibilityŽ .condition for the residual term in b . The regularity requirement for the limit

Ž .homogeneous function H in the condition a is necessary to ensure that H haswell defined asymptotics.

THEOREM 3.3: Let Assumption 2.1 hold, and let F be specified as in Definition3.5. If F is H-regular on a compact set � , then as n��

n1 1�1'Ž . Ž . Ž Ž . .� n , � F x , � � H V r , � drÝ Ht a . s .n 0t�1

Ž .uniformly in �� . Moreoer, if F �, � is H-regular, then

n1 1�1'Ž . Ž . Ž Ž . . Ž .� n , � F x , � u � H V r , � dU rÝ Ht t d'n 0t�1

as n��.

The limit theory for H-regular functions of unnormalized integrated timeseries are essentially identical to those of regular transformations with normal-


ized time series, as given in Theorem 3.1. This is because

xt'Ž . Ž .F x , � � n , � H , �t ž /'n

and the residual is negligible in the limit. Notice that we may write the limit for� Ž . Ž .sample mean asymptotics as H H s, � L 1, s ds using the occupation times��

formula. This expression is analogous in form to that of the sample mean limitof I-regular functions. For I-regular functions, only the local time at the originmatters. Here, the local time at all values of the spatial parameter influences thelimit theory for the H-regular functions. Sample covariance asymptotics alsodiffer between I- and H-regular functions. As noted earlier, sample covariancesfor I-regular functions have mixed normal limits. However, for H-regularfunctions, the limit is given in terms of a stochastic integral and is generallynon-Gaussian.

Just as for regular families of functions, I- and H-regular families of functionsare closed under the operations of addition, subtraction, and multiplication. It isobvious that they are closed under addition and subtraction. That they areclosed under multiplication is proved in Lemma A6 in the Appendix. It is also

Ž .straightforward to show that all the regularity I- and H-regularity conditionsŽ .are preserved, if we compose a regular I- and H-regular family with certain

Ž . � �types of functions. For instance, if F is regular I- and H-regular , then so is F .This property will be used in some of our proofs. In subsequent discussion, wesometimes use the term regularity to mean any of I- and H-regularity as well asregularity in the narrow sense. This should cause no confusion.

4. CONSISTENCY

ˆThis section establishes the consistency of the NLS estimator defined innŽ .11 . The conditions for consistency are easy to verify and, in particular, do notrequire differentiability of the regression function. They are satisfied for most ofthe commonly used nonlinear regression functions, and in these cases consis-tency of the NLS is readily established. However, there are some regressionfunctions that are not covered by the conditions we impose for the consistencyresults in this section. They will be considered in the next section, where wederive the asymptotic distributions of the NLS estimator under stronger assump-tions including differentiability of the regression function.

Ž . Ž . Ž .Define D , �Q �Q . To prove consistency, we show one of then 0 n n 0Ž .following two conditions labelled and numbered CN for reference .

�1 Ž . Ž .CN1: For some sequence � of numbers, � D , � D , uniformlyn n n 0 p 0Ž .in as n��, where D �, is continuous and has unique minimum a.s.0 0

Ž .CN2: For any ��0, lim inf inf D , �0 in probability.n�� n 00

Both consistency conditions CN1 and CN2 are sufficient to ensure thatˆ Ž . Ž . � , as shown in earlier work by Jennrich 1969 and Wu 1981 .n p 0


Ž .For the standard nonlinear regression, Jennrich 1969 proves the consistencyŽ .of the NLS estimator by establishing CN1. On the other hand, Wu 1981

derives the consistency of the NLS estimator for possibly nonstationary nonlin-ear regressions through CN2. Given the results in Section 3, it is not hard toshow that the required conditions hold for regressions with various types ofregular regression functions. The Jennrich approach is more appropriate forregression with I-regular and H -regular regression functions, since the regres-0sion functions converge at the same rate for all values of . We therefore showthat CN1 is satisfied for such functions, under some identifying assumption that

Ž .guarantees that D �, has unique minimum . However, this approach is not0 0applicable for general H-regular functions. These functions have different ratesof convergence for different values of , and the results obtained by Wu arethen more relevant. Therefore, CN2 will be shown to hold for these functions.

THEOREM 4.1: Let Assumption 2.2 hold, and let f be I-regular on � . If� Ž Ž . Ž ..2H f s, � f s, ds�0 for all � , then CN1 holds. In particular, we�� 0 0

hae

� 2Ž . Ž Ž . Ž .. Ž .D , � f s, � f s, ds L 1, 0H0 0ž /��

'with � � n .n

Roughly speaking, the result in Theorem 4.1 holds for all integrable functionsthat are bounded and piecewise smooth over their supports. We only require amild identifying assumption for the consistency result to go through. It holds formany density type regression functions, as well as all linear-in-parameter regres-

Ž . Ž .sion functions of the type f x, � a x with nonzero I-regular a. TheoremŽ .4.1 is also applicable for nonlinear regression with regression function f x, �

e� x 2with ��R , as long as p�8.�

It is interesting to compare two nonlinear nonstationary regressions withŽ . � x 2

f x, �e and different specifications for x : the first with an integratedt 'time series x , as in the paper here, and the second with x � t . The latter ist tsometimes referred to as the first order decay model. The two regressors are

'Ž .comparable, since x O t . However, their asymptotic behaviors are drasti-t pcally different. The NLS estimator is consistent for the former, but for the latter

Ž .it is inconsistent, as noted earlier by Malinvaud 1970 . The reason is simple. Inthe deterministic decay model, the signal from the regressor is asymptoticallynegligible because e� t tends monotonically to zero, so in the limit there is noinformation in the mean function about . On the other hand, in the stochastic

'Ž .trend case, while it is true that the stochastic order of x is O t , the processt pis recurrent rather than monotonic and x 2 keeps returning to the vicinity of thetorigin. In consequence, the regressor continues to carry information about theparameter as t��.


THEOREM 4.2: Let Assumption 2.1 hold, and let f be H -regular on � with0asymptotic order � and limit homogeneous function h. Assume:

Ž . Ž .a � � is bounded away from zero as ��, andŽ . Ž Ž . Ž ..2b for all � and ��0, H h s, �h s, ds�0.0 � s � � � 0

Then CN1 holds. In particular, we hae

� 2Ž . Ž Ž . Ž .. Ž .D , � h s, �h s, L 1, s dsH0 0��

2'Ž .with � �n� n .n

Ž .Condition a ensures that there is sufficient variability in the regressionfunction asymptotically to generate a signal stronger than the noise. ConditionŽ .b is simply an identification condition for the regression with H -regular0regression functions. Unfortunately, this condition fails to hold for some com-monly used H -regular regression functions. These can have a limit homoge-0

Ž .neous function h x , say, that is independent of , so that the true limithomogeneous function is not identified. However, we can usually achieveidentification by properly reformulating the regression function in this case.Indeed, we may simply consider a regression with the transformed regression

Ž . Ž . Ž .function f� x, � f x, �h x to avoid the lack of identification. Clearly, theŽ .regression 10 can then be rewritten as

Ž . Ž .y �h x � f� x , �ut t t 0 t

so the regression is effectively the same as one with the regression function f�.

Ž . Ž .EXAMPLE 4.1: a For the linear-in-parameter regression function f x, �Ž . a x , Theorem 4.2 is particularly easy to apply. Note that this f is H -regular0

Ž . Ž . Ž . Ž . Ž .if, in the representation a � x �� b x � r x, � , b is regular and r x, � isŽ .of order smaller than � � . The asymptotic order and limit homogeneous

Ž . Ž . Ž . Ž .function of f are given respectively by � � and h x, � b x . Condition b� Ž . �of Theorem 4.2 is satisfied whenever H b s ds�0 for all ��0. One may� s � � �

now easily see that Theorem 4.2 is applicable for regression functions such asŽ . x Ž x. � � � � kf x, � e � 1�e , log x , and x , among many others. The asymptotic

Ž . korders of these functions are given respectively by � � �1, log �, and � . TheŽ . � 4corresponding limit homogeneous functions are h x, � 1 x�0 , , and

� � k x .Ž . Ž . Ž .�1 � 4b Consider the regression function given by f x, �x 1� x 1 x�0

with ��R . This is a reparameterized and restricted version of the Michaelis-�Ž .Menten model used in Bates and Watts 1988 to fit the relationship between

the velocity of an enzymatic reaction and the substrate concentration. One mayŽ .easily see that it is H -regular with asymptotic order � � �1 and limit0

Ž . �1 � 4homogeneous function h x, � 1 x�0 . Clearly, it satisfies all the condi-tions of Theorem 4.2.


Ž .c The result in Theorem 4.2 is not directly applicable to the regressionŽ . Ž .2 Ž .function f x, � x� , which was considered in Wu 1981 . Clearly, the

Ž . 2 Ž .function is H -regular with asymptotic order � � �� for which condition a0Ž . 2holds. However, this function has limit homogeneous function h x �x for all

Ž .values of , which fails to satisfy condition b . Nevertheless, as indicated above,Ž .we may reformulate the regression with the regression function f� x, �

Ž . Ž . Ž .2 2 2f x, �h x � x� �x �2 x� to apply Theorem 4.2. Obviously, f�Ž .is H -regular with asymptotic order �� and limit homogeneous function0

Ž .h� x, �2 x, and satisfies the conditions of Theorem 4.2.

Ž . x Ž x.EXAMPLE 4.2: The logistic regression function f x, �e � 1�e has theŽ .same lack of identification problem as the model in Example 4.1 c . The

Ž .function is H -regular with the asymptotic order � � �1, and therefore satis-0Ž . Ž .fies condition a . However, the limit homogeneous function is given by h x �

� 4 Ž .1 x�0 and the identification condition b fails. To analyze such a regressionmodel, we need to reformulate the model in terms of the regression function

e x 1Ž . Ž . Ž . � 4 � 4f� x , � f x , �h x � 1 x�0 � 1 x�0 .

x x1�e 1�e

The reformulated regression function f�, however, is no longer H -regular.0However, it is I-regular and satisfies the conditions of Theorem 4.1. So ourasymptotic theory is applicable in this case also.

THEOREM 4.3: Let Assumption 2.1 hold, and let f be H-regular on � withasymptotic order � and limit homogeneous function h. Then CN2 holds if :

Ž .a for any � and p, q�0, there exist ��0 and a neighborhood N of 0such that as ��

Ž . Ž .inf inf p� �, �q� �, ��;0N� �p�p ��

� �q�q ��

Ž . Ž .2b for all � and ��0, H h s, ds�0.� s � � �

Ž . Ž � � .EXAMPLE 4.3: Consider the Box-Cox transformation f x, � x �1 �with ��R . It is straightforward to see that f is H-regular with asymptotic�

Ž . Ž .order � and limit homogeneous function h given by � �, �� and h x, �� x � , respectively. We may easily show that such an f satisfies the conditions

Ž . Ž .of Theorem 4.3. It is obvious that condition b holds. To see that condition aŽ .is satisfied, set 0��min p, q and, for any given � , let N be any0

neighborhood of such that �N.0

COROLLARY 4.4: Suppose that the assumptions in Theorem 4.1 or Theorem 4.2with �� hold. Then � 2 � � 2, as n��.n̂ p


ˆ2By this Corollary, the error variance estimator is consistent in nonlinearnregressions with regression functions that are I-regular or H -regular with0nonexplosive asymptotic order. Consistency for nonlinear regressions with moregeneral H-regular regression functions will be shown in the next section understronger assumptions.

2 Ž . n 2 2It is interesting to note that � � 1�n Ý y � � for a nonlinearn̂ t�1 t pregression with an I-regular regression function. This follows immediately fromTheorem 3.2. We may therefore estimate the error variance directly from y ,trather than residuals. However, the convergence rate of the resulting estimator

2 2 Ž . n 2 2 2 Ž �1�2 .� is slower. For, if we define � � 1�n Ý u , then � �� O n˜ ˜n n t�1 t n n p2 2 Ž �1�2 .whereas � �� o n . This is shown in the proof of Corollary 4.4.n̂ n p

5. LIMIT DISTRIBUTIONS

This section of the paper derives the asymptotic distribution of the NLSˆ Ž .estimator defined in 11 . As in standard nonlinear regression theory, wen

require conditions on the regression function that ensure it is sufficientlysmooth as a function of the unknown parameter . Assuming differentiability ofthe regression function also allows us to establish the consistency of the NLS inmodels where the results in the previous section are not applicable. For suchmodels, the results in this section will give consistency as well as the asymptoticdistribution of the NLS estimator.

Define

� f � 2 f � 3 f�˙ ¨f� , f� , f�ž / ž / ž /� � � � � �i i j i j k

to be all vectors, arranged by the lexicographic ordering of their indices. It issometimes more convenient to define the second derivatives of f in matrix form

¨ 2 ¨ ¨as in F�� f�� . Clearly, we may obtain f from F by stacking its rows into˙a column vector. In what follows, we denote by h the limit homogeneous�˙ ˙ ¨function of H-regular f. Moreover, the asymptotic orders of f , f , and f of�� ˙ ¨H-regular functions will be written as � , � , and � . Whenever f , f , and f are˙ ¨

introduced, we assume that they exist.˙ ¨Now let Q and Q be the first and second derivatives of Q with respect to n n n

˙ ¨ 2defined in the usual way, i.e., Q �� Q �� and Q �� Q �� . We haven n n n

n˙ ˙Ž . Ž .Ž Ž ..Q �� f x , y � f x , ,Ýn t t t

t�1

n n¨ ˙ ˙ ¨Ž . Ž . Ž . Ž .Ž Ž ..Q � f x , f x , �� F x , y � f x , ,Ý Ýn t t t t t

t�1 t�1

ignoring a constant, which is unimportant. As in standard nonlinear regression,ˆthe asymptotic distribution of in our model can be obtained from the firstn


˙order Taylor expansion of Q , which is written asn

˙ ˆ ˙ ¨ ˆŽ . Ž . Ž .17 Q �Q �Q � ,Ž . Ž .n n n 0 n n n 0

ˆ ˙ ˆŽ .where lies in the line segment connecting and . We have Q �0 ifn n 0 n nˆ Ž . is an interior solution to the minimization problem 11 .n

˙Let f be one of the regular functions introduced in Section 3. For anappropriately chosen normalizing sequence � , it follows immediately from then

�1 ˙ ˙Ž . Ž .sample covariance asymptotics in Section 3 that � Q � Q for somen n 0 d 0˙Ž .random vector Q . Also, if we let0

n0¨ ˙ ˙Ž . Ž . Ž .Q � f x , f x , �,Ýn 0 t 0 t 0

t�1

�1 ¨0 �1� ¨ ¨Ž . Ž . Ž .then � Q � � Q for some random matrix Q , due to Lemman n 0 n p 0 0A6 and sample mean asymptotics in Section 3. Therefore, under suitable

�1 ¨ �1� �1 ¨0 �1� ¨Ž . Ž . Ž . Ž .conditions that ensure � Q � �� Q � �o 1 and Q �0n n n n n n 0 n p 0Ž .a.s., we may expect from 17 that

�1 �1 �1� �1ˆ ¨ ˙Ž . Ž . Ž .18 � � �� Q � � Q Ž . Ž .n n 0 n n n n n n 0

�1�1 0 �1� �1¨ ˙Ž . Ž . Ž .�� Q � � Q �o 1Ž .n n 0 n n n 0 p

�1¨ ˙Ž . Ž .� �Q Q d 0 0

as n��.ŽFor easy reference, we list a set of sufficient conditions labelled and num-

. Ž .bered AD for reference that lead to 18 , using the notation introduced above.�1 ˙ ˙Ž . Ž .AD1: � Q � Q as n��.n n 0 d 0

�1 ¨ �1� �1 ¨0 �1�Ž . Ž . Ž .AD2: � Q � �� Q � �o 1 for large n.n n 0 n n n 0 n p

�1 ¨ �1� ¨Ž . Ž .AD3: � Q � � Q as n��.n n 0 n p 0

¨Ž .AD4: Q �0 a.s.0

˙ ˆŽ .AD5: Q �0 with probability approaching to one as n��.n n

�1 ¨ ¨ �1�Ž Ž . Ž ..AD6: � Q �Q � � 0 as n��.n n n n 0 n p

The asymptotic distribution conditions AD1�AD6 are standard in nonlinearŽ . Ž .regression analysis. Given AD1�AD6, 18 follows immediately from 17 .

It is generally simple to check AD1�AD4 for a given nonlinear regression.For all types of regular regression functions, AD1�AD3 directly follow from theresults in Section 3, if we properly choose the normalizing sequence � . More-nover, given AD2, AD4 can readily be deduced under an identifying assumption

˙to avoid asymptotic multicollinearity in f. For regressions with I- or H -regular0˙ ¨f and f , it is also not difficult to show that AD5 and AD6 hold if we presume the

ˆconsistency of , as established in Theorems 4.1 and 4.2. Clearly, AD5 is ann


immediate consequence of the assumption that is an interior point of � .0Moreover, AD6 can also be easily deduced for this type of regression because,

�1 ¨0 �1�Ž .for a normalizing sequence � independent of , � Q � convergesn n n n¨Ž .uniformly to a continuous function Q , say, of .

THEOREM 5.1: Let Assumption 2.2 hold. Assume

Ž .a f satisfies conditions in Theorem 4.1,˙ ¨Ž .b f and f are I-regular on � , and� ˙ ˙Ž . Ž . Ž .c H f s, f s, � ds�0.�� 0 0

Then we hae�1�2

�4 ˆ ˙ ˙' Ž . Ž . Ž . Ž .n � � L 1, 0 f s, f s, � ds W 1Ž . Hn 0 d 0 0ž /��

as n��.

The conditions in Theorem 5.1 hold for a wide range of integrable regressionfunctions that are used in practical applications. They are satisfied, for instance,

Ž . �� x 2 Ž .for the regression function f x, � , � �� e , where � � , � ��R�R .�Clearly, the results in Theorem 5.1 are also applicable for all nonzero I-regularlinear-in-parameter regression functions. For regressions with I-regular regres-

4'sion functions, the NLS estimator converges at the rate of n , and has a mixedGaussian limiting distribution. The asymptotic theory is not likely to provide agood approximation in small samples for regressions with I-regular regressionfunctions, due to the slower than usual rate of convergence. However, thismatter needs to be investigated in simulations.

As one may well expect from our earlier results, the asymptotic behavior ofthe NLS estimator can be quite different for regressions with other types ofregression functions. For regressions with H -regular regression functions, the0' 'Ž .convergence rate is given by n � n . It therefore converges faster than the˙

' 'Ž .standard n rate, when � n diverges, as is usually the case. The limiting˙distribution theory, however, is not Gaussian, except for the special case where� �0. We have the following result in this case.u


Ž .a f satisfies conditions in Theorem 4.2,˙ ¨Ž .b f and f are H -regular on � ,0

Ž . �Ž .�1 �c �� , and˙ ˙ ¨˙ ˙ Ž . Ž . Ž .d H h s, h s, ds�0 for all ��0.� s � � � 0 0

Then�1

1 1ˆ ˙ ˙ ˙' 'Ž . Ž . Ž . Ž .n � n � � � h V , h V , � h V , dU˙ Ž . H Hn 0 d 0 0 0ž /0 0

as n��.


Theorem 5.2 is applicable for all H -regular regression functions considered0in Example 4.1. It is easy to see that the conditions in Theorem 5.2 hold for all

Ž .the linear-in-parameter regression functions in Example 4.1 a . The zero func-tion can of course be regarded as H -regular with any asymptotic order, since it0

Ž .has zero limit homogeneous function. For the regression function f x, ��1 ˙ 2 �2Ž . � 4 Ž . Ž . Ž . � 4x 1� x 1 x�0 in Example 4.1 b , both f x, ��x 1� x 1 x�0

¨ 3 �3Ž . Ž . � 4 Ž . Ž . Žand f x, �2 x 1� x 1 x�0 are H -regular with � � �� 1 ˙ ¨0˙ �2Ž .. Ž . � 4� � and h x, �� 1 x�0 . One may easily check that all the conditions

Ž .of Theorem 5.2 are met. The reformulated regression function f x, �2 x�2 ˙ ¨Ž . in Example 4.1 c also satisfies conditions in Theorem 5.2. Both f and f are

˙Ž . Ž . Ž .H -regular in this case, respectively with � � �� and h x, �2 x, and � � �1˙ ¨0¨Ž . Ž .and h x, �2. Another example would be the regression function f x, � , �

Ž x. Ž x. Ž . 2� ��e � 1�e with � � , � ��R , which represents smooth tran-sition from level � to level �. It is easily seen that this function satisfies CN2and Theorem 5.2 holds.

It is more difficult to establish AD5 and AD6 for general H-regular functions.If we assume consistency, AD5 would follow immediately. However, we still

¨ Ž .cannot invoke the uniform convergence of Q to prove AD6, since thenconvergence rate is dependent upon . Moreover, the conditions of Theorem

ˆ4.3, where the consistency of is established for general H-regular functions,nare somewhat restrictive and do not allow for some commonly used regressionfunctions. Here we do not presume consistency to derive the asymptotic distri-butions. For our approach, we need to introduce the following condition:

AD7: There is a sequence � such that � ��1 � 0, and such thatn n n a. s.

�1 �1�¨ ¨Ž . Ž .sup � Q �Q � � 0Ž .n n n 0 n pNn

� � Ž .� 4where N � : � � �1 .n n 0

Ž .As shown in Wooldridge 1994 , AD7 implies both AD5 and AD6, givenAD1�AD4. Therefore, we may check AD1�AD4 and AD7, instead of AD1�AD6,

Ž .to deduce 18 .ˆWe now present the asymptotic distribution of for regressions withn

˙ Ž . Ž .H-regular f. For notational brevity, we write � � �� , . Moreover, to˙ ˙0 0properly formulate a sufficient set of conditions for AD7, define a neighborhoodof by0

�1��Ž . � Ž . Ž . 4N � , � � : � � � � ��˙0 0

for ��0 given. We have the following theorem.


˙Ž .a f is H-regular on � ,Ž .b for any s�0 gien, there exists ��0 such that as ��


�1 ¨Ž . Ž .Ž . � Ž . �19 � �� sup f �s, �0,˙ ˙0 0 0ž /� �s �s

�1�1�� ¨Ž . Ž .Ž . � Ž . �20 � � �� sup sup f �s, �0,˙ ˙0 0 ž /Ž .� � N � , �s �s

��1�1��Ž . Ž . � Ž . �21 � � �� sup sup f f �s, �0;˙ ˙ ˙Ž .0 0 0 ž /Ž .� � N � , �s �s

˙ ˙Ž . Ž . Ž .c H h s, h s, � ds�0 for all ��0.� s � � � 0 0

Then

�11 1ˆ ˙ ˙ ˙' 'Ž . Ž . Ž . Ž .n � n � � � h V , h V , � h V , dU˙ Ž . H H0 n 0 d 0 0 0ž /0 0

as n��.

˙ Ž .REMARKS: For the regression function f with H-regular f specified as in a ,Ž . Ž . Ž .the identification condition is given by c . The conditions in a and c areŽ .usually easy to check. The conditions in b , however, are awkward and cumber-

some. We may replace them with a stronger, yet easier to verify, condition asdiscussed below.

Ž .a For many H-regular functions, there exists ��0 such that

�1� ¨Ž . Ž .Ž . � Ž . �22 � � �� sup sup f �s, �0˙ ˙0 0 ž /Ž .� � N � , �s �s

Ž . Ž . Ž .for any s�0. Clearly, 19 and 20 are satisfied if 22 holds. Moreover, weŽ .show in the proof of Theorem 5.3 that thrice differentiability of f and 21 are

Ž . Ž . Ž .unnecessary if 22 holds true. Conditions in 19 � 21 can thus be replaced byŽ .22 , which is much simpler to check.Ž . Ž . � � � 4b Define N � � : � �� for ��0. If there exists ��0 such that0

�1�1��Ž . Ž .23 � � � �0˙0

Ž . Ž .as ��, then we have for any ��0 N � , � �N � when � is sufficientlyŽ .large. Therefore, it suffices to show that there exist ��0 satisfying 23 and

�1� ¨Ž . Ž .Ž . � Ž . �24 � � �� sup sup f �s, �0˙ ˙0 0 ž /Ž .� � N �s �s

Ž .for some ��0, instead of 22 . It is indeed quite easy and straightforward toŽ . Ž .show that 23 and 24 hold for many H-regular functions that are used in

nonlinear analyses.


2 Ž . Ž . �EXAMPLE 5.1: Let ��R and write � � , � . Consider f x, � , � �� x .�It is straightforward to see that f is H-regular with asymptotic order � and limithomogeneous function h given respectively by

Ž . � Ž . �� , � , � �� and h x , � , � �x .

Moreover, we have

� � 0 x �

˙Ž . Ž .� �, � , � � and h x , � , � � .˙ � � �ž / ž /�� log � �� x log x

Ž . Ž .It is obvious that conditions 23 and 24 are satisfied for f. To show this, weŽ . Ž .simply let � and � in 23 and 24 be any numbers such that 0�� , �� for0

any � .0The asymptotic results for regressions with general H-regular regression

functions are essentially identical to those for regressions with H -regular0regression functions given in Theorem 5.2. The only difference is that the

'Ž .convergence rate is now given by � n , which is dependent upon the true˙0value of .0

COROLLARY 5.4: Suppose that the assumptions in Theorem 5.3 or Theorem 5.4hold. Then � 2 � � 2 as n��.n̂ p

Corollaries 4.4 and 5.4 establish the consistency of the error variance estima-tor � 2 for regressions with I- and H-regular regression functions. The estima-n̂tor can therefore be used for consistent estimation of the error variance in awide class of nonlinear regressions. In consequence, hypotheses about can betested using standard procedures like the Wald, Lagrange multiplier, andlikelihood ratio tests. The statistical limit theories for these tests are straightfor-ward given the results in this section. For regressions with I-regular regressionfunctions, test statistics of the usual form all have limiting chi-square distribu-tions, respectively, under the assumptions of Theorem 5.1. However, for regres-sions with H-regular regression functions, they are generally dependent upon anuisance parameter generated by � . These become chi-square under theuassumptions of Theorems 5.2 and 5.3, only when � �0.u

6. CONCLUSION

This paper develops some new technology that makes possible the analysis ofnonlinear regressions with unit root nonstationary time series. The techniquesrely on the spatial properties of Brownian motion and these are used to assist inrepresenting the limiting forms of sample moments and sample covariancefunctions of integrated time series. Under fairly general conditions and for anextensive family of nonlinear regression functions, the paper proves the consis-tency of nonlinear regressions, finds rates of convergence, and obtains forms for

4'Ž .the limit distributions. The convergence rates can be both slower n and


'Ž .faster powers of n than that of traditional nonlinear regression, dependingon whether the signal is attenuated or strengthened by the presence of inte-grated regressors. When the regression function involves exponentials, it isshown that the convergence rates are path dependent. For the regressions withintegrable regression functions, the limit distributions of the nonlinear regres-sion estimators are mixed normal as long as the equation errors are martingaledifferences. In such cases, nonlinear inference procedures apply in the usualmanner, so that although the estimators may have non-Gaussian limit distribu-tions, inference is unaffected.

Some remarks on the limitations of this present work are in order. Broadlystated, our purpose has been to initiate nonlinear econometric analysis forstochastically nonstationary time series and provide new tools that are equal tothe task of providing an asymptotic theory. We have not attempted a completetheory that encompasses all, or even most, of the interesting cases that can beexpected to arise in applied work. As we have seen, one of the distinguishingcharacteristics of this new field is that the spatial features of a time series canplay a significant role in the asymptotics. In some cases, even the rate ofconvergence of a nonlinear estimator can be influenced by the sample path ofthe regressors, making a significant departure from traditional nonlinear asymp-totic theory. The models we have studied cover the case of parametric nonlinearcointegration and should prove useful in some empirical studies of nonlinearcointegrating links between economic time series where martingale differenceerrors can be expected, as in formulations arising from rational expectationsmodels. A general treatment that applies to practical cases in which short rundynamics are unspecified is still to be provided. Moreover, a combination of the

Ž .ideas presented here and those in our other paper, Phillips and Park 1998 , isneeded to form the basis of a nonparametric analysis of cointegration. Theresults of this paper are also limited to nonlinear regressions with a singleintegrated regressor. The theory for general regressions with multiple regressorscan be expected to differ, often in substantial ways, from the theory presentedhere. Indeed, such differences are to be anticipated, because the asymptotics forintegrated time series are naturally approximated by functionals of Brownianmotion, and functionals of vector Brownian motion can behave quite differently

Žfrom those of scalar Brownian motion. For instance, it is well known e.g., RevuzŽ .. dand Yor 1994, Ch. V that while vector Brownian motion is recurrent in R for

d�2, it is transient when d�3, so that the spatial behavior of multivariateBrownian motion involves major changes as the dimension rises above theunivariate case. In a certain sense, the commonly cited curse of dimensionality isworse in the nonstationary case because the number of visits to a spatiallocation are more drastically reduced as the dimension increases than they arein a stationary environment. Nevertheless, some statistical theory for nonlinearregressions with more than one integrated regressor is possible, especially ineconometric models that involve a single index, and this theory is currentlyunder development by the authors. Readers are referred to Park and PhillipsŽ .2000 for some work along these lines with binary choice models.


School of Economics, Seoul National Uniersity, Seoul 151-742, Korea;[email protected]; http:��econ.snu.ac.kr� faculty� jpark� index-e.html

andCowles Foundation for Research in Economics, Yale Uniersity, Box 208281,

New Haen, CT 06520-8281, U.S.A.; Uniersity of Auckland and Uniersity ofYork; [email protected]; http:��korora.econ.yale.edu

Manuscript receied July, 1998; final reision receied Noember, 1999.

Ž .APPENDIX A: TECHNICAL RESULTS FOR I 1 FUNCTIONALS

Useful Lemmas

We give several lemmas that will be used repeatedly in the proofs of the main theorems andcorollaries. The proofs of these lemmas are given in the next section.

LEMMA A1: Let T and T be transformations on R. If T and T are regular, then so are T �T1 2 1 2 1 2and T T .1 2

LEMMA A2: Let Assumption 2.1 hold. If T is regular, then

n1 x 1t Ž Ž ..T � T V r dr ,HÝ a . s .ž /'n n 0t�1

n1 x 1t Ž Ž .. Ž .T u � T V r dU r ,HÝ t dž /' 'n n 0t�1

as n��.

Ž . Ž .LEMMA A3: a If F �, � is a regular family on � and � � , then there is a neighborhood N of0 0Ž . Ž .� such that sup F �, � and inf F �, � are regular for all N�N .0 � N � N 0

Ž . � Ž . �b If F is regular on � and � is compact, then sup F �, � is locally bounded.� �

LEMMA A4: Let Assumption 2.1 hold. Then as n��:

n 'Ž . Ž . Ž . Ž .a 1�n Ý T x � n �O 1 for T�TT .t�1 t a. s. L Bn 0'Ž . Ž . Ž . Ž . Ž .b 1�n Ý T x � n T x �o 1 for T TT , T T .t�1 1 t 2 t a. s. 1 L B 2 B

n 0' 'Ž . Ž . Ž . Ž .c 1� n Ý T x � n u �O 1 for TTT .t�1 t t p L Bn 0 0' 'Ž . Ž . Ž . Ž . Ž .d 1� n Ý T x � n T x u �o 1 for T TT , T TT .t�1 1 t 2 t t p 1 L B 2 B

Ž . Ž . Ž .LEMMA A5: a Let Z, z, � and Z , z , � , i�1, 2, be defined as in Definition 3.4, and leti i im Ž . 0W : R��R be such that sup W �, � TT . If Z is of order smaller than z on � , then W�Zw � L B

is of order smaller than I �z. Moreoer, if Z is of order smaller than z on � for i�1, 2, thenm i i iZ �Z is of order smaller than z �z on � �� .1 2 1 2 1 2

Ž . Ž . Ž .b Suppose that Z, z, � and Z , z , � , i�1, 2, be defined as in Definition 3.4 and that Z is ofi i i ' 'Ž . Ž .order smaller than z on � . Also, let Assumption 2.1 hold, and write Z � �Z x � n , n , � andnt t� 1 � 1 n'Ž . Ž . Ž . Ž .z � � z n , � for short. Then we ha e n z � Ý Z � � 0 andn n t� 1 n t a . s .

�1 Ž .�1 n Ž .n z � Ý Z � u � 0 uniformly in � � . Moreoer, for each � � we haen t�1 nt t p�1 �2 Ž .�1 n Ž .n z � Ý Z � u � 0.n t�1 nt t p


LEMMA A6: Let F : R�� R for i�1, 2, and let �� . Define F : R��R byi i 1 2Ž . Ž . Ž . Ž .F �, � �F �, � �F �, � , where �� , � .1 1 2 2 1 2

Ž .a If F is regular on � for i�1, 2, then so is F on � .i iŽ .b If F is I-regular on compact � for i�1, 2, then so is F on � .i iŽ . Ž . Ž . Ž .c If F is H-regular on compact � with asymptotic order � �, � �� , � �� , � and limiti i 1 1 2 2

Ž . Ž .homogeneous function H for i�1, 2, then so is F on � with asymptotic order � �, � �� , � �i 1 1Ž . Ž . Ž . Ž .� �, � and limit homogeneous function H �, � �H �, � �H �, � .2 2 1 1 2 2

Ž .LEMMA A7: a Let Assumption 2.1 hold. If F is regular on a compact set � , then for large n,�1 n 'Ž . Ž .n Ý F x � n , � u �o 1 uniformly in �� .t�1 t t pŽ .b Let Assumption 2.2 hold. If F is I-regular on a compact set � , then for large n

�1 �2 n Ž . Ž .n Ý F x , � u �o 1 uniformly in �� .t�1 t t pŽ .c Let Assumption 2.1 hold. If F is H-regular on a compact set � , then for large n

�1 n'Ž . Ž . Ž .n � n , � Ý F x , � u �o 1 uniformly in �� .t�1 t t p

Ž . 1 Ž Ž . .LEMMA A8: a If F is regular on a compact set � , then H F V r , � dr is continuous a.s. on � .0Ž . � Ž .b If F is I-regular on a compact set � , H F s, � ds is continuous on � .��

Proofs

Ž .PROOF OF LEMMA A1: It is obvious that T �T and T T satisfy regularity condition a , if T1 2 1 2 1Ž .and T do. To show that they also satisfy regularity condition b , let K�R be compact, and for2

Ž .each ��0, let T , T , and � �0 be given accordingly by regularity condition b for T , i�1, 2.i� i� i� iFor each of T�T �T and T �T , we set1 2 1 2

T �T �T , T �T ,� 1� 2 � 1� 2 �

T �T �T , T �T ,� 1� 2 � 1� 2 �

Ž . Ž . Ž . Ž .and � �min � , � . It is obvious that T and T are continuous, T x �T y �T x for all� 1� 2 � � � � �

� � Ž .Ž .x�y �� on K , and H T �T x dx�0, as ��0, as required for the regularity of T�T �T .� K � � 1 2For T�T T , it suffices to look at the case where T , T �0. Given the above result, we write1 2 1 2

T �T��T�, where T� and T� are the positive and negative parts of T , i�1, 2, respectively, andi i i i i iŽ .then consider each product term separately. To show that regularity condition b holds for T , let

T �T T and T �T T� 1� 2 � � 1� 2 �

Ž . Ž . Ž . Ž .and � �min � , � for each ��0. Clearly, T and T are continuous, and T x �T y �T x� 1� 2 � � � � �

� � Ž .Ž .for all x�y �� on K. Moreover, since T and T , i�1, 2, are bounded on K , H T �T x dx� i� i� K � �

�0, as ��0. This completes the proof. Q.E.D.

Ž .PROOF OF LEMMA A2: The first part is due to Park and Phillips 1999 . To prove the second part,Ž . Ž . Ž .first let s �max V r and s �min V r , and define s �max s , �s �1.max 0 � r � 1 min 0 � r � 1 m max min� 4Since s �� a.s., we have P s �c �0 as c��. Therefore, we may take c�0 large so thatm m

� 4 � �P s �c is arbitrarily small. Fix c�0 large, and define K� �c, c . We then denote by T and Tm � �

Ž .the functions given for each ��0 by regularity condition b on the compact set K. In view ofŽ .regularity condition a , we may find T and T such that they are continuous on R, and T �T is� � � �

bounded.Ž . Ž .Let T �T or T . Since T is continuous, T V � T V . Therefore, by Kurtz and Protter� � � � � n a. s. �

Ž .1991 ,

1 1Ž . Ž . Ž .25 T V dU � T V dUH H� n n d �0 0


as n��. It therefore suffices to show that as ��0

1 1Ž . Ž . Ž .26 T V dU � T V dU � 0,H Hn n � n n p0 0

uniformly for all large n including n��, in which case by convention V and U reduce to V and Un nrespectively.

Define

1 2Ž Ž . Ž .. � � � 4A � T V �T V 1 V �c ,Hn� � n � n n0

1 2Ž Ž . Ž .. � � � 4B � T V �T V 1 V �c .Hn� � n � n n0

Then we have

21 1 22Ž Ž . Ž .. Ž Ž . Ž ..E T V �T V dU �� E T V �T VH Hn � n n � n � nž / ž /0 0

2 Ž .�� E A �B .n� n�

Ž .The result in 26 will therefore follow if we show that E A and E B can be made arbitrarily smalln� n�

for all large n by choosing ��0 sufficiently small.2Ž . Ž Ž . Ž .. � � � 4Let D x � T x �T x 1 x �c . Since D is regular, we have by the result in the first part� � � �

of the lemma that

1 1Ž . Ž .A � D V � D V �A .H Hn� � n a . s . � �0 0

Moreover,

2Ž Ž . Ž .. Ž .A � T s �T s L 1, s dsH� � �K

� � Ž . Ž Ž . Ž ..� T �T sup L 1, s T s �T s ds� 0,H� � � � a . s .ž /KsK

as ��0. It now follows that E A �E A as n��, and E A �0 as ��0, since A and A aren� � � n� �

all bounded. Consequently, E A can be made small for all large n by choosing ��0 appropriately.n�

Finally, notice that

2� � � 4B � T �T 1 s �cn� � � m

for all large n including n��, and therefore,

2� � � 4E B � T �T Pr s �c ,n� � � m

Ž .which, as we noted earlier, can be made arbitrarily small by taking c large. We now have 26 , whichŽ .along with 25 , completes the proof. Q.E.D.

Ž . Ž .PROOF OF LEMMA A3: For part a , let � � and a compact set K�R be given. Since F �, �0 00 0is regular, there exist for each ��0 continuous T , T , and � �0 satisfying� � �

0 0Ž . Ž . Ž .T x �F y , � �T x� 0 �

0 0� � Ž .Ž .for all x�y �� on K , and H T �T x dx�0 as ��0. However, due to the equicontinuity of� K � �

Ž .F x, � , there is a neighborhood N of � such that0 0

0 0Ž . Ž . Ž .T x ��F y , � �T x ��

� �for all �N and all x�y �� on K.0 �


We now let

0 0Ž . Ž . Ž . Ž .T x �T x �� and T x �T x �� .� � � �

It is easy to see that T and T are continuous, and for all N�N ,� � 0

Ž . Ž . Ž . Ž .T x � sup F y , � , inf F y , � �T x� ��N�N

� � Ž .Ž .for all x�y �� on K , and finally, H T �T x dx�0 as ��0 since K is bounded.� K � �

Ž . Ž .To prove part b , use the result in part a to deduce that for every � � there exists a0Ž . Ž .neighborhood N such that sup F �, � and inf F �, � are regular, and therefore locally0 � N � N0 0

� Ž . �bounded. The local boundedness of sup F �, � now follows directly from the compactness� �

of � . Q.E.D.

� � Ž .PROOF OF LEMMA A4: Let K� s �1, s �1 . Part a is trivial, becausemin max

n1 xt� �T � T �� a.s.KÝ ž /'n nt�1

Ž .for large n. Part c is also immediate since

2 2n n1 x 1 xt t2 2 2� �E T u �� E T �� E T ,KÝ Ýtž / ž /ž / ž /' ' 'nn n nt�1 t�1

which is finite, due to that TTT 0 .L BŽ . Ž . 0For the proofs of parts b and d , recall that for all TTTL B

n1Ž .T x � 0,Ý t a . s .n t�1

Ž . Ž .as n��. This is shown in Park and Phillips 1999 . To show part b , we note that

n n1 x 1t Ž . � � � Ž . �T T x � T T x � 0KÝ Ý1 2 t 1 2 t a . s .ž /'n nnt�1 t�1

0 Ž .since T TT and T TT . Finally, we observe for the proof of part d that1 L B 2 L B

2n n1 x 1t 2 22Ž . � � Ž .T T x � T T x � 0KÝ Ý1 2 t 1 2 t a . s .ž /'n nnt�1 t�1

since T TT and T TT 0 . Moreover, we note that1 L B 2 L B

2n1 xt 2 2 2Ž . � � � �T T x � T TKÝ 1 2 t 1 2ž /'n nt�1

0 � 2 �and, since T TT , E T ��. It therefore follows by dominated convergence thatK1 L B 1

2 2n n1 x 1 xt t 22Ž . Ž .E T T x u �� E T T x �0,Ý Ý1 2 t t 1 2 tž / ž /ž / ž /' ' 'nn n nt�1 t�1

which completes the proof. Q.E.D.

Ž . 0 0PROOF OF LEMMA A5: Part a follows directly from Definition 3.4. Note that both TT and TTB L BŽ .are closed under multiplication. To prove the first two results in part b , let

Ž . Ž . Ž . Ž .T x � sup A x , � and S x � sup B x , � ,��


' 'Ž . Ž . Ž . Ž .and write a � �a n , � and b � �b n , � . We haven n

n n1 1 xt�1 �1Ž . Ž . � Ž . Ž . �z � Z � � z � a � TÝ Ýn nt n n ž /ž /'n n nt�1 t�1

n1 xt�1� Ž . Ž . � Ž .or z � b � T S x ,Ýn n tž /ž /'n nt�1

and then the first result follows immediately from Lemma A4. To deduce the second result, notethat

n n1 1 xt�1 �1Ž . Ž . � Ž . Ž . � � �z � Z � u � z � a � T uÝ Ýn nt t n n tž /ž /'n n nt�1 t�1

n1 xt�1� Ž . Ž . � Ž . � �or z � b � T S x u ,Ýn n t tž /ž /'n nt�1

and subsequently observe that

1�2 1�22n n n1 x 1 x 1t t 2� � Ž .T u � T u �O 1 ,Ý Ý Ýt t pž / ž / ž /ž /' 'n n nn nt�1 t�1 t�1

1�2 1�22n n n1 x 1 x 1t t 2 2Ž . � � Ž . Ž .T S x u � T S x u �o 1 ,Ý Ý Ýt t t t pž / ž / ž /ž /' 'n n nn nt�1 t�1 t�1

2 0 2 0 Ž .due to Lemma A4, since T TT and S TT . The third result in part b is an immediateL B Bconsequence of Lemma A4. Q.E.D.

Ž .PROOF OF LEMMA A6: For the proof of part a , assume that F and F are regular. It follows1 2Ž .immediately from Lemma A2 that regularity condition a holds for F, since it is the product of two

Ž .regular functions F and F . To show that F satisfies regularity condition b , we fix x and1 2 0Ž 0 0 .� � � , � arbitrarily, and let ��0 be given. Due to the regularity of F , there exists ��0 such0 1 2 i� Ž . Ž 0 .� � � � �that F x, � �F x, � �� for all x�x �� and � �� . We therefore havei i i i 0 i 0

� Ž . Ž . � � Ž . Ž 0 . � � Ž . �F x , � �F x , � � F x , � �F x , � F x , �0 1 1 1 1 2 2

� Ž 0 . � � Ž . Ž 0 . �� F x , � F x , � �F x , �1 1 2 2 2 2

Ž � Ž 0 . � � Ž 0 . � .�� max F x , � , F x , � ��1 1 2 2

� � � � Ž .for all x�x �� and �� . This establishes regularity condition b for F.0 0Ž . Ž .To prove part b , let F and F be I-regular. To show that I-regularity condition a is satisfied1 2

for F, we choose arbitrary � 0 and � 0. Since the F are I-regular, there exist neighborhoods N 0 of1 2 i i� 0 and bounded and integrable T such thati i

� Ž . Ž 0 . � � 0 � Ž .F x , � �F x , � � � �� T xi i i i i i i

0 Ž 0 0 . 0 0for all � N . Therefore, if we let � � � , � , then it follows for all �N �N �N thati i 0 1 2 0 1 2

� Ž . Ž . �F x , � �F x , � 0

� Ž . Ž 0 . � Ž . Ž . � Ž . Ž 0 . �� F x , � �F x , � S x �S x F x , � �F x , �1 1 1 1 2 1 2 2 2 2

� 0 � Ž . Ž . � 0 � Ž . Ž .� � �� T x S x � � �� S x T x1 1 1 2 2 2 1 2

� � Ž .� �� T x ,0


Ž . � Ž .� Ž .where we set S x �sup F x, � and T�max T , T , S , S . Note that S are bounded andi � � i i 1 2 1 2 ii

integrable, since � are assumed to be compact. Therefore, T is bounded and integrable. Finally, weilet

� Ž . Ž . � � � kF x , � �F y , � �c x�y ,i i i i i

Ž� � � �. Ž .and a�max F , F and b�max c , c . Then it follows immediately that1 2 1 2

� Ž . Ž . �F x , � �F y , �

� Ž . Ž . Ž . Ž . �� F x , � �F x , � �F y , � �F x , �1 1 2 2 1 1 2 2

� Ž . Ž . Ž . Ž . �� F y , � �F x , � �F y , � �F y , �1 1 2 2 1 1 2 2

Ž � Ž . Ž . � � Ž . Ž . �.�a F x , � �F y , � � F x , � �F y , �1 1 1 1 2 2 2 2

� � k�ab x�y ,

Ž .which proves that I-regularity condition b also holds for F.Ž .For part c , let

Ž . Ž . Ž . Ž .F � x , � �� , � H x , � �R x , �, � ,i i i i i i i i

for i�1, 2, and define

Ž . Ž . Ž . Ž . Ž . Ž .� �, � �� , � �� , � and H x , � �H x , � �H x , � .1 1 2 2 1 1 2 2

Ž . Ž .As shown in part a , H is regular, and the H-regularity condition a is satisfied. Moreover, if wewrite

Ž . Ž . Ž . Ž .F � x , � �� , � H x , � �R x , �, � ,

then the residual function R becomes

Ž . Ž . Ž .R x , �, � �R x , �, � �R x , �, �1 1 2 2

Ž . Ž . Ž .�� , � H x , � �R x , �, �1 1 1 1 2 2

Ž . Ž . Ž .�� , � H x , � �R x , �, � .2 2 2 2 1 1

Ž . Ž . Ž .It therefore follows immediately from Lemma A5 a that R x, �, � is of order smaller than � �, �Ž . Ž .on � , and so H-regularity condition b is also met. This completes the proof for part c . Q.E.D.

PROOF OF LEMMA A7: In what follows, we assume w.l.o.g. that F is real-valued by taking eachŽ .component separately. For part a , let � � be chosen arbitrarily. We show that0

n1 xtŽ . Ž .27 sup F , � u �o 1 ,Ý t pž /'n n�N t�10

for some neighborhood N of � , from which the stated result follows immediately because of the0 0compactness of � . From Theorem 3.1

n1 xtŽ . Ž .28 F , � u �O 1 ,Ý 0 t pž /' 'n nt�1

so it suffices to show that

n1 x xt tF , � �F , � uÝ 0 tž / ž /ž /' 'n n nt�1

can be made arbitrarily small a.s. uniformly in �N , which we now set out to do.0


Using Cauchy�Schwarz we have

n1 x xt tŽ .29 F , � �F , � uÝ 0 tž / ž /ž /' 'n n nt�1

1�2 1�22n n1 x x 1t t 2� F , � �F , � u .Ý Ý0 tž / ž /ž / ž /' 'n nž /n nt�1 t�1

Ž .However, it follows from Lemma A6 a and Theorem 3.1 that

2n1 x x 1t t 2Ž Ž Ž . . Ž Ž . ..F , � �F , � � F V r , � �F V r , � dr ,HÝ 0 a . s . 0ž / ž /ž /' 'n n n 0t�1

uniformly in �� . Let N be the �-neighborhood of � . Then, for any xR� 0

Ž . Ž .sup F x , � �F x , � �00�N�

Ž . � Ž . �as ��0, due to the continuity of F x, � . Since sup F �, � is locally integrable as shown in� �

Ž .Lemma A3 b , we may invoke dominated convergence to get

1 2Ž Ž Ž . . Ž Ž . ..F V r , � �F V r , � drH 00

� 2Ž Ž . Ž .. Ž .� F s, � �F s, � L 1, s ds� 0H 0 a . s .��

Ž .uniformly on N , as ��0. It therefore follows from 34 that there exists a neighborhood N of �� 0 0such that

n1 x xt tŽ .30 sup F , � �F , � u �� a.s.Ý 0 tž / ž /ž /' 'n n n�N t�10

Ž . Ž . Ž .for any ��0 given. We may now easily deduce 27 from the results in 28 and 30 . The proof forŽ .part a is therefore complete.

Ž . Ž .We now prove part b . As in the proof of part a , we fix an arbitrary � � . Due to the0compactness of � , it suffices to show that there exists a neighborhood N of � for which0 0

n1Ž . Ž . Ž .31 sup F x , � u �o 1 .Ý t t p'n�N t�10

Since it follows from Theorem 3.2 that

n1Ž . Ž .F x , � u �O 1 ,Ý t 0 t p4' t�1n

it suffices to show that

n1Ž . Ž Ž . Ž .. Ž .32 sup F x , � �F x , � u �o 1Ý t t 0 t p'n�N t�10

Ž .to deduce 31 .


Ž .However, we have by I-regularity condition a that

n n n

Ž . � Ž . Ž . � � � � � � Ž . � � Ž . �33 F x , � �F x , � u � �� T x � T x w ,Ý Ý Ýt t 0 t 0 t t tž /t�1 t�1 t�1

� � Ž � � . Ž � � .2 2where w � u �E u FF . Note that E u FF �� by Jensen’s inequality. Since T ist t t t�1 t t�1bounded and integrable, we have

n1� Ž . � Ž .T x �O 1 ,Ý t p'n t�1

and2n n1 1 22� Ž . � Ž .E T x w �� E T x �0.Ý Ýt t tž / ž /' nn t�1 t�1

Ž . Ž .It is therefore clear from 33 that we may choose N such that 32 holds, which completes the0proof.

Ž .For the proof of part c , note thatn1

Ž . Ž .F x , � u �O 1 ,Ý t 0 t p' 'Ž .n � n t�1

due to Theorem 3.3. Moreover, by Cauchy�Schwarz,n1

Ž Ž . Ž ..F x , � �F x , � uÝ t t 0 t'Ž .n� n t�1

1�2 1�2n n1 12 2Ž Ž . Ž ..� F x , � �F x , � u ,Ý Ýt t 0 t2 ž /ž / n'Ž .n� n t�1 t�1

Ž .and we have from Lemma A6 c and Theorem 3.3 thatn1 1 22Ž Ž . Ž .. Ž Ž Ž . . Ž Ž . ..F x , � �F x , � � H V r , � �H V r , � drHÝ t t 0 a . s . 02

0'Ž .n� n t�1

Ž .uniformly in �� . We may now use the same argument as that in the proof of part a to get thestated result. Q.E.D.

Ž .PROOF OF LEMMA A8: For the proof of part a , it suffices to show that

�Ž . Ž . Ž .34 F s, � L 1, s dsH

��

Ž .is continuous a.s., due to the occupation time formula. The continuity of 34 , however, is animmediate consequence of dominated convergence, and follows immediately from the a.s. integrabil-

� Ž . � Ž . � Ž . �ity of sup F �, � L 1, � . Note that sup F �, � is locally bounded, as shown in Lemma� � � �

Ž . Ž .A3 b , and hence locally integrable, and L 1, � has compact support a.s. We may also easily deduceŽ . Ž .part b from dominated convergence, due to the continuity of F x, � for all xR, and the

� Ž . �integrability of sup F �, � . Q.E.D.� �

APPENDIX B: PROOFS OF THE MAIN RESULTS

Ž . Ž . Ž .PROOF OF LEMMA 2.1: Let U , V � U, V , as given by condition a of Assumption 2.1, andn n dŽ . Ž .denote by � , FF, Pr the probability space where U, V is defined. For each n, we construct a

Ž .n Ž .n Ž .sequence of stopping times and random variables V on � , FF, Pr , from which thent t�0 nt t�0


Ž 0 0 .desired U , V is then defined. In the subsequent construction, we letn n

nt tt0 Ž . Ž . ŽŽ Ž . Ž .. .FF �� U r , r� , V , FF �� U n , V n ,nt ni nt n i n ii�0 i�0ž /ž /n

nt t t�1t�10 Ž . Ž . ŽŽ Ž .. Ž Ž .. .GG �� U r , r� , V , GG �� U n V n ,nt ni nt n i n ii�0 i�0 i�0ž /ž /n

�where n � i�n for 0� i�n. Also, the symbolism ‘� FF ’ is used to signify ‘distribution conditional onithe �-field FF ’.

Ž .Let n be given and fixed. First, we choose any random variable V on � , FF, P , which has then0Ž . Ž . Ž .same distribution as V 0 , i.e., V � V 0 . Second, let � be a stopping time defined on � , FF, P ,n n0 d n n1

Ž . � 0 Ž . �for which U �n FF � U 1�n FF . Such a stopping time exists, as shown in Hall and Heyden1 n0 d n n0Ž . Ž .1980, Theorem A1 . We then define a random variable on � , FF, P , denoted by V , such thatn1

� 0 Ž . � Ž .n Ž .nV GG � V 1�n GG , and so on. It is obvious that we may proceed to find and V onn1 n1 d n n1 nt t�0 nt t�0Ž .� , FF, P successively so that

t tnt 0 0U FF � U FF and V GG � V GGn , t�1 d n n , t�1 nt nt d n ntž / ž / ž /n n n

Ž .n Ž .nin a zig-zag fashion. If we let and V be constructed in this way, and definent t�0 nt t�0

n� n r �0 0Ž . Ž .U r �U and V r �V ,n n n� n r �ž /n

Ž . Ž 0 0 . Ž 0 0 .it follows immediately that U , V � U , V . Such processes U , V can, of course, be found forn n d n n n nall n.

Ž .It is shown by Park and Phillips 1999 that we may choose the stopping times so that theyntŽ .satisfy condition 13 . In particular, it follows from the Holder continuity of the sample path of U¨

that

1�2�� n r �0� Ž . Ž . �U r �U r �c � rn n

Ž .a.s., for some constant c and any ��0. Now we may easily deduce from 13 that

� 0 Ž . Ž . � Ž Ž�1�� .�2�� .sup U r �U r �o n a.s.n� �r 0, 1

Ž . 0 � �for max 1�2, 2�q ��1 and any ��0. In particular, U � U uniformly on 0, 1 . Moreover,n a. s.Ž 0 0 . Ž . 0 Ž 0 0 .since U , V � U, V , we may redefine V , if necessary, so that the distribution of U , V isn n d n n n

0 � �unchanged and V � V uniformly on 0, 1 . This is possible due to the representation theorem ofn a. s.a weakly convergent sequence of probability measures by an almost sure convergent sequencee.g.

Ž .see Pollard 1984, pp. 71�72 . Q.E.D.

Ž .PROOF OF LEMMA 2.2: See Corollary 1.6, p. 215, of Revuz and Yor 1994 . Q.E.D.

PROOF OF THEOREM 3.1: For sample mean asymptotics, we write

n1 x 1t Ž Ž . .F , � � F V r , � dr ,HÝ nž /'n n 0t�1

and show that

1 1Ž . Ž Ž . . Ž Ž . .35 F V r , � dr� F V r , � dr ,H Hn a . s .0 0

Ž .uniformly in �� . Fix an arbitrary � � . Due to Lemma A3 a , there exists a neighborhood N0 0


Ž . Ž .of � such that sup F �, � and inf F �, � are regular for any neighborhood N�N of0 � N � N 0� . Therefore, it follows from Lemma A2 that0

1 1Ž . Ž Ž . . Ž Ž . .36 sup F V r , � dr� sup F V r , � dr ,H Hn a . s .0 0�N �N

1 1Ž . Ž Ž . . Ž Ž . .37 inf F V r , � dr� inf F V r , � dr ,H Hn a . s .�N �N0 0

as n��.Let N �N be the �-neighborhood of � . Then we have� 0 0

Ž . Ž .sup F x , � � inf F x , � �0,�N��N�

Ž . Ž . � Ž . �as ��0, due to the continuity of F x, � . Moreover, as shown in Lemma A3 b , sup F �, � is� �

locally bounded. It therefore follows from the occupation time formula and dominated convergencethat

1Ž . Ž Ž . . Ž Ž . .38 sup F V r , � � inf F V r , � drH ž /�N0 �N

�Ž . Ž . Ž .� sup F s, � � inf H s, � L 1, s ds� 0,H a . s .ž /

�N�� N�

Ž . Ž .as ��0. We may now easily deduce from 36 � 38 that there exists a neighborhood of � where0Ž . Ž .35 holds uniformly in � . Since � was chosen arbitrary and � is compact, 35 holds uniformly on0� , as was to be shown. The sample covariance asymptotics are given in Lemma A2. Q.E.D.

PROOF OF THEOREM 3.2: Fix � � . For any neighborhood N of � , we have by I-regularity0 0Ž .condition b

Ž . Ž . Ž . Ž .inf F x , � � inf F y , � , sup F x , � � sup F y , ��N �N �N �N

� Ž . Ž . �� sup F x , � �F y , ��N

� � k�c x�y .

It follows that

n1 �Ž . Ž . Ž . Ž .39 sup F x , � � sup F s, � ds L 1, 0 ,HÝ t p ž /'n ��N �Nt�1

n1 �Ž . Ž . Ž . Ž .40 inf F x , � � inf F s, � ds L 1, 0 ,HÝ t p ž /' �N �Nn ��t�1

Ž .due to Theorem 5.1 of Park and Phillips 1999 .Ž .Let N be the �-neighborhood of � . By I-regularity condition a , we have� 0

Ž . Ž .sup F x , � � inf F x , � �0,�N��N�

Ž .as ��0, and, in view of I-regularity condition a and dominated convergence,

� �Ž . Ž . Ž .41 sup F s, � ds� inf F s, � ds�0,H H

�N�� N�


Ž . Ž .as ��0. We may now easily deduce from 39 � 41 that there exists a neighborhood of � such0that

n1 �Ž . Ž . Ž .F x , � � F s, � ds L 1, 0 ,HÝ t p ž /'n ��t�1

uniformly in � . Since � was chosen arbitrarily and � is compact, the proof for sample mean0asymptotics is complete.

We now prove the result on sample covariance asymptotics. For notational simplicity, we assumethat F is real-valued. The proof for a vector F follows by considering an arbitrary linearcombination of the components of F. Define

k�1 t�1 4 nt n , t�1' 'Ž . Ž .42 M r � n F n V , � U �UÝn n ž / ž /ž /ž /ž /n n nt�1

k�1 4 n , k�1' ' Ž .� n F n V , � U r �U ,n ž /ž /ž /ž /n n

for �n� r� �n, where , k�1, . . . , n, are the stopping times introduced in Lemma 2.1.n, k�1 n k n kOne may easily see that M is a continuous martingale such thatn

n1 n nŽ . Ž .43 F x , � u �M ,Ý t t n ž /4 n' t�1n

and that

1nt n , t�1Ž . Ž .44 sup � � �o 1 ,a . s .ž /n n n1�t�n

by Lemma 2.1.� �The quadratic variation process M of M is given byn n

2k�1 t�1 nt n , t�1' '� �M � n F n V , � �Ýn nr ž /ž /ž /n n nt�1

2k�1 n , k�1' '� n F n V , � r�n ž /ž /ž /n n

r 2' 'Ž Ž . . Ž Ž ..� n F n V s , � ds 1�o 1 ,H n a . s .0

Ž .due to 44 , and therefore,

� 2Ž . � � Ž . Ž .45 M � F s, � ds L r , 0 ,Hn pr ž /��

� �uniformly in r 0, 1 , from the result obtained in the first part of this theorem. Moreover, if we� �denote by M , V the covariation process of M and V, thenn n

k�1 t�1 4 nt n , t�1' '� �M , V � n F n V , � � �Ýn n ur ž /ž /ž /n n nt�1

k�1 4 n , k�1' '� n F n V , � r� �n už /ž /ž /n n

r4' 'Ž Ž . . Ž Ž ..�� n F n V s , � ds 1�o 1Hu n a . s .0


� � Ž . � �uniformly in r 0, 1 , due to 44 . However, for all r 0, 1 ,

1r4 1' ' ' 'Ž Ž . . � Ž Ž . . �n F n V s , � ds � n F n V s , � ds � 0,H Hn n p4 ž /0 0'n

as n��. It follows that

Ž . � �46 M , V � 0,Ž .� rn pn

Ž . � � � � � 4where � r � inf s 0, 1 : M � r is a sequence of time changes.n n sŽ .The asymptotic distribution of the continuous martingale M in 42 is completely determined byn

Ž . Ž . Ž .45 and 46 , as shown in Revuz and Yor 1994, Theorem 2.3, page 496 . Now define

Ž . Ž Ž ..W r �M � r .n n n

Ž .The process W is called the DDS or Dambis, Dubins-Schwarz Brownian motion of the continuousnŽ Ž . .martingale M see, for example, Revuz and Yor 1994 , Theorem 1.6, page 173 . It now follows thatn

Ž . Ž .V, W converges jointly in distribution to two independent Brownian motions V, W . Therefore,n

n n Ž . Ž .M �M 1 �o 1n n pž /n

� 2Ž . Ž .� W L 1, 0 F s, � dsHd ž /��

Ž .which, in view of 43 , completes the proof for the second part. Q.E.D.

Ž .PROOF OF THEOREM 3.3: Due to Lemma A5 b ,n1 x�1 t' 'Ž .� n , � R , n , � � 0,Ý a . s .ž /'n nt�1

uniformly in �� , andn1 x�1 t' 'Ž .� n , � R , n , � u � 0,Ý t pž /' 'n nt�1

for each �� . We therefore haven n1 1 x�1 t'Ž . Ž . Ž .� n , � F x , � � H , � �o 1 ,Ý Ýt a . s .ž /'n n nt�1 t�1

uniformly in �� , andn n1 1 x�1 t'Ž . Ž . Ž .� n , � F x , � u � H , � u �o 1Ý Ýt t t pž /' ' 'n n nt�1 t�1

for each �� . The stated results follow directly from the application of Theorem 3.1 to the limithomogeneous function H. Q.E.D.

Ž .PROOF OF THEOREM 4.1: It follows readily from Lemma A7 b thatn1 1 2Ž . Ž Ž . Ž .. Ž .D , � f x , � f x , �o 1 ,Ýn 0 t t 0 p' 'n n t�1

Ž .uniformly in � . The stated result now follows immediately from Lemma A6 b and Theorem 3.2.Ž . Ž .Notice that L 1, 0 �0 a.s., and, therefore, D �, has a unique minimum at a.s. when and only0 0

Ž . Ž .when the given identification condition holds. The continuity of D �, follows from Lemma A8 b .0Q.E.D.


Ž .PROOF OF THEOREM 4.2: Due to Lemma A7 c , we have

n1Ž Ž . Ž .. Ž .f x , � f x , u �o 1 ,Ý t t 0 t p'Ž .n� n t�1

Ž .uniformly in � . Furthermore, since � is bounded away from zero by condition a , we have

n1 1 2Ž . Ž Ž . Ž .. Ž .D , � f x , � f x , �o 1 ,Ýn 0 t t 0 p2 2' 'Ž . Ž .n� n n� n t�1

Ž .uniformly in � . The stated result follows directly from Lemma A6 c and Theorem 3.3. SinceŽ . Ž . Ž .L 1, 0 �0 and L 1, � is continuous a.s., there exists a neighborhood of zero on which L 1, � �0 a.s.

Ž . Ž .Therefore, D �, has unique minimum a.s. when and only when the identification condition b0 0Ž . Ž .holds. The continuity of D �, is due to Lemma A8 a . Q.E.D.0

PROOF OF THEOREM 4.3: Let

n12 2'Ž . Ž .m n , � f x , ,Ý t2'Ž .n� n , t�1

�2 2Ž . Ž . Ž .m � h s, L 1, s ds.H��

'Ž . Ž . Ž .We have m n , � m uniformly in � , by Lemma A6 c and Theorem 3.3. It followsa. s.Ž . Ž .from Lemma A8 a that m is continuous a.s. Also, due to condition b , m�0 a.s.

�� 4Let ��0 be given, and define � � � �� . Fix an arbitrary � , and let N be the0 0 0Ž . Ž . Ž .neighborhood of given by the condition a . Also, set p�m and q�m . For large n, we0

have

'� Ž . �sup m n , �p �� ,N

' 'Ž . Ž . � Ž . �since m n , � m uniformly in � and m is continuous. Moreover, m n , �q ��a. s. 0for sufficiently large n. Notice that p, q�0 since m�0. Therefore,

' ' ' 'Ž . � Ž . Ž . � � Ž . Ž . Ž . Ž . �47 inf inf p� �, �q� �, � � n , m n , �� n , m n , 0 0 0N� �p�p ��

� �q�q ��

for large n.Define

n1 2Ž . Ž Ž . Ž ..A , � f x , � f x , ,Ýn 0 t t 0n t�1

n1Ž . Ž Ž . Ž ..B , � f x , � f x , u .Ýn 0 t t 0 tn t�1

n 2 2 Ž .Since Ý u �n�� o 1 , we have by the Cauchy�Schwarz inequalityt�1 t p

1�2 2� Ž . � Ž . Ž Ž ..B , �A , � �o 1 ,n 0 n 0 p

and, therefore,

�1 �2�1 2Ž . Ž � �.Ž . Ž . Ž Ž ..48 A B , �A , � �o 1 ,n n 0 n 0 p


Ž .uniformly in � . However, using 47 and the inequality

21�2 1�2n n n2 2 2Ž .a �b � a � b ,Ý Ý Ýt t t tž / ž /ž /t�1 t�1 t�1

Ž . Ž .which holds for any real-valued sequences a and b , we may deduce thatt t

2' ' ' 'Ž . Ž . Ž Ž . Ž . Ž . Ž ..49 A , � � n , m n , �� n , m n , � �,n 0 0 0 a . s .

uniformly in N.Ž . Ž .Now it follows from 48 and 49 that

�1 Ž . Ž .Ž Ž �1 � �.Ž ..n D , �A , 1�2 A B , n 0 n 0 n n 0

Ž .Ž Ž ..�A , 1�o 1 � �,n 0 p p

uniformly in N. Since � is compact and was chosen arbitrarily, we may easily deduce that0

�1 Ž .n inf D , � �,n 0 p� 0

from which the stated result follows immediately. Q.E.D.

PROOF OF COROLLARY 4.4: Let

n12 2Ž .50 � � u .Ýn tn t�1

Ž . 2 2It follows from Assumption 2.1 b that � � � . Assume first that f satisfies the assumptions inn p�1 �2 Ž . Ž .Theorem 4.1. Then, as shown in the proof of Theorem 4.1, n D , � D , uniformly inn 0 p 0

Ž . Ž .� , with D , given in Theorem 4.1. We have, in particular, that D �, is continuous a.s.,0 0ˆ �1 �2 ˆŽ . Ž . Ž .and therefore D �, �o 1 near . It follows from the consistency of that n D , 0 a. s. 0 n n n 0

Ž . 2 2 Ž �1 �2 .�o 1 , which implies � �� o n , and hence, the stated result follows. For f satisfyingˆp n n p�1 �2 Ž . Ž .the assumptions in Theorem 4.2 with ��, we note that n � D , � D , uniformly inn n 0 p 0

Ž .� , where D �, is an a.s. continuous function given in Theorem 4.2. Therefore, in the same02 2 Ž 2 . 2 Ž .way as above, we have � �� o � . The stated result follows immediately, since � �O 1 .n̂ n p n n p

Q.E.D.

˙ Ž .PROOF OF THEOREM 5.1: Given the I-regularity of f in condition b , AD1�AD3 follow directly4'Ž .from Lemma A6 b and Theorem 3.2 with � � n . Since we have in particularn

�¨ ˙ ˙Ž . Ž . Ž . Ž .Q �L 1, 0 f s, f s, � ds,H0 0 0��

ˆŽ .we may easily deduce AD4 from condition c . Moreover, AD5 holds trivially, since � , due ton p 0Ž .condition a and Theorem 4.1, and we assume that is an interior point of � . It therefore remains0

¨ ¨Ž . Ž .to show AD6. For AD6, we prove that Q � Q uniformly on a neighborhood of . In viewn p 0 0ˆof the consistency of , this establishes AD6.n

We now write

n n n¨ ˙ ˙ ¨ ¨Ž . Ž . Ž . Ž . Ž .Ž Ž . Ž .. Ž .51 Q � f x , f x , �� F x , f x , � f x , � F x , u .Ý Ý Ýn t t t t t 0 t t

t�1 t�1 t�1


Ž .By Lemma A6 b and Theorem 3.2, we have

n1 �˙ ˙ ˙ ˙Ž . Ž . Ž . Ž . Ž . Ž .52 f x , f x , �� L 1, 0 f s, f s, � ds,HÝ t t p'n ��t�1

Ž .uniformly in � . Also, it follows from Lemma A7 b that

n1¨Ž . Ž .53 f x , u � 0,Ý t t p'n t�1

¨� Ž . Ž . �uniformly in � . Finally, since f �, � f �, is I-regular on � and f is bounded, we have0from Theorem 3.2

n n1 1¨ ¨Ž .Ž Ž . Ž .. � � � Ž . Ž . �f x , f x , � f x , � f f x , � f x , Ý Ýt t t 0 t t 0' 'n nt�1 t�1

�¨� � Ž . � Ž . Ž . �� f L 1, 0 f s, � f s, ds,Hp 0��

Ž .uniformly in � . Furthermore, the limit function is continuous in by Lemma A8 b , and we canŽ . Ž .make it arbitrarily small in a neighborhood of . This, together with 52 and 53 , completes the0

proof. Q.E.D.

Ž .PROOF OF THEOREM 5.2: We have AD1�AD3 directly from Lemma A6 c and Theorem 3.3 with˙' 'Ž . Ž .� � n � n , due to the H-regularity of h in condition b . Moreover,˙n

�1¨ ˙ ˙ ˙ ˙Ž . Ž . Ž Ž . . Ž Ž . . Ž . Ž . Ž .54 Q � h V r , h V r , � dr� h s, h s, �L 1, s ds,H H0 0 0 0 00 ��

Ž .and AD4 follows immediately from condition d . As in the proof of Theorem 5.1, AD5 holdsˆ Ž .trivially because of the consistency of implied by condition a . To finish the proof, it thereforen

suffices to show AD6.' ' 'Ž . Ž . Ž .Write � �� n , � �� n , and � �� n for notational simplicity. It follows from Lemma˙ ˙ ¨ ¨n n n

Ž .A6 c and Theorem 3.3 that

n1�1 �1�˙ ˙ ˙Ž . Ž . Ž . Ž Ž . . Ž Ž . .55 � f x , f x , �� h V r , h V r , � dr ,HÝn t t n a . s .

0t�1

Ž .and from Lemma A7 c that

n �1 n�̈n�1�1 ¨ ¨Ž . Ž . Ž . Ž . Ž .56 � �� f x , u � � �� f x , u � 0,˙ ˙ ¨Ý Ýn n t t n n n t t pž /nt�1 t�1

uniformly in � . Therefore, for AD6, we only need to show

n�1 ¨Ž . Ž . Ž .Ž Ž . Ž ..57 � �� f x , f x , � f x , Ýn n t n t n t 0

t�1

n1�1 �1 �1¨ŽŽ . . Ž Ž ..Ž Ž Ž . Ž ... Ž .� � �� f x , � f x , � f x , �o 1 .˙ ˙ ¨ ¨Ýn n n n n t n n t n t 0 pn t�1

Ž . Ž . Ž .It is easily seen from 51 that 55 � 57 imply AD6.


¨Ž . � Ž . Ž . �To prove 57 , apply Theorem 3.3 to f �, � f �, and use the local boundedness of h0Ž .established in Lemma A3 b to deduce that

n1�1 �1¨Ž Ž ..Ž Ž Ž . Ž ...� f x , � f x , � f x , ¨Ý n t n t t 0n t�1

n1¨� � Ž . Ž .� h f x , � f x , K Ý t t 0n�n t�1

1¨� � Ž Ž . . Ž Ž . .� h h V r , �h V r , dr ,KHa . s . 00

� � Ž .uniformly in � , where K� s �1, s �1 �� . To deduce 57 , simply note that the limitmin maxŽ .function is continuous in , due to Lemma A8 a . Q.E.D.

Ž .PROOF OF THEOREM 5.3: We show that AD1�AD4 and AD7 hold to establish 18 . Write � �˙n'Ž . Ž .� n to simplify notation. It follows directly from condition a and Theorem 3.3 that AD1 holds.˙Ž .Also, AD2 is immediate, since we have from 19

n n1�1�1 ¨ 'Ž . Ž . Ž . � Ž . � � �� f x , u � � �� sup f n s, u � 0.˙ ˙Ý Ýn n t 0 t n n 0 t pž / n� �t�1 t�1s �s

Ž .Under AD2, we may easily get AD3 by applying Lemma A6 b and Theorem 3.3. In particular, we¨Ž . Ž . Ž .have Q given by 54 , and therefore, AD4 follows straightforwardly from condition c .0

To show AD7, fix � such that 0��3, and define � �n1�2�� and � �n1�2� so that˙ ˙n n n n� ��1 �0 as required. Let N be defined as in AD7. We first writen n n

¨ ¨ ¨ ¨ ¨ ¨ ¨Ž . Ž . Ž Ž . Ž . . Ž . Ž . Ž .Q �Q � D �D � �D �D �D ,n n 0 1 n 1 n 2 n 3n 4 n

where

n¨ ˙ ˙ ˙Ž . Ž .Ž Ž . Ž ..D � f x , f x , � f x , �,Ý1 n t 0 t t 0

t�1

n¨ ˙ ˙ ˙ ˙Ž . Ž Ž . Ž ..Ž Ž . Ž ..D � f x , � f x , f x , � f x , �,Ý2 n t t 0 t t 0

t�1

n¨ ¨Ž . Ž .Ž Ž . Ž ..D � F x , f x , � f x , ,Ý3n t t t 0

t�1

n¨ ¨ ¨Ž . Ž Ž . Ž ..D �� F x , �F x , u ,Ý4 n t t 0 t

t�1

and define

2 �1 �1�¨Ž . � Ž . �� D � ,i n n in n

for i�1, . . . , 4. For all N , we haven

n�12 �1 ˙ ¨Ž . Ž . � Ž . � �Ž . Ž . �58 � � � f x , � �� f x , ,Ý1 n n t 0 n n t

t�1

n�1 22 ¨Ž . Ž . �Ž . Ž . �59 � � � �� f x , ,Ý2 n n n t

t�1

J. Y. PARK AND P. C. B. PHILLIPS160n

�12 �1 ˙ ¨Ž . Ž . � Ž . � �Ž . Ž . �60 � � � f x , � �� f x , Ý3n n t 0 n n tt�1

n1 �1 �1¨ ¨�Ž . Ž . � �Ž . Ž . �� f x , � �� f x , ,Ý n n t n n t2 t�1

n ��12Ž . Ž . �Ž . Ž . � � �61 � � � �� f x , u ,Ý4 n n n n t tt�1

where lies in the line segment connecting and .0Ž .Let s�max s , �s �1. Then we have for large nmax min

¨ ¨'� Ž . � � Ž . �sup f x , � sup sup f n s, ,tN N� �s �sn n

Ž . Ž .for all t�1, . . . , n. It now follows from 58 � 61 that

3� nn 1�12 �1¨ ˙'Ž . Ž . Ž . � Ž . � � Ž . �62 � � � �� sup sup f n s, � f x , ,˙ ˙ ˙Ý1 n n n n t 0ž /' nn N� � t�1s �s n

24�n �12 ¨'Ž . Ž . Ž . � Ž . �63 � � � �� sup sup f n s, ,˙ ˙2 n n n ž /n N� �s �s n

3� nn 1�12 �1¨ ˙'Ž . Ž . Ž . � Ž . � � Ž . �64 � � � �� sup sup f n s, � f x , ˙ ˙ ˙Ý3n n n n t 0ž /' nn N� � t�1s �s n

24�n �1 ¨'Ž . � Ž . �� sup sup f n s, ,˙ ˙n n ž /2n N� �s �s n

3� nn 1��12Ž . Ž . Ž . � Ž . � � �65 � � � �� sup sup f �s, u ,˙ ˙ ˙ Ý4 n n n n tž /' nn N� � t�1s �s n

2 Ž . Ž . Ž .from which we may easily deduce that � �o 1 , i�1, . . . , 4, uniformly in N , due to 20i n a. s. nŽ . Ž . Ž .and 21 . Now AD7 follows immediately from 62 � 65 . This completes the proof. Q.E.D.

2 Ž . Ž .PROOF OF COROLLARY 5.4: Let � be given as 50 . Due to Assumption 2.1 b , it suffices to shown2 2 Ž .that � �� o 1 . To show this, we definen̂ n p

n2ˆŽ Ž . Ž ..A � f x , � f x , ,Ýn t n t 0

t�1

nˆŽ Ž . Ž ..B � f x , � f x , u ,Ýn t n t 0 t

t�1

ˆŽ .so that D , �A �2 B .n n 0 n n1�2 'Ž .Let � �n � , where � �� n , as defined in the proof of Theorem 5.3. Then we have˙ ˙ ˙n n n 0

n1 2 2�1ˆ ˙� Ž . � � Ž . � Ž .A � � � � f x , �O 1 ,˙Ýn n n 0 n t n pn t�1

�1 n� �B �̇n n ˆ ˙� Ž . � Ž . Ž .� � � f x , u �o 1 ,Ýn n 0 t n t p' nn t�1

�1 �2 ˆŽ . Ž .by Theorem 3.4 and Lemma A7. It therefore follows that n D , �o 1 , from which wen n 0 p2 2 Ž �1 �2 .have � �� o n , as required. Q.E.D.n̂ n p


REFERENCES

Ž .ANDREWS, D. W. K., AND C. J. MCDERMOTT 1995 : ‘‘Nonlinear Econometric Models with Determin-istically Trending Variables,’’ Reiew of Economic Studies, 62, 343�360.

Ž .BATES, D. M., AND D. G. WATTS 1988 : Nonlinear Regression Analysis and Its Applications. NewYork: Wiley.

Ž .BILLINGSLEY, P. 1968 : Conergence of Probability Measures. New York: Wiley.Ž .CHUNG, K. L., AND R. J. WILLIAMS 1990 : Introduction to Stochastic Integration, 2nd ed. Boston:

Birkhauser.¨Ž .GRANGER, C. W. J. 1995 : ‘‘Nonlinear Relationships between Extended-Memory Variables,’’ Econo-

metrica, 63, 265�280.Ž .HALL, P., AND C. C. HEYDE 1980 : Martingale Limit Theory and Its Application. New York: Academic

Press.Ž .HANSEN, B. E. 1992 : ‘‘Convergence to Stochastic Integrals for Dependent Heterogeneous Pro-

cesses,’’ Econometric Theory, 8, 489�500.Ž .HANSEN, L. P. 1982 : ‘‘Large Sample Properties of Generalized Method of Moments Estimators,’’

Econometrica, 50, 1029�1054.Ž .JENNRICH, R. I. 1969 : ‘‘Asymptotic Properties of Non-linear Least Squares Estimation,’’ Annals of

Mathematical Statistics, 40, 633�643.Ž .KURTZ, T. G., AND P. PROTTER 1991 : ‘‘Weak Limit Theorems for Stochastic Integrals and

Stochastic Differential Equations,’’ Annals of Probability, 19, 1035�1070.Ž .MALINVAUD, E. 1970 : ‘‘The Consistency of Nonlinear Regressions,’’ Annals of Mathematical

Statistics, 41, 956�969.Ž .PARK, J. Y., AND P. C. B. PHILLIPS 1988 : ‘‘Statistical Inference in Regressions with Integrated

Processes. Part 1,’’ Econometric Theory, 4, 468�497.Ž . 1999 : ‘‘Asymptotics for Nonlinear Transformations of Integrated Time Series,’’ Economet-

ric Theory, 15, 269�298.Ž .PARK, J. Y., AND P. C. B. PHILLIPS 2000 : ‘‘Nonstationary Binary Choice,’’ Econometrica, 68,

1249�1280.Ž .PHILLIPS, P. C. B. 1986 : ‘‘Understanding Spurious Regressions in Econometrics,’’ Journal of

Econometrics, 33, 311�340.Ž . 1987 : ‘‘Time Series Regression with a Unit Root,’’ Econometrica, 55, 277�301.Ž . 1991 : ‘‘Optimal Inference in Cointegrated Systems,’’ Econometrica, 59, 283�306.

Ž .PHILLIPS, P. C. B., AND S. N. DURLAUF 1986 : ‘‘Multiple Time Series with Integrated Variables,’’Reiew of Economic Studies, 53, 473�496.

Ž .PHILLIPS, P. C. B., AND J. Y. PARK 1998 : ‘‘Nonstationary Density Estimation and Kernel Autore-gression,’’ Yale University, Mimeographed.

Ž .PHILLIPS, P. C. B., AND V. SOLO 1992 : ‘‘Asymptotics for Linear Processes,’’ Annals of Statistics, 20,971�1001.

Ž .POLLARD, D. 1984 : Conergence of Stochastic Processes. New York: Springer-Verlag.Ž .REVUZ, D., AND M. YOR 1994 : Continuous Martingale and Brownian Motion, 2nd ed. New York:

Springer-Verlag.Ž .SAIKKONEN, P. 1995 : ‘‘Problems with the Asymptotic Theory of Maximum Likelihood Estimation in

Integrated and Cointegrated Systems,’’ Econometric Theory, 11, 888�911.Ž .SHORACK, G. R., AND J. A. WELLNER 1986 : Empirical Processes with Applications to Statistics. New

York: Wiley.Ž .WOOLDRIDGE, J. M. 1994 : ‘‘Estimation and Inference for Dependent Processes,’’ in Handbook of

Econometrics, Vol. IV, ed. by R. F. Engle and D. L. McFadden. Amsterdam: Elsevier, pp.2639�2738.

Ž .WU, C. F. 1981 : ‘‘Asymptotic Theory of Nonlinear Least Squares Estimation,’’ Annals of Statistics,9, 501�513.

NONLINEAR REGRESSIONS WITH INTEGRATED TIME SERIES BY...

Documents

Transcript of NONLINEAR REGRESSIONS WITH INTEGRATED TIME SERIES BY...