Cowles Foundation for Research in Economics at Yale University

Cowles Foundation Discussion Paper No. 1897

SIEVE QUASI LIKELIHOOD RATIO INFERENCE ON SEMI/NONPARAMETRIC CONDITIONAL MOMENT MODELS

Xiaohong Chen and Demian Pouzo

May 2013

An author index to the working papers in the Cowles Foundation Discussion Paper Series is located at:

http://cowles.econ.yale.edu/P/au/index.htm

This paper can be downloaded without charge from the Social Science Research Network Electronic Paper Collection:

http://ssrn.com/abstract=2271617

Sieve Quasi Likelihood Ratio Inference on Semi/nonparametricConditional Moment Models1

Xiaohong Chen2 and Demian Pouzo3

First version: March 2009. This version: November 2012.

Abstract

This paper considers inference on functionals of semi/nonparametric conditional moment re-strictions with possibly nonsmooth generalized residuals. These models belong to the difficult(nonlinear) ill-posed inverse problems with unknown operators, and include all of the (nonlinear)nonparametric instrumental variables (IV) as special cases. For these models it is generally difficultto verify whether a functional is regular (i.e., root-n estimable) or irregular (i.e., slower than root-nestimable). In this paper we provide computationally simple, unified inference procedures thatare asymptotically valid regardless of whether a functional is regular or irregular. We establishthe following new results: (1) the asymptotic normality of the plug-in penalized sieve minimumdistance (PSMD) estimators of the (possibly iregular) functionals; (2) the consistency of sieve vari-ance estimators of the plug-in PSMD estimators; (3) the asymptotic chi-square distribution of anoptimally weighted sieve quasi likelihood ratio (SQLR) statistic; (4) the asymptotic tight distribu-tion of a possibly non-optimally weighted SQLR statistic; (5) the consistency of the nonparametricbootstrap and the weighted bootstrap (possibly non-optimally weighted) SQLR and sieve Waldstatistics, which are proved under virtually the same conditions as those for the original-samplestatistics. Small simulation studies and an empirical illustration of a nonparametric quantile IVregression are presented.

Keywords: Nonlinear nonparametric instrumental variables; Penalized sieve minimum distance;Irregular functional; Sieve Riesz representer; Sieve quasi likelihood ratio; Asymptotic normality;Bootstrap; Sieve variance estimator.

1We thank a co-editor and two referees for very useful comments. Earlier versions, some entitled “On PSMDinference of functionals of nonparametric conditional moment restrictions”, were presented in April 2009 at the Banffconference on seminonparametrics, in June 2009 at the Cemmap conference on quantile regression, in July 2009 atthe SITE conference on nonparametrics, in September 2009 at the Stats in the Chateau/France, in June 2010 atthe Cemmep workshop on recent developments in nonparametric instrumental variable methods, in August 2010at the Beijing international conference on statistics and society, in May 2012 at London Cemmap Mastersclass onsemi-nonparametric models and methods, in September 2012 at Toulouse conference celebrating the 65th birthday ofJP Florens, and workshops in UPenn, Purdue/stat, UC Berkeley, Stanford, Seoul National, USC, UCSD, WeierstrassInstitute/Berlin, Monash, Arizona Vanderbilt, Yale and Princeton. We thank D. Andrews, P. Bickel, R. Blundell, T.Christensen, J. Fan, M. Jansson, Z. Liao, O. Linton, J. Powell, A. Santos and other participants of these conferencesand workshops for helpful comments. Chen acknowledges financial support from National Science Foundation grantSES-0838161. Any errors are the responsibility of the authors.

2Cowles Foundation for Research in Economics, Yale University, Box 208281, New Haven, CT 06520, USA. Email:xiaohong.chen@yale.edu.

3Department of Economics, UC Berkeley, 530 Evans Hall 3880, Berkeley, CA 94720, USA. Email:dpouzo@econ.berkeley.edu.

1 Introduction

This paper is about inference on functionals of the unknown true parameters α0 ≡ (θ′0, h0) satisfying

the semi/nonparametric conditional moment restrictions

E[ρ(Yi, Xi; θ0, h01(·), ..., h0q(·))|Xi] = 0 a.s.−Xi, (1.1)

where Yi is a vector of endogenous variables and Xi is a vector of conditioning (or instrumental)

variables. The conditional distribution of Yi given Xi, FYi|Xi, is not specified beyond that it satisfies

(1.1). ρ(·; θ0, h0) is a dρ × 1−vector of generalized residual functions whose functional forms are

known up to the unknown parameters α0 ≡ (θ′0, h0) ∈ Θ × H, with θ0 ≡ (θ01, ..., θ0dθ)′ ∈ Θ

being a dθ × 1−vector of finite dimensional parameters and h0(·) ≡ (h01(·), ..., h0q(·)) ∈ H being a

1 × dq−vector valued function. The arguments of each unknown function hℓ(·) may differ across

ℓ = 1, ..., q, may depend on θ, hℓ′(·), ℓ′ = ℓ, Xi and Yi. The residual function ρ(·;α) could be

nonlinear and pointwise nonsmooth in the parameters α ≡ (θ′, h) ∈ Θ×H.

The general framework (1.1) nests many widely used nonparametric and semiparametric mod-

els in economics, finance and statistics. Well known examples include nonparametric mean in-

strumental variables regressions (NPIV): E[Y1,i − h0(Y2,i)|Xi] = 0 (e.g., Hall and Horowitz (2005),

Carrasco et al. (2007), Blundell et al. (2007), Darolles et al. (2011), Horowitz (2011)); nonpara-

metric quantile instrumental variables regressions (NPQIV): E[1Y1,i ≤ h0(Y2,i) − γ|Xi] = 0

(e.g., Chernozhukov and Hansen (2005), Chernozhukov et al. (2007), Horowitz and Lee (2007),

Chen and Pouzo (2012a), Gagliardini and Scaillet (2011)); the system of shape-invariant mean or

quantile IV Engel curves (e.g., Blundell et al. (2007), Chen and Pouzo (2009)); random coefficient

panel data regressions (e.g., Chamberlain (1992), Graham and Powell (2012)); semi-nonparametric

asset pricing models (e.g., Hansen and Richard (1987), Gallant and Tauchen (1989), Chen and Ludvigson

(2009)); semi-nonparametric static and dynamic game models (e.g., Bajari et al. (2011)); nonpara-

metric optimal endogenous contract models (e.g., Bontempts et al. (2012)). Additional exam-

ples of the general model (1.1) can be found in Chamberlain (1992), Newey and Powell (2003),

Ai and Chen (2003), Chen and Pouzo (2012a), Chen et al. (2011) and the references therein. In

fact, model (1.1) includes all of the (nonlinear) semi-nonparametric IV regressions when the un-

known functions h0(·) depend on the endogenous variables Yi:

E[ρ(Y1,i; θ0, h0(Y2,i))|Xi] = 0, (1.2)

which could lead to difficult (nonlinear) nonparametric ill-posed inverse problems with unknown

operators.

Let Zi ≡ (Y ′i , X

′i)′ni=1 be a strictly stationary ergodic sample with the distribution of Zi the

1

same as the distribution of Z ≡ (Y ′, X ′)′, and FYi|Xi= FY |X . Then we can rewrite model (1.1) as

E[ρ(Z; θ0, h0(·))|X] = 0 a.s.−X. (1.3)

Let ϕ : Θ × H → Rdϕ be a functional with a fixed finite dϕ ≥ 1. Typical functionals include

an Euclidean functional (ϕ(α) = θ), an evaluation functional ϕ(α) = h(y2) (for y2 ∈ supp(Y2)),

weighted derivative functionals ϕ(h) =∫w(y2)∇h(y2)dy2 or

∫w(y2)

[∇2h(y2)

]2dy2 (for a known

positive weight w(·)) and many others. We are interested in simple valid inference on any ϕ(α0) of

the general model (1.3).

As pointed out in Chamberlain (1992) and Ai and Chen (2009), there is generally no closed

form solution for the semiparametric efficiency bound of ϕ(α0) (including θ0) satisfying the model

(1.3), especially so when the residual functions ρ(·; θ0, h0) contain several unknown functions and/or

when the unknown functions h0(·) of endogenous variables enter ρ(·; θ0, h0) nonlinearly. Therefore,it is very difficult for applied researchers to verify whether the semiparametric efficiency bound for

ϕ(α0) is singular or not. Since a non-singular efficiency bound is a necessary condition for ϕ(α0) to

be root-n estimable, it is highly desirable for applied researchers to be able to conduct simple valid

inference on ϕ(α0) regardless of whether it is root-n estimable or not. This is the main goal of our

paper.

In this paper, for the general model (1.3) that could be nonlinearly ill-posed and for any ϕ(α0)

that may or may not be root-n estimable, we first establish the asymptotic normality of the plug-in

penalized sieve minimum distance (PSMD) estimator ϕ(αn) of ϕ(α0). For the model (1.3) with

(pointwise) smooth residuals ρ(Z;α) in α0, we propose a simple sieve variance estimator for possibly

slower than root-n estimator ϕ(αn). However, there is no simple variance estimator for ϕ(αn) when

ρ(Z,α) is not pointwise smooth in α0 (without estimating an extra unknown nuisance function or

using numerical derivatives). We then consider a PSMD criterion based test of the null hypothesis

ϕ(α0) = ϕ0. We show that an optimally weighted sieve quasi likelihood ratio (SQLR) statistic

is asymptotically chi-square distributed under the null hypothesis. This allows us to construct

confidence sets for ϕ(α0) by inverting the optimally weighted SQLR statistic, without the need to

compute a variance estimator for ϕ(αn). Nevertheless, in complicated real data analysis applied

researchers might like to use simple but possibly not optimally weighed PSMD procedures for

estimation of and inference on ϕ(α0). We show that the non-optimally weighted SQLR statistic

still has a tight limiting distribution regardless of whether ϕ(α0) is root-n estimable or not. We

establish these large sample theories allowing for weakly dependent data. In addition, for i.i.d.

data, we establish the consistency of both the nonparametric and weighted bootstrap (possibly

non-optimally weighted) SQLR statistics under virtually the same conditions as those used to

2

derive the limiting distribution of the original-sample SQLR statistic.4 The bootstrap SQLR would

then lead to alternative confidence sets construction for ϕ(α0) without the need to compute a

variance estimator for ϕ(αn).

To the best of our knowledge, our paper is the first to provide a unified theory about criterion

based inference on any ϕ(α0) satisfying the general semi-nonparametric model (1.3) with possibly

nonsmooth residuals. Our results allow applied researchers to obtain limiting distribution of the

plug-in PSMD estimator ϕ(αn) and to construct confidence sets for any ϕ(α0) regardless of whether

it is regular (i.e., root-n estimable) or irregular (i.e., slower than root-n estimable).

Our new results build upon recent literature on identification and estimation of the unknown

true parameters α0 ≡ (θ′0, h0) satisfying the general model (1.3). See, e.g., Newey and Powell

(2003) and Chen et al. (2011) for identification; Newey and Powell (2003), Chernozhukov et al.

(2007), Chen and Pouzo (2012a) and Liao and Jiang (2012) for consistency of their respective esti-

mators; and Chen and Pouzo (2012a) for the rate of convergence of the PSMD estimator of h0(). In

particular, under virtually the same conditions as those in Chen and Pouzo (2012a), we show that

our nonparametric and weighted bootstrap PSMD estimators of α0 ≡ (θ′0, h0) are consistent and

achieve the same convergence rate as that of the original-sample PSMD estimator αn. This result

is then used to establish the consistency of the bootstrap SQLR (and the bootstrap sieve Wald)

statistics under virtually the same conditions as those used to derive the limiting distributions of

the original-sample statistics. As a bonus, our convergence rate of the bootstrap PSMD estimator

is also very useful for the consistency of the bootstrap Wald statistic for semiparametric two step

GMM estimators of regular functionals when the first step unknown functions are estimated via a

PSMD procedure. See Remark 5.1 for details.

There are some published work about estimation of and inference on θ0 satisfying the general

model (1.3) when θ0 is assumed to be regular. See Ai and Chen (2003), Chen and Pouzo (2009)

and Otsu (2011) for the root-n asymptotically normal and efficient estimation of θ0; Ai and Chen

(2003) for consistent variance estimation of the sieve minimum distance (SMD) estimator θn (with

smooth residuals); and Chen and Pouzo (2009) for consistent weighted bootstrap approximation

of the limiting distribution of√n(θn − θ0) for the PSMD estimator θn (with possibly nonsmooth

residuals). However, none of these papers allows for irregular θ0. When specializing our general

theory to inference on θ0 of the model (1.3), we immediately recover the results of Ai and Chen

(2003) and Chen and Pouzo (2009) for regular θ0. Additionally, our results remain valid even when

θ0 might be irregular, and we provide valid bootstrap (possibly non-optimally weighted) SQLR

inference.

When specializing our theory to inference on a specific irregular functional, the evaluation func-

4At the costs of extra heavy notation and additional pages of proofs, we could also establish the consistency ofblock bootstrap (possibly non-optimally weighted) SQLR statistic for strictly stationary weakly dependent data. Weleave it for future research.

3

tional ϕ(α) = h(y2), of the (nonlinear) semi-nonparametric IV model (1.2), we automatically obtain

the pointwise asymptotic normality of the PSMD estimator of h0(y2) and different ways to construct

its confidence set. These results are directly applicable to the NPIV example with ρ(Y1; θ0, h0(Y2)) =

Y1 − h0(Y2) and to the NPQIV example with ρ(Y1; θ0, h0(Y2)) = 1Y1 ≤ h0(Y2) − γ. Horowitz

(2007) and Gagliardini and Scaillet (2011) established the pointwise asymptotic normality of their

kernel based function space Tikhonov regularization estimators of h0(y2) for the NPIV and the

NPQIV examples respectively. As demonstrated in Chen and Pouzo (2012a), the PSMD estima-

tors are easier to compute for the general model (1.3) with possibly nonlinear residuals. In this

paper we illustrate that it is also much easier to conduct the SQLR inference or a sieve Wald infer-

ence on a possibly irregular ϕ(α0) based on its plug-in PSMD estimator. Immediately after the first

version of our paper was presented in April 2009 Banff conference on semiparametrics, the authors

of Horowitz and Lee (2011) informed us that they were independently and concurrently working

on uniform confidence bands for a particular SMD estimator of the NPIV example. Assuming that

h0() belongs to a Sobolev ball with a bounded support and also satisfies some shape restrictions,

their proof strategy depends crucially on the closed form solution of their SMD estimator of the

NPIV example. We consider PSMD based inference on any possibly irregular ϕ(α0) of the general

nonlinear semi-nonparametric model (1.3), in which the parameter space may not be compact and

the PSMD estimator may not have a closed form solution.

The rest of the paper is organized as follows. Section 2 presents the plug-in PSMD estimator

ϕ(αn) of any functional ϕ evaluated at α0 ≡ (θ′0, h0) satisfying the model (1.3). It also illustrates

the asymptotic results that will be established in the subsequent sections through an evaluation

functional ϕ(α) = h(y2) and a weighted integration functional ϕ(h) =∫w(y2)h(y2)dy2 of the

NPQIV example. Section 3 derives the asymptotic normality of ϕ(αn) regardless of whether ϕ(α0)

is regular or not. It also shows that any possibly non-optimally weighted SQLR statistic still

has a tight asymptotic distribution. Section 4 provides inference procedures based on asymptotic

critical values. Section 5 establishes the consistency of the bootstrap SQLR statistic and the

bootstrap sieve Wald statistic for possibly irregular functionals. Section 6 presents simulation

studies and an empirical illustration of the SQLR based confidence sets for the NPQIV regression.

Section 7 briefly concludes with future research. Appendix A presents additional low level sufficient

conditions, useful lemmas, and the consistency of a computationally attractive sieve score bootstrap.

Appendices B and C contain supplementary lemmas and all the proofs.

Notation. We use “≡” to implicitly define a term or introduce a notation. For any column

vector A, we let A′ denote its transpose and ||A||e its Euclidean norm (i.e., ||A||e ≡√A′A, al-

though sometimes we use |A| = ||A||e for simplicity). Let ||A||2W ≡ A′WA for a positive definite

weighting matrix W . Let λmax(W ) and λmin(W ) denote the maximal and minimal eigenvalues

of W respectively. All random variables Z ≡ (Y ′, X ′)′, Zi ≡ (Y ′i , X

′i)′ are defined on a complete

4

probability space (Z,BZ , PZ), where PZ is the joint probability distribution of (Y ′, X ′). We define

(Z∞,B∞Z , PZ∞) as the probability space of the sequences (Z1, Z2, ...). For simplicity we assume that

Y and X are continuous random variables. Let fX (FX) be the marginal density (cdf) of X, and

fY |X (FY |X) be the conditional density (cdf) of Y given X. We use EP [·] to denote the expectation

with respect to a measure P . Sometimes we use P for PZ∞ and E[·] for EPZ[·]. Denote Lp(Ω, dµ),

1 ≤ p < ∞, as a space of measurable functions with ||g||Lp(Ω,dµ) ≡ ∫Ω |g(t)|pdµ(t)1/p < ∞,

where Ω is the support of the sigma-finite positive measure dµ (sometimes Lp(dµ) and ||g||Lp(dµ)

are used for simplicity). For any (possibly random) positive sequences an∞n=1 and bn∞n=1,

an = OP (bn) means that limc→∞ lim supn Pr (an/bn > c) = 0; an = oP (bn) means that for all ε > 0,

limn→∞ Pr (an/bn > ε) = 0; and an ≍ bn means that there exist two constants 0 < c1 ≤ c2 < ∞such that c1an ≤ bn ≤ c2an. Also, we sometimes “wpa1” for an event An, to denote that Pr(An) → 1

as n → ∞. We use An ≡ Ak(n) and Hn ≡ Hk(n) to denote various sieve spaces. To simplify the

presentation, we assume that dim(Ak(n)) ≍ dim(Hk(n)) ≍ k(n), all of which grow to infinity with

the sample size n. We use const., c or C to mean a positive finite constant that is independent of

sample size but can take different values at different places. For sequences, (an)n, we sometimes use

an a (an a) to denote, that the sequence converges to a and that is increasing (decreasing)

sequence.

2 PSMD Estimation and SQLR Inference: An Overview

2.1 The Penalized Sieve Minimum Distance Estimator

Let Zi ≡ (Y ′i , X

′i)′ni=1 be a strictly stationary weakly dependent sample with FZi = FZ and

FYi|Xi= FY |X . Let m(X,α) ≡ E [ρ(Y,X;α)|X] =

∫ρ(y,X;α)dFY |X(y). Let Σ(X) be a positive

definite weighting matrix, and

Q(α) ≡ E[m(X,α)′Σ(X)−1m(X,α)

]≡ E

[||m(X,α)||2Σ−1

]be the population minimum distance (MD) criterion function. Then the semi/nonparametric con-

ditional moment model (1.3) can be equivalently expressed as m(X,α0) = 0 a.s. − X, where

α0 ≡ (θ′0, h0) ∈ A ≡ Θ×H, or as

infα∈A

Q(α) = Q(α0) = 0.

Let Σ0(X) ≡ V ar(ρ(Y,X;α0)|X) be positive definite for almost all X. We call E[||m(X,α)||2

Σ−10

]the population optimally weighted MD criterion function.

Let ϕ : A → Rdϕ be a functional with a fixed finite dϕ ≥ 1. We are interested in inference on

5

ϕ(α0). Let

Qn(α) ≡1

n

n∑i=1

m(Xi, α)′Σ(Xi)

−1m(Xi, α) (2.1)

be a sample estimate of Q(α), where m(X,α) and Σ(X) are any consistent estimators of m(X,α)

and Σ(X) respectively. When Σ(X) = Σ0(X) is a consistent estimator of the optimal weighting

matrix Σ0(X), we call the corresponding Qn(α) the sample optimally weighted MD criterion.

We estimate ϕ(α0) by ϕ(αn), where αn ≡ (θ′n, hn) is an approximate penalized sieve minimum

distance (PSMD) estimator of α0 ≡ (θ′0, h0), defined as

Qn(αn) + λnPen(hn) ≤ infα∈Ak(n)

Qn(α) + λnPen(h)

+OPZ∞ (ηn), (2.2)

where ηn∞n=1 is a sequence of positive real values such that ηn = o(1); λnPen(h) ≥ 0 is a penalty

term such that λn = o(1); and Ak(n) ≡ Θ×Hk(n) is a finite dimensional sieve for A ≡ Θ×H, more

precisely, Hk(n) is a finite dimensional linear sieve for H:

Hk(n) =

h ∈ H : h(·) =k(n)∑k=1

βkqk(·) = β′qk(n)(·)

, (2.3)

where qk∞k=1 is a sequence of known basis functions of a Banach space (H, ∥·∥H) such as wavelets,

splines, Fourier series, Hermite polynomial series, etc. And k(n) → ∞ as n→ ∞.

For the purely nonparametric conditional moment models E [ρ(Y,X;h0(·))|X] = 0, Chen and Pouzo

(2012a) proposed more general approximate PSMD estimators of h0 by allowing for possibly infinite

dimensional sieves (i.e., dim(Hk(n)) = k(n) ≤ ∞). Nevertheless, both the theoretical properties and

Monte Carlo simulations in Chen and Pouzo (2012a) recommend the use of the PSMD procedures

with finite dimensional sieves.

In this paper we first establish the large sample theories under a high level “local quadratic

approximation” (LQA) condition, which allows for weakly dependent data and any consistent

nonparametric estimator m(X,α) such as kernel, local linear regression and series least squares

(LS) estimators. In Appendix A we provide low level sufficient conditions for this LQA assumption

when data is i.i.d. and m(X,α) is a series least squares (LS) estimator of m(X,α):

m(X,α) ≡ pJn(X)′(P ′P )−n∑

i=1

pJn(Xi)ρ(Zi, α), (2.4)

where pj()∞j=1 is a sequence of known basis functions that can approximate any square integrable

functions of X well, pJn(X) = (p1(X), ..., pJn(X))′, P = (pJn(X1), ..., pJn(Xn))

′, and (P ′P )− is the

generalized inverse of the matrix P ′P . To simplify the presentation, we let pJn(X) be a tensor-

6

product linear sieve basis, and Jn be the dimension of pJn(X) such that Jn → ∞ slowly as n→ ∞.

See, e.g., Newey (1997), Huang (1998) and Chen (2007) for more details about tensor-product linear

sieves.

2.2 Preview of the SQLR Inference

For simplicity we let ϕ : A → R be a real-valued functional. Let ϕn ≡ ϕ(αn) be the plug-in

PSMD estimator of ϕ(α0). Under some regularity conditions we establish in Theorem 3.1 that a

self-normalized centered ϕ(αn)− ϕ(α0) is asymptotically normal regardless of whether ϕ(α0) is√n estimable or not. Denote

QLRn(ϕ0) ≡ n

(inf

α∈Ak(n):ϕ(α)=ϕ0

Qn(α)− Qn(αn)

)(2.5)

= n

(inf

α∈Ak(n):ϕ(α)=ϕ0

Qn(α)− infα∈Ak(n)

Qn(α)

)+ oPZ∞ (1)

as the sieve quasi likelihood ratio (SQLR) statistic. It becomes an optimally weighted SQLR statistic

when Qn(α) is the optimally weighted MD criterion. Under some regularity conditions, we show in

Theorem 3.2 that the QLRn(ϕ0) is stochastically bounded under the null hypothesis of ϕ(α0) = ϕ0,

and that the optimally weighted SQLR statistic is asymptotically chi-square distributed regardless

of whether ϕ(α0) is√n estimable or not.

We also consider two bootstrap versions of the SQLR statistic: the nonparametric bootstrap

SQLR and the Bayesian (or weighted) bootstrap SQLR. Let QLRB

n denote a bootstrap version of

the SQLR statistic:

QLRB

n (ϕn) ≡ n

(inf

α∈Ak(n):ϕ(α)=ϕn

QBn (α)− inf

α∈Ak(n)

QBn (α)

), (2.6)

where ϕn ≡ ϕ(αn) and QBn (α) is a bootstrap version of Qn(α):

QBn (α) ≡

1

n

n∑i=1

mB(Xi, α)′Σ(Xi)

−1mB(Xi, α), (2.7)

where mB(x, α) is a bootstrap version of m(x, α) (see Section 5 for details). In Theorem 5.2 we

establish that under the null hypothesis of ϕ(α0) = ϕ0, QLRB

n (ϕn) converges to the same limiting

distribution as that of QLRn(ϕ0), even for possibly non-optimally weighted SQLR statistic and

possibly irregular functionals.

An illustration via the NPQIV example. As an application of their general theory,

7

Chen and Pouzo (2012a) presented the consistency and the rate of convergence of the PSMD esti-

mator hn ∈ Hk(n) of the NPQIV model with i.i.d. data:

Y1 = h0(Y2) + U, Pr(U ≤ 0|X) = γ. (2.8)

In this example we have Σ0(X) = γ(1− γ). So we could use Σ(X) = γ(1− γ) and Qn(α) given in

(2.1) becomes the optimally weighted MD criterion.

Applying Theorem 3.1 of this paper, we immediately obtain:

√nϕ(hn)− ϕ(h0)

||v∗n||sd⇒ N(0, 1), where

||v∗n||2sd =∂ϕ(h0)

∂h[qk(n)(·)]′

Rk(n)

−1 ∂ϕ(h0)

∂h[qk(n)(·)],

Rk(n) =1

γ(1− γ)E(E[fU |Y2,X(0)qk(n)(Y2)|X]E[fU |Y2,X(0)qk(n)(Y2)|X]′

).

For the evaluation functional ϕ(h) = h(y2) we have ∂ϕ(h0)∂h [qk(n)(·)] = qk(n)(y2). For the weighted

integration functional ϕ(h) =∫w(y2)h(y2)dy2 we have ∂ϕ(h0)

∂h [qk(n)(·)] =∫w(y2)q

k(n)(y2)dy2. See

Subsection 6.1 for a Monte Carlo study regarding the finite sample performance of the asymptotic

normality approximation.

Under very mild condition (see, e.g., Chen and Pouzo (2012a)), the conditional expectation

operator Th = E[fU |Y2,X(0)h(Y2)|X

]mapping from h ∈ H to L2(fX) is compact, so is its adjoint

operator T ∗. Thus we have

T ∗Tψj(Y2) = µ2jψj(Y2), µ21 ≥ µ22 ≥ ... and µ2j 0,

where ψj(·) : j ≥ 1 is the eigenfunction sequence of H andµ2j : j ≥ 1

is the eigenvalue sequence

of the self-adjoint compact operator T ∗T . Suppose that (qk(·))k is a Riesz basis for H such that

Hk(n) = clsp ψj(·) : j = 1, ..., k(n). Then Rk(n) =1

γ(1−γ)Diagµ21, ..., µ

2k(n)

and

||v∗n||2sd = γ(1− γ)

k(n)∑j=1

µ−2j

(∂ϕ(h0)

∂h[ψj(·)]

)2

.

For the evaluation functional ϕ(h) = h(y2) we have

||v∗n||2sd = γ(1− γ)

k(n)∑j=1

µ−2j [ψj(y2)]

2.

8

For the weighted integration functional ϕ(h) =∫w(y2)h(y2)dy2 we have

||v∗n||2sd = γ(1− γ)

k(n)∑j=1

µ−2j

(∫w(y2)ψj(y2)dy2

)2

.

Therefore the variance of√n(ϕ(hn)− ϕ(h0)

)is of a complicated form and may diverge to infinity

as k(n) → ∞.

Applying Theorem 3.2 (or more precisely Theorem 4.2(1)), we immediately obtain that the

optimally weighted SQLR statistic QLRn(ϕ0) ⇒ χ21 under the null of ϕ(h0) = ϕ0. Thus we

can compute confidence regions for any functional ϕ(h), such as the evaluation and the weighted

integration functionals, as r ∈ R : QLRn(r) ≤ cχ2

1(τ),

where cχ21(τ) is the (1 − τ)-quantile of the χ2

1 distribution. See Subsection 6.2 for an empirical

illustration of this result to the nonparametric quantile IV Engel curve regression using the British

Family Survey data set that was first used in Blundell et al. (2007).

Instead of using the asymptotic critical values based on a χ21 distribution, we could also construct

a confidence set using bootstrap methods:

r ∈ R : QLRn(r) ≤ cn(τ)

,

where cn(τ) is the bootstrap (1− τ)-quantile of the asymptotic distribution of the SQLR statistic,

computed via either the nonparametric bootstrap or the weighted bootstrap as described in Section

5. See Subsection 6.1 for a Monte Carlo study of the finite sample performance of the bootstrap

based confidence sets.

2.3 A Brief Discussion on the Convergence Rate

Before we could derive the asymptotic distribution of the SQLR statistic, we need some consistency

and convergence rate results that allow us to concentrate on some shrinking neighborhood of the

true parameter value α0 of the semi-nonparametric model (1.3). For the purely nonparametric

conditional moment model E [ρ(Y,X;h0(·))|X] = 0, Chen and Pouzo (2012a) established the con-

sistency and the convergence rates of their various PSMD estimators of h0. Some of their results can

be trivially extended to establish the corresponding properties of our PSMD estimator αn ≡ (θ′n, hn)

defined in (2.2). For the sake of easy reference and to introduce basic assumptions and notation, we

present some sufficient conditions for consistency and the convergence rate here. These conditions

are also needed to establish the consistency and the convergence rate of bootstrap PSMD estimators

9

(see Lemma 5.1). We first impose three conditions on identification, sieve spaces, penalty functions

and sample criterion function. We equip the parameter space A ≡ Θ×H ⊆ Rdθ ×H with a (strong)

norm ∥α∥s ≡ ∥θ∥e + ∥h∥H. Let Πnα ≡ (θ′,Πnh) ∈ Ak(n) ≡ Θ×Hk(n).

Assumption 2.1 (Identification, sieves, criterion). (i) E[ρ(Y,X;α)|X] = 0 if and only if α ∈(A, ∥·∥s) with ∥α− α0∥s = 0; (ii) For all k ≥ 1, Ak ≡ Θ × Hk, Θ is a compact subset in Rdθ ,

Hk : k ≥ 1 is a non-decreasing sequence of non-empty closed subsets of (H, ∥·∥H) such that

H ⊆ cl (∪kHk), and there is Πnh0 ∈ Hk(n) with ||Πnh0 − h0||H = o(1); (iii) Q : (A, ∥·∥s) → [0,∞)

is lower semicontinuous, Q(Πnα0) = o(1). 5 (iv) Σ(x) and Σ0(x) are positive definite, and their

smallest and largest eigenvalues are finite and positive uniformly in x ∈ X .

Assumption 2.2 (Penalty). (i) λn > 0, λn = o(1); (ii) |Pen(Πnh0) − Pen(h0)| = O(1) with

Pen(h0) <∞; (iii) Pen : (H, ∥·∥H) → [0,∞) is lower semicompact. 6

Let ηn∞n=1 and δ2m,n∞n=1 be sequences of positive real values that decrease to zero as n→ ∞.

Let AM0

k(n) ≡ Θ×HM0

k(n) ≡ α = (θ′, h) ∈ Ak(n) : λnPen(h) ≤ λnM0 for a large but finite M0 such

that Πnα0 ∈ AM0

k(n) and that αn ∈ AM0

k(n) with probability arbitrarily close to one for all large n.

Assumption 2.3 (Sample Criterion). (i) Qn(Πnα0) ≤ c0Q(Πnα0)+OPZ∞ (ηn) for some ηn = o(1)

and a finite constant c0 > 0; (ii) Qn(α) ≥ cQ(α) − OPZ∞ (δ2m,n) uniformly over AM0

k(n) for some

δ2m,n = o(1) and a finite constant c > 0.

The following consistency result is a minor modification of Theorem 3.2 of Chen and Pouzo

(2012a).

Lemma 2.1. Let αn be the PSMD estimator defined in (2.2). If Assumptions 2.1, 2.2 and 2.3

hold, then: ||αn − α0||s = oPZ∞ (1) and Pen(hn) = OPZ∞ (1).

Given the consistency result, we can restrict our attention to a shrinking || · ||s−neighborhood

around α0. Let

Aos ≡ α ∈ A : ||α− α0||s ≤ ϵ, λnPen(h) ≤ λnM0 and Aosn ≡ Aos ∩ Ak(n)

for a positive finite constant M0 and a small positive ϵ such that Pr(αn /∈ Aosn) < ϵ eventually.

For any α ∈ Aos we define a pathwise derivative

dm(X,α0)

dα[α− α0] ≡ dE[ρ(Z, (1− τ)α0 + τα)|X]

dτ

∣∣∣∣τ=0

a.s. X

=dE[ρ(Z,α0)|X]

dθ′(θ − θ0) +

dE[ρ(Z,α0)|X]

dh[h− h0] a.s. X.

5A function f is lower semicontinuous at a point x0 iff limx→x0 f(x) ≥ f(x0). We say is lower semicontinuous, ifit is lower semicontinuous at any point.

6A function f is lower semicompact iff for all M , x : f(x) ≤ M is compact.

10

Following Ai and Chen (2003) and Chen and Pouzo (2009), we introduce a pseudo-metric ||α1−α2||for any α1, α2 ∈ Aos, as

||α1 − α2||2 ≡ E

[(dm(X,α0)

dα[α1 − α2]

)′Σ(X)−1

(dm(X,α0)

dα[α1 − α2]

)]. (2.9)

The next assumption is about the local curvature of the population criterion Q(α).

Assumption 2.4 (Local curvature). (i) Aos and Aosn are convex, m(·, α) is continuously pathwise

differentiable with respect to α ∈ Aos, and there is a finite constant C > 0 such that ||α − α0|| ≤C||α − α0||s for all α ∈ Aos; (ii) There are finite constants c1, c2 > 0 such that c1||α − α0||2 ≤Q(α) ≤ c2||α− α0||2 holds for all α ∈ Aosn.

Recall the definition of the sieve measure of local ill-posedness

τn ≡ supα∈Aosn:||α−Πnα0||=0

||α−Πnα0||s||α−Πnα0||

.

The problem of estimating α0 under ||·||s is locally ill-posed in rate if and only if lim supn→∞ τn = ∞.

The following general rate result is a minor modification of Theorem 4.1 and Remark 4.1(1) of

Chen and Pouzo (2012a), and hence we omit its proof. Let δm,n∞n=1 be a sequence of positive

real values that decrease to zero as n→ ∞.

Lemma 2.2. Let αn be the PSMD estimator defined in (2.2) with maxλn, ηn = o(n−1). Let

Assumptions 2.1, 2.2, 2.3 and 2.4 hold, and Qn(α) ≥ cQ(α)−OPZ∞ (δ2m,n) uniformly over Aosn for

some finite constant c > 0. If maxδ2m,n, Q(Πnα0), λn, ηn = δ2m,n then:

||αn − α0|| = OPZ∞ (δm,n) and ||αn − α0||s = OPZ∞ (||α0 −Πnα0||s + τnδm,n) .

The above convergence rate result is applicable to strictly stationary weakly dependent data

and general nonparametric estimator m(X,α) of m(X,α) as soon as one could compute δ2m,n, the

rate at which Qn(α) goes to Q(α). See Chen and Pouzo (2012a) and Chen and Pouzo (2009) for

low level sufficient conditions in terms of i.i.d. data and the series LS estimator (2.4) of m(X,α).

In particular, Lemma C.2 of Chen and Pouzo (2012a) implies that under very mild conditions,

we have Qn(α) ≍ Q(α) − OPZ∞ (δ2m,n) uniformly over Aosn, with δ2m,n ≍ max

Jnn , b

2m,Jn

and

Q(Πnα0) = O(b2m,Jn), where bm,Jn is the L2(fX)−bias order of approximating m(·, α) by the series

basis pJn(·).Lemma 2.2 implies that ||αn − α0|| = OPZ∞ (δn) and ||αn − α0||s = OPZ∞ (δs,n), where

δn : n ≥ 1 and δs,n : n ≥ 1 are real positive sequences such that δn ≍ δm,n = o(1) and δs,n =

11

o(1), δs,n ≥ δn. Thus αn ∈ Nosn ⊆ Nos wpa1, where

Nos ≡ α ∈ Aos : ||α− α0|| ≤Mnδn, ||α− α0||s ≤Mnδs,n, λnPen(h) ≤ λnM0 ,

Nosn ≡ Nos ∩ Ak(n),

withMn ≡ log(log(n)). In the rest of the paper, we can regard Nos as the effective parameter space

and Nosn as its sieve space.

3 Local Asymptotic Theory

In this section, we establish the asymptotic normality of the plug-in PSMD estimator ϕ(αn) of a

possibly irregular functional ϕ : A → R of the general model (1.3) and the limiting distribution of

a properly scaled SQLR statistic.

3.1 Riesz representation

We first provide a representation of the functional of interest ϕ : A → R, which is crucial for all

the subsequent asymptotic theories.

Given the definition of the norm || · || (in equation (2.9)) and the local parameter spaces Aos

or Nos, we can construct a Hilbert space (V, || · ||) with V ≡ clsp(Aos −α0), where clsp(·) is theclosure of the linear span under || · ||. For any v1, v2 ∈ V, we define an inner product induced by

the metric || · ||:

⟨v1, v2⟩ = E

[(dm(X,α0)

dα[v1]

)′Σ(X)−1

(dm(X,α0)

dα[v2]

)],

and for any v ∈ V we call v = 0 if and only if ||v|| = 0 (i.e., functions in V are defined in an

equivalent class sense according to the metric || · ||).For any v ∈ V, we define dϕ(α0)

dα [v] to be the pathwise (directional) derivative of the functional

ϕ (·) at α0 and in the direction of v = α− α0 ∈ V :

dϕ(α0)

dα[v] =

∂ϕ(α0 + τv)

∂τ

∣∣∣∣τ=0

for any v ∈ V.

If dϕ(α0)dα [·] is bounded on the infinite dimensional Hilbert space (V, || · ||), i.e.

supv∈V,v =0

∣∣∣dϕ(α0)dα [v]

∣∣∣∥v∥

<∞,

12

then there is a Riesz representer v∗ ∈ V of the linear functional dϕ(α0)dα [·] on V such that

dϕ(α0)

dα[v] = ⟨v∗, v⟩ for all v ∈ V and ∥v∗∥ ≡ sup

v∈V,v =0

∣∣∣dϕ(α0)dα [v]

∣∣∣∥v∥

;

In this case we say that ϕ (·) is regular (at α = α0). If dϕ(α0)dα [·] is unbounded on the infinite

dimensional Hilbert space (V, || · ||), i.e.

supv∈V,v =0

∣∣∣dϕ(α0)dα [v]

∣∣∣∥v∥

= ∞,

then there does not exist a Riesz representer of the linear functional dϕ(α0)dα [·] on V; In this case we

say that ϕ (·) is irregular (at α = α0).

It is known that ϕ (·) being regular (i.e., dϕ(α0)dα [·] being bounded on V) is a necessary con-

dition for the root-n rate of convergence of ϕ(αn) − ϕ(α0). Unfortunately for complicated semi-

nonparametric models (1.3), it is difficult to compute supv∈V,v =0

∣∣∣dϕ(α0)dα [v]

∣∣∣ / ∥v∥ explicitly; and

hence difficult to verify whether a functional ϕ (·) is regular or not.

3.1.1 Sieve Riesz representer

Define

α0,n ≡ arg minα∈Nosn

||α− α0||. (3.1)

Let Vk(n) ≡ clsp (Nosn − α0,n), where clsp (.) denotes the closed linear span under ∥·∥. Then

Vk(n) is a finite dimensional Hilbert space under ∥·∥. Moreover, Vk(n) is dense in V under ∥·∥.To simplify the presentation, we assume that dim(Vk(n)) = dim(Ak(n)) ≍ k(n), all of which grow

to infinity with n. By definition we have ⟨vn, α0,n − α0⟩ = 0 for all vn ∈ Vk(n). For any vn =

αn − α0,n ∈ Vk(n), we let

dϕ(α0)

dα[vn] =

dϕ(α0)

dα[αn − α0]−

dϕ(α0)

dα[α0,n − α0].

So dϕ(α0)dα [·] is also a linear functional on Vk(n).

Note that Vk(n) is a finite dimensional Hilbert space. As any linear functional on a finite

dimensional Hilbert space is bounded, we can invoke the Riesz representation theorem to deduce

that there is a unique v∗n ∈ Vk(n) such that

dϕ(α0)

dα[v] = ⟨v∗n, v⟩ for all v ∈ Vk(n) (3.2)

13

and

dϕ(α0)

dα[v∗n] = ∥v∗n∥

2 ≡ supv∈Vk(n):∥v∥=0

∣∣∣dϕ(α0)dα [v]

∣∣∣2∥v∥2

<∞. (3.3)

We call v∗n the sieve Riesz representer of the functional dϕ(α0)dα [·] on Vk(n). By definition we have

for any non-zero linear functional dϕ(α0)dα [·],

0 < ∥v∗n∥2 = E

[(dm(X,α0)

dα[v∗n]

)′Σ(X)−1

(dm(X,α0)

dα[v∗n]

)]<∞.

We emphasize that the sieve Riesz representation (3.2) and (3.3) of the linear functional dϕ(α0)dα [·]

on Vk(n) always exists regardless of whetherdϕ(α0)dα [·] is bounded on the infinite dimensional Hilbert

space (V, || · ||) or not. If ∥v∗n∥ = O (1) (in fact ∥v∗n∥ → ∥v∗∥ <∞ and ∥v∗ − v∗n∥ → 0 as k(n) → ∞),

then dϕ(α0)dα [·] is bounded on (V, || · ||) and ϕ (·) is regular (at α = α0). If ∥v∗n∥ → ∞ as k(n) → ∞,

then dϕ(α0)dα [·] is unbounded on (V, || · ||) and ϕ (·) is irregular (at α = α0).

Moreover, we can always compute the sieve Riesz representer v∗n ∈ Vk(n) for any functional

defined in (3.2) and (3.3) explicitly. Let Ak(n) = Θ × Hk(n) where Hk(n) given in (2.3) is a finite

dimensional linear sieve. Let ∥·∥ be the norm defined in (2.9) and Vk(n) = Rdθ×vh (·) = qk(n)(·)′β :

β ∈ Rk(n) be dense in the infinite dimensional Hilbert space (V, ∥·∥). By definition, the sieve

Riesz representer v∗n = (v∗′θ,n, v∗h,n (·))′ = (v∗′θ,n, q

k(n)(·)′β∗n)′ ∈ Vk(n) of dϕ(α0)dα [·] solves the following

optimization problem:

dϕ(α0)

dα[v∗n] = ∥v∗n∥

2 = supv=(v′θ,vh)

′∈Vk(n),v =0

∣∣∣∂ϕ(α0)∂θ′ vθ +

∂ϕ(α0)∂h [vh(·)]

∣∣∣2E

[(dm(X,α0)

dα [v])′

Σ(X)−1(dm(X,α0)

dα [v])]

= supγ=(v′θ,β′)

′∈Rdθ+k(n),γ =0

γ′Fk(n)F′k(n)γ

γ′Rk(n)γ, (3.4)

where

Fk(n) ≡(∂ϕ(α0)

∂θ′,∂ϕ(α0)

∂h[qk(n)(·)′]

)′(3.5)

is a (dθ + k(n))× 1 vector,7 and

γ′Rk(n)γ ≡ E

[(dm(X,α0)

dα[v]

)′Σ(X)−1

(dm(X,α0)

dα[v]

)]for all v =

(v′θ, q

k(n)(·)′β)′

∈ Vk(n),

(3.6)

7When dϕ(α0)dα

[·] applies to a vector (matrix), it stands for element-wise (column-wise) operations. We follow the

same convention for other operators such as dm(X,α0)dα

[·] throughout the paper.

14

with Rk(n) being (dθ + k(n))× (dθ + k(n)) a positive definite matrix, where

Rk(n) ≡

I11 In,12

I ′n,21 In,22

with

I11 = E

[(dm(X,α0)

dθ′

)′Σ(X)−1 dm(X,α0)

dθ′

], In,22 = E

[(dm(X,α0)

dh [qk(n)(·)′])′

Σ(X)−1(dm(X,α0)

dh [qk(n)(·)′])]

,

and In,12 = E

[(dm(X,α0)

dθ′

)′Σ(X)−1

(dm(X,α0)

dh [qk(n)(·)′])]

∈ Rdθ×k(n).

The sieve Riesz representation (3.2) becomes: for all v =(v′θ, q

k(n)(·)′β)′ ∈ Vk(n),

dϕ(α0)

dα[v] = F ′

k(n)γ = ⟨v∗n, v⟩ = γ∗′n Rk(n)γ for all γ = (v′θ, β′)′ ∈ Rdθ+k(n). (3.7)

It is obvious that the optimal solution of γ in (3.4) or in (3.7) has a closed-form expression:

γ∗n =(v∗′θ,n, β

∗′n

)′= R−1

k(n)Fk(n). (3.8)

The sieve Riesz representer is then given by

v∗n = (v∗′θ,n, v∗h,n (·))′ = (v∗′θ,n, q

k(n)(·)′β∗n)′ ∈ Vk(n).

Consequently,

∥v∗n∥2 = γ∗′n Rk(n)γ

∗n = F ′

k(n)R−1k(n)Fk(n), (3.9)

and hence ϕ(·) is regular (or irregular) at α = α0 if and only if limk(n)→∞ F ′k(n)R

−1k(n)Fk(n) <∞ (or

= ∞).

Remark 3.1. Recall that

R−1k(n) ≡

I11n I12n

I21n I22n

=

I11n −I−111 In,12I

22n

−I−1n,22I

′n,21I

11n I22n

with

I11n ≡(I11 − In,12I

−1n,22I

′n,21

)−1, I22n ≡

(In,22 − I ′n,21I

−111 In,12

)−1.

For the Euclidean parameter functional ϕ(α) = λ′θ, we have Fk(n) = (λ′,0′k(n))′ with 0′k(n) =

[0, ..., 0]1×k(n), and hence v∗n = (v∗′θ,n, qk(n)(·)′β∗n)′ ∈ Vk(n) with v

∗θ,n = I11n λ, β

∗n = I21n λ = −I−1

n,22I′n,21v

∗θ,n,

and

∥v∗n∥2 = F ′

k(n)R−1k(n)Fk(n) = λ′I11n λ ≤ λmax(I

11n )× λ′λ.

15

Thus the functional ϕ(α) = λ′θ is regular if limk(n)→∞ λmax(I11n ) <∞. In this case,

limk(n)→∞

∥v∗n∥2 = lim

k(n)→∞λ′I11n λ = λ′I−1

∗ λ = ∥v∗∥2 ,

where

I∗ = infwE

[∥∥∥∥Σ(X)−12

(dm(X,α0)

dθ′− dm(X,α0)

dh[w]

)∥∥∥∥2e

], (3.10)

and v∗ = (v∗′θ , v∗h (·))′ ∈ V where v∗θ ≡ I−1

∗ λ, v∗h ≡ −w∗ × v∗θ , and w∗ solves (3.10). That is,

v∗ = (v∗′θ , v∗h (·))′ becomes the Riesz representer for ϕ(α) = λ′θ previously computed in Ai and Chen

(2003) and Chen and Pouzo (2009). Moreover, if Σ(X) = Σ0(X) and data is i.i.d., then I∗ becomes

semiparametric efficiency bound for θ0 satisfying the model (1.3).

3.1.2 Local characterization of ϕ(α)

As it will become clear later (see Theorem 3.1), the convergence rate of ϕ(αn)− ϕ(α0) depends on

the order of ∥v∗n∥, and

∥v∗n∥2sd ≡ V ar

(n−1/2

n∑i=1

(dm(Xi, α0)

dα[v∗n]

)′Σ(Xi)

−1ρ(Zi, α0)

), (3.11)

which could go to infinity if ∥v∗n∥ → ∞ as k(n) → ∞. Denote

S∗n,i ≡

(dm(Xi, α0)

dα[v∗n]

)′Σ(Xi)

−1ρ(Zi, α0) (3.12)

as the sieve score associated with the i-th observation. Let Σ0(X) ≡ V ar(ρ(Y,X;α0)|X). Then

V ar(S∗n,1

)= E

[(dm(X,α0)

dα[v∗n]

)′Σ(X)−1Σ0(X)Σ(X)−1

(dm(X,α0)

dα[v∗n]

)]. (3.13)

Let ρ∗n(i) ≡ E(S∗n,1S

∗n,i+1

)/V ar

(S∗n,1

)for all i ≥ 1. Then

∥v∗n∥2sd = V ar

(S∗n,1

)×

[1 + 2

n−1∑i=1

(1− i

n

)ρ∗n(i)

].

The triangular array sieve score processS∗n,i

n

i=1is said to be weakly dependent if

∑n−1i=1

(1− i

n

)ρ∗n(i) =

O(1). IfS∗n,i

n

i=1is a martingale difference array, then ρ∗n(i) = 0 for all i ≥ 1 and ∥v∗n∥

2sd =

V ar(S∗n,1

). In general we have ∥v∗n∥

2sd = O

V ar

(S∗n,1

)for strictly stationary weakly dependent

data.

16

Let

u∗n ≡ v∗n∥v∗n∥sd

(3.14)

be the “scaled sieve Riesz representer”. Let Tn ≡ t ∈ R : |t| ≤ 4M2nδn, where Mn and δn are the

ones used in the definition of Nosn.

Assumption 3.1 (Local behavior of ϕ). (i) v 7→ dϕ(α0)dα [v] is a bounded linear functional mapping

from Vk(n) to R; lim infn→∞ ||v∗n|| > 0 and 0 < c ≤ ||v∗n||sd||v∗n||

≤ C <∞;

(ii) sup(α,t)∈Nosn×Tn

∣∣∣∣ϕ (α+ tu∗n)− ϕ(α0)−dϕ(α0)

dα[α+ tu∗n − α0]

∣∣∣∣ = o(n−1/2 ∥v∗n∥

);

(iii)∣∣∣dϕ(α0)

dα [α0,n − α0]∣∣∣ = o

(n−1/2 ∥v∗n∥

).

Assumption 3.1(i) requires that ∥v∗n∥sd is proportional to ||v∗n||, which is satisfied under mild

conditions. To see this, we note that ∥v∗n∥2 ≍ V ar

(S∗n,1

)under Assumption 2.1(iv). We also

have ∥v∗n∥2sd ≍ V ar

(S∗n,1

)for typical semi-nonparametric models with strictly stationary weakly

dependent data. Thus Assumption 3.1(i) is satisfied.

Assumption 3.1(ii) controls the linear approximation error of a possibly nonlinear functional

ϕ (·), which is automatically satisfied when ϕ (·) is a linear functional.

Assumption 3.1(iii) controls the bias part due to the finite dimensional sieve approximation of

α0,n to α0. It is a condition imposed on the growth rate of the sieve dimension dim(Ak(n)) ≍ k(n),

and requires that the sieve approximation error rate is of a smaller order than n−1/2 ∥v∗n∥. Given

Assumption 3.1(i), Assumption 3.1(iii) requires that the sieve bias term,∣∣∣dϕ(α0)

dα [α0,n − α0]∣∣∣, is of a

smaller order than that of the sieve standard deviation term, n−1/2 ∥v∗n∥sd.

Remark 3.2. When ϕ (·) is a regular functional, we have ∥v∗n∥ → ∥v∗∥ <∞, and since ⟨v∗n, α0,n − α0⟩ =0 (by definition of α0,n), we have

∣∣∣dϕ(α0)dα [α0,n − α0]

∣∣∣ ≤ ∥v∗ − v∗n∥×∥α0,n − α0∥, and hence Assump-

tion 3.1(iii) is satisfied if ||v∗−v∗n||× ||α0,n−α0|| = o(n−1/2), which is similar to assumption 4.2 in

Ai and Chen (2003) and Ai and Chen (2007) and assumption 3.2(iii) in Chen and Pouzo (2009)

for regular functionals of the model (1.3).

3.2 Local quadratic approximation (LQA)

The next assumption is about the local quadratic approximation (LQA) to the sample criterion

difference along the scaled sieve Riesz representer direction u∗n = v∗n/ ∥v∗n∥sd.For any tn ∈ Tn, we let Λn(α(tn), α) ≡ 0.5Qn(α(tn))− Qn(α) with α(tn) ≡ α+ tnu

∗n. Denote

Zn ≡ n−1n∑

i=1

(dm(Xi, α0)

dα[u∗n]

)′Σ(Xi)

−1ρ(Zi, α0) = n−1n∑

i=1

S∗n,i

∥v∗n∥sd. (3.15)

17

Assumption 3.2 (LQA). (i) For all (α, t) ∈ Nosn × Tn, α(t) ∈ Ak(n); and with rn(tn) =(maxt2n, tnn−1/2, o(n−1)

)−1,

sup(α,tn)∈Nosn×Tn

rn(tn)

∣∣∣∣Λn(α(tn), α)− tn Zn + ⟨u∗n, α− α0⟩ −Bn

2t2n

∣∣∣∣ = oPZ∞ (1),

where Bn is a Zn measurable positive random variable with Bn = OPZ∞ (1); (ii)√nZn ⇒ N(0, 1).

Assumption 3.2(i) implicitly imposes restrictions on the nonparametric estimator m(x, α) of

the conditional mean function m(x, α) in a shrinking neighborhood of α0. A difficulty in the

verification of Assumption 3.2(i) is that, due to the non-smooth residual function ρ(Z,α), the

estimator m(x, α) (and hence the sample criterion function Qn(α)) could be pointwise non-smooth

with respect to α. Fortunately, under Assumption 2.4(i), Qn(α) could be well approximated by

a “smooth” version of it uniformly in α ∈ Nosn. For the regular functional ϕ(α) = λ′θ of the

model (1.3) with i.i.d. data, Ai and Chen (2003) and Chen and Pouzo (2009) already provide low

level sufficient conditions when m(x, α) is a series LS estimator. Their conditions can be modified

slightly to allow for irregular functionals as well. In Appendix A we present one set of low level

sufficient conditions (Assumption A) for possibly irregular functionals ϕ(·) of the model (1.3) with

possibly non-smooth residuals. The next lemma formally states the result.

Lemma 3.1. Let Zini=1 be i.i.d., m be the series LS estimator (2.4) and conditions for Lemma

2.2 hold. If Assumption A in Appendix A holds, then Assumption 3.2(i) holds.

Assumption 3.2(ii) is a standard one. For i.i.d. data, it is implied by the following Lindeberg

condition: For all ϵ > 0,

lim supn→∞

E

[(S∗n,i

∥v∗n∥sd

)2

1

∣∣∣∣ S∗n,i

ϵ√n ∥v∗n∥sd

∣∣∣∣ > 1

]= 0, (3.16)

which, under Remark 3.2, is automatically satisfied when the functional ϕ(·) is regular (i.e.,

∥v∗n∥ → ∥v∗∥ < ∞). This is why Assumption 3.2(ii) is not imposed in Ai and Chen (2003) and

Chen and Pouzo (2009) in their root-n asymptotically normal estimation of the regular functional

ϕ(α) = λ′θ.

3.3 Asymptotic normality of the plug-in PSMD estimator

We now establish the asymptotic normality of the plug-in PSMD estimator ϕ(αn) of a possibly

irregular functional ϕ(α0) of the general model (1.3).

Theorem 3.1. Let αn be the PSMD estimator (2.2) and conditions for Lemma 2.2 hold. Let

Assumptions 3.1(i) and 3.2(i) hold. Then: (1)√n⟨u∗n, αn − α0⟩ = −

√nZn + oPZ∞ (1).

18

(2) If, in addition, Assumptions 3.1(ii)(iii) and 3.2(ii) hold, then:

√nϕ(αn)− ϕ(α0)

||v∗n||sd= −

√nZn + oPZ∞ (1) ⇒ N(0, 1).

When the functional ϕ(·) is regular at α = α0 (i.e., ∥v∗n∥ = O(1)), we have ∥v∗n∥sd ≍ ∥v∗n∥ = O(1)

typically; so ϕ(αn) converges to ϕ(α0) at the parametric rate of 1/√n. When the functional ϕ(·)

is irregular at α = α0 (i.e., ∥v∗n∥ → ∞), we have ∥v∗n∥sd → ∞ (under Assumption 3.1(i)); so the

convergence rate of ϕ(αn) becomes slower than 1/√n. Regardless of whether the sieve variance

∥v∗n∥2sd (defined in (3.11)) stays bounded asymptotically (i.e., as n→ ∞) or not, it always captures

whatever true temporal dependence exists in finite samples.

For any regular functional of the semi-nonparametric model (1.3), Theorem 3.1 implies that

√n (ϕ(αn)− ϕ(α0)) = −n−1/2

n∑i=1

S∗n,i + oPZ∞ (1) ⇒ N(0, σ2v∗),

with

σ2v∗ = limn→∞

∥v∗n∥2sd = lim

n→∞V ar

(n−1/2

n∑i=1

S∗n,i

)∈ (0,∞).

If the sieve score process S∗n,ini=1 (defined in (3.12)) is a martingale difference array (which is the

case with i.i.d. data), then ∥v∗n∥2sd = V ar

(S∗n,i

)and

σ2v∗ = limn→∞

V ar(S∗n,1

)= E

[(dm(X,α0)

dα[v∗]

)′Σ(X)−1Σ0(X)Σ(X)−1

(dm(X,α0)

dα[v∗]

)].

Thus, our Theorem 3.1 is a natural extension of the asymptotic normality results of Ai and Chen

(2003) and Chen and Pouzo (2009) for the specific regular functional ϕ(α0) = λ′θ0 of the model

(1.3) with i.i.d. data. See Remark 3.1 for further discussion.

Theorem 3.1 is similar to that of Chen et al. (2012) on the asymptotic normality of their plug-

in sieve M estimators for possibly irregular functionals of time series models, except that our

model (1.3) allows for nonparametric endogeneity. In the special case of generalized nonlinear

least squares models, dm(X,α0)dα [v] = dρ(Z,α0)

dα [v] for all directions v, our Theorem 3.1 recovers their

asymptotic normality result on the plug-in sieve generalized nonlinear least squares estimator:√nϕ(αn)−ϕ(α0)

||v∗n||sd⇒ N(0, 1) with ∥v∗n∥

2sd = V ar

(n−1/2

∑ni=1

(dρ(Zi,α0)

dα [v∗n])′

Σ(Xi)−1ρ(Zi, α0)

).

3.4 Asymptotic distribution of the SQLR

We now characterize the asymptotic behavior of the possibly non-optimally weighted SQLR statistic

QLRn(ϕ0) defined in (2.5).

19

Let αRn ∈ α ∈ Ak(n) : ϕ(α) = ϕ0 be the restricted PSMD estimator of α0 ≡ (θ′0, h0), defined

as

Qn(αRn ) + λnPen(h

Rn ) ≤ inf

α∈Ak(n):ϕ(α)=ϕ0

Qn(α) + λnPen(h)

+OPZ∞ (ηn). (3.17)

Then:

QLRn(ϕ0) = n(Qn(α

Rn )− Qn(αn)

)= n

(inf

α∈Ak(n):ϕ(α)=ϕ0

Qn(α)− infα∈Ak(n)

Qn(α)

)+ oPZ∞ (1).

Theorem 3.2. Let αn be the PSMD estimator (2.2), αRn be the restricted PSMD estimator (3.17)

and conditions for Lemma 2.2 hold. Let Assumptions 3.1 and 3.2 with∣∣Bn − ||u∗n||2

∣∣ = oPZ∞ (1)

hold. Then, under the null hypothesis of ϕ(α0) = ϕ0, we have:

||u∗n||2 × QLRn(ϕ0) =(√nZn

)2+ oPZ∞ (1) ⇒ χ2

1.

Compared to Theorem 3.1(2) on the asymptotic normality of ϕ(αn), Theorem 3.2 on the limiting

distribution of the SQLR statistic requires an extra condition∣∣Bn − ||u∗n||2

∣∣ = oPZ∞ (1), which is

also needed even for QLR statistics in parametric extremum estimation and testing problems.

Assumption B below provides a simple sufficient condition for∣∣Bn − ||u∗n||2

∣∣ = oPZ∞ (1).

Remark 3.3. If ϕ(·) is continuous in Ak(n), then α ∈ Ak(n) : ϕ(α) = ϕ0 is a closed set of Ak(n).

Given Assumption 3.1, following the same proof of Lemma 2.2 (see, e.g., Chen and Pouzo (2012a)

or Chen and Pouzo (2009)) with the sieve Ak(n) replaced by α ∈ Ak(n) : ϕ(α) = ϕ0, we obtain

αRn ∈ Nosn wpa1. under the null hypothesis of ϕ(α0) = ϕ0.

4 Inference Based on Asymptotic Critical Values

In this section we provide two simple inference procedures for possibly irregular functionals of the

general model (1.3). The first one is based on the asymptotic normality Theorem 3.1 with a sieve

variance estimator. The second one is based on Theorem 3.2 with the optimally weighted SQLR

statistic.

4.1 Sieve estimation of variance of ϕ(αn)

In order to apply the asymptotic normality Theorem 3.1, we need an estimator of the sieve variance

∥v∗n∥2sd defined in (3.11). In this section we propose a simple consistent estimator of ∥v∗n∥

2sd assuming

i.i.d. data.

The theoretical sieve Riesz representer v∗n is not known and has to be estimated. Let ∥·∥n,W

20

denote the empirical norm induced by the following empirical inner product

⟨v1, v2⟩n,W ≡ 1

n

n∑i=1

(dm(Xi, αn)

dα[v1]

)′W (Xi)

(dm(Xi, αn)

dα[v2]

), (4.1)

for any v1, v2 ∈ Vk(n), whereW (X) is a positive definite weighting matrix for almost allX. Likewise

we introduce a theoretical inner product associated with the weighting matrix W as

⟨v1, v2⟩W ≡ E

[(dm(X,α0)

dα[v1]

)′W (X)

(dm(X,α0)

dα[v2]

)].

To simplify notation we also have:

⟨v1, v2⟩Σ−1 ≡ ⟨v1, v2⟩ and ⟨v1, v2⟩Σ−10

≡ ⟨v1, v2⟩0 for all v1, v2 ∈ Vk(n).

We define an empirical sieve Riesz representer v∗n of the functional dϕ(αn)dα [·] with respect to the

empirical norm || · ||n,Σ−1 as

dϕ(αn)

dα[v∗n] = sup

v∈Vk(n),v =0

|dϕ(αn)dα [v]|2

||v||2n,Σ−1

<∞ (4.2)

anddϕ(αn)

dα[v] = ⟨v∗n, v⟩n,Σ−1 for any v ∈ Vk(n). (4.3)

Recall that ∥v∗n∥2sd = V ar(S∗

n,1) when Zini=1 is i.i.d., we can define a simple sieve variance

estimator as

||v∗n||2n,sd = ||v∗n||2n,Σ−1Σ0Σ−1

where Σ0 is a consistent estimator of Σ0, e.g. En[ρ(Z, αn)ρ(Z, αn)′ | ·], where En is some consistent

estimator of the conditional mean, such as a series, Kernel or local polynomial based estimator. In

the following we denote V1k(n) ≡ v ∈ Vk(n) : ||v|| = 1.

Assumption 4.1. (i) supα∈Nosnsup

v∈V1k(n)

∣∣∣dϕ(α)dα [v]− dϕ(α0)dα [v]

∣∣∣ = oPZ∞ (1); (ii) for any α ∈ Nosn,

v 7→ dm(·,α)dα [v] ∈ L2(fX) is a bounded linear functional measurable with respect to Zn; and

supv1,v2∈V

1k(n)

∣∣⟨v1, v2⟩n,Σ−1 − ⟨v1, v2⟩Σ−1

∣∣ = oPZ∞ (1);

supv∈V1

k(n)

∣∣⟨v, v⟩n,Σ−1Σ0Σ−1 − ⟨v, v⟩Σ−1Σ0Σ−1

∣∣ = oPZ∞ (1);

(iii) For Γ(·) ∈ Σ(·),Σ0(·), supX ||Γ(x) − Γ(x)|| = oPZ∞ (1); Γ(x) is positive definite and its

21

smallest and largest eigenvalues are finite and positive uniformly in x ∈ X wpa1.

Assumption 4.1(i) becomes vacuous if ϕ is linear. Assumption 4.1(ii) implicitly assumes that the

residual function ρ() is “smooth” in α ∈ Nosn (see, e.g., Ai and Chen (2003)) or that dm(X,αn)dα [v]

can be well approximated by numerical derivatives (see, e.g., Hong et al. (2010)).

Theorem 4.1. (1) Let Zini=1 be i.i.d. and conditions for Lemma 2.2 hold. If Assumption 4.1

holds, then: ∣∣∣∣ ||v∗n||n,sd||v∗n||sd− 1

∣∣∣∣ = oPZ∞ (1).

(2) If, in addition, all the assumptions of Theorem 3.1(2) hold, then:

√nϕ(αn)− ϕ(α0)

||v∗n||n,sd= −

√nZn + oPZ∞ (1) ⇒ N(0, 1).

Theorem 4.1 is stated for the semi-nonparametric conditional moment model (1.3) with i.i.d.

data. It says that one could treat the linear sieve approximation to unknown function α as if it

were parametric and the corresponding parametric standard errors are consistent for the PSMD

estimator of the unknown function. This is a natural generalization of the standard error calculation

in Newey (1997) for series LS regression. Recently, Chen et al. (2012) proposed sieve robust long

run variance estimation for a plug-in sieve M estimator of semi-nonparametric time series models.

Their result can be extended to our model (1.3) with strictly stationary weakly dependent data.

Theorem 4.1 (or its time series extension) allows us to construct confidence sets for ϕ(α0) based

on a possibly non-optimally weighted plug-in PSMD estimator ϕ(αn). A potential drawback, is

that it requires a consistent estimator for v 7→ dm(·,α0)dα [v], which may be hard to compute in practice

when the residual function ρ(Z,α) is not pointwise smooth in α ∈ Nosn such as in the NPQIV (2.8)

example.

4.2 Optimally Weighted SQLR

For the specific regular functional ϕ(α) = λ′θ of the semi-nonparametric conditional moment model

(1.3), Chen and Pouzo (2009) established that the optimally weighted SQLR statistic is asymptoti-

cally chi-square distributed. This result is an extension of the earlier results in Murphy and der Vaart

(2000), Fan et al. (2001) and Shen and Shi (2005) on semiparametric likelihood ratio test of regular

functionals. Here we show that the same result remains valid even for irregular functionals.

In this subsection, to stress the fact that we focus on the optimally weighted PSMD procedure,

we use v0n and ||v0n||0 to denote the corresponding v∗n and ||v∗n|| computed using the optimal weighting

22

matrix Σ = Σ0. For instance,

||v0n||20 = E

[(dm(X,α0)

dα[v0n]

)′Σ0(X)−1

(dm(X,α0)

dα[v0n]

)].

We call the corresponding sieve score, S0n,i ≡

(dm(Xi,α0)

dα [v0n])′

Σ0(Xi)−1ρ(Zi, α0), the optimal sieve

score. Note that ||v0n||sd = ||v0n||0 if the optimal sieve score processS0n,i

n

i=1is a martingale

difference array. If ||v0n||sd = ||v0n||0 we call the SQLR statistic the optimally weighted SQLR

statistic. Applying Theorem 3.2, we immediately obtain that the optimally weighted SQLR is

asymptotically chi-square distributed. This result allows us to compute confidence sets for ϕ(α)

without the need of a consistent variance estimator for ϕ(αn).

By Theorem 3.1(2), ||v0n||2sd = ||v0n||20 is the asymptotic variance of the optimally weighted

PSMD estimator ϕ(αn). We could compute a consistent estimator of the variance ||v0n||2sd = ||v0n||20by looking at the “slope” of the optimally weighted SQLR. More precisely, let

||v0n||20 ≡

(Qn(αn)− Qn(αn)

ε2n

)−1

(4.4)

where αn is an approximate minimizer of Qn(α) over α ∈ Ak(n) : ϕ(α) = ϕ(αn)−εn. As discussed

in Remark 3.3, one can show that αn ∈ Nosn wpa1 easily.

We now formally state these results.

Theorem 4.2. Let αn be the optimally weighted PSMD estimator (2.2) with Σ = Σ0. Let all

the conditions of Theorem 3.2 hold with ||v0n||sd = ||v0n||0. Then: (1) under the null hypothesis of

ϕ(α0) = ϕ0, we have:

QLRn(ϕ0) =

(√nϕ(αn)− ϕ(α0)

||v0n||0

)2

+ oPZ∞ (1) =(√nZn

)2+ oPZ∞ (1) ⇒ χ2

1.

(2) If cn−1/2||v0n||0 ≤ εn ≤ Cδn||v0n||0 for finite constants c, C > 0, and αn ∈ Nosn wpa1, then:

||v0n||20||v0n||20

= 1 + oPZ∞ (1).

Theorem 4.2(1) recommends to construct confidence sets for ϕ(α) by inverting the optimally

weighted SQLR statistic:r ∈ R : QLRn(r) ≡ n

(inf

α∈Ak(n):ϕ(α)=rQn(α)− Qn(αn)

)≤ cχ2

1(τ)

,

23

where cχ21(τ) is the (1−τ)-quantile of the χ2

1 distribution. This result extends that of Chen and Pouzo

(2009) to allow for irregular functionals.

When αn is the optimally weighted PSMD estimator of α0, Theorem 4.2(2) suggests ||v0n||20defined in (4.4) as an alternative consistent variance estimator for ϕ(αn). Compared to Theorem

4.1, this alternative variance estimator ||v0n||20 allows for a non-smooth residual function ρ(Z,α),

but is only valid for an optimally weighted PSMD estimator. Theorem 4.2(2) extends the result of

Murphy and der Vaart (2000) on consistent variance estimation for their profile likelihood estimator

of the specific regular functional λ′θ to our semi-nonparametric conditional moment framework

(1.3), allowing for possibly irregular functionals.

5 Inference Based on Bootstrap

The inference procedures described in Section 4 are based on the asymptotic critical values. For

many parametric models it is known that bootstrap based procedures could approximate finite

sample distributions more accurately. In this section we establish the consistency of both the

nonparametric and the weighted bootstrap sieve Wald and SQLR statistics under virtually the

same conditions as those imposed for the original-sample sieve Wald and SQLR statistics.

Throughout this section, we assume that the original sample Zn ≡ Zini=1 is i.i.d. in this

section. A bootstrap procedure is described by an array of positive “weights” ωi,nni=1 for each n

(we omit the n subscript hereafter), where each bootstrap sample is drawn independently of the

original data Zini=1. Different bootstrap procedures correspond to different choices of the weights

ωi,nni=1. For the time being we assume that limn→∞ V ar(ωi,n) = σ2ω ∈ (0,∞) for all i.

To be more precise, we introduce some definitions for the new random variables and the enlarged

probability spaces. Let Ω = ωi,n : i = 1, ..., n; n = 1, ... be the space of weights, defined as a

triangle array with elements in R+, the corresponding σ-algebra and probability are (BΩ, PΩ). Let

V∞ ≡ Z∞ × Ω, B∞ ≡ B∞Z × BΩ be the σ-algebra, and let PV ∞ be the joint probability over V∞.

Finally, for each n, let Bn be the σ-algebra generated by V n ≡ Zn × (ω1,n, ..., ωn,n).

Let An be a random variable that is measurable with respect to Bn. Let LV ∞|Z∞(An|Zn) be

the conditional law of random variable An given Zn. Let L(B) denote the law of a random variable

B. For two real valued random variables, An (measurable with respect to Bn) and B (measurable

with respect to some σ-algebra BB), we say

∣∣LV ∞|Z∞(An|Zn)− L(B)∣∣ = oPZ∞ (1)

24

if for any δ > 0, there exists a N(δ) such that

PZ∞

(sup

f∈BL1

|E[f(An)|Zn]− E[f(B)]| ≤ δ

)≥ 1− δ for all n ≥ N(δ),

which is equivalent to say

supf∈BL1

|E[f(An)|Zn]− E[f(B)]| = oPZ∞ (1),

where BL1 denotes the class of uniformly bounded Lipschitz functions f : R → R such that

||f ||L∞ ≤ 1 and |f(z)− f(z′)| ≤ |z − z′|; see Van der Vaart and Wellner (1996) (henceforth, VdV-

W) ch. 1.12 for more details.

We say ∆n is of order oPV ∞|Z∞ (1) in PZ∞ probability, and denote it as ∆n = oPV ∞|Z∞ (1) wpa1(PZ∞),

if for any ϵ > 0,

PZ∞(PV ∞|Z∞ (|∆n| > ϵ | Zn) > ϵ

)→ 0, as n→ ∞.

We say ∆n is of orderOPV ∞|Z∞ (1) in PZ∞ probability, and denote it as ∆n = OPV ∞|Z∞ (1) wpa1(PZ∞),

if for any ϵ > 0 there exists a M ∈ (0,∞), such that

PZ∞(PV ∞|Z∞ (|∆n| > M | Zn) > ϵ

)→ 0, as n→ ∞.

In this section we first establish the consistency of various bootstrap statistics under a high level

LQA condition for general bootstrap procedures. We then provide low level sufficient conditions for

two widely used bootstrap procedures: the weighted bootstrap and the nonparametric bootstrap

and, which are described below:

Assumption Boot.1 (Weighted Bootstrap). Let (ωi)ni=1 be a sequence such that ωi ∈ R+, ωi ∼

iidPω, E[ω] = 1, V ar(ω) = σ2ω, and∫∞0

√P (|ω − 1| ≥ t)dt <∞.

Assumption Boot.2 (Nonparametric Bootstrap). Let (ωin)ni=1 be a triangular array of random

variables such that (ω1n, ..., ωnn) ∼Multinomial(n;n−1, ..., n−1).

Henceforth we omit the n subscript from the weight series. We note that under Assumption

Boot.2, E[ω1] = 1, V ar(ω1) = (1 − 1/n) → 1 ≡ σ2ω and Cov(ωi, ωj) = −n−1 (for i = j). Finally,

n−1max1≤i≤n(ωi − 1)2 = oPω(1) (see Van der Vaart and Wellner (1996) p. 458 ). We use these

facts in the proofs.

25

5.1 Consistency and convergence rate of the bootstrap PSMD estimators

Let V n ≡ Zn × (ω1,n, ..., ωn,n), where ωi,nni=1 is a sample of positive random variables that are

independent of the original-sample Zn, and each ωi,n acts as a “weight” of Zi. Let

ρB(Vi, α) ≡ ωi,nρ(Zi, α),

be the bootstrap residual function. Let mB(x, α) be a bootstrap version of m(x, α), that is, mB(x, α)

is computed in the same way as that of m(x, α) except that we use ρB(Vi, α) instead of ρ(Zi, α).

For example, if m(x, α) is a series LS estimator (2.4) of m(x, α), then mB(x, α) is a bootstrap series

LS estimator of m(x, α), defined as:

mB(x, α) ≡ pJn(x)′(P ′P )−n∑

j=1

pJn(Xj)ρB(Vj , α). (5.1)

Let QBn (α) ≡ 1

n

∑ni=1 m

B(Xi, α)′Σ(Xi)

−1mB(Xi, α) be a bootstrap version of Qn(α). We denote

the bootstrap PSMD estimator as αBn , i.e., α

Bn is the approximate minimizer of

QB

n (α) + λnPen(h)

on Ak(n).

In this subsection we establish the consistency and the convergence rate of the bootstrap PSMD

estimator αBn under virtually the same conditions as those imposed for the consistency and the

convergence rate of the original-sample PSMD estimator αn.

The next assumption is needed to control the difference of the bootstrap criterion function

QBn (α) and the original-sample criterion function Qn(α); it is analogous to Assumption 2.3 for the

original sample. Let δ∗m,n∞n=1 be a sequence of real valued positive numbers such that δ∗m,n = o(1)

and δ∗m,n ≥ δm,n. Let c

∗0 and c∗ be finite positive constants.

Assumption 5.1 (Bootstrap sample criterion). (i) QBn (Πnα0) ≤ c∗0Qn(Πnα0) + OPV ∞|Z∞ (ηn)

wpa1(PZ∞); (ii) QBn (α) ≥ c∗Qn(α)−OPV ∞|Z∞ ((δ

∗m,n)

2) uniformly over AM0

k(n) wpa1(PZ∞).

Lemma 5.1. Let Assumptions 2.1 - 2.3 and 5.1 hold. Then: (1)

||αBn − α0||s = oPV ∞|Z∞ (1) wpa1(PZ∞) and Pen

(hBn

)= OPV ∞|Z∞ (1) wpa1(PZ∞).

(2) In addition, let Assumption 2.4 hold and QBn (α) ≥ c∗Qn(α) − OPV ∞|Z∞ (δ2m,n) uniformly over

Aosn wpa1(PZ∞). If maxδ2m,n, Q(Πnα0), λn, ηn = δ2m,n, then:

||αBn − α0|| = OPV ∞|Z∞ (δm,n) wpa1(PZ∞);

||αBn − α0||s = OPV ∞|Z∞ (||Πnα0 − α0||s + τn × δm,n) wpa1(PZ∞).

26

Lemma 5.1(2) shows that αBn ∈ Nosn wpa1. Again, when mB(x, α) is the bootstrap series

LS estimator (5.1) of m(x, α), under virtually the same low level sufficient conditions as those in

Chen and Pouzo (2012a) and Chen and Pouzo (2009) for their original-sample PSMD estimator

αBn , one can verify Assumption 5.1 and QB

n (α) ≥ c∗Qn(α) − OPV ∞|Z∞ (δ2m,n) uniformly over Aosn

wpa1(PZ∞). This verification is amounts to follow the proof of Lemma C.2 of Chen and Pouzo

(2012a) except that the original-sample series LS estimator m(x, α) is replaced by its bootstrap

version mB(x, α).

Remark 5.1. Theorem B of Chen et al. (2003) establish the consistency of nonparametric bootstrap

for a general class of semiparametric two step GMM estimators θgmm of root-n estimable Euclidean

parameter θ0:∣∣∣LV ∞|Z∞

(√n(θBgmm − θgmm

)| Zn

)− L

(√n(θgmm − θ0

))∣∣∣ = oPZ∞ (1).

Their theorem is proved under a high level assumption that the first step nonparametric bootstrap

estimator hBn of unknown function h0 satisfies ||hBn − hn|| = oPV ∞|Z∞

(n−1/4

)wpa1(PZ∞). Our

Lemmas 2.2 and 5.1 together imply that ||hBn − hn|| = OPV ∞|Z∞ (δm,n) wpa1(PZ∞). Since δm,n ≍δn = o(n−1/4) under mild smoothness condition on h0 (see, e.g., Chen and Pouzo (2012a)), our

Lemma 5.1 immediately verifies their convergence rate assumption.

5.2 Bootstrap local quadratic approximation (LQAB)

For any tn ∈ Tn, we let ΛBn (α(tn), α) ≡ 0.5QB

n (α(tn)) − QBn (α) with α(tn) ≡ α + tnu

∗n. For any

sequence of non-negative weights (bi)i, let

Zbn ≡ n−1

n∑i=1

bi

(dm(Xi, α0)

dα[u∗n]

)′Σ(Xi)

−1ρ(Zi, α0) = n−1n∑

i=1

biS∗n,i

∥v∗n∥sd.

The next assumption is a bootstrap version of the LQA Assumption 3.2.

Assumption 5.2 (LQAB). (i) For all (α, t) ∈ Nosn × Tn, α(t) ∈ Ak(n), and with rn(tn) =(maxt2n, tnn−1/2, o(n−1)

)−1,

sup(α,tn)∈Nosn×Tn

rn(tn)

∣∣∣∣ΛBn (α(tn), α)− tn Zω

n + ⟨u∗n, α− α0⟩ −Bω

n

2t2n

∣∣∣∣ = oPV ∞|Z∞ (1) wpa1(PZ∞)

where Bωn is a V n measurable positive random variable such that Bω

n = OPV ∞|Z∞ (1) wpa1(PZ∞);

(ii)

∣∣∣∣LV ∞|Z∞

(√nZω−1n

σω| Zn

)− L (Z)

∣∣∣∣ = oPZ∞ (1),

27

where Z is a standard normal random variable.

Assumption 5.2(i) implicitly imposes restrictions on the bootstrap estimator mB(x, α) of the

conditional mean function m(x, α). Below we provide low level sufficient conditions for Assumption

5.2(i) when mB(x, α) is a bootstrap series LS estimator.

Denote g(X,u∗n) ≡ dm(X,α0)dα [u∗n]′Σ(X)−1. Then: E [g(Xi, u

∗n)Σ(Xi)g(Xi, u

∗n)

′] = ||u∗n||2 by

definition.

Assumption B. For Γ(·) ∈ Σ(·),Σ0(·),∣∣∣∣∣n−1n∑

i=1

g(Xi, u∗n)Γ(Xi)g(Xi, u

∗n)

′ −E[g(Xi, u

∗n)Γ(Xi)g(Xi, u

∗n)

′]∣∣∣∣∣ = oPZ∞ (1).

Lemma 5.2. Let Zini=1 be i.i.d., mB(·, α) be the bootstrap series LS estimator (5.1), and con-

ditions of Lemmas 3.1 and 5.1 hold. Let either Assumption Boot.1 or Assumption Boot.2 hold.

Then:

(1) Assumption 5.2(i) holds with Bωn = Bn.

(2) If Assumption B holds, then∣∣Bω

n − ||u∗n||2∣∣ = oPV ∞|Z∞ (1) wpa1(PZ∞) and

∣∣Bn − ||u∗n||2∣∣ =

oPZ∞ (1).

Lemmas 3.1 and 5.2(1) indicate that the low level Assumption A in Appendix A is sufficient

for both the original-sample LQA Assumption 3.2(i) and the bootstrap LQA Assumption 5.2(i).

Assumption 5.2(ii) can be easily verified by applying some central limit theorems. For example,

if the weights are independent (Assumption Boot.1), we can use Lindeberg-Feller CLT; if the weights

are multinomial (Assumption Boot.2) we can apply Hayek CLT (see Van der Vaart and Wellner

(1996) p. 458 ). The next lemma provides some simple sufficient conditions for Assumption 5.2(ii).

Lemma 5.3. Let Zini=1 be i.i.d. and either Assumption Boot.1 or Assumption Boot.2 hold. If

there is a positive real sequence (bn)n such that bn = o (√n) and

lim supn→∞

E

[(g(X,u∗n)ρ(Z,α0))

2 1

(g(X,u∗n)ρ(Z,α0))

2

bn> 1

]= 0. (5.2)

Then: Assumptions 5.2(ii) and 3.2(ii) hold.

5.3 Bootstrap sieve Wald statistic

The following result is a bootstrap version of Theorem 4.1(2).

28

Theorem 5.1. Let αn be the PSMD estimator (2.2) and αBn the bootstrap PSMD estimator. Let

conditions for Lemmas 2.2 and 5.1 hold. If Assumptions 3.1, 3.2(i), 4.1 and 5.2(i) hold, then:

(1)√nϕ(αB

n )− ϕ(αn)

σω||v∗n||n,sd= −

√nZω−1n

σω+ oPV ∞|Z∞ (1) wpa1(PZ∞);

(2) If further Assumptions 3.2(ii) and 5.2(ii) hold, then

∣∣∣∣LV ∞|Z∞

(√nϕ(αB

n )− ϕ(αn)

σω||v∗n||n,sd| Zn

)− L

(√nϕ(αn)− ϕ(α0)

||v∗n||n,sd

)∣∣∣∣ = oPZ∞ (1).

(3) If ϕ(α0) is regular, without imposing Assumption 4.1, we have:

∣∣∣∣LV ∞|Z∞

(√nϕ(αB

n )− ϕ(αn)

σω| Zn

)− L

(√n (ϕ(αn)− ϕ(α0))

)∣∣∣∣ = oPZ∞ (1).

For a regular functional, Theorem 5.1(3) provides one way to construct its confidence sets

without the need to compute any variance estimator. Unfortunately for an irregular functional, we

need to compute a sieve variance estimator ||v∗n||n,sd to apply Theorem 5.1(2). In Appendix A we

establish the consistency of a semiparametric score bootstrap, which does not require to recompute

PSMD estimators using the bootstrap sample and hence is computationally simple.

5.4 Bootstrap SQLR statistic

If Σ = Σ0, the SQLR statistic is no longer asymptotically chi-square; Theorem 3.2, however,

implies that the SQLR statistic converges weakly to a tight limit. In this subsection we show that

the asymptotic distribution of the SQLR can be consistently approximated by those of both the

nonparametric and the weighted bootstrap SQLR statistics.

Recall that

QLRB

n (ϕn) ≡ n

(inf

Ak(n) : ϕ(α)=ϕnQB

n (α)− QBn (α

Bn )

),

with ϕn ≡ ϕ(αn). Denote αR,Bn as the restricted bootstrap PSMD estimator, i.e., the approximate

minimizer of QBn (α) + λnPen(h) on α ∈ Ak(n) : ϕ(α) = ϕn. By Lemma 5.1(2) and Remark 3.3

we can show that αR,Bn ∈ Nosn wpa1 under the null hypothesis of ϕ(α0) = ϕ0.

Then: QLRB

n (ϕn) = n(QB

n (αR,Bn )− QB

n (αBn )).

Theorem 5.2. Let αn be the PSMD estimator (2.2) and αBn be the bootstrap PSMD estimator.

Let conditions for Lemmas 2.2 and 5.1 hold. Let Assumptions 3.1, 3.2(i) and 5.2(i) hold with

29

∣∣Bωn − ||u∗n||2

∣∣ = oPV ∞|Z∞ (1) wpa1(PZ∞). Then: under the null hypothesis of ϕ(α0) = ϕ0,

(1)QLR

B

n (ϕn)

σ2ω=

(√n

Zω−1n

σω||u∗n||

)2

+ oPV ∞|Z∞ (1) wpa1(PZ∞);

(2) If further Assumptions 3.2(ii) and 5.2(ii) hold, then∣∣∣∣∣∣LV ∞|Z∞

QLRB

n (ϕn)

σ2ω| Zn

− L(QLRn(ϕ0)

)∣∣∣∣∣∣ = oPZ∞ (1).

Theorem 5.2 extends the results in Theorems 3.2 and 4.2(1) to the bootstrap SQLR statistic.

It allows us to construct valid confidence sets (CS) for ϕ(α0) based on inverting possibly non-

optimally weighted SQLR statistic without the need to compute a variance estimator. See, e.g.,

Andrews and Buchinsky (2000) for a thorough discussion about how to construct CS via bootstrap.

For regular functionals of parametric time series models, Andrews (2002) and Camponovo (2012)

establish the second order refinement of nonparametric bootstrap Wald and optimally weighted

QLR statistics respectively. In this paper, we recommend the use of bootstrap (possibly non-

optimally weighted) SQLR to construct CS for ϕ(α0) when it is difficult to compute any consistent

variance estimator for ϕ(α), such as in the cases when the residual function ρ(Z;α) is pointwise

non-smooth in α0.

6 Simulation Studies and An Empirical Illustration

In this section, we present two small simulation studies and an empirical illustration of the PSMD

estimation and SQLR based confidence sets for the NPQIV regression E[1Y1 ≤ h0(Y2)−γ|X] = 0.

We use the series LS estimator (2.4) of m(X,h) = E[1Y1 ≤ h(Y2) − γ|X] in the computations.

6.1 Simulation Studies

We run Monte Carlo (MC) studies to assess the finite sample performance of our proposed inference

procedures via a NPQIV model (2.8): Y1 = h0(Y2) + U , Pr(U ≤ 0|X) = γ. We consider two MC

designs to ensure that the results are not sensitive to the specific simulation designs.

MC Design 1

Previously, Chen and Pouzo (2012a) and Chen and Pouzo (2009) designed MC studies to re-

spectively investigate the finite sample performance of the PSMD estimator of h0(·) in a NPQIV

model E[1Y1 ≤ h0(Y2) − γ|X] = 0 and the root-n asymptotic normality of the PSMD estimator

of θ0 in a partially linear quantile IV model E[1Y1 ≤ h0(Y2) + θ′0Y3 − γ|X] = 0. Their MC

designs were drawn from the British Family Expenditure Survey (FES) Engel curve data set that

30

Thanks!

!'"$ !'"# !'"! !'"& ' '"& '"! '"#

!'"%

'

'"%

6789,:-;<=>

)789,:-;<=>

88!?@ABC7!?EG

Figure 6.1: MC Design 1: QQ-Plot for ∇h(µ2) (appropriately centered and scaled), n = 250.

was first used in Blundell et al. (2007). Their simulation studies indicate remarkable finite sam-

ple performances of the PSMD estimator even for a difficult nonlinear, severely ill-posed inverse

problem.

In the first MC design, we generate 1000 i.i.d. samples of n = 250 and 500 observations

from a NPQIV model: Y1 = h0(Y2) +√0.0025U , where U = −Φ−1

(E[h0(Y2)|X]−h0(Y2)

25 + γ)+ V ,

V ∼ N(0, 1), and (Y2, X) ∼ N(µ,Σ), where µ2, µX and σ22, σ2X are set to be the sample estimates

of the means and variances of Y2, X from the “no-kids” subsample of British FES Engel curve data

set of Blundell et al. (2007), and the correlation (in Σ) between Y2 and X is set to be ρ = 0.75.

Finally, h0(y2) = Φ(y2−µ2

σ2

). The parameter of interest is: ϕ(h0) = ∇h0(µ2).

We present the results for γ = 0.5. We estimate h0(·) via the PSMD procedure, using a

polynomial spline (P-spline) sieve Hk(n) with k(n) = 6, Pen(h) = ||∇2h||2L2 with λn = 0.0001, and

pJn(X) is a P-Spline basis with Jn = 15. Figure 6.1 presents a QQ-plot for ϕ(αn) = ∇h(µ2) to verifyour asymptotic normality result. By inspecting this figure, the asymptotic normal approximation

seems to be accurate even for a small sample size of n = 250. The QQ-plot corresponds to the

larger sample size n = 500 is better so we omit it.

Table 6.1 reports the MC bias and standard deviation of the plug-in PSMD estimator ϕ(αn) =

∇h(µ2) for both n = 250 and n = 500. The bias is an order of magnitude lower, reflecting the need

to “undersmooth” since ∇h0(µ2) is an irregular functional parameter.

Bias Std. Dev.

n = 250 0.066 0.236n = 500 0.057 0.133

Table 6.1: MC Design 1: MC bias and standard deviation of the PSMD estimator for ∇h0(µ2).

MC Design 2

The second simulation design is based on the NPIV model MC design of Newey and Powell

31

NS \ Different PSMD (I) (II) (III) (IV)

1% 1.1% 0.5% 1.1% 1.3%5% 4.0% 4.2% 3.6% 5.3%10% 10.8% 11.0% 8.5% 11.8%

Table 6.2: MC Design 2: Size of the SQLR test of ϕ(h0) = 0.

(2003) and Santos (2012), except that we consider the NPQIV model (2.8). Specifically, we generate

450 i.i.d. samples of n = 750 observations from the NPQIV model (2.8): Y2 = 2 sin(πY1) + 0.76U ,

U = 2(Φ(U∗)− γ), Y1 = 2(Φ(Y ∗1 /3)− 0.5) and X = 2(Φ(X∗/3)− 0.5), where

Y ∗1

X∗

U∗

∼ N

0,

1 0.8 0.5

0.8 1 0

0.5 0 1

,

and finally h0(Y1) = 2 sin(πY1). The parameter of interest is: ϕ(h0) = h0(0).

We estimate h0(·) via the PSMD procedure, using a polynomial spline (P-spline) sieve Hk(n)

with k(n) ∈ 3, 4, 6, Pen(h) = ||h||L2 + ||∇h||L2 with λn ∈ 0.0001, 0.0002, 0.002, and pJn(X) is

a Hermite polynomial basis with Jn ∈ 4, 6, 7. We also considered other bases such as B-splines

and results remained essentially the same.

Table 6.1 reports the simulated size of the SQLR test of H0 : ϕ(h0) = 0 as a function of the

nominal size (NS), for different specifications of the tuning parameters. Column (I) corresponds to

k(n) = 4, Jn = 6 and λn = 0.0002; Column (II) corresponds to k(n) = 3, Jn = 4 and λn = 0.0001;

Column (III) corresponds to k(n) = 6, Jn = 7 and λn = 0.0002; Column (IV) corresponds to

k(n) = 6, Jn = 7 and λn = 0.002. We see that for all cases, the simulated size is close to the NS.

We also compute the rejection probabilities of the null hypothesis as a function of r ∈ 2/√n, 4/

√n,

where r : ϕ(0) = r; these are respectively 33% and 88% corresponding to Column (I). We note that

since our functional ϕ(h) = h(0) is estimated at a slower than root-n rate, the deviations considered

for r are indeed “small”.

We study the finite sample behavior of the nonparametric bootstrap SQLR corresponding to

Column (I). We employ 450 nonparametric bootstrap evaluations, and 150 MC repetitions. We

reduce the latter from 450 to 150 to save computation time. For nominal sizes of 10%, 5% and 1%

we obtained a simulated p-value of 13%, 4% and 2% respectively. We expect that the performance

will be much improved if we increase number of bootstrap runs.

32

6.2 An Empirical Application

We compute SQLR based confidence bands for nonparametric quantile IV Engel curves based on

the British FES data set:

E[1Y1,i ≤ h0(Y2,i) | Xi] = 0.5,

where Y1,i is the budget share of the i−th household on a particular non-durable goods, say food-in

consumption; Y2,i is the log-total expenditure of the household, which is endogenous, and hence we

use Xi, the gross earnings of the head of the household, to instrument it. We work with the “no

kids” sub-sample of the data set of Blundell et al. (2007), which consists of n = 628 observations.

See Blundell et al. (2007) for details about the data set.

We estimate h0(·) via the optimally weighted PSMD procedure with Σ = Σ0 = 0.25, using a

polynomial spline (P-spline) sieve Hk(n) with k(n) = 4, Pen(h) = ||h||L2 + ||∇h||L2 with λn =

0.0005, and pJn(X) is a Hermite polynomial basis with Jn = 6. We also considered other bases

such as P-splines and results remained essentially the same.

We use the fact that the optimally weighted SQLR of testing ϕ(h) = h(y2) (for any fixed y2) is

asymptotically χ21 to construct pointwise confidence bands. That is, for each y2 in the sample we

construct a grid of points for the SQLR test; each of these points where the value of SQLR test

corresponding to h(y2) = ri for (ri)30i=1. We then, take the smallest interval that included all points

ri that yield a corresponding value of the SQLR test below the 95% percentile of χ21.8 Figure 6.2

presents the results, where the solid blue line is the point estimate and the red dashed lines are

the 95% pointwise confidence bands. We can see that the confidence bands get wider towards the

extremes of the sample, but are tight enough to reject the hypothesis that the food-in Engel curve

is upward sloping or even constant.

7 Conclusion

In this paper, we provide unified asymptotic theories for estimation and inference on possibly

irregular parameters ϕ(α0) of the general semi-nonparametric conditional moment restrictions

E[ρ(Z;α0)|X] = 0. Under regularity conditions that allow for weakly dependent data and any

consistent nonparametric estimator of the conditional mean function m(X,α) ≡ E[ρ(Z;α)|X], we

establish the asymptotic normality of the plug-in PSMD estimator ϕ(αn) of ϕ(α0), as well as the

asymptotically tight distribution of a possibly non-optimally weighted SQLR statistic under the

8The grid (ri)ni=1 was constructed to have r15 = hn(y2), for all i ≤ 15 ri+1 ≤ ri ≤ r15 decreasing in steps of length

0.002 (approx) and for all i ≥ 15 ri+1 ≥ ri ≥ r15 increasing in steps of length 0.008 (approx); finally, the extremes,r1 and r30, were chosen so the SQLR test at those points was above the 95% percentile of χ2

1. We tried differentlengths and step sizes and the results remain qualitatively unchanged. For some observations, which only accountfor less than 4% of the sample, the confidence interval was degenerate at a point; this result is due to numericalapproximation issues, and thus were excluded from the reported results.

33

5 5.5 6 6.5Y

2

Figure 6.2: PSMD Estimate of the NPQIV food-in Engel curve (blue solid line), with the 95%pointwise confidence bands (red dash lines).

34

null hypothesis of ϕ(α0) = ϕ0. As a simple yet useful by-product, we immediately obtain that an

optimally weighted SQLR statistic is asymptotically chi-square distributed under the null hypoth-

esis. For (pointwise) smooth residuals ρ(Z;α) (in α), we propose a simple consistent sieve variance

estimator for ϕ(αn). Under i.i.d. data and conditions that are virtually the same as those for

the limiting distributions of the original-sample (possibly non-optimally weighted) SQLR statistic,

we establish the consistency of both the nonparametric and the weighted bootstrap (possibly non-

optimally weighted) SQLR statistics. These results are valid regardless of whether ϕ(α0) is regular

or not and whether ρ(Z;α) is pointwise smooth (in α) or not; and they allow applied researchers to

construct confidence sets for ϕ(α0) without computing any consistent estimator of the asymptotic

variance of ϕ(αn). Monte Carlo studies and an empirical illustration of a nonparametric quan-

tile IV regression demonstrate the good finite sample performance of our SQLR based inference

procedures.

In this paper we assume that the semi-nonparametric conditional moment restrictions E[ρ(Z;α0)|X] =

0 uniquely identifies the unknown true parameter value α0 ≡ (θ′0, h0), and conduct inference that is

robust to whether or not the semiparametric efficiency bound of ϕ(α0) is singular. Recently, Santos

(2012) considered Bierens’ type of test of the NPIV model E[Y2−h0(Y1)|X] = 0 without assuming

point identification of h0(·). In Chen et al. (2011) we are currently extending the SQLR inference

procedure to allow for partial identification of the general model E[ρ(Z;α0)|X] = 0.

References

Ai, C. and X. Chen (2003). Efficient estimation of models with conditional moment restrictionscontaining unknown functions. Econometrica 71, 1795–1843.

Ai, C. and X. Chen (2007). Estimation of possibly misspecified semiparametric conditional momentrestriction models with different conditioning variables. Journal of Econometrics 141 (1), 5–43.

Ai, C. and X. Chen (2009). Semiparametric efficiency bound for models of sequential momentrestrictions containing unknown functions. Cowles Foundation Discussion Paper No. 1731 .

Andrews, D. (2002). Higher-order improvements of a computationally attractive k-step bootstrapfor extremum estimators. Econometrica 70, 119–162.

Andrews, D. and M. Buchinsky (2000). A three-step method for choosing the number of bootstraprepetition. Econometrica 68, 23–51.

Bajari, P., H. Hong, and D. Nekipelov (2011). Game theory and econometrics: A survey of somerecent research. Working paper, Stanford and UC Berkeley .

Billingsley, P. (1995). Probability and Measure (3rd ed.). Wiley.

Blundell, R., X. Chen, and D. Kristensen (2007). Semi-nonparametric iv estimation of shapeinvariant engel curves. Econometrica 75, 1613–1669.

35

Bontempts, C., D. Martimort, and A. Thomas (2012). Estimating optimal contracts under dele-gated management, application to the french water industry. Working paper, Toulouse School ofEconomics.

Camponovo, L. (2012). Nonparametric bootstrap for quasi-likelihood ratio tests. Working paper,University of Lugano.

Carrasco, M., J. Florens, and E. Renault (2007). Linear inverse problems in structural econometricsestimation based on spectral decomposition and regularization. The Handbook of Econometrics,J.J. Heckman and E.E. Leamer (eds.), North-Holland, Amsterdam 6B.

Chamberlain, G. (1992). Efficiency bounds for semiparametric regression. Econometrica 60 (3),567–592.

Chen, X. (2007). Large sample sieve estimation of semi-nonparametric models. The Handbook ofEconometrics, J.J. Heckman and E.E. Leamer (eds.), North-Holland, Amsterdam 6B.

Chen, X., V. Chernozhukov, S. Lee, and W. Newey (2011). Local identification of nonparametricand semiparametric models. Cowles Foundation Discussion Paper No. 1795. Yale University .

Chen, X., Z. Liao, and X. Sun (2012). Sieve inference on semi-nonparametric time series models.Cowles Foundation Discussion Paper No 1849. Yale University .

Chen, X., O. Linton, and I. van Keilegom (2003). Estimation of semiparametric models with thecriterion functions is not smooth. Econometrica 71, 1591–1608.

Chen, X. and S. Ludvigson (2009). Land of addicts? an empirical investigation of habit-based assetpricing models. Journal of Applied Econometrics 24, 1057–1093.

Chen, X. and D. Pouzo (2009). Efficient estimation of semiparametric conditional moment modelswith possibly nonsmooth residuals. Journal of Econometrics 152, 46–60.

Chen, X. and D. Pouzo (2012a). Estimation of nonparametric conditional moment models withpossibly nonsmooth generalized residuals. Econometrica 80, 277–321.

Chen, X. and D. Pouzo (2012b). Supplement to ‘estimation of nonparametric conditional momentmodels with possibly nonsmooth generalized residuals’. Econometrica Supplementary Material .

Chen, X., D. Pouzo, and E. Tamer (2011). Sieve qlr inference on partially identified semi-nonparametric conditional moment models. Working paper, Yale, UC Berkeley and Northwest-ern.

Chernozhukov, V. and C. Hansen (2005). An iv model of quantile treatment effects. Economet-rica 73, 245–61.

Chernozhukov, V., G. Imbens, and W. Newey (2007). Instrumental variable estimation of nonsep-arable models. Journal of Econometrics 139, 4–14.

Darolles, S., Y. Fan, J. Florens, and E. Renault (2011). Nonparametric instrumental regression.Econometrica 79, 1541–1566.

Fan, J., C. Zhang, and J. Zhang (2001). Generalized likelihood ratio statistics and wilks phe-nomenon. The Annals of Statistics 29, 640–652.

36

Feller, W. (1970). An Introduction to Probability Theory and its Applications (3rd ed.), Volume I.Wiley.

Gagliardini, P. and O. Scaillet (2011). Nonparametric instrumental variable estimation of quantilestructural effects. Forthcoming in Econometrica.

Gallant, R. and G. Tauchen (1989). Semiparametric estimation of conditional constrained het-erogenous processes: Asset pricing applications. Econometrica 57, 1091–1120.

Graham, B. and J. Powell (2012). Identification and estimation of ”irregular” correlated randomcoefficient models. Econometrica, forthcoming .

Hall, P. and J. Horowitz (2005). Nonparametric methods for inference in the presence of instru-mental variables. The Annals of Statistics 33, 2904–2929.

Hansen, L. and S. Richard (1987). The role of conditioning information in deducing testablerestrictions implied by dynamic asset pricing models. Econometrica 55, 587–613.

Hong, H., A. Mahajan, and D. Nekipelov (2010). Extremum estimation and numerical derivatives.Mimeo, Dept. of Economics, UC at Berkeley .

Horowitz, J. (2007). Asymptotic normality of a nonparametric instrumental variables estimator.International Economic Review 48, 1329–1349.

Horowitz, J. (2011). Applied nonparametric instrumental variables estimation. Econometrica 79,347–394.

Horowitz, J. and S. Lee (2007). Nonparametric instrumental variables estimation of quantile re-gression model. Econometrica 75, 1191–1209.

Horowitz, J. and S. Lee (2011). Uniform confidence bands for functions estimated nonparametricallywith instrumental variables. Journal of Econometrics, forthcoming .

Huang, J. (1998). Projection estimation in multiple regression with application to functional anovamodels. The Annals of Statistics 27, 242–272.

Liao, Y. and W. Jiang (2012). Posterior consistency of nonparametric conditional moment restrictedmodels. Forthcoming in Annals of Statistics.

Murphy, S. and A. V. der Vaart (2000). On profile likelihood. Journal of American StatisticalAssociation 95, 449–465.

Newey, W. (1997). Convergence rates and asymptotic normality for series estimators. Journal ofEconometrics 79, 147–168.

Newey, W. and J. Powell (2003). Instrumental variables estimation for nonparametric models.Econometrica 71, 1565–1578.

Otsu, T. (2011). Empirical likelihood estimation of conditional moment restriction models withunknown functions. Econometric Theory 27, 8–46.

Preastgaard, J. (1991). General-Weights Bootstrap of the Empirical Process. Ph. D. thesis, Uni-versity of Washington.

37

Santos, A. (2012). Inference in nonparametric instrumental variables with partial identification.Econometrica 80, 213–275.

Shen, X. and J. Shi (2005). Sieve likelihood ratio inference on general parameter space. Science inChina 48, 67–78.

Van der Vaart, A. and J. Wellner (1996). Weak Convergence and Empirical Processes. Springer.

38

A Sufficient Conditions and Additional Results

In Appendix A we first provide some low level sufficient conditions for the high level LQA as-sumption 3.2(i) and the bootstrap LQA assumption 5.2(i). We then state useful lemmas whenthe conditional mean function m(·, α) is estimated by series LS estimators. We finally present aconsistency theorem for a computationally attractive sieve score bootstrap. See Appendix C forthe proofs of all the results in this Appendix.

Throughout Appendix A, we maintain the assumptions that the original-sample Zi = (Y ′i , X

′i)′ni=1

is i.i.d., A ≡ Θ ×H, Θ is a compact subset of Rdθ , H ⊆ H, H is a separable Banach space undera norm ∥·∥H. m(x, α) is the series LS estimator (2.4) and mB(x, α) is the bootstrap series LSestimator (5.1) of m(x, α).

A.1 Sufficient conditions for LQA(i) and Bootstrap LQA(i)

Assumption A.1. (i) X is a compact connected subset of Rdx with Lipschitz continuous boundary,and fX is bounded and bounded away from zero over X ; (ii) The smallest and largest eigenvaluesof E[pJn(X)pJn(X)′] are bounded and bounded away from zero for all Jn; (iii) supx∈X |pj(x)| ≤const. < ∞ for all j = 1, ..., Jn; Either J

2n = o(n) or Jn log(Jn) = o(n) for pJn(X) a polynomial

spline sieve; (iv) There is pJn(X)′π such that supx |g(x) − pJn(x)′π| = O(bm,Jn) = o(1) uniformlyin g ∈ m(·, α) : α ∈ AM0

k(n).

Let Oon ≡ ρ(·, α)− ρ(·, α0) : α ∈ Nosn. Denote

1 ≤√Cn ≡

∫ 1

0

√1 + log(N[](w(Mnδs,n)κ,Oon, || · ||L2(fZ)))dw <∞.

Assumption A.2. (i) There is a sequence ρn(Z)n of measurable functions such that supAM0k(n)

|ρ(Z,α)| ≤

ρn(Z) a.s.-Z and E[|ρn(Z)|2|X] ≤ const. < ∞; (ii) there exist some κ ∈ (0, 1] and K : X → Rmeasurable with E[|K(X)|2] ≤ const. such that ∀δ > 0,

E

[sup

α∈N0sn : ||α−α′||s≤δ

∥∥ρ(z, α)− ρ(z, α′)∥∥2e|X

]≤ K(X)2δ2κ ∀α′ ∈ Nosn ∪ α0,

and max(Mnδn)

2, (Mnδs,n)2κ= (Mnδs,n)

2κ; (iii) nδ2n(Mnδs,n)κ√Cnmax

(Mnδs,n)

κ√Cn,Mn

=

o(1); (iv) supX ||Σ(x)−Σ(x)||×(Mnδn) = oPZ∞ (n−1/2); δn ≍√

Jnn = max

√Jnn , bm,Jn = o(n−1/4).

Let m(X,α) ≡ pJn(X)′(P ′P )−∑n

i=1 pJn(Xi)m(Xi, α) be the LS projection of m(X,α) onto

pJn(X), and let g(X,u∗n) ≡ dm(X,α0)dα [u∗n]′Σ(X)−1 and g(X,u∗n) be its LS projection onto pJn(X).

Assumption A.3. (i) EPZ∞

[∥∥∥dm(X,α0)dα [u∗n]−

dm(X,α0)dα [u∗n]

∥∥∥2e

](Mnδn)

2 = o(n−1);

(ii) EPZ∞

[∥g(X,u∗n)− g(X,u∗n)∥

2e

](Mnδn)

2 = o(n−1); (iii) m(·, α) : α ∈ Nosn and g(·, u∗n)m(·, α) :α ∈ Nosn are L2(fX)-Donsker; (iv) E[||g(X,u∗n)m(X,α)−m(X,α0)||2e] = o(1) for all α ∈ Ak(n)

such that ||α− α0||s = o(1).

Assumption A.4. (i) m(X,α) is twice continuously pathwise differentiable in α ∈ Nos, a.s.-X;

(ii) E

[sup

α∈Nosn

∥∥∥∥dm(X,α)

dα[u∗n]−

dm(X,α0)

dα[u∗n]

∥∥∥∥2e

]× (Mnδn)

2 = o(n−1);

39

(iii) E

[supα∈Nosn

∥∥∥d2m(X,α)dα2 [u∗n, u

∗n]∥∥∥2e

]× (Mnδn)

2 = o(1); (iv) Uniformly over α1 ∈ Nos and α2 ∈

Nosn,

E

[g(X,u∗n)

(dm(X,α1)

dα[α2 − α0]−

dm(X,α0)

dα[α2 − α0]

)]= o(n−1/2).

Assumption A.2 is comparable to that imposed in Chen and Pouzo (2009) for a non-smoothresidual function ρ(Z,α). This assumption ensures that the sample criterion function Qn is wellapproximated by a “smooth” version of it. Assumptions A.3 and A.4 are similar to those imposedin Ai and Chen (2003), Ai and Chen (2007) and Chen and Pouzo (2009), except that we use thescaled sieve Riesz representer u∗n ≡ v∗n/ ∥v∗n∥sd. This is because we allow for possibly irregularfunctionals (i.e., possibly ∥v∗n∥ → ∞), while the above mentioned papers only consider regularfunctionals (i.e., ∥v∗n∥ → ∥v∗∥ <∞). We refer readers to these papers for detailed discussions andverifications of these assumptions in examples of the general model (1.3).

A.2 Lemmas for Series LS estimation of m(x, α)

The next lemma (Lemma A.1) extends Lemma C.3 of Chen and Pouzo (2012a) and Lemma A.1 ofChen and Pouzo (2009) to the bootstrap case. Denote

ℓn(x, α) ≡ m(x, α) + m(x, α0) and ℓBn (x, α) ≡ m(x, α) + mB(x, α0).

Lemma A.1. Let mB(·, α) be the bootstrap series LS estimator (5.1). Let Assumptions 2.1(iv),2.4, 4.1(iii), A.1, A.2(i)(ii), and Boot.1 or Boot.2 hold. Then: (1) For all δ > 0, there is aM(δ) > 0 such that for all M ≥M(δ),

PZ∞

(PV ∞|Z∞

(sup

α∈Nosn

τnn

n∑i=1

∥∥mB(Xi, α)− ℓBn (Xi, α)∥∥2e≥M | Zn

)≥ δ

)< δ

eventually, with τ−1n ≡ (δn)

2 (Mnδs,n)2κCn.

(2) For all δ > 0, there is a M(δ) > 0 such that for all M ≥M(δ),

PZ∞

(PV ∞|Z∞

(sup

α∈Nosn

τ ′nn

n∑i=1

∥∥ℓBn (Xi, α)∥∥2e≥M | Zn

)≥ δ

)< δ

eventually, with

(τ ′n)−1 = maxJn

n, b2m,Jn , (Mnδn)

2 = const.× (Mnδn)2.

(3) Let Assumption A.2(iii) hold. For all δ > 0, there is N(δ) such that, for all n ≥ N(δ),

PZ∞

(PV ∞|Z∞

(supNosn

snn

∣∣∣∣∣n∑

i=1

∥∥mB (Xi, α)∥∥2Σ−1 −

n∑i=1

∥∥ℓBn (Xi, α)∥∥2Σ−1

∣∣∣∣∣ ≥ δ | Zn

)≥ δ

)< δ

withs−1n ≤ (δn)

2(Mnδs,n)κ√Cnmax

(Mnδs,n)

κ√Cn,Mn

Ln = o(n−1),

where Ln∞n=1 is a slowly divergent sequence of positive real numbers (such a choice of Ln existsunder assumption A.2(iii)).

40

Recall that

Zωn =

1

n

n∑i=1

ωi,n

(dm(Xi, α0)

dα[u∗n]

)′Σ(Xi)

−1ρ(Zi, α0) =1

n

n∑i=1

g(Xi, u∗n)ωi,nρ(Zi, α0).

Lemma A.2. Let all the conditions for Lemma A.1(2) hold. If Assumptions A.2(iv), A.3 andA.4(i)(ii)(iv) hold, then: for all δ > 0, there is a N(δ) such that for all n ≥ N(δ),

PZ∞

(PV ∞|Z∞

(supNosn

√n

∣∣∣∣∣ 1nn∑

i=1

(dm(Xi, α)

dα[u∗n]

)′

(Σ(Xi))−1ℓBn (Xi, α)− Zω

n + ⟨u∗n, α− α0⟩

∣∣∣∣∣ ≥ δ | Zn

)≥ δ

)< δ.

Lemma A.3. Let all the conditions for Lemma A.1(2) hold. If Assumption A.4(i)(iii) holds, then:for all δ > 0, there is a N(δ) such that for all n ≥ N(δ),

PZ∞

(PV ∞|Z∞

(supNosn

n−1n∑

i=1

(d2m(Xi, α)

dα2[u∗n, u

∗n]

)′(Σ(Xi))

−1ℓBn (Xi, α) ≥ δ | Zn

)≥ δ

)< δ.

Lemma A.4. Let Assumptions 2.1(iv), 2.4(i), 4.1(iii), A.1, A.3(i), A.4(ii) hold. Then: (1) Forall δ > 0 there is a M(δ) > 0, such that for all M ≥M(δ),

PZ∞

(supNosn

1

n

n∑i=1

(dm(Xi, α)

dα[u∗n]

)′Σ−1(Xi)

(dm(Xi, α)

dα[u∗n]

)≥M

)< δ

eventually.(2) If in addition, Assumption B holds, then: For all δ > 0, there is a N(δ) such that for all

n ≥ N(δ),

PZ∞

(supNosn

∣∣∣∣∣ 1nn∑

i=1

(dm(Xi, α)

dα[u∗n]

)′

Σ−1(Xi)

(dm(Xi, α)

dα[u∗n]

)− ||u∗n||2

∣∣∣∣∣ ≥ δ

)< δ.

A.3 Sieve score bootstrap

In the main text we present the consistency of bootstrap SQLR statistic and bootstrap sieve Waldstatistic. Here we consider a sieve score bootstrap, which does not require to recompute PSMDestimators of α0 using the bootstrap sample and hence is computationally attractive.

Recall that αRn is the original-sample restricted PSMD estimator (3.17). Let v∗n be computed

in the same way as that in Subsection 4.1, except that we use αRn instead of αn. Denote

Tn ≡ 1√n

n∑i=1

(dm(Xi, α

Rn )

dα[v∗n/||v∗n||n,sd]

)′

Σ−1(Xi)m(Xi, αRn )

and

TBn ≡ 1√

n

n∑i=1

(dm(Xi, α

Rn )

dα[v∗n/||v∗n||n,sd]

)′

Σ−1(Xi)mB(Xi, αRn )− m(Xi, α

Rn ).

For series LS estimators ofm(x, α), we have: mB(x, αRn )−m(x, αR

n ) = pJn(x)′(P ′P )−∑n

j=1 pJn(Xj)(ωj−

1)ρ(Zj , αRn ). This is not crucial, but it simplifies both the proof and the implementation.

Let ϵn∞n=1 and ζn∞n=1 be real valued positive sequences such that ϵn = o(1) and ζn = o(1).

41

Assumption C. (i) maxϵn, n−1/4Mnδn = o(n−1/2)

supNosn

supu∈Vn : ||u||=1

n−1n∑

i=1

∥∥∥∥dm(Xi, α)

dα[u]− dm(Xi, α)

dα[u]

∥∥∥∥2e

= OPZ∞ (maxn−1/2, ϵ2n);

(ii) there is a continuous mapping Υ : such that maxΥ(ζn), n−1/4Mnδn = o(n−1/2) and

supNosn

supVn : ||u∗

n−u||≤ζn

n−1n∑

i=1

∥∥∥∥dm(Xi, α)

dα[u∗n]−

dm(Xi, α)

dα[u]

∥∥∥∥2e

= OPZ∞ (maxn−1/2, (Υ(ζn))2);

(iii) ||u∗n − u∗n|| = OPZ∞ (ζn) where u∗n ≡ v∗n/||v∗n||sd.

Assumption C(i) can be obtained by similar conditions to those imposed in Ai and Chen (2003).

Assumption C(ii) can be established by

∥∥∥dm(·,α)dα [u]− dm(·,α)

dα [u∗n]∥∥∥2e: α ∈ Nosn; u ∈ Vn : ||u− u∗n|| ≤ ζn

being a Donsker class and E

[∥∥∥dm(X,α)dα [u∗n]−

dm(X,α)dα [u]

∥∥∥2e

]= o(1) for all ||u∗n − u|| < ζn. However,

it can be obtained by weaker conditions, yielding a (Υ(ζn))2 that is slower than O(n−1/2) provided

that Υ(ζn)Mnδn = o(n−1/2). In the proof we show that ||u∗n − u∗n|| = oPZ∞ (1); faster rates ofconvergence will relax the conditions needed to show part (ii).

Theorem A.1. Let αRn be the restricted PSMD estimator (3.17) and conditions for Lemmas 2.2

and 5.1 hold. Let Assumptions 3.1, A, Boot.1 or Boot.2, 4.1, C, 3.2(ii) and 5.2(ii) hold and thatnδ2n (Mnδs,n)

2κCn = o(1). Then, under the null hypothesis of ϕ(α0) = ϕ0,

|LV ∞|Z∞(σ−1ω TB

n | Zn)− L(Tn)| = oPZ∞ (1).

Appendices B and C are available upon request.

42