OPTIMAL MODEL AVERAGING ESTIMATION FOR ......OPTIMAL MODEL AVERAGING ESTIMATION FOR PARTIALLY LINEAR...

OPTIMAL MODEL AVERAGING ESTIMATION FOR

PARTIALLY LINEAR MODELS

Xinyu Zhang1 and Wendun Wang2

1Academy of Mathematics and Systems Science, Chinese Academy of Sciences

2Econometric Institute, Erasmus University Rotterdam, andTinbergen Institute

Abstract: This article studies optimal model averaging for partiallylinear models with het-

eroscedasticity. A Mallows-type criterion is proposed to choose the weight. The resulting model

averaging estimator is proved to be asymptotically optimalunder some regularity conditions.

Simulation experiments show that the proposed model averaging method is superior to other com-

monly used model selection and averaging methods. The proposed procedure is further applied

to study Japan’s sovereign credit default swap spreads.

Key words and phrases:Asymptotic optimality, Heteroscedasticity, Model averaging, Partially

linear model

1. Introduction

Linear regression models have been predominantly popular in a variety of applications,

including biology, economics, psychology, and machine learning. One important reason may

be its simplicity and the clear interpretation of the estimation results. However, an increasing

number of studies have noted that the relationship between the response variable and covari-

ates is not always linear. To list a few examples, Barro (1996) found that democracy can in-

fluence economic development in a nonlinear pattern. Henderson et al. (2012) and Su & Lu

(2013) found a nonlinear effect of initial state on the economic growth rate. Liang et al.

OPTIMAL MODEL AVERAGING ESTIMATION FOR PARTIALLY LINEAR MODELS

(2007), in a study on the effectiveness of antiretroviral medicines, showed that the HIV vi-

ral load depends nonlinearly on treatment time. Ignoring nonlinearity can result in incorrect

estimates and inferences, further resulting in misleadingexplanations and bad decisions. For

example, ignoring the nonlinear effect of global stock markets on the local market may lead

to a lack of awareness of financial contagion; Simply estimating a linear relationship between

inflation and economic growth may lead to inappropriate inflation-targeting policies.

To avoid potential ignorance of nonlinearity, partially linear models (PLMs) have re-

ceived extensive attention in theoretical and applied statistics due to their flexible specifica-

tion. It allows for both linear and nonparametric relationsbetween covariates and the re-

sponse variable. This type of specification is also frequently used when the primary interest

is in the linear component, whereas the relation between themean response and additional

covariates is not easily parameterized. The superiority ofthe partially linear model over the

standard linear models is that it does not require the parametric assumption for all covariates

and allows us to capture potential nonlinear effects. This model is sometimes preferred over

the fully nonparametric models since it preserves the advantages of linear models, e.g., an

easy interpretation of the linear covariates, and suffering less from the dimensionality curse.

PLMs are used in a wide range of applications in the literature; see, for example, Engle et al.

(1986) for an economic application and Liang et al. (2007) for a medical application.

Various methods have been proposed to estimate PLMs, for example, smoothing splines

(Engle et al., 1986; Heckman, 1986), kernel smoothing (Robinson, 1988; Speckman, 1988),

local polynomial estimation (Hamilton & Truong, 1997), andpenalized splines (Ruppert et al.,

2003). See Hardle et al. (2000) for a comprehensive survey.These estimation methods are

all based on the assumption that a correctly specified model is given. In practice, however,


researchers are ignorant of the true model. One needs to decide which covariates are in the

model (covariate uncertainty), and further whether to assign a covariate in the linear or non-

parametric component given that it is in the model (structure uncertainty). The specification

of covariates and the model structure is of fundamental importance as it greatly influences

the estimation and prediction results. These two types of uncertainty are generally referred

to as model uncertainty.

Typical methods to address model uncertainty involve testing and/or selecting the best

model using data-driven approaches. The most popular method may be to use an information

criterion (IC), such as the Akaike information criterion (AIC) or Bayesian information crite-

rion (BIC). To decide which variables to include in the PLMs,Ni et al. (2009), Bunea (2004),

and Xie & Huang (2009), among others, have proposed several variable selection methods.

To further determine the structure of the model (which covariates to include in the (non)linear

function), a commonly used method is to test the linear null hypotheses against nonlinear al-

ternatives for each covariate. Such tests, however, often have low power when the number

of covariates is large (Zhang et al., 2011). In addition, these testing and selection methods

perform model selection and estimation in two separate steps. Thus the uncertainty in the

model selection procedure is ignored in the estimation step, making it difficult to study the

properties of the final estimator (Danilov & Magnus, 2004; Magnus et al., 2016). Zhang et al.

(2011) provided a model selection approach based on smoothing spline ANOVA to automat-

ically and consistently distinguish linear and nonlinear component. This method is useful

if the goal is to identify the correct model structure, but ifthe research purpose is to esti-

mate the parameters or to make predictions, it seems more plausible to take into account all

(potentially) useful models. However, the model selectionapproaches can be rather “risky”


since they require “putting all our inferential eggs in one unevenly woven basket” (Longford,

2005).

In this paper we follow a different approach. Instead of selecting one model, we address

model uncertainty by appropriately averaging the estimates from different models. As an

alternative to model selection, model averaging can substantially reduce risk (Hansen, 2014).

It is an integrated process that accounts for both the model uncertainty and estimation un-

certainty. Model averaging has long been a popular approachwithin the Bayesian paradigm;

see, for example, Hoeting et al. (1999) for a comprehensive review. In recent years, optimal

model averaging methods have been actively developed, for instance, Mallows model aver-

aging (Hansen, 2007), OPT method (Liang et al., 2011), jackknife model averaging (JMA)

(Hansen & Racine, 2012), heteroskedasticity-robust modelaveraging (Liu & Okui, 2013),

optimal averaging method for linear mixed-effects models (Zhang et al., 2014), and optimal

averaging quantile estimators Lu & Su (2015). These methodsare asymptotically optimal in

the sense that they minimize the predictive squared error inthe large sample case, but they

mainly focus on the linear models. To the best of our knowledge, there are no optimal model

averaging estimators for PLMs. The main purpose of this paper is to fill this gap.

Our model averaging approach can simultaneously incorporate the covariate and struc-

ture uncertainty in PLMs, which is not much studied in the PLMliterature. Heteroscedastic

random errors are also allowed. To show the optimality of ourmethod, we first assume that

the covariance matrix of errors is known, and propose a Mallows-type weight choice crite-

rion, which is an unbiased estimator of the expected predictive squared error up to a constant.

We prove that the weights obtained by minimizing this criterion are asymptotically optimal

under some regularity conditions. Next, we replace the unknown covariance matrix with its


estimated counterpart, and show that the plugged-in criterion still leads to asymptotically

optimal weights.

One may naturally formulate this study as an extension of linear regression model av-

eraging. However, we emphasize that such an extension is by no means straightforward and

routine because the existing methods, such as Mallows modelaveraging, typically do not

involve kernel smoothing. To the best of our knowledge, our work is the first to study the op-

timal averaging that involves kernels. One of our main technical contributions is to provide

an optimal weight choice in a kernel smoothing framework.

Our work is also related to Xu et al. (2014), which consideredfrequentist model averag-

ing and post-model-selection inference in an additive partially linear model. Under the local

misspecification setup, their averaging estimator is consistent butmay not beoptimal. We

differ from this study by relaxing the local misspecification assumption, thereby allowing all

candidate models to be possibly misspecified, and we study the optimal averaging estimator.

Moreover, they focus on parameter estimation, while we are interested in prediction. Another

related work is Zhao et al. (2016), which modeled massive heterogeneous data in a partially

linear framework. To estimate the commonality parameter, they proposed to average com-

monality estimators obtained from heterogeneous sub-populations. While the averaging idea

is similar, our candidate estimators are obtained from the same sample but different models,

whereas theirs are from the same model but different sub-populations.

We compare the proposed model averaging estimator with popular model selection and

averaging estimators for PLMs. Our simulation study considers two cases. In the first case,

only the linear component is uncertain, and the candidate models differ in their inclusion

of linear variables. In addition to linear component uncertainty, the second case considers


the situation where there is also uncertainty in choosing which covariates to include in the

(non)linear function. In both cases, the proposed estimator performs best in most of the

cases, especially whenR2 is moderate and low. Only whenR2 is particularly high, our model

averaging estimator is not as good as information-criterion-based methods in the second case.

We also apply our method to study Japan’s sovereign credit default swap spreads. We find

that allowing for nonlinearity indeed provides several newinsights. For example, the effect of

the global stock market performance on the local market is strengthened in the volatile period,

suggesting the existence of financial contagion. The out-of-sample prediction exercise further

illustrates the advantage of partially linear models over the linear models, and we generally

find better prediction performance for our estimator compared to other partially linear model

estimators.

The remainder of this paper is organized as follows. Section2 introduces our model

averaging estimator and presents its asymptotic optimality. Section 3 investigates the finite

sample performance of the proposed estimator. A real data example is studied in Section 4

and Section 5 provides some concluding remarks. Technical proofs, additional simulation

results and additional tables and figure for the real data example can be found in our online

supplement.

2. Model averaging estimation

2.1. Model and estimators

We consider the partially linear model (PLM)

yi =

∞∑

j=1

xijβj + g(Zi) + ǫi, i = 1, . . . , n, (2.1)

whereXi = (xi1, xi2, . . .) is a countably infinite random vector,Zi = (zi1, . . . , ziq)T is a


random vector in some bounded domainD ⊂ Rq, g(·) is an unknown function fromRp toR1,

andǫ1, . . . , ǫn are heteroscedastic random errors with E(ǫi|Xi,Zi) = 0 and E(ǫ2i |Xi,Zi) =

σ2i . We denote the expectation of the response variable asµi = E(yi|Xi,Zi) =

∑∞j=1 xijβj+

g(Zi).

Our goal is to estimateµi, which is of particular use for prediction, and this is also the

typical goal in the optimal model averaging literature (e.g., Hansen, 2007; Lu and Su, 2015).

For this purpose, we useSn candidate PLMs to approximate (2.1), whereSn is allowed to

diverge to infinity asn → ∞. Thesth approximation (or candidate) PLM is

yi = XT(s),iβ(s) + g(s)(Z(s),i) + b(s),i + ǫi, i = 1 . . . , n (2.2)

whereX(s),i is a vector in the linear component,Z(s),i is a vector in the nonparametric

component,gs(·) is an unknown function fromRqs to R1, andb(s),i = µi − XT(s),iβ(s) −

g(s)(Z(s),i) represents the approximation error in thesth model. Here, the linear compo-

nentX(s),i is allowed to contain variables inZi, and reversely the nonparametric covariate

Z(s),i could contain variables inXi. Hence, (2.2) permits two sources of uncertainty: the

uncertainty of which variables to include in the model and the uncertainty of whether a co-

variate should be in the linear or nonparametric component given that it is in the model, i.e.,

the variables in the two components may mutually exchange. See, for example, the second

case in Section 3. LetX(s) = (X(s),1, . . . ,X(s),n)T, Z(s) = (Z(s),1, . . . ,Z(s),n)

T, g(s) =

g(Z(s),1), . . . , g(Z(s),n)T, ǫ = (ǫ1, . . . , ǫn)

T, y = (y1, . . . , yn)T, andµ = (µ1, . . . , µn)

T.

Remark 1. Since estimating the coefficients of the linear component and the non-parametric

component isnot the purpose of this paper, we do not need the conditions for consistency or

asymptotic normality of the coefficient estimates, for example, the conditions in Section 1.3


of Hardle et al. (2000).

To provide an optimal weighting scheme, we first need to estimate each candidate model.

We follow Speckman (1988) and use kernel smoothing estimation. One of the advantages

of this method is its light computational burden, which is crucial in our case since the

number of candidate models is typically substantial. To define Speckman’s (1988) estima-

tor, let k(·) be a kernel function,hs be a bandwidth, andkhs(·) = k(·/hs)/hs. Further-

more, denoteK(s) = K(s),ij as ann × n smoother matrix withK(s),ij = khs(Z(s),i −

Z(s),j)/∑n

j∗=1 khs(Z(s),i − Z(s),j∗). The kernel smoothing estimator ofβ(s) andg(s) can

then be obtained byβ(s) = (XT(s)X(s))

−1XT(s)(In −K(s))y andg(s) = K(s)(y−X(s)β(s)),

whereX(s) = (In − K(s))X(s) andIn is ann × n identity matrix. The estimator ofµ

is then µ(s) = X(s)β(s) + g(s) = X(s)(XT(s)X(s))

−1XT(s)(In − K(s))y + K(s)y. Let-

ting P(s) = X(s)(XT(s)X(s))

−1XT(s) andP(s) = P(s)(In − K(s)) + K(s), we can write

µ(s) = P(s)y. Note that because of the curse of dimensionality,qs (the dimension ofZ(s))

cannot be large.

With the estimators of each model readily available, we can obtain the model averaging

estimator ofµ by µ(w) =∑Sn

s=1wsµ(s) = P(w)y, wherew = (w1, . . . , wSn)T is the

weight vector belonging to the setW = w ∈ [0, 1]Sn :∑Sn

s=1ws = 1 andP(w) =

∑Sn

s=1wsP(s).

Remark 2. We point out that although heteroscedasticity is allowed inthe data generating

process (2.1), we do not immediately take it into account when estimating each candidate

model (using kernel smoothing). Instead, we incorporate heteroscedasticity when estimating

the unknown variance-covariance matrix (for the weight estimation). This is a typical treat-

ment in the literature on model averaging under heteroscedasticity, such as Hansen & Racine


(2012), Liu & Okui (2013), and Zhang et al. (2015). The main reason is that an estimator

that incorporates heteroscedasticity for each candidate model is not necessarily more effi-

cient than an estimator that fails to do so, and the latter is computationally much simpler and

faster.

2.2. Weight choice criterion and asymptotic optimality

Define the predictive squared lossLn(w) = ‖µ(w)− µ‖2 and expected loss

Rn(w) = ELn(w) = ‖P(w)µ − µ‖2 + traceP(w)ΩPT(w), (2.3)

whereΩ = diag(σ21 , . . . , σ

2n). To select the optimal weights in the sense of minimizingLn,

we propose to minimize the following Mallows-type criterion

Cn(w) = ‖µ(w)− y‖2 + 2traceP(w)Ω, (2.4)

as we can show thatRn(w) = ECn(w) − trace(Ω), where trace(Ω) is unrelated tow.

Therefore, if we knowΩ, the weights can be obtained as

w = argminw∈WCn(w). (2.5)

Averaging using this weight choice is called Mallows averaging of partially linear models

(MAPLM). The optimality of such a weight choice holds under some regularity conditions.

Defineξn = infw∈W Rn(w) andwos as a weight vector with thesth element taking on the

value of unity and other elements zeros (model selection weight). Letmaxi

indicate maxi-

mization overi ∈ 1, . . . , n, where all limiting properties here and throughout the textare

undern → ∞.

Condition 1 maxi

∑nj=1 |K(s),ij| = O(1) and max

j

∑ni=1 |K(s),ij | = O(1) uniformly for

s ∈ 1, . . . , Sn, almost surely.


Condition 2 For some integerG ≥ 1,maxi

E(ǫ4Gi |Xi,Zi) < ∞ andSnξ−2Gn

∑Sn

s=1Rn(wos)

G →

0 almost surely.

Condition 1 is the same as assumption (i) of Speckman (1988),which bounds the kernel.

Condition 2 requiresξn → ∞, i.e., there is no finite approximating model whose bias is zero

(Hansen & Racine, 2012 and Liu & Okui, 2013). This condition also constrains the rates of

Sn andRn(wos) going to the infinity, and is widely used in other model averaging studies;

see, for example, Wan et al. (2010), Liu & Okui (2013), and Ando & Li (2014).

Theorem1 Under Conditions 1-2, we have that asn → ∞,

Ln(w)/ infw∈W

Ln(w) → 1 in probability. (2.6)

Theorem 1 shows that the model averaging procedure usingw is asymptotically optimal in

the sense that the resulting squared loss is asymptoticallyidentical to that of the infeasible

best possible model averaging estimator. The proof of Theorem 1 (see the online supple-

ment) takes advantage of several inequalities involving kernels, and it provides a technical

innovation for studying the optimal model averaging in a kernel smoothing framework.

So far we have assumed that the covariance matrixΩ is known. This is not the case in

practice, and the criterion (2.4) is therefore computationally infeasible. To obtain a feasible

criterion, we estimateΩ based on the residues from the largest model indexed bys∗ =

argmaxs∈1,...,Sn(ps + qs), that is

Ω(s∗) = diag(ǫ2s∗,1, . . . , ǫ2s∗,n), (2.7)

where(ǫs∗,1, . . . , ǫs∗,n)T = y − µ(s∗) = y −P(s∗)y. The idea of using the largest model to

estimate the variance parameter or covariance matrix is also advocated by Hansen (2007) and


Liu & Okui (2013). We distinguish between two cases here. First, if the candidate models

have the same nonparametric component but only differ in theinclusion of linear covariates,

the largest model is unambiguously the one with all linear covariates included. In the more

general case with uncertainty in both linear and nonparametric components, the model with

the largest dimension is not uniquely defined since the models with the same dimension can

differ in the structure of linear and nonparametric components. Therefore, we propose to use

the the largestlinear model to estimateΩ in this case. Although the largest linear model

is nested in the largest nonlinear model, including a large number of covariates in the non-

linear component leads to a highly inaccurate estimate of this component due to the curse

of dimensionality. The inaccurate estimate further results in a poor estimator of error vari-

ance. Moreover, in most applications, the dimension of the nonlinear component is typically

low; see, for example, Yatchew and No (2001) and Liang (2006). Hence, the estimated error

variance obtained from the largest linear model is a good approximate in practice. Never-

theless, we point out that when the total number of covariates is particularly small, such that

the largest nonlinear model is of low dimension, it might make more sense to use the largest

nonlinear model to estimate the error variance.

By replacingΩ with its estimatorΩ, the feasible criterion becomes

Cn(w) = ‖µ(w)− y‖2 + 2traceP(w)Ω(s∗), (2.8)

and the weights can be obtained by

w = arg minw∈W

Cn(w). (2.9)

Let H = (µ(1) − y, . . . , µ(Sn) − y) andb = trace(P(1)Ω(s∗)), . . . , trace(P(Sn)Ω(s∗))T.

We can rewriteCn(w) asCn(w) = wTHTHw + 2wTb, which is a quadratic function of


w, and the optimization can be done by standard software packages, such as quadprog in

Matlab, which are generally effective and efficient even whenSn is large.

We now show that the weights obtained by minimizing the feasible criterion (2.8) are

still asymptotically optimal. Denoteρ(s)ii as theith diagonal element ofP(s). Let maxs

(mins

)

represent maximization(minimization) overs ∈ 1, . . . , Sn, p = maxs

ps, andh = mins

hs.

Assume the following conditions hold almost surely.

Condition 3 ‖µ‖2 = O(n).

Condition 4 trace(K(s)) = O(h−1) uniformly fors ∈ 1, . . . , Sn.

Condition 5 There exists a constantc such that|ρ(s)ii | ≤ cn−1|trace(P(s))| for all s ∈

1, . . . , Sn.

Condition 6 n−1h−2 = O(1) andn−1p2 = O(1).

Condition 3 concerns the sum ofn elements ofµ and is commonly used in linear re-

gression models; see, for example, Wan et al. (2010) and Liang et al. (2011). Condition 4

is a natural extension of Condition (h) of Speckman (1988). Condition 5 is commonly used

to ensure the asymptotic optimality of cross-validation; see, for example, Andrews (1991)

and Hansen & Racine (2012). The first part of Condition 6 regards the bandwidth and is less

restrictive than then−1h−2 = o(1) required in Theorem 2 of Speckman (1988). The sec-

ond part of Condition 6, which is the same as Condition (12) ofWan et al. (2010), allows

ps’s to increase asn → ∞, but restricts their increasing rates. Further explanations of these

conditions are provided in the online supplement.


Theorem2 Under Conditions 1-6, we have that asn → ∞,

Ln(w)/ infw∈W

Ln(w) → 1 in probability. (2.10)

The proof of Theorem 2 is provided in the online supplement.

Remark 3. The question of how to choose the optimal bandwidthhs in each candidate model

remains. While this question is of interest, it is especially difficult in our case because each

candidate model is just an approximation of the true model and therefore includes approx-

imation error. In our numerical examples, the bandwidthhs is chosen by minimizing the

generalized cross-validation criterion. As an alternative, we also consider bandwidth selec-

tion using cross-validation, a popular criterion in the presence of heteroscedasticity. The

simulation results show that the two criteria lead to almostidentical relative performance of

their competing methods, but cross-validation is computationally much more expensive than

generalized cross-validation.

Remark 4. Theorem 2 holds no matterΩ is estimated by the largest partially linear model (in

the case with only linear component uncertainty) or the largest linear model (in the case with

structure uncertainty), as long as the number of covariatesis fixed. An alternative strategy to

estimateΩ is based on theaveragedresidualsǫ(w) = ǫ1(w), . . . , ǫn(w)T = y − µ(w).

The motivation of this strategy is to avoid placing too much confidence in a single model.

The use of the averaged residuals does not affect the validity of Theorem 2 and produces

similar numerical results. Detailed results of this alternative estimation strategy and proofs

of this remark are available upon request.

3. Simulation study

3.1. Data generation process


Our setting is similar to the infinite-order regression by Hansen (2007) except that we

have a nonlinear function in addition to the linear component. Specifically, we generate the

data byyi = µi + ǫi =∑500

j=1 βjxij + g(Zi) + ǫi, whereXi = xi1, . . . , xi500T is drawn

from a multivariate normal distribution with mean 0 and covariance0.5|j1−j2| betweenxij1

andxij2 . The corresponding coefficients are set asβj = 1/j. For simplicity, we consider

a nonlinear function of twocorrelatedvariables, i.e.,g(Zi) = g(zi1, zi2), and we generate

zi1 = 0.3u1+0.7u2 andzi2 = 0.7u1+0.3u2 whereu1 andu2 are independent and uniformly

distributed. Two variants of nonlinear functions are studied: g1(Zi) = exp(zi1) + z2i2 and

g2(Zi) = 2(zi1 − 0.5)3+sin(zi2). The errors are normally distributed and heteroscedastic as

ǫi ∼ N(0, η2x2i2). We change the value ofη, so thatR2 = var(µ1, . . . , µn) /var(y1, . . . , yn)

varies from0.1 to 0.9, where var(·) denotes the sample variance. Since all covariates are

correlated with each other,R2 cannot be easily written as a function ofη. We therefore

numericallycomputeR2 based on each chosenη. The sample size is set ton = 50, 100, and

200, and the results ofn = 400 are given in Section S.3 of the online supplement.

In real applications, the model is typically a simplified version of the data generating

process with a number of variables omitted, either because of ignorance or because of data

limitations. To mimic this situation, we omitzi2 and some components ofXi for every

candidate model. We consider two cases with different typesof model uncertainty. In the first

case, it is a priori which variables belong to the nonparametric component (based on existing

theory or the research question of interest), but the specification of the linear component is

uncertain. In this case, all candidate models share a commonnonparametric function ofzi1

(with zi2 being omitted), and their linear components are a subset ofxi1, . . . , xi5T (with

the remainingxij ’s being omitted). We require each candidate model to include at least one


linear covariate, leading to25 − 1 = 31 candidate models.

In the second case, there is no a priori knowledge about whichcovariates should be

chosen as parametric regressors and which belong to the nonparametric component. There-

fore, in addition to the uncertainty of which variables to include, we are also uncertain about

whether a covariate should be included in the linear or nonparametric component. As the

number of covariates increases, the number of candidate models now increases even more

dramatically than in the first case. To facilitate the computation, we assume that only four

covariates(xi1, xi2, xi3, zi1) are observed, whereas the others are omitted. In contrast tothe

first case, the candidate models here allow a subset of(xi1, xi2, xi3, zi1) in the nonparametric

function, and the remaining can be included in the linear component or not in the model at

all. Again, we require each candidate model to contain at least one linear and one nonpara-

metric covariate. This leads to(43

)(23 − 1) +

(42

)(22 − 1) +

(41

)= 50 candidate models.

More simulation designs, such as a diverging number of candidate models, data with a larger

degree of nonlinearity and autoregressive errors, are presented in the supplement. The results

are essentially the same.

3.2. Estimation and comparison

We estimate each candidate model using the quadric kernelk(v) = 15/16(1−v2)2I(|v| ≤

1), whereI(·) is an indicator function. In the first case with only linear component uncer-

tainty, the covariance matrixΩ is estimated using the largest candidate model, i.e., the par-

tially linear model containing all observable linear covariates, and in the second case it is

estimated from the largestlinear model (with all observable variables included linearly and

no nonparametric component). We mainly compare MAPLM with four alternative estima-

tion methods for PLMs including two selection methods and two averaging methods. The two


model selection methods are based on AIC and BIC, and they select the model with the small-

est information criterion, defined respectively, as AICs = log(σ2s) + 2n−1trace(P(s)) and

BICs = log(σ2s)+n−1trace(P(s)) log(n), whereσ2

s = n−1‖y−µ(s)‖2. The two model aver-

aging methods are smoothed AIC (SAIC) and smoothed BIC (SBIC) (Buckland et al., 1997).

The weight of models is constructed bywAICs = exp(−AICs/2)/

∑Ss=1 exp(−AICs/2)

andwBICs = exp(−BICs/2)/

∑Ss=1 exp(−BICs/2).

To evaluate these methods, we compute the mean squared error(MSE) of the predic-

tive variable as500−1∑500

r=1 ‖µ(r) − µ‖2, where500 is the number of replications andµ(r)

denotes the estimator ofµ in the rth replication. For convenient comparison, all MSEs are

normalized by dividing by the MSE produced by AIC model selection.

3.3. Results

We first describe some general observations from the results, and then discuss each case

in detail. In general, the model averaging methods outperform the selection methods. The su-

periority of the averaging methods is particularly obviouswhenR2 is small. AsR2 increases,

the difference between model selection and averaging decreases. The performance of the av-

eraging methods whenR2 is small and moderate is especially good because identifying the

best model is difficult in the presence of substantial noise.In that case, the model chosen by a

selection procedure can be far from ideal, which unsurprisingly leads to inaccurate estimates.

By contrast, model averaging does not rely on a single model and thus provides protection

against choosing a poor model. This observation is also in line with Yuan & Yang (2005) and

Zhang et al. (2012). WhenR2 is large, model selection is sometimes preferred because the

minimal noise in the data allows the selection criterion to choose the correct model.

Figure 1 presents the results when there is uncertainty in only the linear component


specification. Our method yields the smallest MSE in almost all cases, but the information-

criterion model averaging sometimes has a marginal advantage whenR2 is very large. Most

of the figures show that the advantage of our method increasesasR2 decreases. The good

performance of MAPLM is partly because the optimality of MAPLM does not rely on the

correct specification of candidate models. The comparison of the methods for different sam-

ple sizes shows that when we have a relatively small or moderate sample size (n = 50 and

100), MAPLM outperforms all the competing methods over the whole range ofR2. When

the sample size is relatively large (n = 200), MAPLM still dominates the other methods for

a wide range ofR2, but the difference between MAPLM and SAIC decreases. We also note

that all the methods perform almost equally well when the sample size is large andR2 is 0.9.

Further examination suggests that the methods tend to select or impose a large weight on the

same model when there is little noise in the model and the sample size is large. This similarity

can be partly explained by the fact that the bias-variance tradeoff is not so significant in this

situation, so model selection is able to choose the correct model.

Figure 2 compares the estimation results when there is structure uncertainty in addition

to uncertainty in covariate inclusion. In this case, both linear and nonparametric components

vary over the candidate models. MAPLM produces much lower MSE than its rivals in all

cases whenR2 is equal to or less than 0.7, which again demonstrates that our model averaging

approach is preferred when the model is characterized by substantial noise and identifying

the best model is difficult, as in most practical applications. The poorer performance of

MAPLM under particularly largeR2 is mainly a result of allowing for far more uncertainty

than necessary in this case, which prevents MAPLM from assigning a very large weight to

the best model. Specifically, on one hand, allowing for more uncertainty (in both the linear


and nonlinear components) than in the first case causes MAPLMto average over a larger

model space, which generates a larger number of weight parameters to estimate. On the other

hand, when the data are highly informative (with largeR2), there often exists a best model,

and IC are capable of selecting this model. By contrast, simultaneously estimating a large

number of weights clearly prevents MAPLM from assigning a very large weight to the best

model, resulting in the poorer performance of MAPLM in this case.

Figure 1: Mean square error comparison: Uncertainty only inthe linear component

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.6

0.8

1

1.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.6

0.8

1

1.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.8

1

1.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.6

0.8

1

1.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.6

0.8

1

1.2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.8

1

1.2

MAPLM SAIC SBIC BIC

Moreover, model selection and averaging using AIC and SAIC lead to largely similar


results, as do BIC versus SBIC. These results indicate that there is a dominant model that

significantly outperforms the others, and this dominant model is often the one with the most

covariates in the nonparametric component. This further suggests that IC tend to select the

most general model whenever possible, because nonparametric estimation typically fits better

than least squares estimation. However, the dominant modelis not necessarily the best in all

cases. When the data are characterized by substantial noise, a large nonparametric model

mainly fits the noise; thus, the IC-based methods perform much worse than MAPLM. When

the data are highly informative, i.e.,R2 = 0.9, the dominant model coincides with the best

model, leading to the better performance of the IC-based methods than MAPLM.

Figure 2: Mean square error comparison: Uncertainty in bothcomponents

0.2 0.4 0.6 0.80

1

2

3

0.2 0.4 0.6 0.80

1

2

3

0.2 0.4 0.6 0.80

1

2

3

0.2 0.4 0.6 0.80

1

2

3

0.2 0.4 0.6 0.80

1

2

3

0.2 0.4 0.6 0.80

1

2

3

MAPLM

SAIC

SBIC

BIC


To see how much harm can be caused by ignoring nonlinearity, we also compare our

method with linear model averaging (LMA) that considers allcandidate models to be fully

linear. Theoretically, LMA should work better when the model is linear or the degree of

nonlinearity is small since nonparametric estimation converges much slower and is gener-

ally less efficient than least squares estimation. As the degree of nonlinearity increases, the

better fit achieved by nonparametric estimation dominates its efficiency loss and slow con-

vergence; thus MAPLM should outperform LMA under these conditions. Our simulation

results (presented in the supplement) under different degrees of nonlinearity precisely con-

firm this theoretical argument. Moreover, we find that LMA slightly outperforms MAPLM

whenR2 and the sample size are small. AsR2 and the sample size increase, MAPLM quickly

demonstrates its significant superiority over LMA. Detailed simulation designs, results, and

explanations are provided in the online supplement.

4. Empirical application

We apply our method to study Japan’s sovereign credit default swap (CDS) spreads. A

CDS contract is an insurance contract against the credit event specified in the contract. Its

spread is the insurance premium that the buyer under protection has to pay, and it reflects

investors’ expectations about a country’s sovereign credit risk. The likelihood of default typ-

ically depends on the country’s willingness (rather than ability) to repay, and the government

often makes the repayment decision based on a cost-benefit analysis using the information

of the country’s macroeconomic fundamentals. Japan’s sovereign CDS spreads are of world-

wide interest since Japan has long been characterized by itshigh government debt. The ratio


of gross government debt to GDP even reached 237.9% in 2012, the highest in the world.

Furthermore, Japan is the world’s third largest economy, and its financial market plays an

important role in international finance. A crisis in Japan could damage investors’ confidence

in the government debt of many other heavily indebted industrial countries.

In this section, we first examine how macroeconomic indicators affect Japan’s CDS

spreads and then study the predictability of these indicators. We focus on the CDS contract

written on the credit event “complete restructuring”, which is the most popular credit event

insured by a sovereign CDS contract, and we consider the contract maturity of five years,

following Longstaff et al. (2011). Our potential macroeconomic determinants include three

domestic variables that reflect domestic economic performance: the domestic stock market

return (measured by the Dow Jones Japan Total Stock Market Total Return Index), its volatil-

ity, and the nominal Yen-US Dollar exchange rate. We also follow Longstaff et al. (2011)

and consider three global-market determinants: the globalstock market return (measured by

the Morgan Stanley Capital International US Total Return Index), US treasury yield (with the

constant maturity of five years), and the global default riskpremium (approximated by US

investment-grade corporate bond spreads). See Longstaff et al. (2011) and Qian et al. (2017)

for details of the variable construction. We focus on the post-earthquake sample from March

12, 2011 (one day after the Tohuku earthquake) to October 10,2012 to avoid significant

structural breaks, and the number of observations is 388. All data are first-differenced based

on a preliminary unit root analysis and then normalized. Thechange in Japan’s sovereign

CDS spreads (before normalization) is plotted in the left panel of Figure S.6 of the online

supplement, and its sample autocorrelation function is plotted in the right panel. These two

plots show that the differenced series does not exhibit strong serial correlation. Table S.1


of the online supplement provides the descriptive statistics of the first-differenced sovereign

CDS spreads and potential determinants.

4.1. Linear model specification

The existing literature on sovereign CDS spreads mostly considers linear models in

which all the determinants are assumed to have a linear effect on the spreads; see, for exam-

ple, Longstaff et al. (2011) and Dieckmann & Plank (2011). Weinitially follow this conven-

tion to estimate the effect of our six potential determinants using linear models. We consider

ordinary least squares (OLS) estimation and linear model averaging using the heteroscedastic-

robust Mallows criterion (HRCp). Linear model averaging treats all determinants linearly, but

it takes into account the uncertainty of whether a determinant is included in the model.

Table 1: Estimation results of linear models

OLS LMA OLS LMA

Domestic stock return −1.5182*** −1.2790*** Domestic stock volatility 0.6165*** 0.0576

(0.1752) (0.3189) (0.1758) (0.7208)

Foreign exchange rate −0.3250* −0.3777*** Global stock return 1.0107*** 0.9842***

(0.1727) (0.1839) (0.1733) (0.2188)

US treasury yield −0.3672** −0.3649** Global default risk premium −0.0774 −0.0230

(0.1750) (0.2349) (0.1689) (0.0948)

Notes: Standard errors are in parentheses. ***, **, and * denote significance at 1%, 5%, and 10%,

respectively. The significance of LMA is based on bootstrap confidence intervals with 200 random

samplings.

Table 1 presents the estimation results of the linear models. Since all determinants are

normalized, the size of the coefficients reflects the relative importance. We first focus on the


least squares estimation results. The least squares estimates show that the domestic stock

return, its volatility, and the global stock return are the three most important determinants

and have a significant effect on Japan’s CDS spread. More specifically, the domestic stock

return, as a measure of local economic performance, has a strongly negative effect. The do-

mestic stock return can affect the CDS spread by influencing the government’s willingness

to implement fiscal reforms, and effective fiscal reform is typically regarded as an important

tool to reduce default risk. Therefore, when the domestic economy is weak, policy makers

are less willing to implement reforms because the reforms can impose extra pressure on the

distressed economy. This failure to enact reforms thus increases the sovereign CDS spreads.

The strong and negative effect of domestic stock returns is in line with the literature (see, e.g.,

Longstaff et al., 2011 and Dieckmann & Plank, 2011). The domestic stock market volatility

is positively associated with sovereign CDS spreads, whichis in line with the economic the-

ory that higher volatility indicates a less stable economicstatus and thus a higher probability

of default. The other important determinant is the global stock return, which has a positive

effect on Japan’s sovereign CDS spread. Theoretically, theglobal stock market return may

impose two opposite impacts on sovereign CDS spreads. The negative effect is due to the fact

that good global economic performance can positively influence the Japanese government’s

willingness to repay, thus lowering the sovereign CDS spread. On the other hand, a good

global economy would also encourage investment in general,thereby increasing the CDS

spread. The overall impact of the global stock return depends on which effect is dominant.

It is likely that one effect is more prominent in some situations but dominated by the other

effect in other situations. This potential heterogeneity cannot be captured by linear models.

Less significant but still important determinants include the foreign exchange rate and


US treasury yield. The negative effect of the foreign exchange rate is expected because a low

Yen-US Dollar exchange rate reflects weakness in Japan’s current economic situation and

less external demand, which leads to higher sovereign CDS spread. The negative relationship

between US treasury yield and Japan’s CDS spread is also intuitive because a high treasury

yield signals good economic performance in the US, which canpositively influence Japan’s

economy and encourage repayment by the Japanese government.

We also compare the estimates obtained from least squares and model averaging and find

that the signs of all the estimated coefficients are the same for both methods. Nevertheless,

model averaging produces quite different estimates for some determinants, such as the do-

mestic stock return, its volatility, and the global defaultrisk premium, which suggests that

there is a large degree of model uncertainty.

4.2. Partially linear specification

Next, we examine whether the widely used linearity assumption is appropriate here. The

verification is based on both economic theory and statistical tools. First, from the theoretical

perspective, the literature of sovereign CDS spreads generally does not provide firm theory

about nonlinear effects for most covariates, except the domestic and global stock returns.

Qian et al. (2017) found that these two covariates play different roles in tranquil and turbulent

periods. Specifically, global stock returns are more prominent during turbulent periods, and

domestic stock returns are more prominent during tranquil periods. The nonlinear effect of

global stock returns is also supported by extensive literature on financial contagion, which

suggests that the link between domestic markets and the global market is often strengthened

during periods of crisis; see, e.g. Eichengreen et al. (1996) and Bae et al. (2003). Therefore,

it is reasonable to consider the potential nonlinear effectof global stock returns.


Next, we verify the linearity of each determinant by assigning it to the nonparamet-

ric component of partially linear models. We includeonedeterminant in the nonparametric

component each time while keeping others in the linear component. This process enables

us to verify whether each determinant has a nonlinear effecton Japan’s CDS spreads and

also avoids the dimensionality and computational issue of simultaneously considering many

nonparametric covariates. Figure S.7 of the online supplement presents the nonparamet-

ric estimates of each determinant using the proposed MAPLM,i.e., g(w) =∑Sn

s=1 wsg(s),

where g(s) is the nonparametric estimate obtained from each candidatemodel andw =

(w1, . . . , wSn)T is weights estimated by MAPLM. We see that the effects of domestic stock

market volatility and global default risk premium do not exhibit a clear nonlinear pattern.

They either have a relatively flat curve or fluctuate around zero, suggesting that these ef-

fects are almost linear or highly insignificant. In contrast, domestic stock returns, the foreign

exchange rate, global stock returns, and US treasury yield show different degrees of nonlin-

earity.

Finally, we formally test the linearity for each determinant using the test statistic sug-

gested by Li et al. (2010). This test statistic verifies the null hypothesis of the linearity of the

nonparametric component by the fiducial method. Specifically, Li et al. (2010) proposed to

first approximate the nonparametric component by a piecewise linear function. Thus, testing

for linearity is transformed into testing for a linear restriction on the coefficient. Thep-value

of the test is then derived by the classic fiducial method (c.f., Xu & Li (2006)). To validate

this test in our case, we implement the test in thefixedfull model, where only one determi-

nant is included in the nonparametric component each time and the remaining determinants

are included in the linear component. Therefore, no averaging is performed in this testing


procedure. Although it appears to be more general to test thelinearity of a covariate while

assigning others to the nonparametric component, it is difficult in practice because a large

number of covariates in the nonparametric leads to the the curse of dimensionality. The curse

results in poor estimates and unreliable test statistics infinite samples. Thep-values of the

tests are reported in Table 2. We see that the test fails to reject the null hypothesis of linearity

for the domestic stock return, its volatility, and global default risk premium. The reported

p-values also confirm that the effects of the foreign exchangerate and global stock returns

cannot be accurately approximated by linear functions. Thetest statistic for the US treasury

yield is not available because this variable takes only a fewdiscrete values. Thus, it is less

clear whether one can assume a linear effect of the US treasury yield.

Table 2: Linearity test for each determinant

p-value p-value

Domestic stock returns 0.1651 Domestic stock volatility 0.4810

Foreign exchange rate 0.0265 Global stock returns 0.0042

US treasury yield NA Global default risk premium 0.9548

Based on the nonparametric estimation results and the regression diagnostics, we discuss

the (potentially) nonlinear determinants and their economic implications. First, the estimated

effect of the foreign exchange rate has a steeply downward trend when the change in exchange

rates is below average, but the curve becomes relatively flatand close to zero as the change

in exchange rates increases. The negative relationship between the exchange rate and Japan’s

CDS spread is in line with the findings of the linear models. Nevertheless, the nonparametric

estimate shows that this relationship becomes much weaker when the exchange rate is high.


Second, the estimated effect of global returns is characterized by a typical “U-shape”. We

see that the change in Japan’s CDS spreads is particularly high when global returns are at

the extremes, either a large positive change or a large negative change, suggesting that the

negative effect of global stock returns plays a more prominent role in a bear market while

the positive effect is more important when the global financial market is in a bull market.

We also observe that the curve is much steeper when the globalstock market is in a slump,

suggesting that the correlation between Japan’s credit market and the global stock market is

much stronger during periods of crisis. This result is in contrast to Longstaff et al. (2011),

who reported a weak and insignificant effect of global stock returns on Japan. We argue

that the insignificance is possibly a result of ignoring nonlinearity, such that the positive and

negative effects offset each other, leading to an ambiguousoverall effect. Such a nonlinear

effect of global stock returns provides evidence of financial contagion from the global stock

market to Japan’s sovereign credit market, which cannot be captured by linear models. The

finding of financial contagion is of particular importance for both policy makers and investors

since it implies that adapted policies and investment strategies should be implemented under

different situations. Finally, the curve of the US treasuryyield is similar but less nonlinear

than that of the foreign exchange rate. We generally observea negative effect of the US

treasury yield on Japan’s sovereign CDS spreads, in line with the literature and our linear

model estimates, but the effect is relatively stronger whenthe change in treasury yield is

extreme.

Table 3 reports the estimates of the linear coefficients of the partially linear model with

foreign exchange rate and global stock returns in the nonlinear component. To compute these

estimates, letX be the matrix of the linear covariates of the full model (which contains the


domestic stock return and its volatility, US treasury yield, and global default risk premium)

andΠ(s) be a projection matrix such thatX(s) = XΠ(s). Then, the model averaging estimates

of the linear coefficients can be obtained byβ(w) =∑Sn

s=1 wsΠ(s)β(s), similar to Hansen

(2007)’s model averaging estimator. Since no standard inference theories are available for

the optimal model averaging estimates, we provide 99% bootstrap confidence intervals for

the model averaging estimates. The confidence intervals of AIC and BIC are computed with

the selected model based on Theorem 4 of Speckman (1988), ignoring the uncertainty in the

selection procedure. We see that the domestic stock return has the strongest negative associ-

ation with the change in sovereign CDS spreads, as in the linear model; however, compared

to the linear models, the estimated effect of the US treasuryyield is weak and less significant

in the PLMs.

To check whether our empirical results are sensitive to the predetermination of nonlin-

ear covariates, we perform estimation and prediction (discussed in the next section) under

different specifications of nonlinear covariates, and the results are generally quite robust.

4.3. Out-of-sample prediction

Finally, we examine the pseudo out-of-sample predicability of Japan’s CDS spreads us-

ing six alternative methods: three model averaging methods(MAPLM, SAIC, and SBIC) and

two model selection methods (AIC and BIC) for the partially linear models, and one linear

model averaging method.

The linear model averaging is based on HRCp, as above. It considers candidate models

with at least one determinant included, so it averages over26−1 candidate models. For PLM

averaging, the most general specification is to consider allpossibilities, i.e., that a determinant

can be in the linear component, in the nonlinear component, or not in the model. However,


Table 3: Estimates of the linear coefficients in the partially linear models

MAPLM SAIC SBIC AIC BIC

Domestic stock returns −1.2771 −1.5595 −1.5635 −15658 −1.5938

(−2.37,−0.62) (−2.53,−0.65) (−2.52,−0.00) (−1.83,−1.30) (−1.87,−1.32)

Domestic stock vol. 0.0000 0.5330 0.5135 0.5325 0.5928

(−0.56, 2.23) (−0.62, 2.28) (−0.68, 2.26) (0.23, 0.83) (0.28, 0.91)

US treasury yield −0.3246 −0.2441 −0.0858 −0.3292

(−0.78, 0.24) (−0.86, 0.29) (−0.88, 0.03) (−0.65,−0.01)

Global risk premium −0.1284 −0.0876 −0.0144

(−0.47, 0.05) (−0.52, 0.07) (−0.65, 0.02)

this consideration may cause a dimensionality problem by including too many determinants

in the nonlinear component. Thus, we assign determinants tothe nonlinear component only

when necessary. Based on the PLM analysis in the previous subsection, it seems reasonable

to presume a linear relationship between Japan’s CDS spreads and the global default risk pre-

mium and the domestic stock market return and its volatility. It is also clear that the foreign

exchange rate and global stock returns have a nonlinear impact on Japan’s CDS spread; thus

it is necessary to include these two determinants in the nonlinear component when they are

included in the model. As for the US treasury yield, since itseffect only exhibits a moder-

ate degree of nonlinearity and the formal linearity test is not informative, we are less certain

whether to assign this variable to the linear or nonlinear component. Allowing this ambigu-

ous determinant to enter the nonlinear component leads to a more complete model space but

may also result in the dimensionality curse. There is no apriori knowledge of how to make


Table 4: Mean square prediction error of Japan’s CDS spreads

Prediction sample MAPLM SAIC SBIC AIC BIC

Scenario I 5% 0.8608 0.9360 0.9278 0.9403 0.9253

10% 0.8490 1.0162 1.0181 1.0256 1.0190

15% 0.9708 1.0950 1.0830 1.1007 1.1106

20% 0.9927 1.0933 1.1111 1.0751 1.1107

Scenario II 5% 0.8865 0.9723 0.9264 0.9673 0.9253

10% 0.7903 0.9410 1.0175 0.9308 1.0190

15% 0.8119 0.9814 0.8542 0.96520.7770

20% 0.8697 0.9695 1.1073 0.9530 1.1107

an appropriate tradeoff between a more complete model spaceand the dimensionality curse.

Therefore, we compare the prediction performance of six methods in two scenarios. In Sce-

nario I, we allow only the foreign exchange rate and global stock return to be in the nonlinear

component. In other words, the foreign exchange rate and global stock return can either

not be included in the model or be in the nonlinear component of the model. The remain-

ing determinants are either not in the model or in the linear component. Scenario II differs

from Scenario I in that we also allow the US treasury yield to enter the nonlinear component.

Hence, there are three possibilities for the uncertain determinant of the US treasury yield:

not included in the model, included in the linear component,or included in the nonlinear

component. We split the sample into two sub-samples, one forestimation and the other for

prediction and evaluation. We consider the estimation sample varying from 80% to 95% of

the whole period; thus the prediction sample ranging from 20% to 5% correspondingly.

Table 4 presents the mean square prediction error (MSPE) of five PLM methods. All


values are normalized by dividing by the MSPE of the linear model averaging method. We

see that our MAPLM produces the lowest MSPE for all prediction samples in Scenario I.

In Scenario II, MAPLM is the best in most cases, except when the prediction sample is

15%. In all cases, MAPLM outperforms the linear Mallows averaging, demonstrating that

incorporating the necessary nonlinearity improves the prediction performance. Since the

performance of linear model averaging is invariant to the scenario, we can also compare the

predictability of MAPLM in the two scenarios. Interestingly, we observe that allowing the

US treasury yield to enter the nonlinear component improvesthe prediction performance for

all methods when the prediction sample is larger than 5%. However, when we have a small

prediction sample, a smaller model space is better. One possible explanation is that averaging

over a larger model space may offset the additional noise by better diversification. When the

prediction sample is large, the diversification gain from averaging over a larger model space is

substantial and dominates the estimation inaccuracy due tothe dimensionality curse. This is,

however, not the case when the prediction sample is small (or, equivalently, when the training

sample is large) because the predicted values obtained fromdifferent candidate models are

more accurate and more similar to each other; thus, the diversification gain is smaller.

5. Concluding remarks

Partially linear models have become popular in applied econometrics and statistics be-

cause they allow a more flexible specification compared to linear models and provide more

interpretable estimates compared to fully nonparametric models. Estimation of partially lin-

ear models is subjected to at least two types of uncertainty:the uncertainty of which variables

to include in the model and the uncertainty of whether a covariate should be in the linear or

nonlinear component given that it is in the model. Typical model testing and selection meth-


ods do not appropriately address these two types of uncertainty simultaneously, especially

when the research interest is to estimate the parameters or to make predictions. In this paper,

we propose an optimal model averaging procedure for PLMs that jointly incorporates the

two types of model uncertainty. The extension from linear model averaging to partially lin-

ear models is by no means straightforward and routine because it involves kernel smoothing,

which complicates the proof of optimality. We demonstrate the advantages of our methods by

examining the determinants of Japan’s sovereign CDS spreads. Our empirical study suggests

that there exists a large degree of nonlinearity in the effects of macroeconomic determinants,

such as the global stock return and exchange rate. Conventional linear models do not capture

such nonlinearity, and ignoring the nonlinearity can result in a lack of awareness of financial

contagion, which may further lead to inappropriate policies and investment decisions.

At least three issues deserve future research. First, the computational burden of our

method would be substantial when the number of candidate models is large; therefore, a

model screening step prior to model averaging is desirable.Second, although the dimension

ps is allowed to increase with the sample sizen, it must be smaller thann and its increasing

rate is restricted by the second part of Condition 6. How to develop an optimal model averag-

ing method for high- or ultrahigh-dimensional PLMs is an interesting open question. Finally,

if the research interest is to consistently estimate the linear and/or nonlinear component rather

than to make predictions, a consistent model averaging estimator and post-model-averaging

inference are desired. See, for example, Hjort & Claeskens (2003), Zhang & Liang (2011)

and Xu et al. (2014). In these studies, a crucial assumption of local misspecification is re-

quired, and the weights also need to have an explicit form. Bycontrast, we do not utilize

the local misspecification framework, and our weight estimates do not have an explicit form.


Therefore, the development of model averaging estimators for the linear and nonlinear com-

ponents without local misspecification and analytical weights warrants further investigation.

AcknowledgmentsThe authors are grateful to Co-Editor Zhiliang Ying, the Associate Editor

and two referees for their constructive comments, and to Dr.Na Li for providing codes for

nonlinearity test. Zhang’s research was supported by National Natural Science Foundation of

China (Grant nos. 71522004 and 11471324) and a grant from theMinistry of Education of

China (Grant no. 17YJC910011).

Online SupplementOnlineSupp.pdf describes the technical proofs and providemore expla-

nations on the conditions as well as additional simulation studies.

References

ANDO, T. & L I , K.-C. (2014). A model-averaging approach for high-dimensional regression.Journal of theAmerican Statistical Association109, 254–265.

ANDREWS, D. (1991). Asymptotic optimality of generalizedCL, cross-validation, and generalized cross-validation in regression with heteroskedastic errors.Journal of Econometrics47, 359–377.

BAE, K.-H., KAROLYI G. A. & STULZ , R. M. (1996). A new approach to measuring financial contagion. TheReview of Financial Studies16, 717–763.

BARRO, R. J. (1996). Democracy and growth.Journal of Economic Growth1, 1–27.

BUCKLAND , S. T., BURNHAM , K. P. & AUGUSTIN, N. H. (1997). Model selection: An integral part ofinference.Biometrics53, 603–618.

BUNEA, F. (2004). Consistent covariate selection and post model selection inference in semiparametric regres-sion. The Annals of Statistics32, 898–927.

DANILOV, D. & M AGNUS, J. R. (2004). On the harm that ignoring pretesting can cause. Journal of Econo-metrics122, 27–46.

DETTE, H. & M UNK , A. (1998). Validation of linear regression models.The Annals of Statistics26, 778–800.

DIECKMANN , S. & PLANK , T. (2012). Default risk of advanced economies: An empirical analysis of creditdefault swaps during the financial crisis.Review of Finance16, 903–934.

EICHENGREEN, B., ROSE, A. & W YPLOSZ, C. (1996). Contagious currency crises.Scandinavian Journal ofEconomics98, 463–484.

ENGLE, R. F., GRANGER, C. W., RICE, J. & WEISS, A. (1986). Semiparametric estimates of the relationbetween weather and electricity sales.Journal of the American Statistical Association81, 310–320.

HAMILTON , S. A. & TRUONG, Y. K. (1997). Local linear estimation in partly linear models. Journal ofMultivariate Analysis60, 1–19.

HANSEN, B. E. (2007). Least squares model averaging.Econometrica75, 1175–1189.

HANSEN, B. E. (2014). Model averaging, asymptotic risk, and regressor groups. Quantitative Economics5,495–530.


HANSEN, B. E. & RACINE, J. (2012). Jackknife model averaging.Journal of Econometrics167, 38–46.

HARDLE, W., LIANG , H. & GAO, J. (2000).Partially linear models. Springer.

HARDY, G. H., LITTLEWOOD, J. E. & POLYA , G. (1952).Inequalities. Cambridge university press.

HECKMAN , N. E. (1986). Spline smoothing in a partly linear model.Journal of the Royal Statistical Society.Series B (Methodological)48, 244–248.

HENDERSON, D. J., PAPAGEORGIOU, C. & PARMETER, C. F. (2012). Growth empirics without parameters.The Economic Journal122, 125–154.

HJORT, N. L. & CLAESKENS, G. (2003). Frequentist model average estimators.Journal of the AmericanStatistical Association98, 879–899.

HOETING, J. A., MADIGAN , D., RAFTERY, A. E. & VOLINSKY, C. T. (1999). Bayesian model averaging: Atutorial. Statistical Science14, 382–417.

L I , N., XU, X. & JIN , P. (2010). Testing the linearity in partially linear models. Journal of NonparametricStatistics23, 99–114.

L IANG , H., WANG, S. & CARROLL, R. J. (2007). Partially linear models with missing response variables anderror-prone covariates.Biometrika94, 185–198.

L IANG , H., ZOU, G., WAN , A. T. K. & Z HANG, X. (2011). Optimal weight choice for frequentist modelaverage estimators.Journal of the American Statistical Association106, 1053–1066.

L IU , Q. & OKUI , R. (2013). Heteroskedasticity-robustCp model averaging.The Econometrics Journal16,463–472.

LONGFORD, N. T. (2005). Editorial: Model selection and efficiency—is‘which model ...?’ the right question?Journal of the Royal Statistical Society. Series A (Statistics in Society)168, 469–472.

LONGSTAFF, F. A., PAN , J., PEDERSEN, L. H. & SINGLETON, K. J. (2011). How sovereign is sovereigncredit risk?American Economic Journal: Macroeconomics3, 75–103.

LU, X. & SU, L. (2015). Jackknife model averaging for quantile regressions. Journal of Econometrics188,40–58.

MAGNUS, J. R., WANG, W. & ZHANG, X. (2016). Weighted average least square prediction.EconometricReviews35, 1040–1074.

NI , X., ZHANG, H. H. & ZHANG, D. (2009). Automatic model selection for partially linearmodels.Journalof the American Statistical Association100, 2100–2111.

QIAN , Z., WANG, W. & JI , K. (2017). Sovereign credit risk, macroeconomic dynamics, and financial conta-gion: Evidence from Japan.Macroeconomic Dynamics, forthcoming.

ROBINSON, P. M. (1988). Root-n-consistent semiparametric regression. Econometrica56, 931–954.

RUPPERT, D., WAND , M. P. & CARROLL, R. J. (2003).Semiparametric Regression. Cambridge, New York:Cambridge University Press.

SPECKMAN, P. (1988). Kernel smoothing in partial linear models.Journal of the Royal Statistical Society.Series B (Methodological)50, 413–436.

SU, L. & L U, X. (2013). Nonparametric dynamic panel data models: Kernel estimation and specificationtesting.Journal of Econometrics176, 112–133.

WAN , A. T. K., ZHANG, X. & Z OU, G. (2010). Least squares model averaging by Mallows criterion. Journalof Econometrics156, 277–283.

WHITTLE , P. (1960). Bounds for the moments of linear and quadratic forms in independent variables.Theoryof Probability & Its Applications5, 302–305.

X IE, H. & HUANG, J. (2009). SCAD-penalized regression in high-dimensional partially linear models.TheAnnals of Statistics37, 673–696.


XU, G., WANG, S. & HUANG, J. (2014). Focused information criterion and model averaging based on weightedcomposite quantile regression.Scandinavian Journal of Statistics41, 365–381.

XU, X. & L I , G. (2006). Fiducial inference in the pivotal family of distributions. Science in China: Series A49, 410–432.

YUAN , Z. & YANG, Y. (2005). Combining linear regression models: When and how? Journal of the AmericanStatistical Association100, 1202–1214.

ZHANG, H. H., CHENG, G. & L IU , Y. (2011). Linear or nonlinear? Automatic structure discovery for partiallylinear models.Journal of the American Statistical Association106, 1099–1112.

ZHANG, X. & L IANG , H. (2011). Focused information criterion and model averaging for generalized additivepartial linear models.The Annals of Statistics39, 174–200.

ZHANG, X., WAN , A. T. K. & Z HOU, S. Z. (2012). Focused information criteria, model selection and modelaveraging in a Tobit model with a non-zero threshold.Journal of Business & Economic Statistics30,132–142.

ZHANG, X., ZOU, G. & CARROLL, R. (2015). Model averaging based on Kullback-Leibler distance.StatisticaSinica25, 1583–1598.

ZHANG, X., ZOU, G. & L IANG , H. (2014). Model averaging and weight choice in linear mixed-effects models.Biometrika101, 205–218.

ZHAO, T., CHENG, G. & L IU , H. (2016). A partially linear framework for massive heterogeneous data.TheAnnals of Statistics44, 1400–1437.

Xinyu Zhang, Academy of Mathematics and Systems Science, Chinese Academy of SciencesE-mail: [email protected] Wang, Econometric Institute, Erasmus University Rotterdam, and Tinbergen InstituteE-mail: [email protected]

OPTIMAL MODEL AVERAGING ESTIMATION FOR ......OPTIMAL MODEL AVERAGING ESTIMATION FOR PARTIALLY LINEAR...

Documents

Transcript of OPTIMAL MODEL AVERAGING ESTIMATION FOR ......OPTIMAL MODEL AVERAGING ESTIMATION FOR PARTIALLY LINEAR...