A new Difference-based Weighted Mixed Liu Estimator in Semiparametric Regression

Models

Fikri AkdenizFikri Akdeniz Çağ University Department of Mathematics and Computer Science TURKEY 23rd IWMS,Ljubljana, Slonenia 11 June, 2014

OUTLINE

*The problem and objective of presentation*Semiparametric regression model*Difference-based method*Generalized difference-based estimator with correlated errors

*Generalized difference-based weighted mixed estimator Generalized difference–based weighted mixed Liu estimator and efficiency properties

*Generalized difference-based mixed estimator

*A difference based estimation method is given. In particular we introduce the differencing method to semiparametric regression with the focus on estimating the linear component. *A generalized difference-based estimator is introduced for the vector parameter in partially linear model when the errors are correlated. *A generalized difference-based Liu estimator is defined for the vector parameter . *Under the linear stochastic constraint r R e , we introduce a new generalized difference-based weighted mixed Liu estimator. *-The efficiency properties of the difference-based weighted mixed regression method is analyzed

THE PROBLEM and OBJECTIVE OF

PRESENTATION

*In this talk, a commonly used 1 2T

i i i iy X f ( t ) , i , , ...,n

simple semiparametric regression model is considered. The goal is toestimate the unknown parameter vector and nonparametric function f(t) from the data

.

, ,i i iy x t

1. Semiparametric Regression Model (Partially Linear Model)

1 2Ti i i iy X f ( t ) , i , , ...,n

Xi (p-dimensional vectors) and it (real numbers) are known. Xi does not depend on ti . In matrix form

y X f ( t ) ,

1 T

nX ( X , , X ) : n p 1 Tp( , , ) :

p-dimensional parameter vector, f (.): unknown smooth function and to be estimated. : i.i.d. with 0E[ | X,t] , 2Var[ | X,t] I . The 'it s are known and non-random in some bounded domain D R .

Without loss of generality,data rearranged so that 0 1

1 2t t ... t .

n

The explanatory variables are represented separately in two parts: the nonparametric part ( )f t and the parametric linear part ( X )

Some of the relations are believed to be of certain parametric form while

others are not easily parameterized. We shall call f(t) the smooth part of the model and assume that it represents a smooth unparametrized functional relationship.

*SPRM is more flexible than the standard linear regression model since it combines both parametric and nonparametric components. Due to its flexibility, SPRM has been widely used in econometrics, finance, biology, sociology and so on.

*Allows easier interpretation of the effect of each variable compared to a

completely nonparametric regression.

SPRM (or PLM): y X f ( t ) , model generalizes both the parametric linear regression model y X , and nonparametric regression model y f ( t ) which corrrespond to the cases f =0 and =

0, respectively.

Stone (1985) states that the three fundamental aspects of statistical models are filexibility, dimensionality and interpretability.

Flexibility is the ability of the model to provide accurate fits in a wide variety of stuations, inaccuracy here leading to bias in estimation.

Dimensionality can be thought of in terms of the variance in estimation, the curse of dimensionality being that the amount of data required to avoid an acceptable large variance increases rapidly with increasing

dimensionality.

Interpretability lies in the potantial for shedding light on the underlying structure.

To illustrate the ideas in PLM the following example is given. Data for that example consists of the cost of distributing electricity using data on 81 municipal electric utilities in Ontario, Canada during 1993. The main empirical objective is to estimate scale economies of delivering electricity considering the nonparametric and multicollinear structure of the data.Scale economies refer to reductions in unit cost as the size of the facility and the usage levels of the other inputs increase. Therefore, the number of customers (cust) will be considered as a scale effect. A priori, the relationship between the number of customers and unit costs (TC) is unknown. It is therefore proposed to estimate the scale affect using a semiparametric model.

In addition to the number of customers, the hourly wage (WAGE) of linemen of identical grade; and the price of capital (PCAP), which we measure by dividing accumulated gross investment in plant and facilities (TOTPLANT) by total kilometres of distribution wire (KMWIRE) are incorporated. The total quantity of electricity delivered per customer (KWH/CUST) and the remaining lifetime of assets (LIFE) and load factor (LF) which measures capacity utilization relative to peak usage are also included. There is considerable variation in the density of customers across utilities. To capture the efect of the density of customers, the total kilometres of distribution wire divided by the number of customers (KMWIRE/CUST) is also considered. Distributors are grouped as being a part of local Public Utility Commissions (PUC) and plugged into the model as a dummy variable. It is expected that PUCs exhibit some cost savings. Another aspect of the model is to assess whether this is indeed the case. The lower-case names denote the log transformed variables. Thus the covariates used in the model are cust, wage, pcap, PUC, kwh, life, lf and kmwire.

EXAMPLE Aim: Estimate the costs of distributing electricity Source of Data: 81 municipal distributors (during 1993, Ontario, Canada).The data generated by Yatchew (2000) late considered by Tabakan and Akdeniz (2010), Akdeniz Duran and Akdeniz (2013Tabakan (2013), Wu (2014)

The corresponding semiparametric regression model can be written as follows:

1 2 3 4 5

6 7

:

( )

MODEL

tc wage pcap puc kwh life

lf kmwire f cust

tc: log (total cost per customer)-Her bir müşteri için toplam maliyetin logaritması

cust: log (number of customers)- Müşteri sayısının logaritması

wage: log (wage of lineman)- Elektrik şebekesini döşeyen teknisyen ücretinin logaritması

pcap: log (price of capital)-Sermaye miktarının logaritması

PUC: public utility commission dummy- Kamu kuruluşu için yapay değişken(Ekonomik açıdan fayda sağlayabilen ve ek servisler sunabilen)

Kwh: log (kilowatt hour sales per customer)- Müşteri başına düşen ortalama kilowatt saatin logaritması

life: log (remaining lifetime of fixed assets)- Dağıtım varlıklarının geri kalan ömrünün logaritması-

lf: log (load factor)- Bir elektrik santralından alınan ortalama elektrik miktarının elde edilebilecek maksimum miktara oranı

kmwire: log (kilometers of distribution wire per customer)- Her bir müşteri için döşenen elektrik dağıtım kablosunun kilometresinin logaritması

In regression analysis, researchers often encounter the problem of multicollinearity. *In case of multicollinearity we know that the correlation matrix might have one or more small eigenvalues which causes the estimates of the regression coefficients to be large in absolute value. *The least squares estimator performs poorly in the presence of multicollinearity. Multicollinearity is defined as the existence of nearly linear dependency among column vectors of the design matrix X in the linear model . Xy *The existence of multicollinearity may lead to wide confidence intervals for the individual parameters or linear combination of the parameters and may produce estimates with wrong signs.

*Condition number max min( ) ( ) / ( )X X X X X is a measure of the presence of

multicollinearity. If X X is ill conditioned with a large condition number, ridge regression estimator (Hoerl and Kennard, 1970) or Liu estimator (Liu, 1993) can be used to estimate *To apply shrinkage estimators is well-known as an efficient remedial measure in order to solve problems caused by multicollinearity. *We assume that the condition number of the parametric component is large indicating that a biased estimation procedure is desirable. Its parametric part has the same structural form as the classical methods.

MULTICOLLINEARITY PROBLEM

Let us now check whether there is multicollinearity among the parametric variables in model (**).

Let X be a 781 observation matrix consisting of the variables: wage, pcap, puc, kwh, life, lf,

kmwire. The eigenvalues of the matrix 77: XX are given by, 14416.7709 ( max ), 21.7903,

12.6587, 3.3039, 1.8153, 0.7007, 0.3022 ( min ). If we use the spectral norm, then the

corresponding measure of conditioning of X is

max min( ) ( ) / ( ) 47706.06 218.42X X X X X

which is large and therefore X should be considered as being ill-conditioned. As some critical values, (Belsley, D. A., Kuh, E. and Welsch, R. E. (1980) Regression Diagnostics : Identifying Infuential Data and Sources of Collinearity, John Wiley and Sons, New York.) consider condition numbers between 5 and 10 as an indication for weak collinearity, while condition numbers greater than 30 are seen to indicate strong collinearity. In the existence of collinearity, one or more main diagonal

elements of the matrix 1)( XX will be quite large. When strong collinearity is present, the least

squares estimator may not be robust with respect to small changes in values of independent variables. In this case the corresponding element of the least squares estimator has a large variance and the estimate is unreliable. Thus, almost all least squares estimates have greater length than the true parameter vector.

VIF is another measure for diagnose of collinearity. VIF values are calculated as

( ) /( )j j j j j jVIF x x x M x , where 1( )j n j j j jM I X X X X and ( , )j jX x X ; the elements of

the 1n vector jx are the observations of the variable jx and jX is the observation matrix

of the remaining variables (Gross, J. Linear regression,2003, Springer). VIF values for variables are 59.5610, 4.7576, 3.0264, 1.1213, 23.9085, 4.8217, 26.0818, 50.1542, 31.3564, respectively.Some VIF values are larger than 30 which also shows strong collinearity among variables.

Figure 1. Plots of individual explanatory variables versus. dependent variable, linear fit (blue), kernel fit (red), %95 confidence bands (black)

Figure 2. Plots of individual explanatory variables versus dependent variable, linear fit (blue), kernel fit (red), %95 confidence bands (black)

As seen in Figures 1 and 2 , unit cost and number of customers have a U-shaped relation. The hypothesis of the linearity of cust is rejected due to F-test, therefore cust will be taken as a non-parametric variable. For the other variables, it is seen that the linear ordinary least squares fit stays inside the confidence bands except for wage; however, the hypothesis for linearity of wage, cannot be rejected threfore all the variables except cust will be incorporated as parametric variables. Thus semiparametric model will be fitted with cust as a non-parametric variable.

2. The Model and Difference-based Estimator 

(A. Yatchew, An Elemantary Estimator of the Partial Linear Model. Economics Letters, 57, (1997), 135-143) Yatchew suggested estimating on the basis of the m-th order differencing equation

0 0 0 0

m m m m

j i j j i j j i j j i jj j j j

d y ( d x ) ( d f ( u ) ( d ) 1 i m , ...,n (2.2)

Now let ),...,,( 10 mdddd be a (m+1)-vector, where m is the order of differencing

and mddd ,...,, 10 are differencing weights minimize 0

2

1 0

m

m m l

j l jd ,...,dl j

min ( d d )

satisfying the conditions

0

0

m

jj

d and2

0

1

m

jj

d . (2.3)

In this section we use a difference-based technique to esatimate linear regression coefficient vector . Consider the following semiparametric regression model y X f , (2.1)

0

0m

jj

d

implies differencing removes np variables as n ,

2

0

1m

jj

d

implies transformed residual has variance 2.

0 1, ,..., md d d are optimal differencing weights (Hall, P., Kay, J.W. and

Titterington , D. M. On the estimation of noise variance in two-dimensional signal processing , Advanced in Appl.Prob. 23, 476-495,1990)

Let us define the nmn )( differencing matrix D to have first and last rows 10, mnd ,

dmn ,0 1 respectively, with i-th row 10,,0 imni d , ),1(,...,2 mni where r0 indicates

a r-vector or all zero elements.

0 1 2 m

0 1 2 m

0 1 m

0 1 m

d d d . . . d 0 0 . . . 0

0 d d d . . . d 0 0 . . . 0

. . .

D .. . .

. . .

0 0 0 . . . d d . . d 0

0 0 0 . . . d d . . d

Applying the differencing matrix to model (2.1), permits direct estimation of the parametric effect.

Since the data have been ordered so that the values of the nonparametric variable(s) are close, the application of the differencing matrix D in model (2.1) removes the non-parametric effect in large samples (Yatchew, 2000). If f is an unknown function that is the inferential object and has a bounded first derivative, then Df is close to 0, so that applying the differencing matrix we have

Dy DX Df D (2.4) This ignores the presence of Df(t). Thus we may write Equation (2.4) as Dy DX D , or ~~~ Xy (2.5) where DXXDyy ~

,~ and .~ D

Thus, standart linear models considerations suggest estimating by

1ˆ ( ) ( ) ( ) ( ),diff DX DX DX Dy (2.6) or

1ˆ ( )diff X X X y . This estimator was first proposed in Yatchew (1997).

The matrix : 7 7X X X D DX has eigenvalues 19.49 (= max ),

15.14, 3.42, 2.62, 1.54, 0.23, 0.11 (= min ). The condition

number for the differenced matrix X is ( ) 19.49 / 0.11 13.11X which implies the existence of moderate multicollinerity in the difference dataset.

nonparametric variable(s) are made close by reordering the data

‘difference’ the data to ‘remove’ the effect of the nonparametric variable(s)

run OLS regression of the differenced dependent variable on the differenced parametric explanatory variables

Applying the differencing matrix permits direct estimation of the parametric effect.

Estimation Procedure

The estimation procedure is optimal in the sense that the estimator of the linear component is asymptotically efficient (Yatchew, 2003)

What is the advantage of differencing?

An important advantage of differencing procedures is their simplicity.

Increasing the order of differencing as sample size increases, the estimator of the linear component becomes asymptotically efficient (Yatchew 2003, p.72)

Using Eq. (2.2) for m=1, then we have the first differencing: 1 1 1 1

0 0 0 0

2 20 1 0 1

0 1

0 1

( ) ( ) ,

2,..., .

0, 1

0

0

j i j j i j j i j j i jj j j j

d y d x d f t d

i n

d d d d

d dD

d d

D: n-mxn

How does the approximation work?

Suppose it equally spaced on the unit interval and (.) .f L

For some *1,i i it t t we have

1 1 *i i i i i

Lf ( t ) f ( t ) f ( t )( t t ) .n

(by the mean value theorem). Thus, we have

1 1 1 1 i i i i i i i iy y ( x x ) f ( t ) f ( t )

= 1 11 i i i i( x x ) O( / n )

1 1 i i i i( x x ) .

We then estimate the linear regression coefficient by the ordinary least- squares estimator based on the differences. Then we obtain the least squares estimate:

1 12

1

( )( )ˆ( )

i i i iD

i i

y y x x

x x

Biased Estimation in SemiparametricRegression Models

under Multicollinearity

Consider the standard multiple linear regression model.

Xy

where y is an nx1 vector of observationson on the dependent variable β is a 1p vector of regression coefficients X is an pn matrix and of full column rank, and is an 1n random vector which distributed as E(ε)=0 and variance-covariance matrix Var(ε) = σ2I, I is the identity matrix of order n.

The ordinary least squares (OLS) estimator of β is

The ridge estimators originally proposed by Hoerl and Kennard (1970) are of the highly specialized form:

(00)

The Lagrange equation with multiplier k for minimizing the objective function that a vector of values b of subject to the

restriction that the squared length of b is 2Cbb can be written as

.

Equating bL / to zero then yields equation (00).

1ˆ ( )X X X y

1ˆ ( ) ( )R k X X kI X y

2( ) ( ) ( ) ( )L b y Xb y Xb k b b C

Augmented model:

0

y X

I

.

Define a vector 0*

, then we have

* * ( ) ( ) ( ) ( )y X y X

0.F

The solution of this system is

1( ) ( )T TX X I X y , 0 1 . (***)

where η is biasing parameter. Since there is a formal resemblance between (***) and the Liu estimator of the linear model, we call it a Liu-type estimator of the semiparametric regression model.

3. Generalized difference-based estimator with correlated errors In this section we consider the following semiparametric model

fXy with ,0)( E and .)( 2VE So, ~ = D is a )( mn vector of disturbances distributed with

0)~( E and DVDDVE 22)~~( , (3.1) where mnD IDDVV is a known )()( mnmn symmetric positive definite (p.d.) matrix and 02 is an unknown parameter. It is well known that adopting the linear model (2.5), the unbiased estimator of is the following generalized difference-based estimator given by 1 1 1ˆ ( ) .GDE D DX V X X V y (3.2)

It is observed from Equation (3.2) that the properties of the generalized difference-based estimator of depends on the characteristics of the information matrix XVX D

~~ 1 = G.

If G: pp , p n m matrix is ill-conditioned with a large condition number, then the GD produces large sampling variances. Moreover, some regression coefficients may be statistically insignificant and meaningful statistical inference becomes difficult for the researcher. We assume that the condition number of the G matrix is large indicating that a biased estimation procedure is desirable.

To avoid identifiability problems we assume that a variable contained in X is not contained also in t, and in general, that no component of X can be mapped to any component of t .

In this section, we will discuss a biased estimation technique when the matrix G appears to be ill-conditioned. In literature, there are various biased estimation methods to combat the multicollinearity problem, such as ridge

regression estimator 1ˆ , 0k Z Z kI Z y k , and Liu estimator

1ˆ ˆ , 0 1OLSZ Z I Z Z I As a remedy, we suggest to use the

following estimator, namely, generalized difference-based Liu estimator:

1 1 1 GD D D GD

ˆ ˆ( ) ( X V X I ) ( X V y ) (3.3)

Applying a penalizing function of the squared norm 2ˆ GD to the

generalized least squares objective )~~()

~~( 1 XyVXy D for the vector of regression coefficients yields a conditional objective:

21

D GDˆF( ) arg min ( y X ) V ( y X )

(3.4)

3.1 Generalized Difference-based Liu Estimator

The first order condition of objective (3.4) minimized by vector is

F

0.

(3.5)

From condition (3.5) of minimizing (3.4) by the vector , we obtain the generalized difference-based Liu estimator given by (3.3).

Now, we consider the linear nonstochastic constraint R r . (…) For a given pq matrix R with rank pq and a given 1q known vector r. We will call an estimator )(* y for a restricted estimator with respect to rR , if it satisfies ryR )(* for all 1n vectors y (see Gross, 2003). Subject to the linear rectriction (…), the generalized difference-based restricted estimator is given by 1 1 1

GRD GD GDˆ ˆ ˆG R ( RG R ) ( r R )

We can obtain generalized difference-based restricted Liu estimator to improve the generalized difference-based estimator by minimizing the sum of squared residuals with a restriction .rR

4. Weighted Mixed Regression and Estimation of Parameters The use of prior information in linear regression analysis is well known to provide more efficient estimators of regression coefficients. The available prior information sometimes can be expressed in the form of exact, stochastic or inequality restrictions. We consider the model (2.5):

y X , 2 20 0 D( , DVD ) ( , V ) (4.1) When a set of stochastic linear constraints binding the regression coefficients in a linear regression model is available, Theil and Goldberger (1961) have proposed the method of mixed regression estimation. Their method typically assumes that the prior information in the form of stochastic linear constaints and sample information in the form of observations on the study variable and explanatory vaiables are equally important and therefore receive equal weights in the estimation procedure. Totally, we do not have exact prior information such as R r , involving estimation of economic relations, industrial structures, production planning, etc.

In addition to the sample model (4.1), it is supposed that a set of stochastic linear constraints binding the regression coefficients is available in the form of independent prior information: ,r R e 2(0, )e W (4.2) where R is a q p known matrix with rank ( )R q , e is a 1n vector of disturbances. W is assumed to be known and positive definite, the 1q vector r can be interpreted as a random variable with expectation ( )E r R . Therefore the restriction (4.2) does not hold exactly but in the mean and we assume r to be known, that is to be a realized value of the

random vector, so that all expectations are conditional on r as, for example, ˆ( )E r (Rao et

al.,2008). In order to take the information (4.2) into account while constructing estimators

for , we require that ˆ( ) .E R r r Toutenburg et al., (2003)

In model (4.2), we have assumed the structure of the dispersion matrix of e, 2( )E ee W ,

that is, with the same factor of proportionality 2 as occured in the sample model. Therefore, it may some times be more realistic to suppose that ( )E ee W . It is also assumed that the random vector is stochastically independent of e . When the sample information given by (4.1) and prior information is described by (4.2) are to be assigned not necessarily equal weights on the basis of some extraneous considerations in the estimation of regression parameters, Schaffrin and Toutenburg (1990) have proposed the method of weighted mixed regression estimation. Following their technique we obtain the generalized difference-based weighted mixed estimator of .

In order to incorporate the restrictions (4.2) in the estimation of parameters, we minimize 1 1

D( y X )V ( y X ) ( r R )W ( r R ) (4.3)

with respect to . This leads to the following solution for :

1 1 1 1 GDWME D

ˆ ˆ( ) ( ) (G R W R ) ( X V y R W r ) , (4.4)

where 1

DG X V X and is a non-stochastic and non-negative scalar weight with 0 1 ,

( 0 would lead to ˆGDE ). It is seen that a value of between 0 and 1 specifies an estimator in

which the prior information receives less weight in comparison to the sample information. On the other hand, a value of greater than 1 implies higher weight to the prior information which, of course, may be of little practical interest.

Since

1 1 1 1 1 1 1 (G R W R ) G G R (W RG R ) RG (4.5) we have

GDWMEˆ ( ) = GDE + 1 1 1 GDE

ˆG R (W RG R ) ( r R ) . (4.6)

If we substitute 1 in (4.4), we get

1 1 1 1 GDME D

ˆ (G R W R ) ( X V y R W r ) , (4.7)

which is the generalized difference-based mixed estimator. The ordinary mixed estimator is proposed by Theil and Goldberger (1961). This estimator gives equal weight to sample and prior information.

(Generalized) Liu estimator proposed by Liu (1993) and Akdeniz and Kaçıranlar (1995) is defined as

1 1 GDLE D GDE

ˆ ˆ( ) (G I ) ( X V y ) , 0 1 ,

1 GDE GDE

ˆ ˆ(G I ) (G I ) F (4.8)

where 1 1 1ˆ ( )GDE D DX V X X V y is the generalized difference-based estimator of and 1( ) ( )F G I G I . Observing that F and 1G are commutative, we have

GDLEˆ ( ) 1 1 1 1

D DF G X V y G F X V y . (4.9)

Hubert and Wijekoon (2006) grafted the Liu estimator into the mixed estimation procedure and obtained a new family of stochastic restricted Liu estimator.

Substituting ˆGDE with ˆ ( )GDLE in ˆ ( )GDWME , we describe a generalized difference–based

weighted mixed Liu estimator (GDWMLE), as follows: Since

1 1 1 1 1 1 1( ) ( )W RG R W W R G R W R RW (4.10) and

1 1 1( )G R W RG R r 1 1 1 1 1 1( )G G R W RG R RG R W r

1 1 1( )G R W R R W r (4.11) Using the equalities in (4.9), (4.10) and (4.11), we have

ˆ ˆ( , ) ( , )GDWMLE = ˆ ( )GDLE + 1 1 1 ˆ( ) ( ( ))GDLEG R W RG R r R

1 1 1 1 1 1 1 1 1 1( ) ( )D DG F X V y G R W RG R r G R W RG R RG F X V y

1 1 1 1 1 1( ( ) ) DG G R W RG R RG F X V y 1 1 1( )G R W RG R r

1 1 1 1 1 1( ) ( )DG R W R F X V y G R W RG R r 1 1 1 1( ) ( ).DG R W R F X V y R W r (4.12)

In fact, from the definition of ˆ( , ) , we can see that it is a general estimator which includes the ˆ ( )GDWME , ˆ ( )GDLE and the mixed regression estimator as special cases. Namely, if =1 then ˆ( , 1) ˆ ( )GDWME , ((4.12)(4.6)) if 1 and 1 then ˆ( 1, 1) ˆ

GDME , ((4.12)(4.7)) if 0 then ˆ( 0, ) = ˆ ( )GDLE ((4.12)(4.9))

5. Efficiency Properties In this section, we compare the underlying estimators. For the convenience of the following discussions, we give some lemmas here. Lemma 5.1 (Farebrother,1976) Let A be a positive definite matrix, namely A > 0, and let be some vector, then 0A if and only if 1 1.A Lemma 5.2 Let n n matrices M > 0, 0N , then M > N if and only if 1

max ( )NM < 1.(see

Rao et al., 2008)

Lemma 5.3 (Trenkler and Toutenburg, 1990) Let ˆ , 1, 2j jA y j be two competing

estimators of . Suppose that 1 2ˆ ˆ( ) ( )Cov Cov > 0. Then

1 2ˆ ˆ( ) ( )MSEM MSEM 0 if and only if 1

2 1 1 2( ) 1,b b b b where jb denotes bias

vector of ˆj .

Now we study the eficiency properties of generalized difference-based weighted mixed estimator and the dominance conditions for the mean squared error matrix (MSEM)

superiority of ˆ ( )GDWME and ˆ( , ) over ˆGDE . It is easy to compute that the expectation

and dispersion matrix of the ˆ( , ) are

ˆE( ( , )) BA and Var( 2 ˆ( ( , )) BAB (5.1) where

1 1: ( )B G R W R , 1: ( )A F G R W R and 2 1: ( )A F GF R W R .

The bias of ˆ( , ) is

Bias ˆ( ( , ) ) = ˆE( ( , )) = B( F I )G (5.2)

and the mean squared error matrix of ˆ( , ) is ˆMSEM( ( , )) (Var ˆ( ( , )) + Bias ˆ( ( , ) ) Bias ˆ( ( , ))

= 2 BAB + 1 1b b (5.3)

where 1 ( )b B F I G . So it is obvious that ˆ( , ) is always biased unless 1.

We can easily obtain the MSEM of the estimators ˆGDE , ˆ

GDME , ˆ ( )GDLE , ˆ ( )GDWME as

follows

2 1ˆ ˆ( ) ( )GDE GDEMSEM Var G (5.4) 2 1 1ˆ ˆ( ) ( ) ( )GDME GDMEMSEM Var G R W R (5.5)

2 12 2

ˆ( ( )) ,GDLEMSEM F G F b b (5.6)

with 2ˆ( ) ( )GDLEb Bias F I .

2 2 1ˆ ˆ ˆ( ) ( ( )) ( ( )) ( )GDWMEMSEM MSEM Var B G R W R B (5.7)

5.1 MSEM Comparison between ˆ( ) and ˆ( , ) Theorem 5.1 The generalized difference-based weighted mixed Liu estimator ˆ( , ) is superior

to the generalized difference-based weighted mixed estimator ˆ( ) in the MSEM sense, namely

ˆ ˆ( ( )) ( ( , )) 0MSEM MSEM if and only if 2 * 1

1 1 1b B b .

Proof. In order to compare ˆ( ) with ˆ( , ) in the MSEM sense we consider the difference

2 2 1 2 2 11 1

ˆ ˆ( ( )) ( ( , )) ( ) ( )MSEM MSEM B G R W R B B F GF R W R B b b

2 21 1 1 1 1( )B G F GF B b b B B b b (5.8)

where 1 G F GF . For 1

DG X V X > 0, there exists some orthogonal matrix Q , such that

,G Q Q where 1( ,..., )pdiag .

Therefore, we can easily compute that

1 2( , ,..., )pG F GF Qdiag Q

where 2

2 2

( )( ( ) ) (1 )(1 2 ( ) ) ( )( ) .

( ( ) 1) ( ( ) 1)i i i i

i ii i

G G G GG

G G

Since 0< < 1 and

( )i G > 0, then i > 0. Observing that 1 1: ( )B G R W R > 0 we get 1 > 0 and

1* :B B B > 0 . By applying the Lemma 5.1, we get ˆ ˆ( ( )) ( ( , ))MSEM MSEM 0

if and only if 2 * 11 1 1b B b . By Lemma 5.3 the proof is completed.

5.2 MSEM Comparison between ˆ ( )GDLE and ˆ( , ) Theorem 5.2 When 1

max ( )NM < 1, the generalized difference-based weighted mixed Liu

estimator ˆ( , ) is superior to the generalized difference-based Liu estimator ˆ ( )GDLE in

the MSEM sense, namely ˆ ˆ( ( )) ( ( , )) 0GDLEMSEM MSEM if and only if

2 11 2 2 2 1( ) 1.b b b b

Proof. In order to compare ˆ ( )GDLE with ˆ( , ) in the MSEM sense, we similarly consider

the difference 2 1 2 2 1

2 2 1 1ˆ ˆ( ( )) ( ( , )) ( )GDLEMSEM d MSEM F G F b b B F GF R W R B b b

2 1 2 12 2 1 1

2 22 2 1 1 2 2 2 1 1

( )

( ) .....(5.9)

F G F B F GF R W R B b b b b

M N b b b b b b b b

where 1M F G F , 2 1( )N B F GF R W R B and 2 M N .

It is obvious that, M 1F G F > 0, N 2 1( )B F GF R W R B > 0. Therefore, when

1max ( )NM < 1, we get 2 > 0 by applying Lemma 5.2. Furthermore, by Lemma 5.3, we have

ˆ ˆ( ( )) ( ( , ))GDLEMSEM MSEM 0 if and only if 2 11 2 2 2 1( ) 1.b b b b This

assertation completes the proof.

5.3 Variance Comparison between ˆGDE and ˆ

GDWME Theorem 5.3 The generalized difference-based weighted mixed estimator ˆ

GDWME is superior

to the generalized difference-based estimator ˆGDE in the sense that

0 GDE GDWMEˆ ˆVar( ) Var( ) if and only if 12

1 0

( )W RG R .

Proof: The covariance matrix of ˆGDE is

2 1GDE

ˆVar( ) G (5.10)

The covariance matrix of ˆGDWME is

2 2 1 GDWME

ˆVar( ) B(G R W R )B . (5.11)

The difference in the covariance matrices of ˆGDE and ˆ

GDWME is

3 ˆ( )GDEVar 2 1 2ˆ( )GDWMEVar G B 2 1( )G R W R B

2 1 1 1 2 1

2 2 1 1 12( 1)

B B G B G R W R B

BR W W RG R W RB

. (5.12)

The difference in (5.12) is positive definite when B is positive definite and

12( 1) 0W RG R

,

which is positive definite as long as 2 . When q p , R has full row rank, therefore R has full column rank and it follows that in this case we can only conclude that 3 0.

6.Conclusions We considered the method of weighted mixed regression estimation to estimate the regression coefficients in genralized difference-based semiparametric partially linear model. The

generalized difference-based weighted mixed Liu estimator, ˆ( , )d is derived and its

dominance over both the generalized difference-based weighted mixed estimator, ˆ( ) and

the generalized difference-based Liu estimator, ˆ ( )GDLE d is studied under the criterion of

mean squared error matrix.

REFERENCES 1 P. Green, C. Jennison and A. Seheult, Analysis of field experiments by least squares smoothing, J. Roy. Statist. Soc. Ser. B. 47, (1985), 299-315. 2 R.F. Engle, C.W.J. Granger, J. Rice and A. Weiss, Semiparametric estimates of the relation between weather and electricity sales, J. Amer. Statist. Assoc. 81, (1986), 310-320. 3 R.L. Eubank, E. L. Kambour, J.T. Kim, K. Klipple, C.S. Reese and M. Schimek, Estimation in partially linear models, Computational Statistics and Data Analysis 29, (1988), 27-34. 4 P. Speckman, Kernel somoothing in partial linear models. J. Roy. Statist. Soc. Ser. B. 50(3): (1988), 413-436. 5 R.L. Eubank, Nonparametric regression and Spline Smoothing. Marcel Dekker, New York, 1999. 6 D. Ruppert, M.P. Wand and R.C. Carroll, Semiparametric regression. Cambridge University Pres, 2003. 7 W. Härdle, M. Müller and S. Sperlich, A. Werwatz, Nonparametric and semiparmetric models. Springer-Verlag, Berlin, 2004.

8 A. Yatchew, An Elemantary Estimator of the Partial Linear Model. Economics Letters, 57, (1997), 135-143.

9 A. Yatchew. Semiparametric regression for the applied econometrican. Cambridge University Pres, 2003.

10 A.E. Hoerl and R.W. Kennard, Ridge regression: biased estimation for orthogonal problems. Technometrics 12, ( 1970), 55-67. 11 K.J. Liu, A new class of biased estimate in linear regression, Communications in

Statistics Theory and Methods 22, (1993), 393-402. 12 C. Stein, Inadmissibility of the usual estimator for the mean of a multivariate

normal distribution. Proc. Third Berkeley Symp. Math. Statist. Prob. 1, (1956), 197-206. 13 F. Akdeniz and S. Kaçıranlar, On the almost unbiased generalized Liu estimator

and unbiased estimation of the Bias and MSE, Communications in Statistics- Theory and Methods 24(7), (1995), 1789-1797. 14 M.H.J. Gruber, Improving efficiency by shrinkage: The James-Stein and ridge

regression estimators, New York, Marcell Dekker, Inc., 1998. 15 Akdeniz, F. and Tabakan, G. Restricted ridge estimators of the parameters in semiparametric regression model Communications in Statistics- Theory and Methods. 38(11), (2009), 1852-1869 16 Akdeniz, F. and Akdeniz Duran, E. Liu–type estimator in semiparametric regression models. Journal of Statistical Computation and Simulation 80(8), (2010), 853-871. 17 Akdeniz Duran, E., Akdeniz, F. and Hongchang, Hu . Efficiency of estimators in semiparametric regression models, Journal of Computational and Applied Mathematics 235(5), (2011),1418-1428. 18 Akdeniz Duran. E. and Akdeniz, F. New difference-based estimator of parameters in semiparametric regression models, Journal of Statistical Computation and Simulation DOI:10.1080/00949655.2011.638633

19 Tabakan, G. and Akdeniz, F. Difference-based ridge estimator of parameters in

partial linear model, Statistical Papers, 51: (2010), 357-368. 20 K. Klipple and R.L. Eubank, Difference-based variance estimators for partially linear models. Festschrift in honor of Distinguished Professor Mir Masoom Ali on the occasion of his retirement. May 18-19, (2007), 313-323. 21 L.D. Brown and M. Levine, Variance estimation in nonparametric regression via the difference sequence method. The Annals of Statistics, 35, (2007), 2219-2232. 22 A. Yatchew, Scale economics in electricity distribution, Journal of Applied Econometrics, 15, (2000), 187-210. 23 M. Roozbeh, M. Arashi and H. A. Niroumand, Ridge regression methodology in partial linear models with correlated errors Journal Statist. Comp. Simul. Vol 81(4), (2011), 517-528.

24 H, Theil and Goldberger, A.S. On pure and mixed statistical estimation in

economics, Int. Econ. Review 2, (1961), 65-78.

25 C. R. Rao, and Toutenburg, H., Shalab and Heumann, C. Linear Models: Least

Squares and Alternatives. Springer, New York, 1995.

26 H, Toutenburg, Srivastava, V. K., Schaffrin , B.,Heumann, C. Efficiency properties of

weighted mixed regression estimation, METRON-Int. J. Statist. LXI (1), 2003), 91-103.

27 B, Schaffrin, and Toutenburg, H. Weighted mixed regression, Zeitschrift fur

Angewandte Mathematik und Mechanik, Vol.70, (1990), 735-738.

28 M. H. Hubert, and Wijekoon, P. Improvement of the Liu estimator in linear

regression model, Stat. Pap. 47, (2006), 471-479.

29 H. Yang, Chang, X. and Liu, D. Improvement of the Liu estimator in weighted

mixed regression, Commun. Statis. Theor. Meth., 38, (2009), 285-292.

30 R. W. Farebrother, Further results on the mean square error of ridge regression J.

R. Stat. Soc. B 38, (1976), 248-250.

31 G. Trenkler and Toutenburg, H. Mean squared error matrix comparisons between

biased estimators: an overview of recent results Stat. Pap. 31, (1990),165-179. Wang, L. Brown, L.D. and Cai, T. T. (2011) A difference based approach to

semiparametric partial linear model Electronic J. Statistics , 5, 619-641

Articles on the Semiparametric Regression Models (2009-2014) 1) Akdeniz, F. and Tabakan, G. (2009) Restricted ridge estimators of the

parameters in semiparametric regression model Communications in Statistics- Theory

and Methods. 38(11), 1852-1869. 2) Tabakan, G. and Akdeniz, F. (2010) Difference-based ridge estimator of parameter in partial linear model. Statistical Papers 51:357-368 3) Akdeniz, F. and Akdeniz Duran, E. (2010) Liu–type estimator in semiparametric regression models. Journal of Statistical Computation and Simulation 80(8),853- 871. 4) Akdeniz Duran, E., Akdeniz, F. and Hongchang Hu (2011) Efficiency of estimatorsin semiparametric regression models, Journal of Computational and Applied Mathematics 235(5),1418-1428. 5) Akdeniz Duran, E. and Akdeniz F. (2012) Efficiency of Modified Jacknife Liu- type estimator. Statistical Papers 53 (2), 265-280. 6) Erdugan, F. and Akdeniz, F. (2012) Computational method for Jackknifed Generalized RidgeTuning Parameter based on Generalized Maximum Entropy Commun.Statist Simul. Comput. 41 (8), 1411-1429. 7) Akdeniz Duran, E. and F. Akdeniz (2013) Difference based Liu-type estimators of parameters in semiparametric regression models Journal of Statistical Computation and Simulation 83(5), 808-822. 8) Akdeniz, F., Esra Akdeniz Duran, M. Roozbeh and M. Arashi, Efficiency of the Generalized Difference-based Liu Estimators in Semiparametric Regression Models with Correlated Errors Journal of Statistical Computation and Simulation and Simulation (Accepted 17 may,2013 http://dx.doi.org/10.1080/00949655.806924)

THANK YOU