8.4 Weighted Least Squares EstimationBefore the existence of heteroskedasticity-robust statistics, one needed to

know the form of heteroskedasticity

-Het was then corrected using WEIGHTED LEAST SQUARES (WLS)

-This method is still useful today, as if heteroskedasticity can be correctly modeled, WLS becomes more efficient than OLS

-ie: WLS becomes BLUE

8.4 Known Heteroskedasticity-Assume first that the form of heteroskedasticity is known and expressed as: )()|( 2 XhXuVar

-Where h(X) is some function of the independent variables-since variance must be positive, h(X)>0 for all valid combinations

of X-given a random sample, we can write:

iiii hXuVar 22 )|(

8.4 Known Het Example-Assume that sanity is a function of econometrics knowledge and other factors: ursotherfactoeconcrazy 10

-However, by studying econometrics two things happen: either one becomes more sane as one understands the world, or one becomes more crazy as one is pulled into a never-ending vortex of causal relationships. Therefore:

iii econXuVar 2)|(

8.4 Known Heteroskedasticity-Since h is a function of x, we know that:

ii

ii

hXuEX

XhuE22

i )|()|Var(u

and 0)|/(

-Therefore

h

|)(])|/[( 2i2

22

iiiii hhuEXhuE

-So inclusion of the h term in our model can solve heteroskedasticity

8.4 Fixing Het – And Stay Down!-We therefore have the modified equation:

i

i

i

ikk

i

i

ii

i

h

u

h

x

h

x

hh

y

...110

-Or alternately:

(8.26) ... ***110

*iikkii uxxy

-Note that although our estimates for BJ will change (and their standard errors become valid), their interpretation is the same as the straightforward OLS model (don’t try to bring h into your interpretation)

8.4 Het Fixing – “I am the law”-(8.26) is linear and satisfied MLR.1-if the original sample was random, nothing chances so MLR.2 is satisfied-If no perfect collinearity existed before, MLR.3 is still satisfied now-E(ui*|Xi*)=0, so MLR.4 is satisfied

-Var(ui*|Xi*)=σ2, so MLR.5 is satisfied

-if ui has a normal distribution, so does ui*, so MLR. 6 is satisfied

-Thus if the original model satisfies everything but het, the new model satisfies MLR. 1 to 6

8.4 Het Fix – Control the Het Pop-These BJ* estimates are different from typical OLS estimates and are examples of GENERALIZED LEAST SQUARES (GLS) ESTIMATORS

-this GLS estimation provides standard errors, t statistics and F statistics that are valid-Since these estimates satisfy all 6 CLM assumptions, and because they are BLUE, GLS is always more efficient than OLS-Note that OLS is a special case of GLS where hi=1

8.4 Het Fix – Who broke it anyhow?-Note that the R2 obtained from this regression is useful for F statistics but is NOT useful for its typical interpretation

-this is due to the fact that it explains how much X* explains y*, not how much X explains y-when GLS estimators are used to correct for heteroskedasticity, they are called WEIGHTED LEAST SQUARES (WLS) ESTIMATORS-most econometric programs have commands to minimize the weighted sum of squared residuals:

iikkii hxxy /)...(min 2110

8.4 Incorrect Correcting?What happens if h(x) is misspecified and

WLS is run (ie: if one expects x1 to cause het but x3 actually causes het)

1) WLS is still unbiased and consistent (similar to OLS)

2) Standard Errors (thus t and F tests) are no longer valid

-to avoid this, one can always apply a fully robust inference for WLS (as we say for OLS in 8.2)-this can be tedious

8.4 Incorrect Correcting?WLS is often criticized as being better than

OLS ONLY IF the form of het is correctly chosen

-one may argue that making some correction for het is better than none at all

-there is always the option of using robust WLS estimation

-in cases of doubt, both robust WLS and robust OLS results can be reported

8.4 Averages and HetHeteroskedasticity will always exist when

AVERAGES are used-when using averages, each observation is

the sum of all individual observations divided by group size:

iii mxx /

-Therefore in our true regression, our error term is the sum of all individual observations’ error terms divided by group size:

iii muu /

8.4 Averages and HetIf the individual model is homoskedastic, and no

correlation exists between groups, then the average equation is heteroskedastic with a weight of hi=1/mi

-In this way larger groups receive more weight in the regression and is due to the fact that

ii muVar /)( 2For example, assume that we run a regression on

how math knowledge impacts grades in econ classes. Bigger classes (Econ 299) would be weighted to give more information than smaller classes (Econ 349.5 – Love and Econ.)

8.4 Feasible GLS-In the previous section we assumed that we knew the form of the heteroskedasticity, hi(x)

-often this is not the case an we need to use data to estimate hihat

-this yields an estimator called FEASIBLE GLS (FGLS) or ESTIMATED GLS (EGLS)-Although h(x) can be measured many ways, we assume that

kk

kk

xx

xx

eXh

eXuVar

...

...2

110

110

)(

(8.30) )|(

8.4 Feasible GLS-Note that while the BP test for Het assumed Het was linear, here we allow for non-linear Het-although testing for linear Het is effective, correcting for Het has issues with linear models as h(X) could be negative, making Var(u|X) negative-since delta is unknown, it must be estimated

-using (8.30),veu kk xx ...22 110

-Where v, conditional on X, has a mean of unity

8.4 Feasible GLS-If we assume v is independent of X,

exxu kk ...)log( 1102

-Where e has zero mean and is independent of X-note that the intercept changes, which is unavoidable but not drastically important-as usual, we only have residuals, not errors, so we run the regression and obtain fitted values

kk xxu ˆ...ˆˆ)ˆg(ol 1102

-To obtain:)g(ol 2ˆ i

u

i eh

8.4 FGLSTo use FGLS to correct for

Heteroskedasticity,1) Regress y on all x’s and obtain residuals

uhat2) Create log(uhat2)3) Regress log(uhat2) on all x’s and obtain

fitted values ghat4) Estimate hhat=exp(ghat)5) Run WLS using weights 1/hhat

8.4 FGLSIf we used the actual h(X), our estimator would be unbiased and BEST-since h(X) is estimated using the same data as FGLS, it is biased and therefore not BEST-however, FGLS is consistent and asymptotically more efficient than OLS-therefore FGLS is a good alternative to OLS in large samples-note that FGLS estimates are interpreted the same as OLS-note also that heteroskedasticity-robust standard errors can always be calculated in cases of doubt

8.4 FGLS AlternativeOne alternative is to estimate ghat as:

2210

2 ˆˆˆˆˆ)ˆg(ol yyu Using fitted y values from the OLS equation

-This changes step 3 above, but the remaining steps are the same

-Note that the Park (1996) test is based on FGLS but is inferior to our previous tests due to FGLS only being consistent

8.4 F Tests and WLSWhen conducting F tests using WLS,

1) First estimate the restricted and unrestricted model using OLS

2) After determining weights, use these weights on both the restricted and unrestricted model

3) Conduct F tests

Luckily most econometric programs have commands for joint tests

8.4 WLS vs. OLS – Cage MatchIn general, WLS and OLS estimates should

always differ due to sampling error-However, some differences are problematic:1) If significant variables change signs2) If significant variables drastically change

magnitudes-This usually indicates a violation of a Gauss-

Markov assumption, generally the zero conditional mean assumption (MLR.4)

-this violation would cause bias-the Hausman (1978) test exists to test for this,

but “eyeballing” is generally sufficient

8.5 Linear Probability ModelWe’ve already seen that the Linear Probability Model (LPM), where y is a Dummy Variable, is subject to Heteroskedasticity

-the simplest way to deal with this Het is to use OLS estimation with heteroskedastic-robust standard errors

-since OLS estimators are generally inefficient in LPM, we can use FGLS:

8.5 LPM and FGLSWe know that:

kk xxXp

XpXpXyVar

...)(

)](1)[()|(

110

Where p(X) is the response probability; probability that y=1-OLS gives us fitted values and estimates variance using

]ˆ1[ˆˆiii yyh

Given that we now have hhat, we can apply FGLS, except for one catch…

8.5 LPM and FGLSIf our fitted values, yhat, are outside our (0,1) range, hhat becomes negative or zero-if this happens WLS cannot be done as each observation i is multiplied by

ih/1The easiest way to fix this is to use OLS and heteroskedasticity-robust statistics-One alternative is to modify yhat to fit in the range, for example, let yhat=0.01 if yhat is too low and yhat=0.99 if yhat is too high-unfortunately this is very arbitrary and thus not the same among estimations

8.5 LPM and FGLSTo estimate LPM using FGLS,1) Estimate the model using OLS to obtain yhat2) If some values of yhat are outside the unit interval

(0,1), adjust those yhat values3) Estimate variance using:

]ˆ1[ˆˆiii yyh

4) Perform WLS estimation using the weight hhat

8. Heteroskedasticic Review1) Heteroskedasticity does not affect consistency or biasedness, but does affect standard errors and all tests2) 2 ways to test for Het are:

a) Breuch-Pagan Testb) White Test

3) If the form of Het is know, WLS is superior to OLS4) If the form of Het is unknown, FGLS can be run and is asymptotically superior to OLS5) Failing 3 or 4, heteroskedastic-robust standard errors can be used in OLS