PROBABLITY

STATICS&

PROJECT.1Assuming that the error terms are distributed as:

Please derive the maximum likelihood estimator for the simple linear regression model,assuming the regressor X is given (that is, not random -- this is also commonly referred to as conditioning on X = x). One must check whether the MLE of the model parameters ( and ) are the same as their LSE’s.

Solution

The p.d.f. is

We derive the Maximum Likelihood Estimator for the simple linear regression model:

Derive the total probability function L

Derive lnL

We compute the and , namely that

Note:

We also set up

We will still use them in next projects.

It's easy to get the result:

As we can see, they are the same as their LSE's

PROJECT.2Errors in Variable (EIV) regression.

In this case, let's derive its distribution.

X and Y follow a bivariate normal distribution:

And we need to derive the MLE of the regression slope :

In order to simplify the calculation,we also bulit a simpleEIV model.

In this model, are not ramdon any longer.It means that Xand Y are independent.And we can also derive the MLE of the regression slope as same expression .

SolutionLet's derive the MLE for simple EIV model:

Let 、 and

then we have

Here, stands for .

So we can get

Now,we will pin-point which special cases correspond to the OR and the GMR(they are 2 kinds of way to derive the linear regression).

the Orthogonal Regression

So,we have

the Geometric Mean Regression

So,we have

PROJECT.3Our third project is to derive a class of non-parametricestimators of the EIV model for simple linear regressionbased on minimizing the sum of the following distance fromeach point to the line as illustrate din the figure below:

Please also show whether there is a 1-1 relationship between this class of estimators and those in Project 2(A/B). That is, try to ascertain whether there is a 1-1 relationship between c and .

We know that

Therefore ,we need to minimize the sum

and

Here, stands for

Let and ,we can get that

Solving these two equations, we can get s.t.

When we have the examples, we can put them in this founction,then we can compute the .

PROJECT.4In the project 3,we have found that and are 1-1 relationship.

In this case, we can easily derive that and are 1-1 relationship, so and are 1-1 relationship.

PROJECT.6For those who have finished Projects 2 & 3 & 4 above, youmay also examine how to estimate the error variance ratio , when we have two repeated measures on each sample.

Let be the material and measurement.Therefore, we have

It’s easy to know these elements are independent.

In order to simplify the calculation,we also bulit a simpleEIV model.

Let and ,we can get

and

Let and ,we can get

Therefore