P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive...

27
PROBABLITY STATICS &

Transcript of P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive...

Page 1: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

PROBABLITY

STATICS&

Page 2: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

PROJECT.1Assuming that the error terms are distributed as:

Please derive the maximum likelihood estimator for the simple linear regression model,assuming the regressor X is given (that is, not random -- this is also commonly referred to as conditioning on X = x). One must check whether the MLE of the model parameters ( and ) are the same as their LSE’s.

Page 3: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Solution

The p.d.f. is

We derive the Maximum Likelihood Estimator for the simple linear regression model:

Page 4: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Derive the total probability function L

Derive lnL

Page 5: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

We compute the and , namely that

Note:

Page 6: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

We also set up

We will still use them in next projects.

Page 7: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

It's easy to get the result:

As we can see, they are the same as their LSE's

Page 8: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

PROJECT.2Errors in Variable (EIV) regression.

In this case, let's derive its distribution.

Page 9: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.
Page 10: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

X and Y follow a bivariate normal distribution:

And we need to derive the MLE of the regression slope :

Page 11: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

In order to simplify the calculation,we also bulit a simpleEIV model.

In this model, are not ramdon any longer.It means that Xand Y are independent.And we can also derive the MLE of the regression slope as same expression .

Page 12: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

SolutionLet's derive the MLE for simple EIV model:

Page 13: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Let 、 and

then we have

Here, stands for .

Page 14: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

So we can get

Now,we will pin-point which special cases correspond to the OR and the GMR(they are 2 kinds of way to derive the linear regression).

Page 15: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

the Orthogonal Regression

So,we have

Page 16: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

the Geometric Mean Regression

So,we have

Page 17: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

PROJECT.3Our third project is to derive a class of non-parametricestimators of the EIV model for simple linear regressionbased on minimizing the sum of the following distance fromeach point to the line as illustrate din the figure below:

Page 18: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Please also show whether there is a 1-1 relationship between this class of estimators and those in Project 2(A/B). That is, try to ascertain whether there is a 1-1 relationship between c and .

Page 19: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

We know that

Therefore ,we need to minimize the sum

and

Here, stands for

Page 20: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Let and ,we can get that

Page 21: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Solving these two equations, we can get s.t.

When we have the examples, we can put them in this founction,then we can compute the .

Page 22: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

PROJECT.4In the project 3,we have found that and are 1-1 relationship.

In this case, we can easily derive that and are 1-1 relationship, so and are 1-1 relationship.

Page 23: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

PROJECT.6For those who have finished Projects 2 & 3 & 4 above, youmay also examine how to estimate the error variance ratio , when we have two repeated measures on each sample.

Page 24: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Let be the material and measurement.Therefore, we have

It’s easy to know these elements are independent.

Page 25: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

In order to simplify the calculation,we also bulit a simpleEIV model.

Page 26: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Let and ,we can get

and

Let and ,we can get

Page 27: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.

Therefore