10. Simple Regression

8/11/2019 10. Simple Regression

1/35

Simple Linear Regression


2/35


3/35


Regression analysis can be used to develop anequation showing how the variables are related.

Managerial decisions often are based on therelationship between two or more variables.

The variables being used to predict the value of thedependent variable are called the independent

variables and are denoted by x.

The variable being predicted is called the dependentvariable and is denoted by y.


4/35


The relationship between the two variables isapproximated by a straight line.

Simple linear regression involves one independentvariable and one dependent variable.

Regression analysis involving two or moreindependent variables is called multiple regression.


5/35

Simple Linear Regression Model

y = b 0 + b 1x +e

where: b 0 and b 1 are called parameters of the model,e is a random variable called the error term.

The simple linear regression model is:

The equation that describes how y is related to x andan error term is called the regression model.


6/35

Simple Linear Regression Equation

The simple linear regression equation is:

E(y) is the expected value of y for a given x value.

b 1 is the slope of the regression line.

b 0 is the y intercept of the regression line.

Graph of the regression equation is a straight line.

E(y) = b 0 + b 1x


7/35

Estimated Simple Linear Regression Equation

The estimated simple linear regression equation

0 1

y b b x

is the estimated value of y for a given x value.

y

b1 is the slope of the line.

b0 is the y intercept of the line.

The graph is called the estimated regression line.


8/35

Estimation Process

Regression Modely = b 0 + b 1x +e

Regression EquationE(y) = b 0 + b 1x

Unknown Parameters b 0, b 1

Sample Data:x yx1 y1

. .

. . xn yn

b0 and b

1

provide estimates of b 0 and b 1

EstimatedRegression Equation

Sample Statisticsb0, b1

0 1

y b b x


9/35

Least Squares Method

Least Squares Criterion

min ( y yi i

)2

where:

yi = observed value of the dependent variablefor the ith observation

yi = estimated value of the dependent variablefor the ith observation


10/35

Slope for the Estimated Regression Equation

1 2

( )( )( )

i i

i

x x y yb

x x


where:xi = value of independent variable for ith

observation

_y = mean value for dependent variable

_x = mean value for independent variable

yi = value of dependent variable for ithobservation


11/35

y-Intercept for the Estimated Regression Equation


0 1b y b x


12/35

Kataria Auto periodically has a special week-longsale. As part of the advertising campaign Kataria runsone or more television commercials during theweekend preceding the sale. Data from a sample of 5previous sales are shown on the next slide.


Example: Kataria Auto Sales


13/35


Example: Kataria Auto Sales

Number ofTV Ads ( x)

Number ofCars Sold ( y)

1

3213

14

24181727

S x = 10 S y = 1002x 20y


14/35

Estimated Regression Equation

10 5y x

1 2

( )( ) 20 5( ) 4

i i

i

x x y yb

x x

0 1 20 5(2) 10b y b x

Slope for the Estimated Regression Equation

y-Intercept for the Estimated Regression Equation

Estimated Regression Equation


15/35

Coefficient of Determination

Relationship Among SST, SSR, SSE

where:SST = total sum of squares SSR = sum of squares due to regression SSE = sum of squares due to error

SST = SSR + SSE

2( )iy y2( )iy y

2( )i iy y


16/35

The coefficient of determination is:


where:

SSR = sum of squares due to regressionSST = total sum of squares

r 2 = SSR/SST


17/35


r 2 = SSR/SST = 100/114 = .8772

The regression relationship is very strong; 87.72%of the variability in the number of cars sold can be

explained by the linear relationship between thenumber of TV ads and the number of cars sold.


18/35

Sample Correlation Coefficient

21 )of (sign r br xy

xyr 1(sign of ) Coefficient of Determination b

where: b1 = the slope of the estimated regressionequation xbb y 10


19/35

21 )of (sign r br xy

The sign of b1 in the equation is +.

10 5y x

= + .8772xyr

Sample Correlation Coefficient

r xy = +.9366


20/35

Assumptions About the Error Term e

1. The error e is a random variable with mean of zero.2. The variance of e , denoted by 2, is the same for

all values of the independent variable.

3. The values of e are independent.

4. The error e is a normally distributed randomvariable.


21/35

Testing for Significance

To test for a significant regression relationship, wemust conduct a hypothesis test to determine whetherthe value of b 1 is zero.

Two tests are commonly used:t Test and F Test

Both the t test and F test require an estimate of 2,the variance of e in the regression model.


22/35

An Estimate of 2


2102 )()

(SSE iiii xbb y y y

where:

s 2 = MSE = SSE/( n 2)

The mean square error (MSE) provides the estimateof 2, and the notation s2 is also used.


23/35


An Estimate of

2SSE

MSEn

s

To estimate we take the square root of 2. The resulting s is called the standard error of

the estimate.


24/35

Hypotheses

Test Statistic

Testing for Significance: t Test

0 1: 0H b

1: 0aH b

1

1

b

bts

where1 2

( )

b

i

ss

x xS


25/35

Rejection Rule


where:t is based on a t distributionwith n - 2 degrees of freedom

Reject H 0 if p-value < or t < -t or t > t


26/35

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

= .05

4. State the rejection rule. Reject H 0 if p-value < .05or | t| > 3.182 (with3 degrees of freedom)


0 1: 0H b 1: 0aH b

1

1

b

bts


27/35


5. Compute the value of the test statistic.

6. Determine whether to reject H 0.t = 4.541 provides an area of .01 in the uppertail. Hence, the p-value is less than .02. (Also,t = 4.63 > 3.182.) We can reject H 0.

1

1 5 4.631.08b

bts


28/35

Confidence Interval for b 1

H 0 is rejected if the hypothesized value of b 1 is notincluded in the confidence interval for b 1.

We can use a 95% confidence interval for b 1 to testthe hypotheses just used in the t test.


29/35

The form of a confidence interval for b 1 is:


11 /2 bb t s

where is the t value providing an area

of /2 in the upper tail of a t distributionwith n - 2 degrees of freedom

2/ t

b1 is thepoint

estima tor

is themarginof error

1/2 bt s


30/35


Reject H 0 if 0 is not included inthe confidence interval for b 1.

0 is not included in the confidence interval.Reject H 0

= 5 +/- 3.182(1.08) = 5 +/- 3.4412/1 b st b or 1.56 to 8.44

Rejection Rule

95% Confidence Interval for b 1

Conclusion


31/35

Hypotheses

Test Statistic

Testing for Significance: F Test

F = MSR/MSE

0 1: 0H b

1: 0aH b


32/35

Rejection Rule


where:F is based on an F distribution with1 degree of freedom in the numerator andn - 2 degrees of freedom in the denominator

Reject H 0 if p-value F


33/35

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

= .05

4. State the rejection rule. Reject H 0 if p-value < .05

or F > 10.13 (with 1 d.f.in numerator and3 d.f. in denominator)


0 1: 0H b

1: 0aH b

F = MSR/MSE


34/35


5. Compute the value of the test statistic.

6. Determine whether to reject H 0.

F = 17.44 provides an area of .025 in the uppertail. Thus, the p-value corresponding to F = 21.43is less than 2(.025) = .05. Hence, we reject H 0.

F = MSR/MSE = 100/4.667 = 21.43

The statistical evidence is sufficient to concludethat we have a significant relationship between thenumber of TV ads aired and the number of cars sold.

S C i b h


35/35

Some Cautions about theInterpretation of Significance Tests

Just because we are able to reject H 0: b

1 = 0 and

demonstrate statistical significance does not enableus to conclude that there is a linear relationshipbetween x and y.

Rejecting H 0: b 1 = 0 and concluding that therelationship between x and y is significant doesnot enable us to conclude that a cause-and-effectrelationship is present between x and y.

10. Simple Regression

Documents

Transcript of 10. Simple Regression