of 35

himanshu-jain
• Category

## Documents

• view

234

0

### Transcript of 10. Simple Regression

• 8/11/2019 10. Simple Regression

1/35

Simple Linear Regression

• 8/11/2019 10. Simple Regression

2/35

• 8/11/2019 10. Simple Regression

3/35

Simple Linear Regression

Regression analysis can be used to develop anequation showing how the variables are related.

Managerial decisions often are based on therelationship between two or more variables.

The variables being used to predict the value of thedependent variable are called the independent

variables and are denoted by x.

The variable being predicted is called the dependentvariable and is denoted by y.

• 8/11/2019 10. Simple Regression

4/35

Simple Linear Regression

The relationship between the two variables isapproximated by a straight line.

Simple linear regression involves one independentvariable and one dependent variable.

Regression analysis involving two or moreindependent variables is called multiple regression.

• 8/11/2019 10. Simple Regression

5/35

Simple Linear Regression Model

y = b 0 + b 1x +e

where: b 0 and b 1 are called parameters of the model,e is a random variable called the error term.

The simple linear regression model is:

The equation that describes how y is related to x andan error term is called the regression model.

• 8/11/2019 10. Simple Regression

6/35

Simple Linear Regression Equation

The simple linear regression equation is:

E(y) is the expected value of y for a given x value.

b 1 is the slope of the regression line.

b 0 is the y intercept of the regression line.

Graph of the regression equation is a straight line.

E(y) = b 0 + b 1x

• 8/11/2019 10. Simple Regression

7/35

Estimated Simple Linear Regression Equation

The estimated simple linear regression equation

0 1

y b b x

is the estimated value of y for a given x value.

y

b1 is the slope of the line.

b0 is the y intercept of the line.

The graph is called the estimated regression line.

• 8/11/2019 10. Simple Regression

8/35

Estimation Process

Regression Modely = b 0 + b 1x +e

Regression EquationE(y) = b 0 + b 1x

Unknown Parameters b 0, b 1

Sample Data:x yx1 y1

. .

. . xn yn

b0 and b

1

provide estimates of b 0 and b 1

EstimatedRegression Equation

Sample Statisticsb0, b1

0 1

y b b x

• 8/11/2019 10. Simple Regression

9/35

Least Squares Method

Least Squares Criterion

min ( y yi i

)2

where:

yi = observed value of the dependent variablefor the ith observation

yi = estimated value of the dependent variablefor the ith observation

• 8/11/2019 10. Simple Regression

10/35

Slope for the Estimated Regression Equation

1 2

( )( )( )

i i

i

x x y yb

x x

Least Squares Method

where:xi = value of independent variable for ith

observation

_y = mean value for dependent variable

_x = mean value for independent variable

yi = value of dependent variable for ithobservation

• 8/11/2019 10. Simple Regression

11/35

y-Intercept for the Estimated Regression Equation

Least Squares Method

0 1b y b x

• 8/11/2019 10. Simple Regression

12/35

Kataria Auto periodically has a special week-longsale. As part of the advertising campaign Kataria runsone or more television commercials during theweekend preceding the sale. Data from a sample of 5previous sales are shown on the next slide.

Simple Linear Regression

Example: Kataria Auto Sales

• 8/11/2019 10. Simple Regression

13/35

Simple Linear Regression

Example: Kataria Auto Sales

Number ofTV Ads ( x)

Number ofCars Sold ( y)

1

3213

14

24181727

S x = 10 S y = 1002x 20y

• 8/11/2019 10. Simple Regression

14/35

Estimated Regression Equation

10 5y x

1 2

( )( ) 20 5( ) 4

i i

i

x x y yb

x x

0 1 20 5(2) 10b y b x

Slope for the Estimated Regression Equation

y-Intercept for the Estimated Regression Equation

Estimated Regression Equation

• 8/11/2019 10. Simple Regression

15/35

Coefficient of Determination

Relationship Among SST, SSR, SSE

where:SST = total sum of squares SSR = sum of squares due to regression SSE = sum of squares due to error

SST = SSR + SSE

2( )iy y2( )iy y

2( )i iy y

• 8/11/2019 10. Simple Regression

16/35

The coefficient of determination is:

Coefficient of Determination

where:

SSR = sum of squares due to regressionSST = total sum of squares

r 2 = SSR/SST

• 8/11/2019 10. Simple Regression

17/35

Coefficient of Determination

r 2 = SSR/SST = 100/114 = .8772

The regression relationship is very strong; 87.72%of the variability in the number of cars sold can be

explained by the linear relationship between thenumber of TV ads and the number of cars sold.

• 8/11/2019 10. Simple Regression

18/35

Sample Correlation Coefficient

21 )of (sign r br xy

xyr 1(sign of ) Coefficient of Determination b

where: b1 = the slope of the estimated regressionequation xbb y 10

• 8/11/2019 10. Simple Regression

19/35

21 )of (sign r br xy

The sign of b1 in the equation is +.

10 5y x

= + .8772xyr

Sample Correlation Coefficient

r xy = +.9366

• 8/11/2019 10. Simple Regression

20/35

Assumptions About the Error Term e

1. The error e is a random variable with mean of zero.2. The variance of e , denoted by 2, is the same for

all values of the independent variable.

3. The values of e are independent.

4. The error e is a normally distributed randomvariable.

• 8/11/2019 10. Simple Regression

21/35

Testing for Significance

To test for a significant regression relationship, wemust conduct a hypothesis test to determine whetherthe value of b 1 is zero.

Two tests are commonly used:t Test and F Test

Both the t test and F test require an estimate of 2,the variance of e in the regression model.

• 8/11/2019 10. Simple Regression

22/35

An Estimate of 2

Testing for Significance

2102 )()

(SSE iiii xbb y y y

where:

s 2 = MSE = SSE/( n 2)

The mean square error (MSE) provides the estimateof 2, and the notation s2 is also used.

• 8/11/2019 10. Simple Regression

23/35

Testing for Significance

An Estimate of

2SSE

MSEn

s

To estimate we take the square root of 2. The resulting s is called the standard error of

the estimate.

• 8/11/2019 10. Simple Regression

24/35

Hypotheses

Test Statistic

Testing for Significance: t Test

0 1: 0H b

1: 0aH b

1

1

b

bts

where1 2

( )

b

i

ss

x xS

• 8/11/2019 10. Simple Regression

25/35

Rejection Rule

Testing for Significance: t Test

where:t is based on a t distributionwith n - 2 degrees of freedom

Reject H 0 if p-value < or t < -t or t > t

• 8/11/2019 10. Simple Regression

26/35

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

= .05

4. State the rejection rule. Reject H 0 if p-value < .05or | t| > 3.182 (with3 degrees of freedom)

Testing for Significance: t Test

0 1: 0H b 1: 0aH b

1

1

b

bts

• 8/11/2019 10. Simple Regression

27/35

Testing for Significance: t Test

5. Compute the value of the test statistic.

6. Determine whether to reject H 0.t = 4.541 provides an area of .01 in the uppertail. Hence, the p-value is less than .02. (Also,t = 4.63 > 3.182.) We can reject H 0.

1

1 5 4.631.08b

bts

• 8/11/2019 10. Simple Regression

28/35

Confidence Interval for b 1

H 0 is rejected if the hypothesized value of b 1 is notincluded in the confidence interval for b 1.

We can use a 95% confidence interval for b 1 to testthe hypotheses just used in the t test.

• 8/11/2019 10. Simple Regression

29/35

The form of a confidence interval for b 1 is:

Confidence Interval for b 1

11 /2 bb t s

where is the t value providing an area

of /2 in the upper tail of a t distributionwith n - 2 degrees of freedom

2/ t

b1 is thepoint

estima tor

is themarginof error

1/2 bt s

• 8/11/2019 10. Simple Regression

30/35

Confidence Interval for b 1

Reject H 0 if 0 is not included inthe confidence interval for b 1.

0 is not included in the confidence interval.Reject H 0

= 5 +/- 3.182(1.08) = 5 +/- 3.4412/1 b st b or 1.56 to 8.44

Rejection Rule

95% Confidence Interval for b 1

Conclusion

• 8/11/2019 10. Simple Regression

31/35

Hypotheses

Test Statistic

Testing for Significance: F Test

F = MSR/MSE

0 1: 0H b

1: 0aH b

• 8/11/2019 10. Simple Regression

32/35

Rejection Rule

Testing for Significance: F Test

where:F is based on an F distribution with1 degree of freedom in the numerator andn - 2 degrees of freedom in the denominator

Reject H 0 if p-value F

• 8/11/2019 10. Simple Regression

33/35

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

= .05

4. State the rejection rule. Reject H 0 if p-value < .05

or F > 10.13 (with 1 d.f.in numerator and3 d.f. in denominator)

Testing for Significance: F Test

0 1: 0H b

1: 0aH b

F = MSR/MSE

• 8/11/2019 10. Simple Regression

34/35

Testing for Significance: F Test

5. Compute the value of the test statistic.

6. Determine whether to reject H 0.

F = 17.44 provides an area of .025 in the uppertail. Thus, the p-value corresponding to F = 21.43is less than 2(.025) = .05. Hence, we reject H 0.

F = MSR/MSE = 100/4.667 = 21.43

The statistical evidence is sufficient to concludethat we have a significant relationship between thenumber of TV ads aired and the number of cars sold.

S C i b h

• 8/11/2019 10. Simple Regression

35/35

Some Cautions about theInterpretation of Significance Tests

Just because we are able to reject H 0: b

1 = 0 and

demonstrate statistical significance does not enableus to conclude that there is a linear relationshipbetween x and y.

Rejecting H 0: b 1 = 0 and concluding that therelationship between x and y is significant doesnot enable us to conclude that a cause-and-effectrelationship is present between x and y.