Regression:Regression:Independent Independent

and Dependent and Dependent VariablesVariables

Euro Cup Round 4(Final Results)

1. From the Homeworks page.

2. Do at least 1 ANOVA problem and at least 1 Regression problem by the end of class.

In-Class Activity(Due By: 5pm today)

• Go to Homeworks page, select Survey 2.

• Answer the 10 simple rating questions for the four familiar characters.

• Pay attention to the questions and your answers – there are attention checks…

Applications ofMultiple Regression

Forecasting:

Weather

Sales

Stock prices

Dating Services

Rating Responses:

What is your major?

How much money do you want to make?

Is religion important to you?

Do you prefer video games over movies?

Are you looking for a serious relationship?

Are you a morning person or a night person?

Dependent Variable: Length of relationship.

r R2 R2

-1.0 1.0 100%

-0.9 0.81 81%

-0.75 0.56 56%

-0.5 0.25 25%

-0.25 0.06 6%

0 0 0%

0.25 0.06 6%

0.5 0.25 25%

0.75 0.56 56%

0.9 0.81 81%

1.0 1.0 100%

One Predictor Variable

2 || Rr

Issues

Adding predictor variables: overfitting (or, “fitting the noise”) and cross-validation.

Mood ≈ Mood in this dataset(Approx. = )

Scatterplot

One Predictor VariableSimple (Linear) Regression

Perfect Fit, R2 = 100% !

Horrible Prediction!

Adding predictor variables to your model gives the model more flexibility.

The model can (and will) always ignore a predictor if it doesn’t help improve the fit to available data.

So, adding a predictor variable CANNOT cause R2 to decrease. R2 will almost always INCREASE, even if the predictor is NOT correlated to the Dependent Variable.

The penalty for adding useless predictors to your model will come when you use the fitted model to PREDICT NEW datapoints.

What’s Going On?

Check the p Value for the coefficient of the predictor in the Regression Table. It should be ‘small’ (e.g., less than 0.05).

Cross Validation Test: Use half of your dataset to fit the model. Use the other half to see how good you can PREDICT new data.

Compare “Adjusted R Square” values for models with and without that predictor.

How Do Know if a Predictor is‘Fitting to Noise’?

Issues

Adding predictor variables: overfitting (or, “fitting the noise”) and cross-validation.

Multicolinearity. Predictor variables are correlated among themselves (NOT a good thing!).

Miles Per Gallon Predicts Tank Size

Kilometers Per Gallon Predicts Tank Size

Kilometers AND Miles Per Gallon do NOT predict Tank

Size?