Linear Discriminant Analysis and Logistic Regression

Background

Linear Discriminant Analysis predicts a categorical variable based on one or more metric independent variables

Example

Age Purchase18 019 020 021 022 124 025 026 027 129 130 0

65 168 170 1

Data

Age

Pur

chas

e

Consider purchase data compared to a person’s age. A 0 value for Purchase represents someone who didn’t buy, while a 1 represents someone who did.

Graph Interpretation

Potential customers who did purchase

Age

Pur

chas

e

Potential customers who did not purchase

Graphical Representation

Age

Pur

chas

e

A discriminant analysis fits a linear regression to this data as though the categorical variable was numerical.

Graphical Representation ctd.

Age

Pur

chas

e

Then the Discriminant Analysis determines a cutoff score.

For a single predictor variable, this score is where the regression line is equal to.5. Any data points to the left of the line are predicted to be 0, while those to the right are predicted to be 1.

For this data, any potential customer below the age of 41 is predicted not to buy, while anyone older is predicted to buy.

A 100% Accurate Discriminate Analysis

Even a discriminant analysis that provides perfect separation between purchasers and non-purchasers does not have a perfect R .2

Classification Accuracy

Standard Error measures the distance of the predicted value (the regression line) from the observed values. Even data points that are correctly predicted will contribute to the error calculation.

Classification accuracy is a better measure.

This distance will lower the total R , even though it is a correct classification.

2

Discriminant Analysis in StatTools

Discriminant Analysis in StatTools

StatTools – Interpreting OutputA

ctu

al valu

es

Predicted Values

Correct Predictions

StatTools – Interpreting Output ctd.A

ctu

al valu

es

Predicted Values

False Negatives

False Positives

Overall Accuracy

Logistic Regression

A logistic regression fits a sigmoid, or S-shaped curve instead of a straight line. On some datasets, this will provide greater classification accuracy.

pp

pp

XXX

XXX

e

e

....

....

22110

22110

1)(x

Logistic Regression in StatTools

Logistic Regression in StatTools

StatTools – Interpreting Output

Age is highly statistically significant

Overall Accuracy

Comparison

Discriminant Analysis Can be used for

dependent variables with more than 2 possible values

Logistic Regression Less reliant on basic

assumptions of the data like normality and constant variance

More accurate on borderline points for some datasets