Exploratory Factor Analysis. Principal components analysis seeks linear combinations that best...
-
Upload
may-katherine-pearson -
Category
Documents
-
view
220 -
download
2
Transcript of Exploratory Factor Analysis. Principal components analysis seeks linear combinations that best...
Exploratory Factor Analysis
• Principal components analysis seeks linear combinations that best capture the variation in the original variables.
• Factor analysis seeks linear combinations that best capture the correlations among the original variables.
Review: FA vs. PCA
Factor Analysis: Seeks to identify common factors that influence correlations among items, it is not a correlation matrix that is analyzed thus, the main diagonal is replaced by the communalities - the variances in X that are due only to the common factors. But, communalities are not known in advance of the factor analysis, giving rise to the communality problem and the need to solve for the common factors iteratively.
Review: FA vs. PCAPrincipal Components Analysis: The correlation matrix contains ones on the main diagonal, accounting for all of the variance in X—the goal of principal components analysis.
In the principal axes approach to factor analysis, the only difference compared to principal components is that the matrix being analyzed is a correlation matrix (which is also a variance-covariance matrix for standardized variables) in which the main diagonal contains the communalities rather than the variances.
FA: Principal Axes Factoring
• Most of the loadings on any given factor are small and a few loadings are large in absolute value
• Most of the loadings for any given variable are small, with ideally only one loading being large in absolute value.
• Any pair of factors have dissimilar patterns of loadings.
Simple Structure
The ideal loadings are the simple structure, which might look like this for the factor loadings:
To get closer to the ideal, we rotate the factors
F1 F2 F3
X1 1 0 0
X2 1 0 0
X3 1 0 0
X4 0 1 0
X5 0 1 0
X6 0 1 0
X7 0 0 1
X8 0 0 1
X9 0 0 1
Varimax One way to approach this ideal pattern is to find the rotation that maximizes the variance of the loadings in the columns of the factor structure matrix. This approach was suggested by Kaiser and is called varimax rotation.
F1 F2 F3
X1 1 0 0
X2 1 0 0
X3 1 0 0
X4 0 1 0
X5 0 1 0
X6 0 1 0
X7 0 0 1
X8 0 0 1
X9 0 0 1
Quartimax A second way to approach this ideal pattern is to find the rotation that maximizes the variance of the loadings in the rows of the factor structure matrix. This approach is called quartimax rotation.
F1 F2 F3
X1 1 0 0
X2 1 0 0
X3 1 0 0
X4 0 1 0
X5 0 1 0
X6 0 1 0
X7 0 0 1
X8 0 0 1
X9 0 0 1
Equimax For those who want the best of both worlds, equimax rotation attempts to satisfy both goals.
Varimax is the most commonly used and the three rarely produce results that are very discrepant.
F1 F2 F3
X1 1 0 0
X2 1 0 0
X3 1 0 0
X4 0 1 0
X5 0 1 0
X6 0 1 0
X7 0 0 1
X8 0 0 1
X9 0 0 1
Correlation Matrixa
1.000 .447 .422 .435 .114 .203 .091 .082 .513 .304 .245 .101 .245
.447 1.000 .619 .604 .068 .146 .103 .063 .445 .318 .203 .088 .199
.422 .619 1.000 .583 .053 .139 .110 .066 .365 .240 .183 .074 .184
.435 .604 .583 1.000 .115 .258 .122 .097 .482 .368 .255 .139 .293
.114 .068 .053 .115 1.000 .349 .209 .321 .186 .303 .272 .279 .278
.203 .146 .139 .258 .349 1.000 .221 .355 .315 .377 .323 .367 .545
.091 .103 .110 .122 .209 .221 1.000 .201 .150 .163 .310 .232 .232
.082 .063 .066 .097 .321 .355 .201 1.000 .154 .219 .288 .320 .314
.513 .445 .365 .482 .186 .315 .150 .154 1.000 .534 .301 .204 .394
.304 .318 .240 .368 .303 .377 .163 .219 .534 1.000 .302 .368 .467
.245 .203 .183 .255 .272 .323 .310 .288 .301 .302 1.000 .340 .392
.101 .088 .074 .139 .279 .367 .232 .320 .204 .368 .340 1.000 .511
.245 .199 .184 .293 .278 .545 .232 .314 .394 .467 .392 .511 1.000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .003 .000 .000 .005 .000 .000 .000 .000 .000
.000 .000 .000 .016 .000 .000 .004 .000 .000 .000 .001 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .003 .016 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .005 .004 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .001 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
CIGS
BEER
WINE
LIQUOR
COCAINE
TRANQS
DRUGSTR
HEROIN
MARIJUAN
HASHISH
INHALE
HALLUCIN
AMPHETS
CIGS
BEER
WINE
LIQUOR
COCAINE
TRANQS
DRUGSTR
HEROIN
MARIJUAN
HASHISH
INHALE
HALLUCIN
AMPHETS
Correlation
Sig. (1-tailed)
CIGS BEER WINE LIQUOR COCAINE TRANQS DRUGSTR HEROIN MARIJUAN HASHISH INHALE HALLUCIN AMPHETS
Determinant = 1.742E-02a.
Data on drug use reported by 1634 students in Los Angeles. Participants rated their use on a 5-point scale: 1 = never tried, 2 = only once, 3 = a few times, 4 = many times, 5 = regularly.
Example: Principal Components Analysis
FACTOR /VARIABLES cigs beer wine liquor cocaine tranqs drugstr heroin marijuan hashish inhale hallucin amphets /PRINT INITIAL KMO EXTRACTION ROTATION /PLOT EIGEN ROTATION /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION .
Syntax for Factor Analysis
PC = Principal ComponentsPAF = Principal Axis Factoring
KMO asks for a test of multicollinearity – this will tell whether there is enough shared variance to warrant a factor analysis
KMO and Bartlett's Test
.868
6592.802
78
.000
Kaiser-Meyer-Olkin Measure of SamplingAdequacy.
Approx. Chi-Square
df
Sig.
Bartlett's Test ofSphericity
Should the matrix be analyzed?
This can vary between 0 and 1, indicating whether there is sufficient multicollinearity to warrant an analysis. Higher values indicate the desirability of a principal components analysis.
Bartlett's test indicates the correlation matrix is clearly not an identity matrix
Total Variance Explained
4.382 33.705 33.705 4.382 33.705 33.705
2.046 15.736 49.440 2.046 15.736 49.440
.953 7.328 56.769 .953 7.328 56.769
.817 6.288 63.057 .817 6.288 63.057
.766 5.889 68.946 .766 5.889 68.946
.687 5.286 74.231 .687 5.286 74.231
.644 4.950 79.182 .644 4.950 79.182
.615 4.729 83.911 .615 4.729 83.911
.561 4.314 88.225 .561 4.314 88.225
.399 3.067 91.293 .399 3.067 91.293
.395 3.037 94.330 .395 3.037 94.330
.374 2.879 97.209 .374 2.879 97.209
.363 2.791 100.000 .363 2.791 100.000
Component1
2
3
4
5
6
7
8
9
10
11
12
13
Total % of Variance Cumulative % Total % of Variance Cumulative %
Initial Eigenvalues Extraction Sums of Squared Loadings
Extraction Method: Principal Component Analysis.
Two principal components account for nearly half of the information in the original variables
Communalities
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
1.000 1.000
CIGS
BEER
WINE
LIQUOR
COCAINE
TRANQS
DRUGSTR
HEROIN
MARIJUAN
HASHISH
INHALE
HALLUCIN
AMPHETS
Initial Extraction
Extraction Method: Principal Component Analysis.
Initial Extraction All principal components are extracted so all of the variance in the variables is accounted for.
Component Matrix
.582 -.400 -5.81E-02 1.457E-02
.598 -.567 .127 9.098E-02
.555 -.561 .215 .128
.665 -.465 5.158E-02 5.725E-02
.436 .412 5.078E-02 .534
.613 .370 -.167 7.766E-02
.369 .271 .710 -.299
.422 .451 .143 .481
.710 -.234 -.231 -9.89E-02
.688 7.239E-02 -.346 -.107
.578 .242 .306 -.176
.520 .471 -.107 -.260
.688 .332 -.227 -.239
CIGS
BEER
WINE
LIQUOR
COCAINE
TRANQS
DRUGSTR
HEROIN
MARIJUAN
HASHISH
INHALE
HALLUCIN
AMPHETS
1 2 3 4
Component
Extraction Method: Principal Component Analysis.
13 components extracted.a.
Part of the loading matrix. How should the first two principal components be interpreted?
Scree Plot
Component Number
13121110987654321
Eig
enva
lue
5
4
3
2
1
0
Only two principal components are indicated by the scree test.
Total Variance Explained
4.382 33.705 33.705 4.382 33.705 33.705
2.046 15.736 49.440 2.046 15.736 49.440
.953 7.328 56.769
.817 6.288 63.057
.766 5.889 68.946
.687 5.286 74.231
.644 4.950 79.182
.615 4.729 83.911
.561 4.314 88.225
.399 3.067 91.293
.395 3.037 94.330
.374 2.879 97.209
.363 2.791 100.000
Component1
2
3
4
5
6
7
8
9
10
11
12
13
Total % of Variance Cumulative % Total % of Variance Cumulative %
Initial Eigenvalues Extraction Sums of Squared Loadings
Extraction Method: Principal Component Analysis.
Only two principal components are indicated by the Kaiser rule as well.
Communalities
1.000 .499
1.000 .680
1.000 .622
1.000 .658
1.000 .360
1.000 .513
1.000 .209
1.000 .381
1.000 .559
1.000 .479
1.000 .393
1.000 .492
1.000 .583
CIGS
BEER
WINE
LIQUOR
COCAINE
TRANQS
DRUGSTR
HEROIN
MARIJUAN
HASHISH
INHALE
HALLUCIN
AMPHETS
Initial Extraction
Extraction Method: Principal Component Analysis.
With only two principal components, much less of the variance in each variable is accounted for.
Component Matrixa
.582 -.400
.598 -.567
.555 -.561
.665 -.465
.436 .412
.613 .370
.369 .271
.422 .451
.710 -.234
.688 7.239E-02
.578 .242
.520 .471
.688 .332
CIGS
BEER
WINE
LIQUOR
COCAINE
TRANQS
DRUGSTR
HEROIN
MARIJUAN
HASHISH
INHALE
HALLUCIN
AMPHETS
1 2
Component
Extraction Method: Principal Component Analysis.
2 components extracted.a.
The loadings for the first two components do not change when they are the only ones extracted.
Correlation Matrixa
1.000 .447 .422 .435 .114 .203 .091 .082 .513 .304 .245 .101 .245
.447 1.000 .619 .604 .068 .146 .103 .063 .445 .318 .203 .088 .199
.422 .619 1.000 .583 .053 .139 .110 .066 .365 .240 .183 .074 .184
.435 .604 .583 1.000 .115 .258 .122 .097 .482 .368 .255 .139 .293
.114 .068 .053 .115 1.000 .349 .209 .321 .186 .303 .272 .279 .278
.203 .146 .139 .258 .349 1.000 .221 .355 .315 .377 .323 .367 .545
.091 .103 .110 .122 .209 .221 1.000 .201 .150 .163 .310 .232 .232
.082 .063 .066 .097 .321 .355 .201 1.000 .154 .219 .288 .320 .314
.513 .445 .365 .482 .186 .315 .150 .154 1.000 .534 .301 .204 .394
.304 .318 .240 .368 .303 .377 .163 .219 .534 1.000 .302 .368 .467
.245 .203 .183 .255 .272 .323 .310 .288 .301 .302 1.000 .340 .392
.101 .088 .074 .139 .279 .367 .232 .320 .204 .368 .340 1.000 .511
.245 .199 .184 .293 .278 .545 .232 .314 .394 .467 .392 .511 1.000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .003 .000 .000 .005 .000 .000 .000 .000 .000
.000 .000 .000 .016 .000 .000 .004 .000 .000 .000 .001 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .003 .016 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .005 .004 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .001 .000 .000 .000 .000 .000 .000 .000 .000 .000
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
Cigarettes
Beer
Wine
Liquor
Cocaine
Tranquilizers
Drug Store Medication
Heroin
Marijuana
Hashish
Inhalants
Hallucinogenics
Amphetamines
Cigarettes
Beer
Wine
Liquor
Cocaine
Tranquilizers
Drug Store Medication
Heroin
Marijuana
Hashish
Inhalants
Hallucinogenics
Amphetamines
Correlation
Sig. (1-tailed)
Cigarettes Beer Wine Liquor Cocaine TranquilizersDrug StoreMedication Heroin Marijuana Hashish Inhalants
Hallucinogenics
Amphetamines
Determinant = 1.742E-02a.
Drug use data by 1634 students. Participants rated their use on a 5-point scale: 1 = never tried, 2 = only once, 3 = a few times, 4 = many times, 5 = regularly.
Common Factor Analysis
KMO and Bartlett's Test
.868
6592.802
78
.000
Kaiser-Meyer-Olkin Measure of SamplingAdequacy.
Approx. Chi-Square
df
Sig.
Bartlett's Test ofSphericity
The analysis begins in the same way as principal components analysis. It would make little sense to search for common factors in an identity matrix:
Communalities
.354 .380
.504 .617
.465 .524
.498 .589
.215 .246
.382 .440
.136 .135
.217 .258
.476 .478
.415 .416
.278 .310
.337 .393
.478 .545
Cigarettes
Beer
Wine
Liquor
Cocaine
Tranquilizers
Drug Store Medication
Heroin
Marijuana
Hashish
Inhalants
Hallucinogenics
Amphetamines
Initial Extraction
Extraction Method: Principal Axis Factoring.
Unlike principal components analysis, factor analysis will not attempt to explain all of the variance in each variable. Only common factor variance is of interest. This creates the need for some initial estimates of communalities.
Total Variance Explained
4.382 33.705 33.705 3.832 29.477 29.477 2.687 20.670 20.670
2.046 15.736 49.440 1.498 11.526 41.003 2.643 20.333 41.003
.953 7.328 56.769
.817 6.288 63.057
.766 5.889 68.946
.687 5.286 74.231
.644 4.950 79.182
.615 4.729 83.911
.561 4.314 88.225
.399 3.067 91.293
.395 3.037 94.330
.374 2.879 97.209
.363 2.791 100.000
Factor1
2
3
4
5
6
7
8
9
10
11
12
13
Total % of Variance Cumulative % Total % of Variance Cumulative % Total % of Variance Cumulative %
Initial Eigenvalues Extraction Sums of Squared Loadings Rotation Sums of Squared Loadings
Extraction Method: Principal Axis Factoring.
The number of factors to extract is guided by the size of the eigenvalues, as it was in principal components analysis. But, not all of the variance can be accounted for in the variables.
Total Variance Explained
4.382 33.705 33.705 3.832 29.477 29.477 2.687
2.046 15.736 49.440 1.498 11.526 41.003 2.643
.953 7.328 56.769
Factor1
2
3
Total % of Variance Cumulative % Total % of Variance Cumulative % Total
Initial Eigenvalues Extraction Sums of Squared Loadings Rotation Sums of Squared Loadings
To the extent there is random error in the measures, the eigenvalues for factor analysis will be smaller than the corresponding eigenvalues in principal components analysis.
Remember, this is because PCA assumes no error of measurement, whereas FA assumes there is measurement error.
Total Variance Explained
33.705 3.832 29.477 29.477 2.687 20.670 20.670
49.440 1.498 11.526 41.003 2.643 20.333 41.003
56.769
Cumulative % Total % of Variance Cumulative % Total % of Variance Cumulative %
Initial Eigenvalues Extraction Sums of Squared Loadings Rotation Sums of Squared Loadings
The location of the factors might be rotated to a position that allows easier interpretation. This will shift the variance, but preserve the total amount accounted for.
Scree Plot
Factor Number
13121110987654321
Eig
enva
lue
5
4
3
2
1
0
Two factors appear to be sufficient. The attenuated eigenvalues will generally tell the same story.
Factor Matrixa
.541 -.296
.594 -.514
.539 -.483
.653 -.403
.379 .320
.566 .346
.312 .194
.368 .350
.672 -.160
.637 9.973E-02
.516 .209
.472 .414
.654 .343
Cigarettes
Beer
Wine
Liquor
Cocaine
Tranquilizers
Drug Store Medication
Heroin
Marijuana
Hashish
Inhalants
Hallucinogenics
Amphetamines
1 2
Factor
Extraction Method: Principal Axis Factoring.
2 factors extracted. 6 iterations required.a.
Component Matrix
.582 -.400 -5.81E-02
.598 -.567 .127
.555 -.561 .215
.665 -.465 5.158E-02
.436 .412 5.078E-02
.613 .370 -.167
.369 .271 .710
.422 .451 .143
.710 -.234 -.231
.688 7.239E-02 -.346
.578 .242 .306
.520 .471 -.107
.688 .332 -.227
CIGS
BEER
WINE
LIQUOR
COCAINE
TRANQS
DRUGSTR
HEROIN
MARIJUAN
HASHISH
INHALE
HALLUCIN
AMPHETS
1 2 3
Component
Extraction Method: Principal Component Analysis.
13 components extracted.a.
The loadings will be reduced in FA compared to PCA because not all of the variance in X is due to common factor variance.
PCA FA
Rotated Factor Matrixa
.593 .168
.784 4.961E-02
.723 3.232E-02
.749 .170
4.598E-02 .494
.162 .643
8.689E-02 .357
1.704E-02 .507
.592 .356
.385 .517
.222 .511
4.697E-02 .626
.226 .703
Cigarettes
Beer
Wine
Liquor
Cocaine
Tranquilizers
Drug Store Medication
Heroin
Marijuana
Hashish
Inhalants
Hallucinogenics
Amphetamines
1 2
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 3 iterations.a.
Rotating the factors to simple structure makes the interpretation easier. The first factor appears to be minor “recreational” drug use. The second factor appears to be major “abusive” drug use.
A sample of 303 MBA students were asked to evaluate different makes of cars using 16 different adjectives rated on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree):
“This car is an exciting car.”
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
New Example: FA
Descriptive Statistics
3.21 1.218 294
3.65 1.020 294
3.48 1.107 294
2.68 1.398 294
3.60 1.016 294
3.44 1.237 294
3.70 .851 294
2.53 1.329 294
3.48 1.263 294
3.75 .979 294
3.44 1.181 294
3.46 1.286 294
3.08 1.093 294
3.03 1.318 294
3.57 1.183 294
3.35 1.106 294
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
Mean Std. Deviation Analysis N
Are all 16 individual ratings required to understand product evaluation, or, is there a simpler measurement model?
Communalities
.773 .784
.414 .437
.644 .672
.687 .747
.595 .600
.767 .778
.520 .550
.701 .753
.761 .763
.539 .596
.678 .651
.720 .778
.529 .537
.671 .690
.734 .751
.581 .571
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
Initial Extraction
Extraction Method: Principal Axis Factoring.
The analysis must begin with some idea of the proportion of variance in each variable that can be attributed to the common factors. The most common initial estimate is the squared multiple correlation between a given measure and all the remaining measures.
Scree Plot
Component Number
16151413121110987654321
Eig
enva
lue
7
6
5
4
3
2
1
0
Three common factors appear to underlie the 16 evaluative ratings.
Total Variance Explained
5.927 37.042 37.042 5.644 35.275 35.275 5.632 35.198 35.198
3.187 19.920 56.961 2.791 17.446 52.721 2.624 16.403 51.601
2.542 15.887 72.849 2.222 13.885 66.606 2.401 15.005 66.606
.648 4.052 76.901
.589 3.684 80.585
.477 2.983 83.568
.406 2.539 86.107
.369 2.307 88.414
.337 2.107 90.521
.296 1.852 92.373
.277 1.729 94.102
.242 1.510 95.613
.205 1.282 96.895
.185 1.154 98.049
.169 1.054 99.103
.144 .897 100.000
Factor1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Total % of Variance Cumulative % Total % of Variance Cumulative % Total % of Variance Cumulative %
Initial Eigenvalues Extraction Sums of Squared Loadings Rotation Sums of Squared Loadings
Extraction Method: Principal Axis Factoring.
The three common factors account for two-thirds of the common factor variance.
Factor Matrixa
.863 9.283E-03 .199
.123 .512 -.400
.636 .244 -.457
4.746E-02 .439 .743
.695 .290 .178
.873 9.610E-02 -8.02E-02
.148 .617 -.385
5.676E-02 .486 .716
.860 6.511E-02 .140
-7.07E-02 .671 -.376
.772 6.326E-02 -.227
-.675 .565 -5.54E-02
-.237 .634 .281
.691 2.867E-03 .461
.829 .109 -.229
-.439 .613 -5.37E-02
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring.
3 factors extracted. 6 iterations required.a.
Without rotation, the interpretation of the factors is not readily apparent, especially the second and third factors.
Rotated Factor Matrixa
.864 -.142 .132
.150 .641 -5.82E-02
.644 .425 -.276
8.497E-02 -4.57E-02 .859
.715 .111 .276
.876 8.355E-02 -5.42E-02
.182 .719 1.137E-02
9.692E-02 7.389E-03 .862
.864 -6.28E-02 .113
-3.23E-02 .769 5.788E-02
.772 .142 -.190
-.637 .533 .295
-.192 .384 .594
.695 -.284 .355
.831 .178 -.170
-.399 .560 .312
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 5 iterations.a.
The varimax rotation makes the interpretation a bit easier. What would you name these factors?
Factor 1?
Factor 2?
Factor 3?
It is rare for the different rotational criteria to produce different results
Rotated Factor Matrixa
.861 -.142 .147
.152 .640 -6.41E-02
.649 .421 -.272
7.177E-02 -3.48E-02 .860
.711 .114 .285
.877 8.168E-02 -4.19E-02
.183 .719 4.946E-03
8.375E-02 1.832E-02 .863
.862 -6.25E-02 .127
-3.20E-02 .770 4.752E-02
.775 .138 -.180
-.641 .537 .279
-.201 .392 .586
.689 -.280 .369
.834 .175 -.159
-.403 .565 .299
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Quartimax with Kaiser Normalization.
Rotation converged in 4 iterations.a.
Rotated Factor Matrixa
.864 -.142 .132
.150 .641 -5.82E-02
.644 .425 -.276
8.497E-02 -4.57E-02 .859
.715 .111 .276
.876 8.355E-02 -5.42E-02
.182 .719 1.137E-02
9.692E-02 7.389E-03 .862
.864 -6.28E-02 .113
-3.23E-02 .769 5.788E-02
.772 .142 -.190
-.637 .533 .295
-.192 .384 .594
.695 -.284 .355
.831 .178 -.170
-.399 .560 .312
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 5 iterations.a.
Rotated Factor Matrixa
.866 -.141 .120
.148 .642 -5.50E-02
.640 .429 -.281
9.554E-02 -5.22E-02 .857
.718 .111 .268
.875 8.611E-02 -6.42E-02
.180 .719 1.477E-02
.107 9.192E-04 .861
.865 -6.15E-02 .102
-3.34E-02 .769 6.427E-02
.769 .145 -.198
-.635 .529 .307
-.186 .379 .599
.700 -.285 .344
.829 .181 -.178
-.397 .557 .321
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Equamax with Kaiser Normalization.
Rotation converged in 6 iterations.a.
• Advocates of factor analysis often claim that it is inappropriate to apply principal components procedures in the search for meaning or latent constructs.
• But, does it really matter all that much?
• To the extent that the communalities for all variables are high the two procedures should give very similar results.
• When commonalities are very low factor analysis results may depart from principal components.
PCA vs. FA
Communalities
1.000 .806
1.000 .572
1.000 .728
1.000 .807
1.000 .669
1.000 .798
1.000 .652
1.000 .808
1.000 .790
1.000 .686
1.000 .703
1.000 .794
1.000 .659
1.000 .750
1.000 .778
1.000 .656
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
Initial Extraction
Extraction Method: Principal Component Analysis.
Communalities
.773 .784
.414 .437
.644 .672
.687 .747
.595 .600
.767 .778
.520 .550
.701 .753
.761 .763
.539 .596
.678 .651
.720 .778
.529 .537
.671 .690
.734 .751
.581 .571
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
Initial Extraction
Extraction Method: Principal Axis Factoring.
PCA FA
Component Matrixa
.873 -5.03E-03 .209
.134 .609 -.427
.658 .285 -.463
4.710E-02 .394 .806
.728 .296 .228
.885 .101 -6.72E-02
.157 .697 -.375
5.653E-02 .443 .780
.873 5.610E-02 .154
-7.26E-02 .744 -.357
.801 8.172E-02 -.233
-.684 .572 -9.99E-03
-.251 .671 .382
.710 -2.92E-02 .494
.845 .124 -.221
-.462 .665 -2.53E-03
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Component
Extraction Method: Principal Component Analysis.
3 components extracted.a.
Factor Matrixa
.863 9.283E-03 .199
.123 .512 -.400
.636 .244 -.457
4.746E-02 .439 .743
.695 .290 .178
.873 9.610E-02 -8.02E-02
.148 .617 -.385
5.676E-02 .486 .716
.860 6.511E-02 .140
-7.07E-02 .671 -.376
.772 6.326E-02 -.227
-.675 .565 -5.54E-02
-.237 .634 .281
.691 2.867E-03 .461
.829 .109 -.229
-.439 .613 -5.37E-02
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring.
3 factors extracted. 6 iterations required.a.
PCA FA
Rotated Component Matrixa
.876 -.143 .133
.159 .736 -7.42E-02
.661 .452 -.294
9.146E-02 -6.30E-02 .892
.750 .114 .306
.887 8.631E-02 -5.43E-02
.189 .785 1.353E-02
.103 -8.27E-03 .893
.879 -6.26E-02 .116
-3.71E-02 .825 6.459E-02
.798 .156 -.203
-.648 .526 .311
-.201 .400 .678
.720 -.300 .376
.845 .185 -.174
-.421 .595 .353
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Component
Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 6 iterations.a.
Rotated Factor Matrixa
.864 -.142 .132
.150 .641 -5.82E-02
.644 .425 -.276
8.497E-02 -4.57E-02 .859
.715 .111 .276
.876 8.355E-02 -5.42E-02
.182 .719 1.137E-02
9.692E-02 7.389E-03 .862
.864 -6.28E-02 .113
-3.23E-02 .769 5.788E-02
.772 .142 -.190
-.637 .533 .295
-.192 .384 .594
.695 -.284 .355
.831 .178 -.170
-.399 .560 .312
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 5 iterations.a.
PCA FA
• Factor analysis offers a more realistic model of measurement than principal components analysis by admitting the presence of random and systematic error.
• Another way to make the model more realistic is to relax the restriction that factors be orthogonal. Allowing oblique factors has two potential benefits:
• It allows the model to better match the actual data
• It allows the possibility of higher order factors
FA: Oblique Rotation
A sample of 303 MBA students were asked to evaluate different makes of cars using 16 different adjectives rated on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree):
“This car is an exciting car.”
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
FA Example, Oblique Rotation
Rotated Factor Matrixa
.864 -.142 .132
.150 .641 -5.82E-02
.644 .425 -.276
8.497E-02 -4.57E-02 .859
.715 .111 .276
.876 8.355E-02 -5.42E-02
.182 .719 1.137E-02
9.692E-02 7.389E-03 .862
.864 -6.28E-02 .113
-3.23E-02 .769 5.788E-02
.772 .142 -.190
-.637 .533 .295
-.192 .384 .594
.695 -.284 .355
.831 .178 -.170
-.399 .560 .312
Exciting
Dependable
Luxurious
Outdoorsy
Powerful
Stylish
Comfortable
Rugged
Fun
Safe
Performance
Family
Versatile
Sports
Status
Practical
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 5 iterations.a.
• Oblique rotation relaxes the requirement that factors be independent. This requires the addition of a new matrix—the factor pattern matrix.
• With orthogonal rotation, because the factors are independent, the weights are simply correlations---just the elements of the factor loading (structure) matrix.
• When the factors are not orthogonal, the correlations are not the only weights considered (structure matrix) and a separate matrix containing partial weights is necessary (pattern matrix).
Because the factor pattern matrix represents the unique contribution of each factor to the reconstruction of any variable in X, it provides a better basis for judging simple structure.
Available techniques for achieving simple structure for oblique rotation (e.g., promax, direct oblimin) confront an additional problem---the specification for the amount of correlation among the factors. Unlike orthogonal rotation in which these correlations are fixed at 0, in oblique rotation, simple structure can be sought for any correlations among the factors.
Hypothetical data (N = 500) were created for individuals completing a 12-section test of mental abilities. All variables are in standard form.
Descriptive Statistics
-.0076 .97648 500
-.0038 1.06328 500
-.0148 .99894 500
-.0878 1.08634 500
.0106 .97891 500
-.0286 .96150 500
-.0151 .98903 500
-.0063 .96384 500
-.0168 .93611 500
.0008 .97184 500
-.0046 .98339 500
-.0629 1.01653 500
Grammar
ParagraphComprehension
Vocabulary
Sentence Completion
Geometry
Algebra
Numerical Puzzles
Series Completion
Practical Problem Solving
Symbol Manipulation
Analytical Ability
Formal Logic
Mean Std. Deviation Analysis N
KMO and Bartlett's Test
.856
1943.111
66
.000
Kaiser-Meyer-Olkin Measure of SamplingAdequacy.
Approx. Chi-Square
df
Sig.
Bartlett's Test ofSphericity
The correlation matrix is not an identity matrix:
Scree Plot
Factor Number
121110987654321
Eig
enva
lue
5
4
3
2
1
0
The scree test clearly shows the presence of three factors:
Communalities
.428 .535
.410 .518
.387 .471
.426 .547
.367 .473
.375 .475
.357 .483
.399 .508
.397 .482
.382 .498
.409 .525
.417 .525
Grammar
ParagraphComprehension
Vocabulary
Sentence Completion
Geometry
Algebra
Numerical Puzzles
Series Completion
Practical Problem Solving
Symbol Manipulation
Analytical Ability
Formal Logic
Initial Extraction
Extraction Method: Principal Axis Factoring.
On average, the three factors extracted can account for about half of the variance in the individual subtests.
Total Variance Explained
4.216 35.137 35.137 3.722 31.013 31.013 2.751
1.717 14.308 49.445 1.209 10.072 41.085 2.566
1.594 13.287 62.733 1.110 9.247 50.332 2.820
.618 5.150 67.883
.566 4.720 72.603
.541 4.512 77.115
.535 4.461 81.576
.470 3.918 85.494
.462 3.850 89.343
.450 3.751 93.094
.436 3.633 96.727
.393 3.273 100.000
Factor1
2
3
4
5
6
7
8
9
10
11
12
Total % of Variance Cumulative % Total % of Variance Cumulative % Total
Initial Eigenvalues Extraction Sums of Squared Loadings RotationSums ofSquaredLoadings
a
Extraction Method: Principal Axis Factoring.
When factors are correlated, sums of squared loadings cannot be added to obtain a total variance.a.
Factor analysis accounts for less variance than principal components and rotation shifts the variance accounted for by the factors.
Why does the sum of the squared rotated loadings not equal the sum of the squared unrotated loadings?
Factor Matrixa
.592 -.165 -.397
.590 -8.05E-02 -.404
.570 -.162 -.347
.579 -.135 -.440
.498 .469 6.766E-02
.554 .403 7.396E-02
.466 .511 6.591E-02
.551 .434 .127
.568 -.263 .300
.525 -.287 .374
.583 -.267 .336
.590 -.275 .319
Grammar
ParagraphComprehension
Vocabulary
Sentence Completion
Geometry
Algebra
Numerical Puzzles
Series Completion
Practical Problem Solving
Symbol Manipulation
Analytical Ability
Formal Logic
1 2 3
Factor
Extraction Method: Principal Axis Factoring.
3 factors extracted. 6 iterations required.a.
The initial extraction . . .
Structure Matrix
.343 .278 -.731
.305 .333 -.718
.349 .270 -.685
.302 .284 -.739
.242 .687 -.276
.315 .685 -.330
.199 .692 -.243
.325 .710 -.292
.693 .278 -.342
.704 .238 -.271
.724 .290 -.335
.724 .287 -.352
Grammar
ParagraphComprehension
Vocabulary
Sentence Completion
Geometry
Algebra
Numerical Puzzles
Series Completion
Practical Problem Solving
Symbol Manipulation
Analytical Ability
Formal Logic
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser Normalization.
Pattern Matrixa
2.531E-02 -2.44E-02 -.730
-3.52E-02 6.073E-02 -.709
5.689E-02 -1.89E-02 -.667
-3.50E-02 -7.43E-03 -.758
-2.10E-02 .692 -7.24E-03
5.106E-02 .647 -4.76E-02
-6.35E-02 .723 1.929E-02
7.498E-02 .689 1.862E-02
.673 1.029E-02 -3.65E-02
.733 -1.64E-02 5.129E-02
.713 1.881E-02 -8.38E-03
.706 9.512E-03 -3.13E-02
Grammar
ParagraphComprehension
Vocabulary
Sentence Completion
Geometry
Algebra
Numerical Puzzles
Series Completion
Practical Problem Solving
Symbol Manipulation
Analytical Ability
Formal Logic
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser Normalization.
Rotation converged in 6 iterations.a.
Oblique rotation is much clearer in the pattern matrix than in the structure matrix:
Factor Correlation Matrix
1.000 .376 -.448
.376 1.000 -.402
-.448 -.402 1.000
Factor1
2
3
1 2 3
Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser Normalization.
The correlations among the factors suggest the presence of a higher order factor:
Why are some of the correlations negative when one would expect all mental abilities to be positively correlated?
Pattern Matrixa
.731 2.489E-02 -2.67E-02
.710 -3.66E-02 5.939E-02
.668 5.664E-02 -2.12E-02
.759 -3.59E-02 -9.24E-03
4.785E-03 -2.72E-02 .696
4.504E-02 4.560E-02 .649
-2.17E-02 -7.01E-02 .727
-2.15E-02 6.933E-02 .692
3.360E-02 .676 4.904E-03
-5.45E-02 .737 -2.22E-02
5.235E-03 .717 1.322E-02
2.829E-02 .710 3.868E-03
Grammar
ParagraphComprehension
Vocabulary
Sentence Completion
Geometry
Algebra
Numerical Puzzles
Series Completion
Practical Problem Solving
Symbol Manipulation
Analytical Ability
Formal Logic
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Promax with Kaiser Normalization.
Rotation converged in 5 iterations.a.
Pattern Matrixa
2.531E-02 -2.44E-02 -.730
-3.52E-02 6.073E-02 -.709
5.689E-02 -1.89E-02 -.667
-3.50E-02 -7.43E-03 -.758
-2.10E-02 .692 -7.24E-03
5.106E-02 .647 -4.76E-02
-6.35E-02 .723 1.929E-02
7.498E-02 .689 1.862E-02
.673 1.029E-02 -3.65E-02
.733 -1.64E-02 5.129E-02
.713 1.881E-02 -8.38E-03
.706 9.512E-03 -3.13E-02
Grammar
ParagraphComprehension
Vocabulary
Sentence Completion
Geometry
Algebra
Numerical Puzzles
Series Completion
Practical Problem Solving
Symbol Manipulation
Analytical Ability
Formal Logic
1 2 3
Factor
Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser Normalization.
Rotation converged in 6 iterations.a.
An alternative oblique rotation---Promax---provides much the same answer:
The order of the factors may vary, and one gets reflected, but the essential interpretation is the same.
Factor Correlation Matrix
1.000 .455 .410
.455 1.000 .392
.410 .392 1.000
Factor1
2
3
1 2 3
Extraction Method: Principal Axis Factoring. Rotation Method: Promax with Kaiser Normalization.
Factor Correlation Matrix
1.000 .376 -.448
.376 1.000 -.402
-.448 -.402 1.000
Factor1
2
3
1 2 3
Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser Normalization.
The correlations among the factors are similar for the two procedures.
Exploratory factor analytic methods are sometimes used as a crude way to confirm hypotheses about the latent structure underlying a data set. As a first pass, these methods do just fine. But, more powerful confirmatory factor analytic procedures exist that can better address questions about data that are strongly informed by theory.