Principal Components Factor Analysis
-
Upload
irfan-u-shah -
Category
Documents
-
view
239 -
download
0
Transcript of Principal Components Factor Analysis
-
8/6/2019 Principal Components Factor Analysis
1/11
Principal Components Factor Analysis
The purpose of principal components factor analysis is to reduce the number of variables in the analysis
by using a surrogate variable or factor to represent a number of variables, while retaining the variance
that was present in the original variables. The data analysis indicates the relationship between the
original variables and the factors, so that we know how to make the substitutions.
Principal components are frequently used to simplify a data set prior to conducting a multiple regressionor discriminates analysis.
To demonstrate principal components analysis, we will use the sample problem in the text which begins
on page 120.
Stage 1: Define the Research Problem
The perceptions of HATCO on seven attributes (Delivery speed, Price level, Price flexibility, Manufacturer
image, Service, Salesforce image, and Product quality) are examined to (1) understand if these
perceptions can be "grouped" and (2) reduce the seven variables to a smaller number. (Text, page 120)
Stage 2: Designing a Factor Analysis
In this stage, we address issues of sample size and measurement issues. Since missing data has an
impact on sample size, we will analyze patterns of missing data in this stage.
Sample size issues
Missing Data Analysis
There is no missing data in the HATCO data set.
Sample size of 100 or more
There are 100 subjects in the sample. This requirement is met.
Ratio of subjects to variables should be 5 to 1
There are 100 subjects and 7 variables in the analysis for a ratio of 14 to 1. This requirement is met.
Variable selection and measurement issues
Dummy code non-metric variables
All variables in the analysis are metric so no dummy coding is required.
Parsimonious variable selection
The variables included in the analysis are core elements of the business. There do not appear to be any
extraneous variables.
Stage 3: Assumptions of Factor Analysis
In this stage, we do the tests necessary to meet the assumptions of the statistical analysis. For factor
analysis, we will examine the suitability of the data for a factor analysis.
-
8/6/2019 Principal Components Factor Analysis
2/11
Metric or dummy-coded variables
All variables are metric in this analysis.
Departure from normality, homoscedasticity, and linearity diminish correlations
In the last class, we conducted an exploratory analysis of these variables. We know from this several
variables are not normally distributed. Non-normality will diminish the correlations among the
variables, such that the relationships between variables might be stronger than what we are able to
represent in this analysis.
Multivariate Normality required if a statistical criteria for factor loadings is used
We will use the criteria of 0.40 for identifying substantial loadings on factors, rather than a statistical
probability, so this criteria is not binding on this analysis.
Homogeneity of sample
There is nothing in the problem to indicate that subgroups within the sample have different patterns of
scores on the variables included in the analysis. In the absence of evidence to the contrary, we will
assume that this assumption is met.
Use of Factor Analysis is Justified
The determination that the use of factor analysis is justifiable is obtained from the statistical output that
SPSS provides in the request for a factor analysis.
Requesting a Principal Components Factor
Analysis
Specify the Variables to Include in the Analysis
C l i ck on th e 'Da ta Red u c t i on |
F ac to r . . .' c omm and in th e
A n a l y z e m e n u .
F i r s t ,
i g
l i g
t t
v
r i
b l
s : X 1 '
l i v
r
S p
' , X 2 ' P r i
L
v
l ' , X 3 ' P r i
l
x i b i l i t
' ,
X 4 ' M
f
t
r
r I
g
' , X 5 ' S
r v i
' , X 6
' S
l
s f
r
I
g
' ,
X 7 ' P r
t Q
l i t
' .
S e c o n d ,
l i
t
v
r r
w t
v
t
i g
l i g
t
v
r i
b l
s t
t
' V
r i
b l
s : ' l i s t .
-
8/6/2019 Principal Components Factor Analysis
3/11
Specify the
Descriptive
Statistics to
include in the
Output
Specify the
Extraction
Method and
Number of
Factors
Specify the
Rotation
Method
F i r s t , !
li!
"
#
$
t%
&
''
&
s!
r i p t i v&
s . . . 'b
(
t t#
$
.
S e c o n d , )
0
r"
t%
&
!
%
& !
"
b#
x f#
r'I
$
it i0
l s#
l(
t i#
$
' i$
t%
&
'S t0
t i s t i!
s '
p0
$
&
l.1
l& 0
r0
ll#
t %&
r!
%
& !
" b#
x&
s .
T h i r d , )
0
r"
t%
&
!
%
& !
"
b#
x&
s f#
r '1
# &
f f i!
i&
$
t s ' ,' K M O
0
$
2
B0
r t l&
t t ' s t&
s t#
f s p%
&
r i!
it3
' , 0
$
2
'4
$
t i - i)
0
g&
'#
$
t%
&
'1
#
r r&
l0
t i#
$
M0
t r i x ' p0
$
&
l.1
l& 0
r0
ll#
t%
&
r!
%
& !
"
b#
x&
s .
F o u r t h , !
li!
"
#
$
t%
&
'1
#
$ t i $ (&
' b ( t t#
$ t#
! #
)
p l&
t&
t%
&
'5
0 !
t#
r4
$
0
l3
s i s :'
&
s!
r i p t i v&
s2
i0
l#
g b#
x .
F i r s t,6
li6
7
8
9 t @A
'ExtrB
6
ti8
9
...'
bC
tt8
9
.
S e c o n d , sA
lA 6
t
'Pri9
6
ipB
l6
8
D
p8
9
A
9
ts '
fr8
D
t@
A
'MA
t@
8
E
'
E
r8
pE8
w9
D
A
9 C
.
T h ir d,D
B
r7
t@
A
'F
8
rrA
lB
ti8
9
D
B
trix'8
pt i8
9
i9
t@
A
'G
9
B
lH I
A
' pB
9
A
l.
F o u rt h ,D
B
r7
t@
A
6
@
A 6
7
b8
xA
s f8
r 'U9
r8
tB
tA
E
fB
6
t8
r s8
l C ti8
9 'B
9
E
'S6
rA A
p l8
t'8
9
t@
A
'P
isplB
H
' pB
9
A
l.
F i f t h ,B
6 6
A
pt t@
A
E
A
fB
C
lt vB
lC
A
s8
f 'EigA
9
vB
lC
A
s8
vA
r: 1'8
9
t@
A
ExtrB
6
t pB
9
A
lB
9
E
t@
A
'MB
x iD C D
ItA
rB
ti8
9
s f8
r6
8
9
vA
rgA
9
6
A
: 25'.
S ix t h ,6
li6
7
8
9
t@
A
'F
8
9
ti9 C
A
'
bC
tt8
9
t8
6
8
D plA
tA
t @A
E
iB
l8
g b8
x.
First, Q li Q R S T t U V
'W S
tX
tiS T
...' bY
ttS T
.
Second,` X
rR
tU V
'VX
ri` X
x'S
ptiS T
S T
tU V
'MV
tU S a
' pX T V
l.
Th ird,` X
rR
tU V
Q U V Q R
bS
x fS
r 'W S
tX
tV a
sS
lY
tiS T
'S T
tU V
'b
isplX
c
' pX T V
l.d
lV X
rX
llS
tU V
r
Q U V Q R
bS
xV
s.
Fou rth, Q li Q R S T t U V
' d S T ti T Y V ' b Y tt S T t S
Q S
plV
tV
tU V
a
iX
lS
g
bS
x.
-
8/6/2019 Principal Components Factor Analysis
4/11
Complete the Factor Analysis Request
Count the Number of
Correlations Greater than
0.30
Measures of
Appropriateness of Factor
Analysis
Assessing the Sampling Adequacy Problem
C lie
f g
h t ip
O K b q t tg
h
tg
e g
r p lp
tp
t ip
fs e
tg
rs
h
s
lt
s i s rp
q qp
s t .
Niu v
w
f tx v
21y w
rrv
l
tiw u
s iu
tx v
trix
rv
l rgv r tx u 0.30,x igx ligx tv iu
v llw w iu tx v
Corrv l tionM trix. Wv v v t tx is y ritv ri fortx v s it bilit
of tx v t for f y tor nal
sis.
In t rp r ti v a d j e c ti ves fo r t e K a is er -M e y e r-O l in M e a s re o f S a m p ling
d eq a c y a re : in t e 0.90 as m a rve lo u s , in t e 0.80's as m er it ori o us ,in t
e 0.70's a s m id d lin g , in t
e 0.60's a s m e d ioc re , in t
e 0.50's as
m is erab le ,a n d b e low 0 .50 a s u n a c c e ptab le . T
e va lu e o f t
e KMOM e as u re o f S a m p lin g
d eq u a c y for t is s e t o f var iab le s is .446 , fa ll ing
b e lo w t e a c c eptab le leve l. W e will exa m in e t e a n ti-im a g e co rr e la ti o n
m a tr ix to se e if it pro v id e s us w i t
a n y p oss ib le re m e d ies .
-
8/6/2019 Principal Components Factor Analysis
5/11
Removing X5 'Service' from the Analysis
The Revised
Measures of
Appropriateness
of Factor Analysis
T e
n t i - i m a g e C o r r e l a t i o n M a t r i x c o n t a i n s t e m e a s u r e s o f s a m p l i n g
a d e q u a c y fo r t e i n d iv id u a l v a r i a b l e s o n t e d i a g o n a l o f t e m a t r i x ,h i g h l ig h t e d i n c y a n . T h e m e a s u r e s fo r t h r e e v a r i a b l e s fa l l b e l o w t h ea c c e p t a b l e l e v e l o f 0 . 5 0 : X 1 ' e l i v e r y S p e e d ' ( . 3 4 4 ) , X 2 ' P r ic e L e v e l '
( . 3 3 0 ) , a n d X 5 'S e r v i c e ' ( . 2 8 8 ) . T h e c o r r e c t i v e a c t i o n i s t o d e l e t e t h e
v a r i a b l e s o n e a t a t i m e , s t a r t i n g w i t h t h e o n e w i t h t h e s m a l l e s t v a l u e ,
u n t i l t h e p r o b l e m i s c o r r e c t e d .
F i r s t , c li c k o n t h e '
ia lo g
e c a l l ' t o o l
b u t t o n .
S e c o n d , in t h e d r o p d o w n m e n u o f r e c e n t ly u s e d
d ia lo g s , h ig h li g h t t h e '
a c t o r
n a ly sis' i t e m .
T h ir d , h ig h li g h t 'S e r v ic e (X5) ' in t h e l ist o f 'V a r ia b le s: ' .
F o u rt h , c li c k o n t h e
m o v e a r r o w t o r e t ur n
'S e r v ic e (X5) ' t o t h e l ist
o f a v a il a b le b u t t o n s .
F i f t h , c li c k o n t h e O Kb u t t o n t o r e q u e s t t h e
r e v is e d a n a ly sis.
h e re v is e d K
O
e a s u r e o f S a m p li n g
d e q u a c y h a s a v a lu e o f . 5 , in t h e ra n g e o fa c c e p t a b le v a lu e s .
a r t le t t ' s t e s t o f s p h e r ic it y t e s t s t h e h y p o t h e s i s t h a t t h e c o r r e la t io nm a t r i
is a n id e n t ify m a t r i
; i. e . a ll d ia g o n a l e le m e n t s a r e
a n d a ll o f f -d ia g o n a l e le m e n t s a r e , im p ly in g t h a t a ll o f t h e v a r ia b le s a r eu n c o r r e la t e d . I f t h e S i g v a lu e fo r t h is t e s t i s le s s t h a n o u r a lp h a le v e l,w e r e j e c t t h e n u ll h y p o t h e s i s t h a t t h e p o p u la t io n m a t r i
is a n id e n t it ym a t r i
.
h e S ig . v a lu e fo r t h is a n a ly s i s le a d s u s t o r e j e c t t h e n u llh y p o t h e s is a n d c o n c lu d e t h a t t h e re a re c o r re la t io n s in t h e d a t a s e t t h a ta r e a p p r o p r ia t e fo r fa c t o r a n a ly s i s .
-
8/6/2019 Principal Components Factor Analysis
6/11
The Revised Anti-image CorrelationMatrix
Stage 4: Deriving Factors and Assessing Overall Fit 1
In the stage, several criteria are examined to determine the number of factors that represent the data.
If the analysis is designed to identify a factor structure that was obtained in or suggested by previous
research, we are employing an a priori criterion, i.e. we specify a specific number of factors. The three
other criteria are obtained in the data analysis: the latent root criterion, the percentage of variance
criterion, and the Scree test criterion. A final criterion influencing the number of factors is actually
deferred to the next stage, the interpretability criterion. The derived factor structure must make
plausible sense in terms of our research; if it does not we should seek a factor solution with a different
number of components.
It is generally recommended that each strategy be used in determining the number of factors in the data
set. It may be that multiple criteria suggest the same solution. If different criteria suggest different
conclusions, we might want to compare the parameters of our problem, i.e. sample size, correlations,
and communalities to determine which criterion should be given greater weight.
Stage 4: Deriving Factors and Assessing Overall Fit 2
One of the most commonly used criteria for determining the number of factors or components to
include is the latent root criterion, also known as the eigenvalue-one criterion or the Kaiser criterion.
With this approach, you retain and interpret any component that has an eigenvalue greater than 1.0.
The rationale for this criterion is straightforward. Each observed variable contributes one unit of
variance to the total variance in the data set (the 1.0 on the diagonal of the correlation matrix). Any
component that displays an eigenvalue greater than 1.0 is accounting for a greater amount of variance
than was contributed by one variable. Such a component is therefore accounting for a meaningful
amount of variance and is worthy of being retained. On the other hand, a component with an
eigenvalue less than 1.0 is accounting for less variance than had been contributed by one variable.
The latent root criterion has been shown to produce the correct number of components when the
number of variables included in the analysis is small (10 to 15) or moderate (20 to 30) and the
T h e n e
a n t i - im aj
e c o r r e l a t io n m a t r ik
i n d i c a t e
l
t h a t t h e
l
a mm
li n
j
a d e
n
u a c y f o r e a c ho a r i a le il
a oo
e t h e . t h r el
h o l d .
-
8/6/2019 Principal Components Factor Analysis
7/11
communalities are high (greater than 0.70). Low communalities are those below 0.40 (Stevens, page
366).
Stage 4: Deriving Factors and Assessing Overall Fit 3
Proportion of Variance Accounted For
Another criterion in determining the number of factors to retain involves retaining a component if it
accounts for a specified proportion of variance in the data set, i.e. at least 5% or 10%. Alternatively, one
can retain enough components to explain some cumulative total percent of variance, usually 70% to
80%.
While this strategy has intuitive appeal (our goal is to explain the variance in the data set), there is not
agreement about what percentages are appropriate to use and the strategy is criticized for being
subjective and arbitrary. We will employ 70% as the target total percent of variance.
Stage 4: Deriving Factors and Assessing Overall Fit 4
The Scree Test
With a Scree test, the eigenvalues associated with each component are plotted against their ordinal
numbers (i.e. first eigenvalue, second eigenvalue, etc.). Generally what happens is that the magnitude of
successive eigenvalues drops off sharply (steep descent) and then tends to level off.
The recommendation is to retain all the eigenvalues (and corresponding components) before the first
one on the line where they start to level off. (Hatcher and Stevens both indicate that the number to
accept is the number before the line levels off; Hair, et. al. say that the number of factors is the point
where the line levels off, but also state that this tends to indicate one or two more factors that are
indicated by the latent root criteria.)
The Scree test has been shown to be accurate in detecting the correct number of factors with a sample
size greater than 250 and communalities greater than 0.60.
Stage 4: Deriving Factors and Assessing Overall Fit 5
The Interpretability Criteria
Perhaps the most important criterion in solving the number of components problem is the
interpretability criterion: interpreting the substantive meaning of the retained components and verifying
that this interpretation makes sense in terms of what is know about the constructs under investigation.
The interpretability of a component is improved if it is measured by at least three variables, when all of
the variables that load on the component have the same conceptual meaning, when the conceptual
meaning of other components appear to be measuring other constructs, and when the rotated factor
pattern shows simple structure, i.e. each variable loads significantly on only one component.
-
8/6/2019 Principal Components Factor Analysis
8/11
The Latent Root Criterion
Percentage of
Variance Criterion
Scree Test Criterion
Stage 5: Interpreting the Factors 1
All three of the criteria for determining the number of components indicate that two components
should be retained. If this number matches the number of components derived by SPSS, we can
continue with the analysis. If this number does not match the number of components that SPSS
derived, we need to request the factor analysis, specifying the number of components that we want
SPSS to extract.
I n t h e l a te n t r o o t c r i t e r io n ,
e i d e n t if y t h e
n u m e r o f e i
e n
a l u e
t h a t a r e l a r
e r t h a n
. . I n t h i
t a l e ,
e h a
e t
o ,
o t h i
c r i t e r i o n
u z z o r t
t h e z r e
e n c e o f t
o
c o m z o n e n t
o r f a c t o r
.
In thi { criterion, | e count thenum
}
er of com~
onent{
that
| ould } e nece { { ary to ex ~ lain7 % or more of the ariance
in the ori inal { et of aria } le { .In thi
{
analy{
i{
,|
e reach the
7
% minimum|
ith t|
ocom
~
onent{
.
I n m y a n a ly
i
o f t h e
c r e e z lo t , t h e
e i
e n
a l u e
le
e l o f f e
i n n i n
i th t h e
t h i r d e i
e n
a l u e . T h e n u m e r o f
c o m z o n e n t
t o r e t a i n c o r r e
z o n d
to
t h e n u m e r o f e i
e n
a l u e
e f o r e t h e
l ine l e
e l
o f f . T h e r e f o r e ,
e
o u l dr e t a i n t
o c o m z o n e n t
,
h i c h
c o r r e
z o n d
t o t h e n u m e r d e t e r m i n e d
y t h e l a t e n t r o o t c r i te r i o n . ( N O T E : in
a z z l y in
t h i
t e
t , t h e t e x t i d e n t if ie
t h r e e c o m z o n e n t
u
in
t h e i r
i n t e rz
r e t a t i o n o f t h e c r i t e r ia ) .
-
8/6/2019 Principal Components Factor Analysis
9/11
Once the extraction of factors has been completed satisfactorily, the resulting factor matrix, which
shows the relationship of the original variables to the factors, is rotated to make it easier to interpret.
The axes are rotated about the origin so that they are located as close to the clusters of related variables
as possible. The orthogonal VARIMAX rotation is the one found most commonly in the literature.
VARIMAX rotation keeps the axes at right angles to each other so that the factors are not correlated
with each other. Oblique rotation permits the factors to be correlated, and is cited as being a more
realistic method of analysis. In the analyses that we do for this class we will specify an orthogonal
rotation.
The first step in this stage is to determine if any variables should be eliminated from the factor solution.
A variable can be eliminated for two reasons. First, if the communality of a variable is low, i.e. less than
0.50, it means that the factors contain less than half of the variance in the original variable, so we might
want to exclude that variable from the factor analysis and use it in its original form in subsequent
analyses. Second, a variable may have loadings below the criteria level on all factors, i.e. it does not
have a strong enough relationship with any factor to be represented by the factor score, so that the
variables information is better represented by the original form of the variable.
Stage 5: Interpreting the Factors 2
Since factor analysis is based on a pattern of relationships among variables, elimination of one variable
will change the pattern of all of the others. If we have variables to eliminate, we should eliminate them
one at a time, starting with the variable that has the lowest communality or pattern of factor loadings.
Once we have eliminated variables that do not belong in the factor analysis, we complete the analysis by
naming the factors which we obtained. Naming is an important piece of the analysis, because it assures
us that the factor solution is conceptually valid.
Analysis of the Communalities
Once the extraction of factors has been completed, we examine the table of 'Communalities' which tells
us how much of the variance in each of the original variables is explained by the extracted factors. For
example, in the table shown below, 65.8% of the variance in the original X1 'Delivery Speed' variable is
explained by the two extracted components. Higher communalities are desirable. If the communality
for a variable is less than 50%, it is a candidate for exclusion from the analysis because the factor
solution contains less that half of the variance in the original variable, and the explanatory power of that
variable might be better represented by the individual variable.
The table of Communalities for this analysis shows communalities for all variables above 0.50, so we
would not exclude any variables on the basis of low communalities. If we did exclude a variable for a
low communality, we should re-run the factor analysis
without that variable before proceeding.
The table of Communalities for this analysis shows
communalities for all variables above 0.50, so we would not
exclude any variables on the basis of low communalities. If
we did exclude a variable for a low communality, we should re-run the factor analysis without that
variable before proceeding.
-
8/6/2019 Principal Components Factor Analysis
10/11
Analysis of the Factor Loadings 1
When we are satisfied that the factor solution explains sufficient variance for all of the variables in the
analysis, we examine the 'Rotated Factor Matrix ' to see if each variable has a substantial loading on
one, and only one, factor. The size of the loading termed substantial is a subject about which there are
a lot of divergent opinions. We will use a time-honored rule of thumb that a substantial loading is 0.40
or higher. Whichever method is employed to define substantial, the process of analyzing factor loadings
is the same.
The methodology for analyzing factor loading is to underline or mark all of the loadings in the rotated
factor matrix that are higher than 0.40. For the rotated factor matrix for this problem, the substantial
loadings are highlighted in green.
Analysis of the Factor Loadings 2
We examine the pattern of loadings for
what is called 'simple structure' which means that each variable has a substantial loading on one andonly one factor.
In this component matrix, each variable does have one substantial loading on a component. If one or
more variables did not have a substantial loading on a factor, we would re-run the factor analysis
excluding those variables one at a time, until we have a solution in which all of the variables in the
analysis load on at least one factor.
In this component matrix, each of the original variables also has a substantial loading on only one
factor. If a variable had a substantial loading on more than one variable, we refer to that variable as
"complex" meaning that it has a relationship to two or more of the derived factors. There are a variety
of prescriptions for handling complex variables. The simple prescription is to ignore the complexity and
treat the variable as belonging to the factor on which it has the highest loading. A second simple
solution to complexity is to eliminate the complex variable from the factor analysis. I have seen other
instances where authors chose to include it as a variable in multiple factors, or to arbitrarily assign it to a
factor for conceptual reasons. Other prescriptions are to try different methods of factor extraction and
rotation to see if a more interpretable solution can be found.
Naming the Factors
Once we have an interpretable pattern of loadings, we name the factors or components according to
their substantive content or core. The factors should have conceptually distinct names and content.
Variables with higher loadings on a factor should play a more important role in naming the factor.
If the factors are not conceptually distinct and cannot be named satisfactorily, the factor solution may
be a mathematical contrivance that has not useful application.
The naming should take into account the signs of the factor loadings, i.e. negative signs imply an inverse
relationship to the factor. For example, X1 'Delivery Speed', X2 'Price Level', X3 'Price Flexibility', and
X7 'Product Quality' load on the first factor. Two of these variables: X1 'Delivery Speed' and X3 'Price
Flexibility' have a negative sign meaning that they vary inversely to the two variables which have a
-
8/6/2019 Principal Components Factor Analysis
11/11
positive loading: 'Price Level' and X7 'Product Quality'. The name for this factor, which the authors term
'basic value' on page 126 of the text, attempts to take into account the direction of the relationships
among all of these variables.
The two variables loading on the second factor X4 'Manufacturer Image' and X6 'Salesforce Image' both
have positive signs and are named 'HATCO image' by the authors on page 127 of the text.
Stage 6: Validation of Factor Analysis
In the validation stage of the factor analysis, we are concerned with the issue generalizability of the
factor model we have derived. We examine two issues: first, is the factor model stable and
generalizable, and second, is the factor solution impacted by outliers.
The only method for examining the generalizability of the factor model in SPSS is a split-half validation.
To identify outliers, we will employ a strategy proposed in the SPSS manual.
Split Half Validation
As in all multivariate methods, the findings in factor analysis are impacted by sample size. The larger the
sample size, the greater the opportunity to obtain significant findings that are present only because of
the large sample. The strategy for examining the stability of the model is to do a split-half validation to
see if the factor structure and the communalities remain the same.