Factor Analysis

19-2

What is factor analysis ?

Factor analysis is a general name denoting a class of Procedures primarily used for data reduction and summarization.

Variables are not classified as either dependent or independent. Instead, the whole set of interdependent relationships among variables is examined in order to define a set of common dimensions called Factors.

19-3

Purpose of Factor Analysis

To identify underlying dimensions called Factors, that explain the correlations among a set of variables.

-- lifestyle statements may be used to measure the psychographic profile of consumers.

To identify a new, smaller set of uncorrelated variables to replace the original set of correlated variables for subsequent analysis such as Regression or Discriminant Analysis.

-- psychographic factors may be used as independent variables to explain the difference between loyal and non loyal customers.

19-4

Assumptions

Models are usually based on linear relationships

Models assume that the data collected are interval scaled

Multicollinearity in the data is desirable because the objective is to identify interrelated set of variables.

The data should be amenable for factor analysis. It should not be such that a variable is only correlated with itself and no correlation exists with any other variables. This is like an Identity Matrix. Factor analysis cannot be done on such data.

19-5

An Example

A study conducted to determine customers perception and attributesof an airline. A set of 10 statements were constructed and respondentswere asked to rate in a 7 point scale ( 1= completely agree, 7 = completely disagree ) Statements were as follows:

1. The Airline is always on time2. The seats are very comfortable3. I love the food they provide4. Their air-hostesses are very courteous5. My boss/friend flies with the same airline6. The airlines have younger aircrafts7. I get the advantage of a frequent flyer

program8. It suits my schedule9. My mom feels safe when I fly in this airline 10. Flying by this airline compliments my lifestyle

and social standing in the society

19-6

Example Contd..

Do the ten different statements indicate 10 different factors which influence a customer to fly by this airline ?

OR Is there any correlations between

these statements so that we can identify

only a few factors such that some of these statements can be associated to these factors.

19-7

Factor Analysis – basic ideas

Each of the statement indicated in the example is considered as a Variable. Hence for each respondent there will be a score against each variable. Ex: V1 V2 V3 V4 V5 V6 V7 V8 V9 V10respondent 1 2 2 4 3 5 3 5 7 6 2

We can attach suitable weights to each of the variable scores and aWeighted sum of these can be calculated. Ex: weight for V1 = 0.3 , weight for V2 = 0.1 etcHence a score called Factor Score can be calculated as

Factor Score ( Resp 1) = W1x2 + W2x2+ W3x4+w4x3+……….

Similarly factor score can be calculated for each respondent. If there were 20 respondents, we would get a table containing 20 factor scores.

19-8Factor Analysis – basic ideas contd

The weights which are assigned to each of the variables are not taken arbitrarily but are chosen such that the variance in the factor scores obtained is the maximum.

Once the first set of weights are obtained, a new set of weights are obtained so that the new set of factor scores shows the maximum variance but keeping in mind that these set of factor scores are uncorrelated with the first set of factor scores.

This process is repeated till all the variance is explained by these factors.

The first set of factor scores obtained is now correlated with the data for the variable 1 to 10 . This is called factor loadings. Thus factor loading is the correlation between the factor scores and the variables.

19-9Factor Analysis – basic ideas contd

An example would clarify what we have discussed so far.

A file in excel data sheet can now be looked at to understand what we have just discussed.

The factors thus extracted are done using a technique calledPrincipal – Component Analysis.

19-10

It is possible to extract as many factors as there are variables but the very purpose of factor analysis will be defeated and hence a smaller number of factors need to be found.

Question is --- how many?

Several procedures are available: -- Determine based on Eigenvalues. An eigenvalue represents the amount of

variance associated with the factor. Generally only factors with an Eigenvalue of >1.0 is included.

Determining the number of factors

19-11

Determining the number of factors

Determination based on Scree Plot. A scree plot is a plot of the eigenvalues against the number of factors. Typically, the plot has a distinct break with a gradual trailing off with the rest of the factors. This trailing off is referred to as Scree.

19-12

Scree Plot

0.5

2 5 4 3

6 Component Number

0.0

2.0

3.0

Eig

envalu

e

1.0

1.5

2.5

1

19-13Determining the number of factors

Determination based on percentage of Variance.

The number of factors extracted is determined so that the

cumulative percentage of variance reaches a satisfactory level. The amount of variance explained can vary with situation but above 60% is considered satisfactory.

19-14How to check suitability for Factor

Analysis

Kaiser-Meyer-Olkin ( KMO ) measure of sampling adequacy . This index compares the magnitude of observed correlation coefficients to the magnitude of partial correlation coefficients. Typically it should be

> 0.5 is considered as good enough for conducting

Factor analysis for the data under consideration.

Bartlett test of sphericity : It is a test used to examine the hypothesis that the variables are uncorrelated in the population. If the hypothesis can be rejected then the data is suitable for factor analysis.

19-15

Conducting Factor AnalysisRESPONDENT

NUMBER V1 V2 V3 V4 V5 V61 7.00 3.00 6.00 4.00 2.00 4.002 1.00 3.00 2.00 4.00 5.00 4.003 6.00 2.00 7.00 4.00 1.00 3.004 4.00 5.00 4.00 6.00 2.00 5.005 1.00 2.00 2.00 3.00 6.00 2.006 6.00 3.00 6.00 4.00 2.00 4.007 5.00 3.00 6.00 3.00 4.00 3.008 6.00 4.00 7.00 4.00 1.00 4.009 3.00 4.00 2.00 3.00 6.00 3.00

10 2.00 6.00 2.00 6.00 7.00 6.0011 6.00 4.00 7.00 3.00 2.00 3.0012 2.00 3.00 1.00 4.00 5.00 4.0013 7.00 2.00 6.00 4.00 1.00 3.0014 4.00 6.00 4.00 5.00 3.00 6.0015 1.00 3.00 2.00 2.00 6.00 4.0016 6.00 4.00 6.00 3.00 3.00 4.0017 5.00 3.00 6.00 3.00 3.00 4.0018 7.00 3.00 7.00 4.00 1.00 4.0019 2.00 4.00 3.00 3.00 6.00 3.0020 3.00 5.00 3.00 6.00 4.00 6.0021 1.00 3.00 2.00 3.00 5.00 3.0022 5.00 4.00 5.00 4.00 2.00 4.0023 2.00 2.00 1.00 5.00 4.00 4.0024 4.00 6.00 4.00 6.00 4.00 7.0025 6.00 5.00 4.00 2.00 1.00 4.0026 3.00 5.00 4.00 6.00 4.00 7.0027 4.00 4.00 7.00 2.00 2.00 5.0028 3.00 7.00 2.00 6.00 4.00 3.0029 4.00 6.00 3.00 7.00 2.00 7.0030 2.00 3.00 2.00 4.00 7.00 2.00

19-16

Correlation Matrix

Variables V1 V2 V3 V4 V5 V6 V1 1.000 V2 -0.530 1.000 V3 0.873 -0.155 1.000 V4 -0.086 0.572 -0.248 1.000 V5 -0.858 0.020 -0.778 -0.007 1.000 V6 0.004 0.640 -0.018 0.640 -0.136 1.000

19-17Results of Principal Components Analysis

Communalities

Variables Initial Extraction V1 1.000 0.926 V2 1.000 0.723 V3 1.000 0.894 V4 1.000 0.739 V5 1.000 0.878 V6 1.000 0.790

Initial Eigen values

Factor Eigen value % of variance Cumulat. % 1 2.731 45.520 45.520 2 2.218 36.969 82.488 3 0.442 7.360 89.848 4 0.341 5.688 95.536 5 0.183 3.044 98.580 6 0.085 1.420 100.000

19-18Results of Principal Components Analysis

Extraction Sums of Squared Loadings

Factor Eigen value % of variance Cumulat. % 1 2.731 45.520 45.520 2 2.218 36.969 82.488

Factor Matrix

Variables Factor 1 Factor 2 V1 0.928 0.253 V2 -0.301 0.795 V3 0.936 0.131 V4 -0.342 0.789 V5 -0.869 -0.351 V6 -0.177 0.871

19-19

Although the initial or unrotated factor matrix indicates the relationship between the factors and individual variables, it seldom results in factors that can be interpreted, because the factors are correlated with many variables. Therefore, through rotation the factor matrix is transformed into a simpler one that is easier to interpret.

In rotating the factors, we would like each factor to have nonzero, or significant, loadings or coefficients for only some of the variables. Likewise, we would like each variable to have nonzero or significant loadings with only a few factors, if possible with only one.

The rotation is called orthogonal rotation if the axes are maintained at right angles.

Conducting Factor AnalysisRotate Factors

19-20

The most commonly used method for rotation is the varimax procedure. This is an orthogonal method of rotation that minimizes the number of variables with high loadings on a factor, thereby enhancing the interpretability of the factors. Orthogonal rotation results in factors that are uncorrelated.

The rotation is called oblique rotation when the axes are not maintained at right angles, and the factors are correlated. Sometimes, allowing for correlations among factors can simplify the factor pattern matrix. Oblique rotation should be used when factors in the population are likely to be strongly correlated.

Conducting Factor AnalysisRotate Factors

19-21Results of Principal Components Analysis

Rotated Factor Matrix

Variables Factor 1 Factor 2 V1 0.962 -0.027 V2 -0.057 0.848 V3 0.934 -0.146 V4 -0.098 0.845 V5 -0.933 -0.084 V6 0.083 0.885

Rotation Sums of Squared Loadings

Factor Eigenvalue % of variance Cumulat. % 1 2.688 44.802 44.802 2 2.261 37.687 82.488

19-22

A factor can then be interpreted in terms of the variables that load high on it.

Another useful aid in interpretation is to plot the variables, using the factor loadings as coordinates. Variables at the end of an axis are those that have high loadings on only that factor, and hence describe the factor.

Conducting Factor AnalysisInterpret Factors

19-23

Factor Loading Plot

1.0

0.5

0.0

-0.5

-1.0

Facto

r 2

Factor 1

factor Variable 1 2

V1 0.962 -2.66E-02

V2 -5.72E-02 0.848

V3 0.934 -0.146

V4 -9.83E-02 0.854

V5 -0.933 -8.40E-02

V6 8.337E-02 0.885

Factor Plot in Rotated Space

-1.0 -0.5 0.0 0.5 1.0

V1

V3

V6 V2

V5

V4

Rotated Component Matrix

19-24

A few examples

We can now take few examples

with hypothetical data and run

factor analysis using SPSS package.

19-25