Multivariate Analysis and Data Reduction

Multivariate Analysis

Multivariate analysis tries to find patterns and relationships among multiple dependent variables.

Situations where multiple dependent variables are common: Test validation and development of scales. Multi-measure paradigms (e.g., physiological

psychology). Complex sociological studies, including surveys

and archival research.

Extending Correlation

Correlation examines the type and extent of relationship between two variables: Positive (direct), negative (inverse), no relation. A correlation matrix shows relationships among

multiple pairs of variables (average interrcorrelation can be calculated for the matrix).

What if latent causes exist that affect many of the variables in a study? What if several different causes have different

impacts on different of the variables?

Factor Analysis

Factors are underlying constructs (causes) for the variability in a set of multiple variables.

Factor analysis applies matrix algebra to a correlation matrix in order to extract a set of common factors. The relationship between each variable and the

extracted factors is quantified by “factor loadings”. Eigenvalues show how much of the variance is

explained by each factor.

Goals of Factor Analysis

The goal of factor analysis is to make sense of the overlap and relationships among multiple dependent variables (measures).

The procedure seeks a more economical explanation of the observed behavior. The number of important factors should be less

than the number of variables input. It is up to the investigator to make sense out

of the factors identified by the analysis.

Rotation

Factor analysis is an iterative process. It begins by trying to find the factor that

produces the highest correlations among a set of variables. It then locates the factor that produces the second

highest correlations, and so forth. You decide how many factors are relevant.

This original factor structure may be difficult to interpret, so factors are rotated to find a meaningful interpretation.

Interpretation of Results

Orthogonal factors are at right angles to each other. Orthogonal factors are assumed to be

independent of each other. Varimax rotation produces orthogonal rotations.

Ideally, a variable should load high on one factor and low or not at all on other factors. In reality, factors may be complex to interpret.

The emergent structure depends on the input.

Two Different Rotations of the Same Intelligence Data Spearman’s G

The main factor that emerges when minimum residual factor analysis is applied to multiple measures of intelligence (e.g., on an IQ test).

Gardner’s Multiple Intelligences A factor structure consisting of the maximum

orthogonal factors emerging from the same analysis of multiple measures of intelligence with Varimax rotation.

Both are valid solutions for the same data.

Cluster Analysis

A method of reducing multiple input variables (or cases/subjects) into larger groups based on similarity of scores.

Cluster analysis is used when you want to group cases or variables but don’t already know which belong together.

Similarity of features, not shared variance is the basis for clustering.

Interpretation of Cluster Analysis Results Two approaches – putting items together

(agglomorative) or dividing large groups into smaller ones (divisive).

The technique provides the solution but it must be interpreted by the investigator.

The structure of the solution is determined by the number and type of input variables or cases (subjects). Systematic or valid sampling is essential.

Discriminant Analysis

Discriminant analysis uses the characteristics of variables to predict membership in defined groups. Used when the group membership is already

known. The relationship between the groups and the

variables is analyzed, resulting in factor loadings for discriminant functions.

Input variables can be changed to test different models for predicting groups.

Multidimensional Scaling

Similarities and differences among items are used to create a plot showing relationships among them. Input is similarity judgments, distances, or

correlation matrix for multiple variables. Similarities are converted to distances for plotting.

An iterative calculation figures out the plot that best retains the relative distances among all items.

Dimensions can be inferred from the plot.

Important Measures of Model Fit Frequently, more than two dimensions are

needed to accurately display relationships among cases. Stress shows how well the locations of the items

fit within the number of dimensions requested. The more items, the greater the stress. The more dimensions, the lower the stress.

Higher dimensionality is difficult to interpret. The highest dimensions are noise (error).

Individual Differences Scaling Some methods take into account that

different individuals emphasize different dimensions when making judgments about items.

The relative weights for each dimension, for each subject, can be analyzed.