What is Factor Analysis?

FA

and PCA (principal components analysis) are methods of

data reduction
Take

many variables and explain them with a few factors

or components

Correlated

variables are grouped together and separated

from other variables with low or no correlation

What is Factor Analysis?

Factor

analysis is all about simplification

Allows

us to understand large quantities of observable

variables in terms of a smaller number of unobservable
variables

These

unobservable variables are called latent variables

What is Homer Simpson like?

Homer likes to stick his

hand in beehives

Homer encourages his

daughter to smoke

Homer is distracted
during a reactor
meltdown

These
It

are all directly observable phenomena

might be easier just to say hes stupid

Stupidity

is a latent variable.

Ta-daaaaa!
Weve

just conducted a Factor Analysis!

We

explained a range of observable characteristics in terms of

something simpler which isnt directly observable.

Notice

that all of these individual observable characteristics

seem to be highly related to each other.

We

are usually interested in studying elusive phenomena

anxiety, psychological stress, social identity, irrationality,
personality, etc.

What is FA?
Patterns

of correlations are identified and either used as

descriptives (PCA) or as indicative of underlying theory (FA)

Process

of providing an operational definition for latent

construct (through regression equation)

What is FA?
FA

and PCA are not much different than canonical correlation

in terms of generating canonical variates from linear
combinations of variables

Although
And

there are now no sides of the equation

your not necessarily correlating the factors,

components, variates, etc.

General Steps to FA
Step

1: Selecting and Measuring a set of variables in a given

domain

Step

2: Data screening in order to prepare the correlation

matrix

Step

3: Factor Extraction

Step

4: Factor Rotation to increase interpretability

Step

5: Interpretation

What is a Good Factor?

A

good factor:

Makes
will

sense

be easy to interpret

simple
Lacks

structure

complex loadings

Problems w/ FA
It

is more art than science

There

are a number of extraction methods (PCA, FA, etc.)

There

are a number of rotation methods (Orthogonal, Oblique)

Number

of factors to extract

Communality

estimates

ETC
This

is what makes it great

Types of FA
Exploratory

FA

Summarizing

data by grouping correlated variables

Investigating

sets of measured variables related to

theoretical constructs

Usually
The

done near the onset of research

type of FA and PCA we are talking about

Types of FA
Confirmatory

FA

More

advanced technique

When

factor structure is known or at least theorized

Testing
This

generalization of factor structure to new data, etc.

is tested through SEM methods

Terminology
Orthogonal Rotation
Loading Matrix correlation

and the factor

Oblique Rotation
Factor Correlation

between each variable

Matrix correlation between the

factors
Structure Matrix correlation between factors and
variables
Pattern Matrix unique relationship between each
factor and variable uncontaminated by overlap
between the factors

Terminology
Factor

Coefficient matrix coefficients used to

calculate factor scores (like regression coefficients)

FA vs. PCA conceptually

FA produces factors; PCA produces components

Factors cause variables; components are aggregates of the variables

FA analyzes only the variance shared among the variables (common

variance without error or unique variance); PCA analyzes all of the
variance

FA: What are the underlying processes that could produce these
correlations?; PCA: Just summarize empirical associations, very data
driven

FA

Is a variable reduction technique

which identifies the number of latent
constructs and the underlying factor
structure of a set of variables
Hypothesizes an underlying
construct, a variable not measured
directly
Estimates factors which influence
responses on observed variables
Allows you to describe and identify
the number of latent constructs
(factors)
Includes unique factors, error due to
unreliability in measurement
Traditionally has been used to
explore the possible underlying factor
structure of a set of measured
variables without imposing any
preconceived structure on the
outcome (Child, 1990).

PCA

Is a variable reduction technique

Is used when variables are highly
correlated
Reduces the number of observed

variables to a smaller number of

principal components which account
for most of the variance of the
observed variables
Is a large sample procedure

Factor Analysis in SPSS

Factors are the underlying constructs that describe a set of variables
Factor analysis begins with a measure of the total amount of variation
observed (similar to sum of squares) in all variables selected.

SPSS will select the combination of variables whose shared correlations

explain the greatest amount of the total variance.

Factor 1 explains the largest proportion of total variance.

The remaining variation after Factor 1 has been extracted is combined
and called Factor 2.

This procedure continues until as many factors have been extracted, as

there are variables

Factor Analysis in SPSS

SPSS will calculate a factors associated Eigenvalue.
Eigenvalues are designed to show the proportion of variance
accounted for by each factor.

The first eigenvalue will always be the largest since it is associated

with the factor that explains the largest proportion of variance.

We select the number of factors to retain using eigenvalues and

parallel analyses.

The rotation of the factor matrix is determined by the nature of the

relationships in your data - either orthogonal or oblique.