Scaling

Measuring the
Unobservable

Scaling
Scaling

involves the construction of an

instrument that associates qualitative
constructs with quantitative metric units.

Scaling

evolved out of efforts to measure

"unmeasurable" constructs like
authoritarianism and self esteem.

Note: We are talking about constructed scales involving

multiple items, not a response scale for a particular question.

Scaling

How do we define or capture or measure a nebulous

concept?

By taking stabs from several directions, we can get a

more complete picture of a concept we know exists but
cannot see.

Scaling
In

scaling, we have several items that

are intended to capture a piece of the
underlying concept.
The items are then combined in some
form to create the scale.
Quite technically, we will talk about scales and indexes
interchangeably. Scales are composed of items caused by an
underlying construct, whereas indexes are composed of items that
indicate the level of a construct and might be useful together to predict
outcomes.

Scaling
Graphical depiction of a scale:
Latent
Variable

Observed
Item 1

e1

Observed
Item 2

e2

Observed
Item 3

e3

Observed
Item 4

e4

Scaling
Graphical depiction of
an index:

Observed
Item 1
+
Observed
Item 2
+
Observed
Item 3
+
Observed
Item 4
Form an Index

Scaling
In

most scaling,
the objects are text
statements,
usually statements
of attitude or
belief.

Scaling
A

scale can have any number of

dimensions in it. Most scales that we
develop have only a few dimensions.

What's
If

a dimension?

you think you can measure a person's

self-esteem well with a single ruler that
goes from low to high, then you probably
have a unidimensional construct.

Scaling

Scaling

Many familiar concepts (height, weight,

temperature) are actually unidimensional.
But, if the concept you are studying is in fact
multidimensional in nature, a unidimensional
scale or number line won't describe it well.
E.g., academic achievement: how do you
score someone who is a high math achiever
and terrible verbally, or vice versa?
A unidimensional scale can't capture that type
of achievement.

Scaling

Factor analysis can tell you whether you have

a unidimensional or multidimensional scale
helping you discover the number of dimensions
or scales that exist among a group of
variables.
Factor analysis is typically an exploratory
process, but it can be confirmatory.
Exploratory factor analysis helps you reduce
data by grouping variables into sets that tap
the same phenomena.

Scaling

Steps in factor analysis (what the computer does):

1. Assumes one factor and checks the correlation
of each item with the proposed factor and
compares the proposed inter-item correlations
with the actual inter-item correlations.
Compared

Proposed Model
Item 1

A
Factor
Sum of 1,2

Actual Data

with
Do they

Item 2
Match?
A = 1s correlation with factor
B = 2s correlation with factor
By definition, Item 1 & 2s correlation is A * B

Item 1
Correlation
Item 2

Scaling

Steps in factor analysis (what the computer does):

2. If the single concept is not a good model, the
computer rejects one factor and forms a residual
correlation matrix (real 1,2 proposed A*B)
3. Identifies a second concept that may explain
some of the remaining correlation and checks
the proposed inter-item correlation against the
real correlations.
4. And so on until the correlations match.

Scaling

In actuality, factor analysis will give K factors

for K variables. The last residual correlation
matrix will result in zeros.
So, how many factors should you use?

You could use statistical criteria: extract factors until

matrix is not statistically significant from zero.
Historically, number of factors has been determined by
substantial needs, intuition, and theory.

Scaling
Guideline

for subjective analysis: A

group of factors should be able to
explain a high proportion of total
covariance among a set of items.

Eigenvalue test
Scree Test

Scaling

Eigenvalues

An eigenvalue represents the number of units of

information that a factor explains in a k set of variables
with k units of information.

E.g., when k = 10, an eigenvalue of 3 represents 30% of

information is explained by the factor.

An eigenvalue of 1 corresponds with a variables worth

of information. Therefore, factors with an eigenvalue
of 1 or less do not help to reduce data.
Get rid of factors with eigenvalues less than 1

Scaling
Scree

Test

Most researchers are looking for stronger,

fewer factors (they want to reduce data).

Therefore, they tend to use the scree plot.

Plot the eigenvalues relative to each other

Strong factors form a steep slope, weaker factors
2
1

form a plateau
Retain those factors that lie above the elbow of
the plotlike with gangrene, cut off the elbow!
Scree plot for 5 variables

Scaling
In

addition, factors should be

composed of similar, logically linked
items. This is an especially helpful rule
when the number of factors is not that
obvious.

Scaling
Factor

Rotation

Factor rotation involves using an algorithm

to maximize the correlation of items to a

factormaking each item appear most
relevant to a single factor.
The point is to identify variables that most
similarly form indicators of the same factor
each factors variables being most clearly
highlighted.

Scaling
Factor

Rotation

The best-scenario (never happens) is when

all items load (correlate with) as 1 on a

single factor and 0 on all the rest. This is
called simple structure.
Factor rotation mathematically takes the
items as close as possible to simple
structure.

Scaling

Factor Rotation

Orthogonal versus oblique rotation

Orthogonal rotation makes factors completely independent of each

other.

This is preferred for finding the most unique factors. Use if factors
ought not be related.
Any items variation explained by one factor can be added to that of
another factor to get the total variation explained by the two.
If you find lots of cross-loading, you should consider Oblique.

Use if the factors ought to be related.

There is redundancy in the variation of any item explained by one
factor versus another, such that they have overlapping explanatory
power.

Oblique rotation makes factors that are allowed to be correlated with

each other to some degree.

You might want to try both and look for simple structure.
Strong loadings on two factors may indicate a single factor, high
correlation of two factors may indicate a single factor.

Scaling
Factor

Rotation

Items with a high loading on (high

correlation with) a factor form the factors

variable for research purposes.
Common elements of the items is likely
what the factor represents.

Scaling
Type

of analysis in extracting factors:

Principal components analysis produces

specified proportion of total variance
among items explained.
Common factor analysis produces
specified proportion of shared variance
among items explained.
Bottom line: report which you used.

Scaling

Exploratory versus Confirmatory Analysis

Exploratory is that which we have been

discussing. If using exploratory, with new

samples you rediscover a structure in each
sampleyou have persuasive evidence of the
structure.
Confirmatory typically refers to models
generated by Structural Equation Modeling
where items are specified to form a factor in
advance. The question becomes, How well do
the data fit a specified model using statistical
inference? You have to be careful not to
overproduce many meaningless factors.

Scaling

Validity and Reliability

Like other measures, scales and indexes must

be valid and reliable to be useful.

Validity:

Face, Content, Criterion, Construct

A particular kind of reliability that is particularly

useful for scales and indexes is inter-item

reliability (internal consistency or high inter-item
correlation)
To the degree that the items are correlated, the
common correlation is attributable to the true
score of the latent variable.

Scaling

Inter-item ReliabilityAlpha

Variation in each item is caused by the latent

variable and error (unique for each)

Common variation is caused by the latent
variable.
Using the variance/covariance matrix, you can
see total variance in the sum of components.
The diagonal (variance) represents unique
variation for each item.
The off-diagonal represents co-variation of
items. This also equals 1 (Unique/Total)

Scaling

Inter-item ReliabilityAlpha

The off-diagonal represents co-variation of

items. This also equals 1 (Unique/Total)

To correct for the ways variance/covariance
matrices change with number of items, the
formula above is adjusted by k/k-1, where k =
number of items. This constrains alpha to
range from 0 to 1.
k
Unique variance
= k 1 Total variance

Scaling
Inter-item

ReliabilityAlpha

Some characteristics of alpha

Holding correlation constant, alpha goes up

with more scale items

To improve a scale, look for effect on alpha if
an item were dropped.
Reliability is not good unless it is .65 or
above. Best reliability would be around .9.
Good scales require a balance between
reliability and length.

Scaling

1.

2.

Creating a scale
Determine what you want to measure

Clarity
Specificity

Scales may be generated from 40 to 100 items

Good reason to reuse scales
Writing

Generate an item pool

Positive and negative items

Stay on topic
Avoid lengthy items
Keep wording simple
Avoid multiple negatives
No double-barreled items

Scaling

3.

4.
5.
6.
7.

Creating a scale
Determine format for measurement.

Response options
Broad versus narrow

Differences between items

You need item variance
Look for means in the middle of the scale
Item-scale correlations

Review of item pool by experts

Include scale validation items
Administer to a development sample
Evaluate items