Shackman Psyc210 Module04 Measuring

Nuts & Bolts Plan for
Today
Cumulative review
Lecture (Block 95; Tomarken 95)

Measuring T&P
Some problems with words
Some problems with physiological measures of T&P,
Just Because Youre Measuring the Brain (or HPA axis, or
facial musculature, etc) Doesnt Mean You Can Stop Using
Your Head John Cacioppo, past present of the Assoc for
Psychol Science
Take-home critical thinking questions

Today

Measuring T&P
Problems with words
Problems with physiological measures of T&P
Just Because Youre Measuring the Brain (or HPA
axis, or facial musculature, etc) Doesnt Mean You
Can Stop Using Your Head
John Cacioppo, past present of the Assoc for
Psychol Science

Today

Measuring T&P
Problems with words
Just Because Youre Measuring the Brain Doesnt
Mean You Can Stop Using Your Head
John Cacioppo, past present of the
Assoc for Psychol Science (APS)

Today

Measuring T&P
Problems with words
Just Because Youre Measuring the Brain Doesnt
Mean You Can Stop Using Your Head
John Cacioppo, past present of the
Assoc for Psychol Science (APS)
PSYC 210:
How Can We Discover and Measure
Individual Differences in T&P?
AJ Shackman
10 February 2015
Todays Conceptual Roadmap

Where did the fundamental dimensions
of T&P come from? How were they
discovered? What are their limitations?
How should we measure T&P? What is
the value of adopting biological
measures?
What statistical properties do we need
to assess if we turn to biological
measures of T&P?

measures?
measures of T&P?

measures?
measures of T&P?

measures?
to assess if we use biological measures
of T&P?
From Measures to Models

The Case of the 5 Factor Model, or,
How did we get
the factors that
we were given?
Why 5 Factors? Words, words, words

Lexical Hypothesis,"All aspects of human personality which are or
have been of importance, interest, or utility have already become
recorded in the substance of language Cattell
Common speech is a poor guide to psychological subtleties
Allport

Lexical Hypothesis,"All aspects of human personality which are or
have been of importance, interest, or utility have already become
recorded in the substance of language Cattell
Common speech is a poor guide to psychological subtleties
Allport

Allport 1930s
400,000 words in the unabridged dictionary 18,000 trait-relevant
adjectives 4,500 key traits
Cattell 1940s
Used a combination of ad hoc subjective and statistical techniques to
whittle it down to 35 scales
Used factor analysis to reduce it to 12
Set the stage for the discovery of the Big 5 (OCEAN)

Allport 1930s
400,000 words in the unabridged dictionary 18,000 trait-relevant
adjectives 4,500 key traits
Cattell 1940s
Used a combination of subjective and statistical techniques to whittle
it down to 35 scales
Used factor analysis to reduce it to 12
Set the stage for the discovery of the Big 5 (OCEAN)
Whats Factor Analysis

Factor Analysis
Statistical technique for compressing or reducing the number of
dimensions in a dataset
e.g., 100 items on a questionnaire 5 latent factors or dimensions
(95% compression)
Lossy compression technique: Retention of a small number of
dimensions typically captures <100% of the variance in the observed
data
Key limitations
Subjective
How many dimensions is sufficient? Should the factors be
orthogonal (uncorrelated) or oblique (correlated)? Analyst
gets to pick.
Garbage In/Garbage Out
Results are data-dependent. If the analyst includes 30
anxiety items and 2 thought-disturbance items, the factor
analysis will likely tell you that a single anxiety dimension is
sufficient, but this simply reflects the choice of inputs

Factor Analysis
e.g., 100 items on a questionnaire 5 latent factors or dimensions
(95% compression)
data
Key limitations
Subjective
gets to pick.

Factor Analysis
e.g., 100 items on a questionnaire 5 factors or dimensions (95%
compression)
data
Key limitations
Subjective
gets to pick.

Factor Analysis
e.g., 100 items on a questionnaire 5 factors or dimensions (95%
compression)
data
Key limitations
Subjective
gets to pick.
Non-Mathematical Example
Recall the survey that
you completed in class
Choices, choices, choices
tive: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors
factor explains a great

eal of variance in scores
2nd factor explains

much less
Factors 3-18 account for
less and less variance
tive: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors
Loss of 56.3%
Loss of 13.5%
Varying degrees of information loss,

depending on the number of factors we
choose to retainfewer factors, more loss
Loss of 0.0%
oss: 100 Cumulative
Loss of 56.3%
Loss of 13.5%
Varying degrees of information loss,

depending on the number of factors we
choose to retainfewer factors, more loss
Loss of 0.0%
oss: 100 Cumulative
Pick one big factor, four factors, many fact
If we retain 4 factors (~2/3

variance)
F1: Distress
F2: Worry
F3: Depression
F4: Junk
Limitations of Factor
Analysis
Subjective, the Analyst
Has to Choose
The number of factors to retain
The rotation, should the factors be statistically uncorrelated
(orthogonal) or correlated (oblique)
"...each orientation is equally acceptable mathematically. But
different theories proved to differ as much in terms of the [choice of
rotation] as in terms of anything else, so that model fitting did not
prove to be useful in distinguishing among theories." (Sternberg,
1977).
All rotations are equally valid mathematical outcomes. There is no
unique or optimal solution. Different interpretations are equally valid.

Results are data-dependent. If the analyst includes 30 anxiety
items and 2 thought-disturbance items, the factor analysis will
likely tell you that a single anxiety dimension is sufficient, but
this simply reflects the choice of inputs
Analysis
Has to Choose
"...each orientation is equally acceptable mathematically. But
prove to be useful in distinguishing among theories." (Sternberg,
1977).
All rotations are equally valid mathematical outcomes. There is no
unique or optimal solution. Different interpretations are equally valid.

Analysis
Has to Choose
F1 F2 But
"...each orientation is equally acceptable mathematically.
prove to we
be useful
in distinguishing
among
re concretely,
have to
choose whether
we theories." (Sternberg,
1977).
nk that
F1 (distress) and F2 (worry) are
ated or
in terms
their
Allunrelated,
rotations are
equallyofvalid
mathematical outcomes. There is no
derlying
psychology
and
neurobiology
unique
or optimal
solution.
Different interpretations are equally valid.

Analysis
Has to Choose
All rotations are equally valid mathematical outcomes there
is nothing inherent in the mathematics of factor analysis that
tells you which rotation is the right one

analysis will likely tell you that a single anxiety dimension
is sufficient, but this simply reflects the choice of inputs
Analysis
Has to Choose
All rotations are equally valid mathematical outcomes there
is nothing inherent in the mathematics of factor analysis that
tells you which rotation is the right one

Results are data-dependent. If you include 30 anxiety items but
only 2 thought-disturbance items, the factor analysis will indicate
that a single anxiety dimension is sufficient to describe T&P, but
this one dimensional solution is an artifact of the inputs
Back to Blocks history

lesson

Tupes & Christal 1940s
personnel selection psychologists employed by the Air Force to
improve officer selection and promotion procedures
factor analyses based on officers ratings of one another
Can the psychological perceptiveness of Air Force officers
and officer candidates, as quickly expressed by 3-point
ratings on 30 or so scales in an officially required research
program regarding 12 to 30 of their peers known for such
short periods [as few as 3 days], provide a fundamental data
basis for discerning the essential dimensions for the
scientifically sufficient description of personality? Block 95
5 factors:
Surgency/Extroversion, Agreeableness, Dependability, Emotional
Stability, and Culture.
Norman 1960s
Again, various subjective (but herculean) ad hoc approaches to
turning very large lists of words (adjectives or trait descriptors) into
a manageable set of dimensions

Can the psychological perceptiveness of Air Force officers
and officer candidates, as quickly expressed by 3-point
ratings on 30 or so scales in an officially required research
program regarding 12 to 30 of their peers known for such
short periods [as few as 3 days], provide a fundamental data
basis for discerning the essential dimensions for the
scientifically sufficient description of personality? Block 95
5 factors:
udents Any concerns with this approac
Norman 1960s

Discovered 5 factors:
Norman 1960s

Discovered 5 factors:
Norman 1960s
Again, various subjective approaches for turning very large lists of
words (adjectives or trait descriptors) into a manageable set of
dimensions

Culminated in the work of Goldberg 1970s 1990s and Costa &
McCrae 1980s 1990s
Similar concerns
In designing their questionnaire, Costa and McCrae distinguished and
permanently fixed upon [case closed!] a half dozen facets each for their
broad constructs of Neuroticism, Extraversion, and Openness.
The facet distinctions they offered were not rooted in factor
analysis, formal theorizing, or ineluctable empirical findings.
Rather, the facets derived from their personal thinking about how
the three domains could be further articulated.
The six facets Costa and McCrae nominated to represent the Neuroticism
domain were Depression, Impulsiveness, Anxiety, Hostility, Selfconsciousness, and Vulnerability.
Jack Blocks Critique

The Lexical Hypothesis
udents: What are some potentia

oblems with the lexical hypothes

#1. The Lexical Hypothesis




The premise is false
Meaningful aspects of T&P may not be captured by single word
adjectives
No guarantee that our language naturally includes every
scientifically
crucial aspect of T&P
If you were trying to reverse engineer cars, radios, computers would
you use this strategy?

#2. Reliance on Untrained Lay Raters to Specify Personality
Descriptors and to Provide Personality Descriptions

dents:
at are some potential problems

h using untrained raters (e.g.,
dents enrolled in college psychol
rses)?

Lay language may not make scientifically important distinctions
E.g., Fear (imminent threat) vs. anxiety (apprehension/distress
when threat is distal or uncertain)
Lay language is relatively gross, casual, unreflective, and surprisingly
inconsistent
E.g., "aggressive" - assertive vs. hostile
E.g., "critical" analytic vs. hostile

E.g., Fear (imminent threat) vs. Anxiety (apprehension/distress
when threat is remote, uncertain, or diffuse)
inconsistent

inconsistent

inconsistent

#3. Factor Analysis

#3. Factor Analysis
dents: Can factor analysis be use

ctively discover the fundamenta
ensions of T&P?

#3. Factor Analysis
Big 5 advocates suggest that the five factors are real because they
emerge for very large lists of adjectives across different factor
analytic methods

#3. Factor Analysis
analytic methods
Block counter-argues that if you closely scrutinize the results of
these most compelling demonstrations
The big list was actually chosen by earlier investigators
(Norman) to reflect a preconceived model of personality
There is still a good deal deal of subjective choices about which
words to use, how they should be clustered, what is important,
how many factors are sufficient, and so on basically, he argues
that the Big 5 advocates oversell the claim that they have
objectively discovered The Most Important Factors
The sequence of empirical procedures that repeatedly issued
similar five-factor
structures may have been constrained to produce the results
obtained; and the five-factor solution is not as stable as often
claimed

#3. Factor Analysis
analytic methods
Block counter-argues that if you closely scrutinize the results of
these most compelling demonstrations
The big list was actually chosen by earlier investigators
(Norman) to reflect a preconceived model of personality
Subjective, undocumented choices about which words to use,
how they should be clustered, what is important, how many
factors are sufficient, and so on basically, he argues that the
Big 5 advocates oversell the claim that they have objectively
discovered The Most Important Factors
The sequence of empirical procedures that repeatedly issued
similar five-factor
structures may have been constrained to produce the results
obtained; and the five-factor solution is not as stable as often
claimed
Not Just Block

[The purported fundamental dimensions of T&P]are
derived top-down from pools of lexically-chosen
questionnaire itemsnot from biological anchors.
They also depend on factor analysis, which determines the
number of dimensions,but not location of trait axesof the
personality space that items occupyIt is little more than
an act of faith to believe that the causal [i.e., real] structure
of personality is isomorphic with its lexical factor structure.
Further, not only is there no reason to suppose that
biologically accurate scales should have simple structure but
also current scale systems, even though designed to have
this, often do not...
McNaughton & Corr Frontiers in Sys Neurosci 20
Not Just Block

It is little more than an act of faith
to believe that the causal [i.e., real
scientific] structure of personality is
isomorphic with [i.e. the same as] its
lexical factor structure. to have this,
often d
But what does this even

mean?
It is little more than an act of faith
to believe that the causal [i.e., real
scientific] structure of personality is
isomorphic with [i.e. the same as] its
lexical factor structure. to have this,
often d
An explanation by
means of a automobile analogy
Epstein Psychol Inquiry 199
Like people, we can measure

individual differences in car
phenotypes
Parameters like horsepower,
maximum speed, fuel
consumption, off-road clearance,
deep-snow performance, crash
ratings, pollution
We could use factor analysis to
crunch these parameters into
broad-band factors
e.g. sport-performance

phenotypes
Parameters like horsepower, top
speed, fuel consumption, offroad clearance, deep-snow
performance, crash ratings,
pollution, size, and reliability
broad-band factors

phenotypes
Parameters like horsepower, top
speed, fuel consumption, offroad clearance, deep-snow
performance, crash ratings,
pollution, size, and reliability
broad-band factors that
summarize the phenotype
But a sport-performance score

tells you nothing about the
systems that CAUSE car-to-car
differences in the phenotype
Power plant
Powertrain and
Transmission
Suspension
Emissions
Its unlikely that a factor
analysis would discover the key
systems that are familiar to
every mechanic and automobile
engineer

Power plant
Powertrain and
Transmission
Suspension
Emissions
analysis of self-reported car
ratings would discover the key
engineer

Power plant
Powertrain and
Transmission
Suspension
Emissions
analysis of self-reported car
ratings would discover the key
engineer
The fundamental dimensions

are descriptive and superficial
not explanatory.
They give
you 199
Epstein Psychol
Inquiry
[Factor-analytic] Traits do notexplain behavior.

They describe patterns and consistencies in
behavior, but they don't explain where those
patterns and consistencies come from either
developmentally or in terms of their proximal
causation [i.e., underlying psychological or
neural systems]. What traits do not tell you, in
particular, is why somebody is doing something
David Funder (Psychol Inquiry, 1994)
Where Do We Go From Here?

Block offers a number of suggestions for moving beyond the FFM,
among them:
personality psychologists [should] not limit their thinking and research by
considering only what is or will be convenient to index via simple self-report
or layperson-report measures.
to study [T&P, we]will need to turn, or return, to more complicated and
complex ways of studying personsfor example, behavioral observations,
psychophysiological measures, individual differences in various
standardized situational contexts, the garnering of life facts about the
persons studied, truly intimate interviews, and the longitudinal study of
personality development.

among them:
complex ways of studying personsfor example, behavioral observations,
psychophysiological measures, individual differences in various
standardized situational contexts, the garnering of life facts about the
persons studied, truly intimate interviews, and the longitudinal study of
personality development.

among them:
[That is, do not consider the case of How many dimensions or Which
Dimensions closed]
complex ways of studying personsfor example,
behavioral observations,
psychophysiological measures,
individual differences in various standardized situational contexts,
the garnering of life facts about the persons studied,
truly intimate interviews, and
the longitudinal study of personality development.

among them:
Dimensions closed]
behavioral observations,
psychophysiological measures,
individual differences in various standardized situational contexts,
the garnering of life facts about the persons studied,
truly intimate interviews, and
the longitudinal study of personality development.

among them:
Dimensions closed]
Objective behavioral observations by trained raters
Psychophysiological and neurobiological measures
Individual differences in response to standardized challenges
Where do we go from here?

How should we measure T&P?
e Way to Measure Trait-Like

Individual Differences
What Are Some Potential Advantages

of Using Physiology to Assess T&P?
Students???

Objective, not subject to biases in reporting,
introspection, memory
Reporting Bias: Desire to look good (social
desirability); outright lying or malingering
Mnemonic Bias: Peak-End Rule
Limits of introspection: some processes may
occur outside of the focus of attention (e.g., habits)
or be opaque to introspection we may only be
aware of the downstream product or read-out of
more elementary processes
E.g., how much cortisol is in your blood stream?
How exactly did you drive from campus to your home?
Behavior is normally guided by both conscious and

pre-conscious processes (lie outside of awareness)

Reporting Bias: Desire to look good (social
desirability); outright lying or malingering
Mnemonic Bias: Peak-End Rule
Limits of introspection: some processes may occur
outside of the focus of attention (e.g., habits) or be
opaque to introspection we may only be aware of the
downstream product or read-out of more elementary
processes
Behavior is normally guided by both conscious and

pre-conscious processes (lie outside of awareness)

Reporting Bias: Desire to look good (social desirability);
outright lying or malingering
Mnemonic Bias: Peak-End rule of retrospective ratings
processes
Behavior is normally guided by both conscious and preconscious processes (lie outside of awareness)

peakonly be aware of the
opaque to introspection we may
processes
end
Ratherby
than
the conscious
average experience
Behavior is normally guided
both
and pre-
conscious processes (lie outside of awareness)

processes

processes
Some mental processes are too fast for self-report or lie outside of conscious
awareness
Some mental processes are too fast for self-report or lie outside of conscious
awareness
tudents???
Whats a possible solution?
Shackman et al. 2007; Shackman et al. 2011; Shackman Shackman et al. under
Note: We will discuss some other interesting

aspects of the unconscious mind/brain in
Module 5
Implication
Understanding aspects of T&P
(N/NE, E/PE, C/SC)
that lie outside of conscious awareness
mandates the use of behavioral or physiological assays
Tasks
Resting Physiology
ask-Evoked Physiology
Tasks
Cog Tasks
Standardized Behl Challenges
Resting Physiology
Tasks
Cog Tasks
Resting Physiology
PET
rs-fMRI
EEG
MRS
Tasks
Cog Tasks
Resting Physiology
PET
rs-fMRI
EEG
MRS
Startle Reflex
ERPs
PET During Behl Challenge
Just because youre

measuring the brain
Students?
sychometric = statistical
tionnaire measures of T&P, we need to establish that biological measure
over time (trait-like; test-retest)
sychometric = statistical
2 Kinds of Reliability
Degree to which a scale measures a single latent construct
Items hang together
Necessary but not sufficient condition for unidimensionality
requires factor analysis
Very high ICR is undesirable: items are redundant
Internal Consistency Reliability (Cronbachs Alpha)
Items hang together
Items hang together (covary) or provide the same rank order of Ss
Items hang together (covary) or provide the same rank order of Ss
Very high Alpha is undesirable: items are redundant
Brief examples
Items hang together or provide the same rank order of Ss
Temporal Stability / Test-Rest Reliability (Correlation or ICC)
Individual differences in T&P should be trait-like (stable)
Rank order consistency (mean neednt be stable)
Over more than ~30 days, however, you might expect genuine
change
For broad-band traits (e.g., neuroticism) long-term coefficients of
~.70 (.55 to .85) or about 50% variance
Individual differences in T&P should be stable and consistent
change
Rank order consistency (not absolute or mean value)
change
change
change
Construct Validity:
Linking Physiological Measures to T&P
What does a measure actually measure? (functional significance)
What inferences can we legitimately draw from a measure?
E.g., Is amygdala activation specifically indicative of anxiety
or is it activated by a range of socially and motivationally
significant stimuli?
E.g., Is electrodermal activity (SCR) indicative of stress,
arousal, cognitive load?
E.g., Is the startle reflex sensitive to any strong emotional
state or is it linearly related to valence (NEG > NEU > POS)?
Students???
Construct Validity:
state or is it linearly related to valence (NEG > NEU > POS)?
Students???
Construct Validity:
state or is it linearly related to emotional valence (NEG >
NEU > POS)?
Construct Validity:
state or is it linearly related to emotional valence (NEG >
NEU > POS)?
Students???
How might we forge this link ?
Construct Validity:
In order to establish the functional significance or construct
validity of a measure
1. Psychological Sensitivity
E.g., probability of amygdala activation, given fear-arousing
stimuli
AND
2. Psychological Specificity
Reverse
E.g., Probability of fear, given amygdala activation
Oftentimes, low specificity is not explicitly acknowledged
In both cases, mechanistic studies are helpful directly manipulate the
measure (lesions, TMS, drugs, etc.) and asses consequences for
construct of interest (e.g., fear) and other constructs (e.g., attention)
Construct Validity:
stimuli
AND
Reverse
Construct Validity:
stimuli
AND
Reverse inference
Construct Validity:
stimuli
AND
Reverse inference
because researchers focus on a small number of psychological
processes (e.g., emotion or cognition)
Construct Validity:
stimuli
AND
Reverse inference
because researchers focus on a small number of psychological
processes (e.g., emotion or cognition)
measure (lesions, TMS, drugs, etc.) and assess consequences for
Construct Validity Has

Fundamental Implications
As cognitive neuroscientists who use the same brain imaging

technology, we know that it is not possible to definitively determine
whether a person is anxious or feeling connected
simply by looking at activity in a particular brain region. This is so
because brain regions
are typically engaged by many mental states, and thus a one-to-one
mapping between a
brain region and a mental state is not possible.
As cognitive neuroscientists who usebrain imaging technology, we

know that it is not possible to definitively determine whether a person is
anxious or feeling connected
simply by looking at activity in a particular brain region.
This is so because brain regions are typically engaged by many mental
states, and thus a one-to-one mapping between a brain region and a
mental state is not possible.
As cognitive neuroscientists who usebrain imaging technology, we

know that it is not possible to definitively determine whether a person is
anxious or feeling connected
simply by looking at activity in a particular brain region.
This is so because brain regions are typically engaged by many mental
states, and thus a one-to-one mapping between a brain region and a
mental state is not possible.
Measuring T&P - Key Take-Home Points

1. There are serious problems with the 5 Factor Model
2. In particular, they do not represent an empirically discovered set of
natural kinds. They instead strongly reflect the theoretical perspectives,
methods, and assumptions of key investigators
3. Key assumptions underlying the 5 Factor Model may not be warranted
Words / Lexical Hypothesis
Lay assessments and lay language
Factor analysis
4. The best level of description for T&P (wide-band <-> narrow-band)
remains unclear
5. Expert behavioral and physiological assessments may prove illuminating
6. Especially given other known biases and limitations of introspective selfreports
Reporting biases (e.g., social desirability); Mnemonic biases (peakend rule)
Limits of introspection (too fast, outside of conscious awareness)
7. Just because you use biological measures doesnt mean you can forget
about psychometric reliability (internal-consistency and test-retest)

Factor analysis
remains unclear

Factor analysis
remains unclear

Factor analysis
remains unclear

Factor analysis
remains unclear

Factor analysis
remains unclear

Factor analysis
remains unclear

Factor analysis
remains unclear

Factor analysis
remains unclear

Factor analysis
remains unclear
Critical Thinking
Questions
Pick any 2 options

If low on time, read at home
Critical Thinking
Questions
1. Jack Block argued that,
we should not limit [our] thinking and research by considering
only what is or will be convenient to index via simple self-report
or layperson-report measuresTo study [T&P, we]will need to
turn, or return, to more complicated and complex ways of
studying persons
What do you think?
Based on what you know about the Big 3 traits (see the slides
from last time) or their facets, briefly describe a novel
experiment that exploits a measure other than self-report to
discover and understand some important aspect of T&P.
For example, you could think about ways to clarify whether C/SC
involves heightened anxiety about order, rules, or deadlines.
Critical Thinking
Questions
1. Jack Block argued that,
we should not limit [our] thinking and research by considering
only what is or will be convenient to index via simple self-report
or layperson-report measuresTo study [T&P, we]will need to
turn, or return, to more complicated and complex ways of
studying persons
What do you think?
Based on what you know about the Big 3 traits (see the slides
from last time) or their facets, briefly describe a novel
experiment that exploits a measure other than self-report to
discover and understand some important aspect of T&P.
For example, you could think about ways to clarify whether C/SC
involves heightened anxiety about order, rules, or deadlines.
Critical Thinking
Questions
2. If traits are superficial and descriptive (rather than explanations of behavior), what good
are they?
David Funder (1994) described what I call The Neurotic Professor,
I have a friend and colleague who has served as a faculty member in several departments.
In each job, he's been miserable. He can enumerate very persuasively why each of his
department chairs was an unfair tyrant, why each teaching load was excessive, how office
space and salary were unfair and paltryHe really is persuasive. But you know what? I
think he'd be unhappy anywhere. I think he's got a trait. He exhibits a behavioral pattern
(of complaining) from which I infer an emotional pattern (of feeling miserable) that I think
does explain why he is saying such nasty things about his latest department.
To explain a behavior in terms of the broader pattern of which it is a part-as I have just
done-can be a legitimate and useful step in the (infinite) explanatory regress. I know, it's
not complete. We still don't know why my friend is such a negativistic cynic. But identifying
him as such provides (a) insight into his current actions (and feelings, which we infer from
his actions), (b) a basis for predicting his future actions and feelings, and (c) a useful
analytic rest stop. We have explained his behavior; take a breath; now we need to explain
the explanation
What do you think?
Briefly describe a strategy or two for addressing the origins of The Neurotic Professor?
Critical Thinking
Questions
are they?
the explanation
What do you think?
Critical Thinking
Questions
are they?
the explanation
What do you think?
Critical Thinking
Questions
are they?
I have a friend and colleague who has served as a faculty member in several departments. In
each job, he's been miserable. He can enumerate very persuasively why each of his
space and salary were unfair and paltryHe really is persuasive. But you know what? I think
he'd be unhappy anywhere. I think he's got a trait. He exhibits a behavioral pattern (of
complaining) from which I infer an emotional pattern (of feeling miserable) that I think does
explain why he is saying such nasty things about his latest department.
To explain a behavior in terms of the broader pattern of which it is a part-as I have just donecan be a legitimate and useful step in the (infinite) explanatory regress. I know, it's not
complete. We still don't know why my friend is such a negativistic cynic. But identifying him
as such provides (a) insight into his current actions (and feelings, which we infer from his
actions), (b) a basis for predicting his future actions and feelings, and (c) a useful analytic
rest stop. We have explained his behavior; take a breath; now we need to explain the
explanation
What do you think?
Briefly describe a strategy or two for addressing the origins of The Neurotic Professor, that is,
for reverse engineering the systems that underlie N/NE or some other dimension of T&P.
Critical Thinking
Questions
3. Andy Tomarken underscored the importance of reliability for
behavioral and biological measures of T&P
Recent work suggests that the dot-probe task (a widely used
measure of anxious individuals vigilance for threat) is not reliable
What do you think?

Skim the paper
(http://journal.frontiersin.org/Journal/10.3389/fpsyg.2014.01368/fu
ll) and and briefly comment on the key take-home points. What
are the implications for understanding the cognitive underpinnings
of anxiety and other key facets of T&P
Critical Thinking
Questions
What do you think?

Skim the paper
are the broader implications for understanding the cognitive
underpinnings of anxiety and other key facets of T&P
Critical Thinking
Questions
What do you think?

Skim the paper
are the implications for understanding the cognitive underpinnings
of anxiety and other key facets of T&P
Critical Thinking
Questions
4. Andy Tomarken underscored
the importance of establishing the
sensitivity and specificity (construct validity) of biological measures of
T&P (e.g., fMRI, PET, EEG, ERP, GSR, startle, HR, cortisol, etc.)
What do you think?
a) Briefly describe how you might test the specificity of one of these
measures, the probability that it will be activated or engaged given a
particular process or perturbation (such as the induction of anxiety vs.
other.
-orb) Play with Neurosynth.org, an on-line tool that allows you to dynamically
assess the sensitivity and specificity of different brain regions. Then
write a short paragraph describing what you did and what you learned.
It need not be a deep insight, its sufficient to just go play with it and
see what you uncover about sensitivity (forward inference) and
specificity (reverse inference). The next few slides provide some
pointers for interested students.
Critical Thinking
Questions
What do you think?
other states).
Critical Thinking
Questions
4. Scientists can be sloppy in their use of language. Does fear in a rat exposed to cues
paired with shock reflect the same processes involved in N/NE? In anxiety disorders? In
specific phobias? Is Neuroticism in adults synonymous with Behavioral Inhibition in kids?
What about Harm Avoidance?
In a recent review focused on Theory of Mind, Schaafsma and colleagues (TiCS 2015)
described this kind of communication break down:
The term X [e.g. self-control] is used interchangeably with Q, R, and S, to name only a
few. This diversity of terms used is probably telling: different investigators have different
concepts in mind.
Different levels of description (e.g., neural vs. self-report vs. behavior), together with the
different terms used, make it difficult even for experts from different fields to navigate
both what is meant by X and how to study it using scientific methods...to the uninitiated,
the topic becomes bewildering.
The problem is that these differences [are implicit and unspoken]
[Thus] it has been [extraordinarily difficult] to triangulate the actual [psychological,
genetic, and neuobiological] processes involved.
What do you think? Briefly describe a strategy for addressing this road block.
Critical Thinking
Questions
concepts in mind.
Critical Thinking
Questions
concepts in mind.
Critical Thinking
Questions
concepts in mind.
Critical Thinking
Questions
concepts in mind.
Critical Thinking
Questions
What do you think?
other states).
Critical Thinking
Questions
What do you think?
other states).
Critical Thinking
Questions
What do you think?
other states).
Time-Permitting
Review Questions
Note: NeuroSynth.org slides follow
Moffitt et al showed that childhood self-control predicts health, wealth &

public safety in midlife. What was one intervening mechanism during
adolescence that partially explained the link from kid temperament to
deleterious adult outcomes?
A. Smoking
B. Becoming a parent
C. Excessive video
game playing
D. Violence in the
media
E. High-caffeine energy
drinks
F. A & B
G. C & D
Correlation and variance explained: If

two variables are correlated R = .50, the
amount of variance accounted for is:
A. 0.50 * 0.50 = .25 =

25%
B. 0.50 / 0.50 = 1 =
100%
C. Sqrt(.50) = .7071 =
70%
T&P reflect trait-like individual differences in

emotional and cognitive biases that
A. First emerge early in life
B. Continue to evolve for many
years
C. Account for consistency in
behavior, inner experience,
and risk across time and
contexts. Can be relatively
simple (e.g., anxious distress)
or complex and multiply
determined (orderliness).
Excessive video game playing
D. Can be relatively simple
E. Can be complex and
multidimensional
F. All of the above
T&P are not different in kind (according to Shackman)

because they are both
A. Biological
B. Emotional
C. Cognitive
D. Somewhat
heritable
E. All of the above
What are the 3 fundamental dimensions of T&P?
A. N/NE
B. P/TA
C. E/PE
D. S/RE
E. C/SC
F. A, C, and E
G. A, B, and C
N/NE can be dissected into which 2 facet traits
A. Distress
(fear/anxiety)
and Irritation
(anger)
B. Guilt and Shame
Which statistical test is used to quantify

the continuity (temporal stability) of traits
A. Students t test
B. ANOVA
C. Correlation
T&P is
A. Fixed and
immutable
B. Moderately stable
(R = 0.4 to 0.6
over periods of
one to several
years)
C. Completely plastic
and malleable
NeuroSynth.org Slides
can click around with your mouse on the brain and the coordinates will be autofi
The End
Material for Future

Semesters
This could be incorporated into lecture

or- as a take home question
Incorporate some of Grubers material @ https://

www.youtube.com/watch?v=g1lXq5VIKS4&list=PLh9mgdi4rNewie
O9Dsj-OhNBC9bF4FoRp
Gruber interview of Mauss could be critical q
https://www.youtube.com/watch?v=dDMdE3MnBFU
Note that chapter 1 of matthews deary and whiteman is pretty

good; optional or even assigned reading
Alex you have the damasio material in the next lecture,
#5.copy and paste that version into this version and make it
more explicit that if low on time, skip.
Future Semester Thought

Question
Just out .. earlyview ..

Park, G., Schwartz, H.A., Eichstaedt, J.C., Kern, M.L., Kosinski, M., Stillwell, D.J., Ungar, L.H., & Seligman, M.E.P. (2014).Automatic personality assessment through social media language.Journal of Personality and Social
Psychology, EarlyView, , 1-20.
Abstract
Language use is a psychologically rich, stable individual difference with well-established correlations to personality. We describe a method for assessing personality using an open-vocabulary analysis of language from
social media. We compiled the written language from 66,732 Facebook users and their questionnaire-based self-reported Big Five personality traits, and then we built a predictive model of personality based on their
language. We used this model to predict the 5 personality factors in a separate sample of 4,824 Facebook users, examining (a) convergence with self-reports of personality at the domain- and facet-level; (b)
discriminant validity between predictions of distinct traits; (c) agreement with informant reports of personality; (d) patterns of correlations with external criteria (e.g., number of friends, political attitudes,
impulsiveness); and (e) testretest reliability over 6-month intervals. Results indicated that language-based assessments can constitute valid personality measures: they agreed with self-reports and informant reports of
personality, added incremental validity over informant reports, adequately discriminated between traits, exhibited patterns of correlations with external criteria similar to those found with self-reported personality, and
were stable over 6-month intervals. Analysis of predictive language can provide rich portraits of the mental life associated with traits. This approach can complement and extend traditional methods, providing
researchers with an additional measure that can quickly and cheaply assess large groups of participants with minimal burden.

The key results can be seen in Table 1:

From this, the authors conclude (1stsentence of their Conclusions):
In this article, we provided evidence that the language in social media can be harnessed to create a valid and reliable measure of personality.

Some context: A correlation of 0.40, sampling a random normal bivariate distribution, 4,000 cases, integer scale varying between 0 and 20, mean = 10, SD = 3,
and plotted as a frequency scatterplot (the bigger circles indicate higher frequencies of occurrence):

What the authors do not do is engage with that word accuracy, using the kinds of analyses (in the metric of the criterion data) that better inform the reader about the kinds and patterns of errors associated with
using a Language Based Assessment (LBA) of Personality rather than a Questionnaire Based Assessment. And the authors do make a recommendation for usage:
LBAs offer a practical, cost-effective alternative, allowing assessment of psychological characteristics when questionnaires are impractical. Researchers could reduce participant burden by replacing some
questionnaires with a single link and sign-in procedure, allowing a research application to access participant social media language and quickly assess personality and other characteristics of interest.

Correlation is not the correct statistic to be used here. You need estimates of errors that are sensitive to actual score magnitudes, not monotonicity relations. You also need to conduct a fair few actuarial and
discrepancy-based analyses/graphics to properly understand both accuracy and error, and where LBAs are more accurate over a range of personality variation (do they predict high scorers better than low scorers?).

All in all, sophisticated, innovative, and very, very clever work; but, for someone who aspires to greater accuracy of psychological assessment, I am puzzled by the seeming indifference to accuracy shown within this
entire line of research.

Anyway, this is already hot stuff.. as Michal Kosinski describes on his website:
http://www.michalkosinski.com/home/publications
Private traits and attributes are predictable from digital records of human behaviorby M. Kosinski, D. Stillwell, T. Graepel,Proceedings of the National Academy of Sciences (PNAS),2013.
The 9th most discussed paper of 2013 and the 4th most influential article ever published by PNAS (Altmetric.com)

Regards .. Paul

Chief Research Scientist
Cognadev.com
Existing theories of personality provide a number of competing surface level, lexically-derived,

systems with trait measures that relate to approach and avoidance either indirectly via
constructs such as Extraversion and Neuroticism (Eysenck, 1957) or directly via constructs such
as Harm Avoidance (Cloninger et al., 1993). Each system is stable, with links to mental disorder
(Strelau and Zawadzki, 2011;Gomez et al., 2012;Mullins-Sweatt and Lengel, 2012;Trull, 2012)
and brain structure (Gardini et al., 2009;DeYoung et al., 2010). But even when starting with
approach and avoidance as primary constructs, they are derived top-down from pools of
lexically-chosen questionnaire items (Carver and White, 1994;Elliot and Thrash, 2010) not from
biological anchors. They also depend on factor analysis, which determines the number of
dimensions,but not location of trait axesof the personality space that items occupy (Lykken
, 1971;Corr and McNaughton, 2008). It is little more than an act of faith to believe that the
causal structure of personality is isomorphic with its lexical factor structure. So, even if we knew
for certain that there were only two dimensions within a particular measured personality space,
one questionnaire system could have a single simple trait anxiety dimension (orthogonal to,
say, impulsiveness) that was a combination of neuroticism and introversion in another (
Gray, 1970)the two systems differing only on which items from an original pool were used to
create scales. Factor analytically derived trait measures can also easily meet the criterion of
having simple structure (in the sense that a set of items loads highly on only one factor so
factors can be clearly identified by unique item loadings) while implying improbable causation (
Lykken, 1971). Further, not only is there no reason to suppose that biologically accurate scales
should have simple structure but also current scale systems, even though designed to have
this, often do not (DeYoung, 2006,2010).

Goldberg 1970s 1990s and Costa & McCrae 1980s 1990s
Similar concerns
In designing their questionnaire, Costa and McCrae distinguished and
permanently fixed upon [case closed!] a half dozen facets each for their
broad constructs of Neuroticism, Extraversion, and Openness.
The facet distinctions they offered were not rooted in factor
analysis, formal theorizing, or ineluctable empirical findings.
Rather, the facets derived from their personal thinking about how
the three domains could be further articulated.
The six facets Costa and McCrae nominated to represent the Neuroticism
domain were Depression, Impulsiveness, Anxiety, Hostility, Selfconsciousness, and Vulnerability.
change
Reliability (Alpha and Test-Retest)
Sets an upper limit on the magnitude of correlations with external
measures
Can be enhanced by aggregation
Average out random noise or state-specific variance
change
measures
Can be enhanced by aggregation (increase the number of
measurements)
change
measures
measurements)
change
measures
measurements)

Shackman Psyc210 Module04 Measuring

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Shackman Psyc210 Module04 Measuring

Hochgeladen von

Copyright:

Verfügbare Formate

Nuts & Bolts Plan for

Lecture (Block 95; Tomarken 95)

Take-home critical thinking questions

Nuts & Bolts Plan for

Lecture (Block 95; Tomarken 95)

Take-home critical thinking questions

Nuts & Bolts Plan for

Lecture (Block 95; Tomarken 95)

Take-home critical thinking questions

Nuts & Bolts Plan for

Lecture (Block 95; Tomarken 95)

Take-home critical thinking questions

Todays Conceptual Roadmap

Todays Conceptual Roadmap

Todays Conceptual Roadmap

Todays Conceptual Roadmap

From Measures to Models

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Whats Factor Analysis

Whats Factor Analysis

Whats Factor Analysis

Whats Factor Analysis

Choices, choices, choices

Choices, choices, choices

tive: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors

Choices, choices, choices

factor explains a great

2nd factor explains

tive: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors

Choices, choices, choices

Varying degrees of information loss,

oss: 100 Cumulative

Choices, choices, choices

Varying degrees of information loss,

oss: 100 Cumulative

Pick one big factor, four factors, many fact

If we retain 4 factors (~2/3

Garbage In/Garbage Out

Garbage In/Garbage Out

Garbage In/Garbage Out

Garbage In/Garbage Out

Garbage In/Garbage Out

Back to Blocks history

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

udents Any concerns with this approac

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Jack Blocks Critique

udents: What are some potentia

Jack Blocks Critique

udents: What are some potentia

Jack Blocks Critique

udents: What are some potentia

Jack Blocks Critique

Jack Blocks Critique

Jack Blocks Critique

at are some potential problems

Jack Blocks Critique