Shackman Psyc210 Module04 Measuring 021015

Nuts
& Bolts Plan for Today

Cumula4ve review
Lecture (Block 95; Tomarken 95)
Measuring T&P
Some problems with words
Some problems with physiological measures of T&P,
Just Because Youre Measuring the Brain (or HPA axis, or facial
musculature, etc) Doesnt Mean You Can Stop Using Your Head
John Cacioppo, past present of the Assoc for Psychol Science
Take-home cri4cal thinking ques4ons
Nuts & Bolts Plan for Today

Measuring T&P
Problems with words
Problems with physiological measures of T&P
Just Because Youre Measuring the Brain (or HPA axis, or
facial musculature, etc) Doesnt Mean You Can Stop Using
Your Head
John Cacioppo, past present of the Assoc for Psychol
Science

Measuring T&P
Problems with words
Just Because Youre Measuring the Brain Doesnt Mean You
Can Stop Using Your Head
John Cacioppo, past present of the
Assoc for Psychol Science (APS)

Measuring T&P
Problems with words
Just Because Youre Measuring the Brain Doesnt Mean You
Can Stop Using Your Head
John Cacioppo, past present of the
Assoc for Psychol Science (APS)
PSYC 210:

How Can We Discover and Measure
Individual Dierences in T&P?

AJ Shackman
10 February 2015
Todays Conceptual Roadmap

Where did the fundamental dimensions of T&P
come from? How were they discovered? What are
their limita4ons?
How should we measure T&P? What is the value
of adop4ng biological measures?
What sta4s4cal proper4es do we need to assess if
we turn to biological measures of T&P?

their limita4ons?

their limita4ons?

their limita4ons?
we use biological measures of T&P?
From Measures to Models

The Case of the 5 Factor Model, or,

How did we get
the factors that
we were given?
Why 5 Factors? Words, words, words

Lexical Hypothesis,"All aspects of human personality which are or have been of importance,
interest, or u4lity have already become recorded in the substance of language Ca[ell

Common speech is a poor guide to psychological subtle4es Allport


Lexical Hypothesis,"All aspects of human personality which are or have been of importance,
interest, or u4lity have already become recorded in the substance of language Ca[ell

Common speech is a poor guide to psychological subtle4es Allport


Allport 1930s
400,000 words in the unabridged dicJonary 18,000 trait-relevant adjecJves
4,500 key traits

Ca[ell 1940s
Used a combinaJon of ad hoc subjecJve and staJsJcal techniques to whiQle it down
to 35 scales
Used factor analysis to reduce it to 12
Set the stage for the discovery of the Big 5 (OCEAN)

Allport 1930s
400,000 words in the unabridged dicJonary 18,000 trait-relevant adjecJves
4,500 key traits

Ca[ell 1940s
Used a combinaJon of subjecJve and staJsJcal techniques to whiQle it down to 35
scales
Used factor analysis to reduce it to 12
Set the stage for the discovery of the Big 5 (OCEAN)
Whats Factor Analysis

Factor Analysis
StaJsJcal technique for compressing or reducing the number of dimensions in a
dataset
e.g., 100 items on a quesJonnaire 5 latent factors or dimensions (95%
compression)
Lossy compression technique: RetenJon of a small number of dimensions typically
captures <100% of the variance in the observed data
Key limitaJons
SubjecJve
How many dimensions is sucient? Should the factors be orthogonal
(uncorrelated) or oblique (correlated)? Analyst gets to pick.
Garbage In/Garbage Out
Results are data-dependent. If the analyst includes 30 anxiety items and 2
thought-disturbance items, the factor analysis will likely tell you that a
single anxiety dimension is sucient, but this simply reects the choice of
inputs

Factor Analysis
dataset
e.g., 100 items on a quesJonnaire 5 latent factors or dimensions (95%
compression)
Key limitaJons
SubjecJve
inputs

Factor Analysis
dataset
e.g., 100 items on a quesJonnaire 5 factors or dimensions (95% compression)
Key limitaJons
SubjecJve
inputs

Factor Analysis
dataset
e.g., 100 items on a quesJonnaire 5 factors or dimensions (95% compression)
Key limitaJons
SubjecJve
inputs
Non-MathemaCcal Example

Recall the survey that
you completed in class
Choices, choices, choices
CumulaJve: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors

1st factor explains a great
deal of variance in scores
2nd factor explains

much less
Factors 3-18 account for
less and less variance
CumulaJve: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors

Loss of 56.3%
Loss of 13.5%

Varying degrees of informaCon loss,
depending on the number of factors we
choose to retainfewer factors, more loss
Loss of 0.0%
Loss: 100 CumulaJve

Loss of 56.3%
Loss of 13.5%
Loss of 0.0%
Loss: 100 CumulaJve

Varying degrees of informaCon loss,
depending on the number of factors we
choose to retainfewer factors, more loss

Pick one big factor, four factors, many factors?
If we retain 4 factors (~2/3 variance)

F1: Distress
F2: Worry
F3: Depression
F4: Junk
Limita4ons of Factor Analysis
Subjec4ve, the Analyst Has to Choose

The number of factors to retain
The rotaJon, should the factors be staJsJcally uncorrelated
(orthogonal) or correlated (oblique)
"...each orientaJon is equally acceptable mathemaJcally. But dierent

theories proved to dier as much in terms of the [choice of rotaJon] as
in terms of anything else, so that model rng did not prove to be
useful in disJnguishing among theories." (Sternberg, 1977).
All rotaJons are equally valid mathemaJcal outcomes. There is no
unique or opJmal soluJon. Dierent interpretaJons are equally valid.

Results are data-dependent. If the analyst includes 30 anxiety
items and 2 thought-disturbance items, the factor analysis will
likely tell you that a single anxiety dimension is sucient, but this
simply reects the choice of inputs


"...each orientaJon is equally acceptable mathemaJcally. But dierent

All rotaJons are equally valid mathemaJcal outcomes. There is no
unique or opJmal soluJon. Dierent interpretaJons are equally valid.



"...each orientaJon is equally acceptable mathemaJcally.

F1 F2 But dierent
More concretely, we have to choose whether we
tAll
are
equally
valid am
think
hat rFotaJons
1 (distress)
and
F2 (worry)
re athemaJcal outcomes. There is no
unique
or opJmal
soluJon.
related
or unrelated,
in terms
of their Dierent interpretaJons are equally valid.
underlying psychology and neurobiology



All rotaJons are equally valid mathemaJcal outcomes there
is nothing inherent in the mathemaJcs of factor analysis that
tells you which rotaJon is the right one

Results are data-dependent. If the analyst includes 30
anxiety items and 2 thought-disturbance items, the factor
analysis will likely tell you that a single anxiety dimension is
sucient, but this simply reects the choice of inputs

All rotaJons are equally valid mathemaJcal outcomes there
is nothing inherent in the mathemaJcs of factor analysis that
tells you which rotaJon is the right one

Results are data-dependent. If you include 30 anxiety items but
only 2 thought-disturbance items, the factor analysis will indicate
that a single anxiety dimension is sucient to describe T&P, but
this one dimensional soluJon is an arJfact of the inputs
Back to Blocks history lesson

Tupes & Christal 1940s
personnel selecJon psychologists employed by the Air Force to improve ocer
selecJon and promoJon procedures
factor analyses based on ocers raJngs of one another
Can the psychological percep4veness of Air Force ocers and ocer candidates, as
quickly expressed by 3-point ra4ngs on 30 or so scales in an ocially required
research program regarding 12 to 30 of their peers known for such short periods [as
few as 3 days], provide a fundamental data basis for discerning the essen4al
dimensions for the scien4cally sucient descrip4on of personality? Block 95
5 factors:
Surgency/Extroversion, Agreeableness, Dependability, EmoJonal Stability, and
Culture.
Norman 1960s
Again, various subjecJve (but herculean) ad hoc approaches to turning very large lists
of words (adjecJves or trait descriptors) into a manageable set of dimensions


Can the psychological percep4veness of Air Force ocers and ocer candidates, as
quickly expressed by 3-point ra4ngs on 30 or so scales in an ocially required
research program regarding 12 to 30 of their peers known for such short periods [as
few as 3 days], provide a fundamental data basis for discerning the essen4al
dimensions for the scien4cally sucient descrip4on of personality? Block 95
5 factors:
Culture.
Students Any concerns with this approach?
Norman 1960s


Discovered 5 factors:
Culture.
Norman 1960s


Discovered 5 factors:
Culture.
Norman 1960s
Again, various subjecJve approaches for turning very large lists of words (adjecJves
or trait descriptors) into a manageable set of dimensions


Culminated in the work of Goldberg 1970s 1990s and Costa & McCrae 1980s 1990s
Similar concerns
In designing their quesJonnaire, Costa and McCrae disJnguished and permanently xed
upon [case closed!] a half dozen facets each for their broad constructs of NeuroJcism,
Extraversion, and Openness.

The facet dis4nc4ons they oered were not rooted in factor analysis, formal theorizing, or
ineluctable empirical ndings. Rather, the facets derived from their personal thinking about
how the three domains could be further ar4culated.

The six facets Costa and McCrae nominated to represent the NeuroJcism domain were
Depression, Impulsiveness, Anxiety, HosJlity, Self-consciousness, and Vulnerability.

Jack Blocks Cri4que

The Lexical Hypothesis
Students: What are some potenCal

problems with the lexical hypothesis?
Jack Blocks Cri4que

#1. The Lexical Hypothesis

Jack Blocks Cri4que


Jack Blocks Cri4que

The premise is false
Meaningful aspects of T&P may not be captured by single word adjecJves
No guarantee that our language naturally includes every scienJcally
crucial aspect of T&P

If you were trying to reverse engineer cars, radios, computers would you use this
strategy?
Jack Blocks Cri4que

#2. Reliance on Untrained Lay Raters to Specify Personality Descriptors and to Provide
Personality Descrip4ons
Jack Blocks Cri4que

Students:

What are some potenCal problems
with using untrained raters (e.g.,
students enrolled in college psychology
courses)?
Jack Blocks Cri4que

Lay language may not make scienJcally important disJncJons
E.g., Fear (imminent threat) vs. anxiety (apprehension/distress when threat is
distal or uncertain)
Lay language is relaJvely gross, casual, unreecJve, and surprisingly inconsistent
E.g., "aggressive" - asserJve vs. hosJle
E.g., "criJcal" analyJc vs. hosJle
Jack Blocks Cri4que

E.g., Fear (imminent threat) vs. Anxiety (apprehension/distress when threat is
remote, uncertain, or diuse)
Jack Blocks Cri4que

Jack Blocks Cri4que


Jack Blocks Cri4que

#3. Factor Analysis
Jack Blocks Cri4que

#3. Factor Analysis
Students: Can factor analysis be used to

objecCvely discover the fundamental
dimensions of T&P?
Jack Blocks Cri4que

#3. Factor Analysis
Big 5 advocates suggest that the ve factors are real because they emerge for very
large lists of adjecJves across dierent factor analyJc methods
Jack Blocks Cri4que

#3. Factor Analysis
Block counter-argues that if you closely scruJnize the results of these most
compelling demonstraJons
The big list was actually chosen by earlier invesJgators (Norman) to reect a
preconceived model of personality
There is sJll a good deal deal of subjecJve choices about which words to use,
how they should be clustered, what is important, how many factors are
sucient, and so on basically, he argues that the Big 5 advocates oversell the
claim that they have objecJvely discovered The Most Important Factors
The sequence of empirical procedures that repeatedly issued similar ve-factor
structures may have been constrained to produce the results obtained; and the
ve-factor soluJon is not as stable as oyen claimed
Jack Blocks Cri4que

#3. Factor Analysis
Block counter-argues that if you closely scruJnize the results of these most
compelling demonstraJons
The big list was actually chosen by earlier invesJgators (Norman) to reect a
preconceived model of personality
SubjecJve, undocumented choices about which words to use, how they should
be clustered, what is important, how many factors are sucient, and so on
basically, he argues that the Big 5 advocates oversell the claim that they have
objecJvely discovered The Most Important Factors
The sequence of empirical procedures that repeatedly issued similar ve-factor
structures may have been constrained to produce the results obtained; and the
ve-factor soluJon is not as stable as oyen claimed
Not Just Block

[The purported fundamental dimensions of T&P]are
derived top-down from pools of lexically-chosen
quesJonnaire itemsnot from biological anchors.

They also depend on factor analysis, which determines the
number of dimensions, but not locaGon of trait axes of the
personality space that items occupyIt is liQle more than an
act of faith to believe that the causal [i.e., real] structure of
personality is isomorphic with its lexical factor structure.

Further, not only is there no reason to suppose that
biologically accurate scales should have simple structure but
also current scale systems, even though designed to have this,
oyen do not...
McNaughton & Corr FronJers in Sys Neurosci 2014
Not Just Block

It is liale more than an act of faith to believe that
the causal [i.e., real scienCc] structure of
personality is isomorphic with [i.e. the same as] its
lexical factor structure. to have this, oyen d
But what does this even mean?

It is liale more than an act of faith to believe that
the causal [i.e., real scienCc] structure of
personality is isomorphic with [i.e. the same as] its
lexical factor structure. to have this, oyen d
An explanaCon by
means of a automobile analogy
Epstein Psychol Inquiry 1994

Like people, we can measure individual
dierences in car phenotypes

Parameters like horsepower, maximum
speed, fuel consumpJon, o-road
clearance, deep-snow performance,
crash raJngs, polluJon

We could use factor analysis to crunch
these parameters into broad-band
factors

e.g. sport-performance


Parameters like horsepower, top speed,
fuel consumpJon, o-road clearance,
deep-snow performance, crash raJngs,
polluJon, size, and reliability

factors



Parameters like horsepower, top speed,
fuel consumpJon, o-road clearance,
deep-snow performance, crash raJngs,
polluJon, size, and reliability

factors that summarize the phenotype


But a sport-performance score tells
you nothing about the systems that
CAUSE car-to-car dierences in the
phenotype

Power plant
Powertrain and Transmission
Suspension
Emissions

Its unlikely that a factor analysis would
discover the key systems that are
familiar to every mechanic and
automobile engineer

phenotype

Power plant
Suspension
Emissions

Its unlikely that a factor analysis of self-
reported car raJngs would discover the
key systems that are familiar to every
mechanic and automobile engineer


phenotype

Power plant
Suspension
Emissions

Its unlikely that a factor analysis of self-
reported car raJngs would discover the
key systems that are familiar to every
mechanic and automobile engineer

The fundamental dimensions are
descripJve and supercialnot
explanatory. They give you no clues
about how a car works or what to do
when it breaks down
[Factor-analyGc] Traits do notexplain behavior.

They describe paPerns and consistencies in
behavior, but they don't explain where those
paPerns and consistencies come from either
developmentally or in terms of their proximal
causaGon [i.e., underlying psychological or neural
systems]. What traits do not tell you, in parGcular,
is why somebody is doing something

David Funder (Psychol Inquiry, 1994)

Where Do We Go From Here?

Block oers a number of sugges4ons for moving beyond the FFM, among them:

personality psychologists [should] not limit their thinking and research by considering only
what is or will be convenient to index via simple self-report or layperson-report measures.

to study [T&P, we]will need to turn, or return, to more complicated and complex ways of
studying personsfor example, behavioral observaJons, psychophysiological measures,
individual dierences in various standardized situaJonal contexts, the garnering of life facts
about the persons studied, truly inJmate interviews, and the longitudinal study of personality
development.



studying personsfor example, behavioral observaJons, psychophysiological measures,
individual dierences in various standardized situaJonal contexts, the garnering of life facts
about the persons studied, truly inJmate interviews, and the longitudinal study of personality
development.



[That is, do not consider the case of How many dimensions or Which Dimensions closed]

studying personsfor example,

behavioral observaJons,
psychophysiological measures,
individual dierences in various standardized situaJonal contexts,
the garnering of life facts about the persons studied,
truly inJmate interviews, and
the longitudinal study of personality development.





behavioral observaJons,
psychophysiological measures,
individual dierences in various standardized situaJonal contexts,
the garnering of life facts about the persons studied,
truly inJmate interviews, and
the longitudinal study of personality development.





ObjecJve behavioral observaJons by trained raters
Psychophysiological and neurobiological measures

Individual dierences in response to standardized challenges
Where do we go from here?

How should we measure T&P?
One Way to Measure Trait-Like

Individual Dierences
What Are Some Poten4al Advantages

of Using Physiology to Assess T&P?
Students???

ObjecJve, not subject to biases in reporJng,
introspecJon, memory
Repor4ng Bias: Desire to look good (social

desirability); outright lying or malingering
Mnemonic Bias: Peak-End Rule
Limits of introspec4on: some processes may occur
outside of the focus of aQenJon (e.g., habits) or be
opaque to introspecJon we may only be aware of
the downstream product or read-out of more
elementary processes
E.g., how much corJsol is in your blood stream?
How exactly did you drive from campus to your home?
Behavior is normally guided by both conscious and

pre-conscious processes (lie outside of awareness)

Repor4ng Bias: Desire to look good (social desirability);

outright lying or malingering
Mnemonic Bias: Peak-End Rule
opaque to introspecJon we may only be aware of the
downstream product or read-out of more elementary
processes
Behavior is normally guided by both conscious and pre-

conscious processes (lie outside of awareness)


Mnemonic Bias: Peak-End rule of retrospecJve raJngs
processes



opaque to introspecJon we peak
may only be aware of the
processes
E.g., how much corJsol is in your blood stream? end
Rather than the average experience



processes



processes


Some mental processes are too fast
for self-report or lie outside of
conscious awareness

Some mental processes are too fast
for self-report or lie outside of
conscious awareness

Students???
Whats a possible soluCon?
Shackman et al. 2007; Shackman et al. 2011; Shackman Shackman et al. under review
Note: We will discuss some other

interes4ng aspects of the unconscious
mind/brain in Module 5
Implica4on
Understanding aspects of T&P
(N/NE, E/PE, C/SC)

that lie outside of conscious awareness

mandates the use of behavioral or
physiological assays
Tasks
Res4ng Physiology
Task-Evoked Physiology
Tasks
Cog Tasks
Standardized Behl Challenges
Res4ng Physiology
Tasks
Cog Tasks
Res4ng Physiology
PET
rs-fMRI
EEG
MRS
Tasks
Cog Tasks
Res4ng Physiology
PET
rs-fMRI
EEG
MRS
Startle Reex
ERPs
PET During Behl Challenge
Just because youre

measuring the brain

Students?
psychometric = staJsJcal

Like quesConnaire measures of T&P, we need to establish that biological measures are:

- Reliable
- Stable over Cme (trait-like; test-retest)
psychometric = staJsJcal
2 Kinds of Reliability

Degree to which a scale measures a single latent construct

Items hang together
Necessary but not sucient condiJon for unidimensionality requires factor analysis
Very high ICR is undesirable: items are redundant

Internal Consistency Reliability (Cronbachs Alpha)
Items hang together


Items hang together (covary) or provide the same rank order of Ss


Items hang together (covary) or provide the same rank order of Ss
Very high Alpha is undesirable: items are redundant

Brief examples

Items hang together or provide the same rank order of Ss

Temporal Stability / Test-Rest Reliability (Correla4on or ICC)
Individual dierences in T&P should be trait-like (stable)
Rank order consistency (mean neednt be stable)
Over more than ~30 days, however, you might expect genuine change
For broad-band traits (e.g., neuroJcism) long-term coecients of ~.70 (.55 to .85) or
about 50% variance


Individual dierences in T&P should be stable and consistent
about 50% variance


Rank order consistency (not absolute or mean value)
about 50% variance


about 50% variance


about 50% variance
Construct Validity:
Linking Physiological Measures to T&P

What does a measure actually measure? (func4onal signicance) What inferences can we
legi4mately draw from a measure?
E.g., Is amygdala ac4va4on specically indica4ve of anxiety or is it ac4vated by a
range of socially and mo4va4onally signicant s4muli?
E.g., Is electrodermal ac4vity (SCR) indica4ve of stress, arousal, cogni4ve load?
E.g., Is the startle reex sensi4ve to any strong emo4onal state or is it linearly
related to valence (NEG > NEU > POS)?
Students???
Construct Validity:

related to valence (NEG > NEU > POS)?
Students???
Construct Validity:

related to emo4onal valence (NEG > NEU > POS)?
Construct Validity:

related to emo4onal valence (NEG > NEU > POS)?
Students???
How might we forge this link ?
Construct Validity:

In order to establish the func4onal signicance or construct validity of a measure
1. Psychological Sensi4vity
E.g., probability of amygdala acJvaJon, given fear-arousing sJmuli

AND

2. Psychological Specicity
Reverse
E.g., Probability of fear, given amygdala acJvaJon
OyenJmes, low specicity is not explicitly acknowledged
In both cases, mechanisJc studies are helpful directly manipulate the measure (lesions,
TMS, drugs, etc.) and asses consequences for construct of interest (e.g., fear) and other
constructs (e.g., aQenJon)
Construct Validity:


AND

Reverse
Construct Validity:


AND

Reverse inference
Construct Validity:


AND

Reverse inference
OyenJmes, low specicity is not explicitly acknowledged because researchers
focus on a small number of psychological processes (e.g., emoJon or cogniJon)
Construct Validity:


AND

Reverse inference
OyenJmes, low specicity is not explicitly acknowledged because researchers
focus on a small number of psychological processes (e.g., emoJon or cogniJon)
TMS, drugs, etc.) and assess consequences for construct of interest (e.g., fear) and other
Construct Validity Has

Fundamental ImplicaCons
As cogniGve neuroscienGsts who use the same brain imaging technology, we know that it
is not possible to deniGvely determine whether a person is anxious or feeling connected
simply by looking at acGvity in a parGcular brain region. This is so because brain regions
are typically engaged by many mental states, and thus a one-to-one mapping between a
brain region and a mental state is not possible.
As cogniGve neuroscienGsts who usebrain imaging technology, we know that it is not

possible to deniGvely determine whether a person is anxious or feeling connected
simply by looking at acGvity in a parGcular brain region.

This is so because brain regions are typically engaged by many mental states, and thus a
one-to-one mapping between a brain region and a mental state is not possible.
As cogniGve neuroscienGsts who usebrain imaging technology, we know that it is not

possible to deniGvely determine whether a person is anxious or feeling connected
simply by looking at acGvity in a parGcular brain region.

This is so because brain regions are typically engaged by many mental states, and thus a
one-to-one mapping between a brain region and a mental state is not possible.
Measuring T&P - Key Take-Home Points

1. There are serious problems with the 5 Factor Model
2. In par4cular, they do not represent an empirically discovered set of natural kinds. They instead
strongly reect the theore4cal perspec4ves, methods, and assump4ons of key inves4gators
3. Key assump4ons underlying the 5 Factor Model may not be warranted
Words / Lexical Hypothesis
Lay assessments and lay language
Factor analysis

4. The best level of descrip4on for T&P (wide-band <-> narrow-band) remains unclear

5. Expert behavioral and physiological assessments may prove illumina4ng
6. Especially given other known biases and limita4ons of introspec4ve self-reports
Repor4ng biases (e.g., social desirability); Mnemonic biases (peak-end rule)
Limits of introspec4on (too fast, outside of conscious awareness)
7. Just because you use biological measures doesnt mean you can forget about psychometric
reliability (internal-consistency and test-retest)
8. Determining construct validity (func4onal signicance) of physiological measures is hard
Sensi4vity, specicity

Factor analysis



Factor analysis



Factor analysis



Factor analysis



Factor analysis



Factor analysis



Factor analysis



Factor analysis



Factor analysis


Cri4cal Thinking Ques4ons
Pick any 2 opCons

If low on Cme, read at home

1. Jack Block argued that,

we should not limit [our] thinking and research by considering
only what is or will be convenient to index via simple self-report or
layperson-report measuresTo study [T&P, we]will need to turn,
or return, to more complicated and complex ways of studying
persons

What do you think?

Based on what you know about the Big 3 traits (see the slides from
last Jme) or their facets, briey describe a novel experiment that
exploits a measure other than self-report to discover and
understand some important aspect of T&P.

For example, you could think about ways to clarify whether C/SC
involves heightened anxiety about order, rules, or deadlines.


1. Jack Block argued that,

we should not limit [our] thinking and research by considering
only what is or will be convenient to index via simple self-report or
layperson-report measuresTo study [T&P, we]will need to turn,
or return, to more complicated and complex ways of studying
persons

What do you think?

Based on what you know about the Big 3 traits (see the slides from
last Jme) or their facets, briey describe a novel experiment that
exploits a measure other than self-report to discover and
understand some important aspect of T&P.

For example, you could think about ways to clarify whether C/SC
involves heightened anxiety about order, rules, or deadlines.


2. If traits are supercial and descripJve (rather than explanaJons of behavior), what good
are they?

David Funder (1994) described what I call The NeuroJc Professor,

I have a friend and colleague who has served as a faculty member in several departments. In
each job, he's been miserable. He can enumerate very persuasively why each of his
department chairs was an unfair tyrant, why each teaching load was excessive, how oce
space and salary were unfair and paltryHe really is persuasive. But you know what? I think
he'd be unhappy anywhere. I think he's got a trait. He exhibits a behavioral paPern (of
complaining) from which I infer an emoGonal paPern (of feeling miserable) that I think does
explain why he is saying such nasty things about his latest department.

To explain a behavior in terms of the broader paPern of which it is a part-as I have just done-
can be a legiGmate and useful step in the (innite) explanatory regress. I know, it's not
complete. We sGll don't know why my friend is such a negaGvisGc cynic. But idenGfying him
as such provides (a) insight into his current acGons (and feelings, which we infer from his
acGons), (b) a basis for predicGng his future acGons and feelings, and (c) a useful analyGc rest
stop. We have explained his behavior; take a breath; now we need to explain the explanaGon

What do you think?

Briey describe a strategy or two for addressing the origins of The NeuroJc Professor?

are they?




What do you think?


are they?




What do you think?


are they?




What do you think?

Briey describe a strategy or two for addressing the origins of The NeuroJc Professor, that is,
for reverse engineering the systems that underlie N/NE or some other dimension of T&P.

3. Andy Tomarken underscored the importance of reliability for behavioral
and biological measures of T&P

Recent work suggests that the dot-probe task (a widely used measure of
anxious individuals vigilance for threat) is not reliable

What do you think?
Skim the paper (hQp://journal.fronJersin.org/Journal/10.3389/fpsyg.
2014.01368/full) and and briey comment on the key take-home points.
What are the implicaJons for understanding the cogniJve underpinnings of
anxiety and other key facets of T&P



What do you think?
What are the broader implicaJons for understanding the cogniJve
underpinnings of anxiety and other key facets of T&P



What do you think?
What are the implicaJons for understanding the cogniJve underpinnings of
anxiety and other key facets of T&P

4. Andy Tomarken underscored the importance of establishing the sensiJvity and
specicity (construct validity) of biological measures of T&P (e.g., fMRI, PET, EEG,
ERP, GSR, startle, HR, corJsol, etc.)

What do you think?

a) Briey describe how you might test the specicity of one of these measures,
the probability that it will be acJvated or engaged given a parJcular process or
perturbaJon (such as the inducJon of anxiety vs. other.
-or-
a)
Play with Neurosynth.org, an on-line tool that allows you to dynamically assess
the sensiJvity and specicity of dierent brain regions. Then write a short
paragraph describing what you did and what you learned. It need not be a
deep insight, its sucient to just go play with it and see what you uncover
about sensiJvity (forward inference) and specicity (reverse inference).
The next few slides provide some pointers for interested students.



What do you think?

perturbaJon (such as the inducJon of anxiety vs. other states).
-or-
a)


4. ScienJsts can be sloppy in their use of language. Does fear in a rat exposed to cues
paired with shock reect the same processes involved in N/NE? In anxiety disorders? In
specic phobias? Is NeuroJcism in adults synonymous with Behavioral InhibiJon in kids?
What about Harm Avoidance?

In a recent review focused on Theory of Mind, Schaafsma and colleagues (TiCS 2015)
described this kind of communicaJon break down:

The term X [e.g. self-control] is used interchangeably with Q, R, and S, to name only a few.
This diversity of terms used is probably telling: dierent invesGgators have dierent concepts
in mind.

Dierent levels of descripGon (e.g., neural vs. self-report vs. behavior), together with the
dierent terms used, make it dicult even for experts from dierent elds to navigate both
what is meant by X and how to study it using scienGc methods...to the uniniGated, the topic
becomes bewildering.

The problem is that these dierences [are implicit and unspoken]

[Thus] it has been [extraordinarily dicult] to triangulate the actual [psychological, geneGc,
and neuobiological] processes involved.

What do you think? Briey describe a strategy for addressing this road block.




in mind.








in mind.








in mind.








in mind.







What do you think?

-or-
a)



What do you think?

-or-
a)



What do you think?

-or-
a)

Time-Permi|ng
Review Ques4ons

Note: NeuroSynth.org slides follow
MoQ et al showed that childhood self-control predicts health,

wealth & public safety in midlife. What was one intervening
mechanism during adolescence that parJally explained the link
from kid temperament to deleterious adult outcomes?
0%
0%
0%
0%
0%
0%
0%
Ex
Be
Sm
ce
co
ss
ok
m
iv
i
in
e v
g a ng
id
p
eo
Vi
ga are n
ol
t
m
Hi
en
e
gh
ce
p
la
-ca
i
...
ffe n th
e
in
m
e e
ed
ne
ia
rg
y d
r in
ks
A
&
B
C
&
D
A. Smoking
B. Becoming a parent
C. Excessive video game
playing
D. Violence in the media
E. High-caeine energy
drinks
F. A & B
G. C & D
CorrelaJon and variance explained: If two

variables are correlated R = .50, the amount
of variance accounted for is:
.7
07
1
= 7
0%
0%
Sq
rt(
.5
0)
=
1
=
10
0%
0%
0.
50
/
0.
50
=
0.
50
=
.2
5 =
2
5%
0%
0.
50
*
A. 0.50 * 0.50 = .25 =

25%
B. 0.50 / 0.50 = 1 =
100%
C. Sqrt(.50) = .7071 =
70%
T&P reect trait-like individual dierences in emoJonal and

cogniJve biases that
0%
0%
0%
0%
0%
t e
m
er
ge
in
ea
ue

rly
t
o
Ac
in
ev
co
lif
ol
un
v
e
e
t f
fo
or
r m
co
...
ns
Ca
ist
n
en
be
Ca
cy

r
e
n
i..
l
a
be
tiv
.
co
el
y s
m
pl
im
ex
pl
a
e
nd
m
ul
Al
t..
l o
.
f t
he
ab
ov
e
0%
Co
nt
Fir
s
A. First emerge early in life

B. ConJnue to evolve for many
years
C. Account for consistency in
behavior, inner experience,
and risk across Jme and
contexts. Can be relaJvely
simple (e.g., anxious distress)
or complex and mulJply
determined (orderliness).
Excessive video game playing
D. Can be relaJvely simple
E. Can be complex and
mulJdimensional
F. All of the above
T&P are not dierent in kind (according to Shackman)

because they are both
A. Biological
B. EmoJonal
C. CogniJve
D. Somewhat heritable
E. All of the above
0%
Al
l
of
t
he
ab
bl
ov
e
0%
er
ita
t h
Co
gn
iti
ve
0%
So
m
ew
ha
0%
io
na
Em
ot
Bi
o
lo
gic
a
0%
What are the 3 fundamental dimensions of T&P?
0%
C
0%
A,
B
, a
nd

0%
nd
E
0%
A,
C
, a
0%
C/
SC
0%
S/
RE
P/
TA
N/
NE
0%
E/
PE
A. N/NE
B. P/TA
C. E/PE
D. S/RE
E. C/SC
F. A, C, and E
G. A, B, and C
N/NE can be dissected into which 2 facet traits
A. Distress (fear/
anxiety) and
IrritaJon (anger)
B. Guilt and Shame
0%
Di
st
re
ss
Sh
a
an
d
ilt

Gu
(f
ea
r/
an
xie
ty
) a
n
d.
..
m
e
0%
Which staJsJcal test is used to quanJfy

the conJnuity (temporal stability) of traits
A. Students t test
B. ANOVA
C. CorrelaJon
St
u
io
0%
Co
rr
el
at
0%
AN
OV
de
nt
s
t
te
st
0%
T&P is
0%
d
...
as
bl
Co
m
pl
et
el
y
p
l
ta
y s
el
od
er
at
M
ti c
an
e (
R
=
m
ut
im
an
d
0%
0.
...
ab
le

0%
Fix
ed

A. Fixed and
immutable
B. Moderately stable
(R = 0.4 to 0.6 over
periods of one to
several years)
C. Completely plasJc
and malleable
NeuroSynth.org Slides
You can click around with your mouse on the brain and the coordinates will be autolled!
The End
Material for Future Semesters
This could be incorporated into lecture or- as a

take home quesJon

Incorporate some of Grubers material @
hQps://www.youtube.com/watch?
v=g1lXq5VIKS4&list=PLh9mgdi4rNewieO9Dsj-OhNBC9bF4FoRp

Gruber interview of Mauss could be criJcal q
hQps://www.youtube.com/watch?v=dDMdE3MnBFU

Note that chapter 1 of maQhews deary and whiteman is preQy good; opJonal
or even assigned reading

Alex you have the damasio material in the next lecture, #5.copy and paste
that version into this version and make it more explicit that if low on Jme,
skip.

Future Semester Thought

QuesCon
Just out .. earlyview ..

Park, G., Schwartz, H.A., Eichstaedt, J.C., Kern, M.L., Kosinski, M., SJllwell, D.J., Ungar, L.H., & Seligman, M.E.P. (2014). AutomaJc personality assessment through social media
language. Journal of Personality and Social Psychology, EarlyView, , 1-20.
Abstract
Language use is a psychologically rich, stable individual dierence with well-established correlaJons to personality. We describe a method for assessing personality using an open-
vocabulary analysis of language from social media. We compiled the wriQen language from 66,732 Facebook users and their quesJonnaire-based self-reported Big Five personality traits,
and then we built a predicJve model of personality based on their language. We used this model to predict the 5 personality factors in a separate sample of 4,824 Facebook users,
examining (a) convergence with self-reports of personality at the domain- and facet-level; (b) discriminant validity between predicJons of disJnct traits; (c) agreement with informant
reports of personality; (d) paQerns of correlaJons with external criteria (e.g., number of friends, poliJcal artudes, impulsiveness); and (e) testretest reliability over 6-month intervals.
Results indicated that language-based assessments can consJtute valid personality measures: they agreed with self-reports and informant reports of personality, added incremental
validity over informant reports, adequately discriminated between traits, exhibited paQerns of correlaJons with external criteria similar to those found with self-reported personality, and
were stable over 6-month intervals. Analysis of predicJve language can provide rich portraits of the mental life associated with traits. This approach can complement and extend
tradiJonal methods, providing researchers with an addiJonal measure that can quickly and cheaply assess large groups of parJcipants with minimal burden.

The key results can be seen in Table 1:

From this, the authors conclude (1st sentence of their Conclusions):
In this arJcle, we provided evidence that the language in social media can be harnessed to create a valid and reliable measure of personality.

Some context: A correlaJon of 0.40, sampling a random normal bivariate distribuJon, 4,000 cases, integer scale varying between 0 and 20, mean = 10, SD = 3,
and ploQed as a frequency scaQerplot (the bigger circles indicate higher frequencies of occurrence):

What the authors do not do is engage with that word accuracy, using the kinds of analyses (in the metric of the criterion data) that beQer inform the reader about the kinds and
paQerns of errors associated with using a Language Based Assessment (LBA) of Personality rather than a QuesJonnaire Based Assessment. And the authors do make a recommendaJon
for usage:
LBAs oer a pracJcal, cost-eecJve alternaJve, allowing assessment of psychological characterisJcs when quesJonnaires are impracJcal. Researchers could reduce parJcipant burden
by replacing some quesJonnaires with a single link and sign-in procedure, allowing a research applicaJon to access parJcipant social media language and quickly assess personality and
other characterisJcs of interest.

CorrelaJon is not the correct staJsJc to be used here. You need esJmates of errors that are sensiJve to actual score magnitudes, not monotonicity relaJons. You also need to conduct a
fair few actuarial and discrepancy-based analyses/graphics to properly understand both accuracy and error, and where LBAs are more accurate over a range of personality variaJon (do
they predict high scorers beQer than low scorers?).

All in all, sophisJcated, innovaJve, and very, very clever work; but, for someone who aspires to greater accuracy of psychological assessment, I am puzzled by the seeming indierence to
accuracy shown within this enJre line of research.

Anyway, this is already hot stu.. as Michal Kosinski describes on his website:
hQp://www.michalkosinski.com/home/publicaJons
Private traits and aQributes are predictable from digital records of human behavior by M. Kosinski, D. SJllwell, T. Graepel, Proceedings of the NaGonal Academy of Sciences
(PNAS), 2013.
The 9th most discussed paper of 2013 and the 4th most inuen4al ar4cle ever published by PNAS (Altmetric.com)

Regards .. Paul
ExisJng theories of personality provide a number of compeJng surface level, lexically-derived,

systems with trait measures that relate to approach and avoidance either indirectly via constructs
such as Extraversion and NeuroJcism (Eysenck, 1957) or directly via constructs such as Harm
Avoidance (Cloninger et al., 1993). Each system is stable, with links to mental disorder (
Strelau and Zawadzki, 2011; Gomez et al., 2012; Mullins-SweaQ and Lengel, 2012; Trull, 2012) and
brain structure (Gardini et al., 2009; DeYoung et al., 2010). But even when starJng with approach
and avoidance as primary constructs, they are derived top-down from pools of lexically-chosen
quesJonnaire items (Carver and White, 1994; Elliot and Thrash, 2010) not from biological anchors.
They also depend on factor analysis, which determines the number of dimensions, but not locaGon
of trait axes of the personality space that items occupy (Lykken, 1971;
Corr and McNaughton, 2008). It is liQle more than an act of faith to believe that the causal
structure of personality is isomorphic with its lexical factor structure. So, even if we knew for
certain that there were only two dimensions within a parJcular measured personality space, one
quesJonnaire system could have a single simple trait anxiety dimension (orthogonal to, say,
impulsiveness) that was a combinaJon of neuroJcism and introversion in another (Gray, 1970)
the two systems diering only on which items from an original pool were used to create scales.
Factor analyJcally derived trait measures can also easily meet the criterion of having simple
structure (in the sense that a set of items loads highly on only one factor so factors can be clearly
idenJed by unique item loadings) while implying improbable causaJon (Lykken, 1971). Further,
not only is there no reason to suppose that biologically accurate scales should have simple
structure but also current scale systems, even though designed to have this, oyen do not
(DeYoung, 2006, 2010).

Goldberg 1970s 1990s and Costa & McCrae 1980s 1990s
Similar concerns
In designing their quesJonnaire, Costa and McCrae disJnguished and permanently xed
upon [case closed!] a half dozen facets each for their broad constructs of NeuroJcism,
Extraversion, and Openness.

The facet dis4nc4ons they oered were not rooted in factor analysis, formal theorizing, or
ineluctable empirical ndings. Rather, the facets derived from their personal thinking about
how the three domains could be further ar4culated.

The six facets Costa and McCrae nominated to represent the NeuroJcism domain were
Depression, Impulsiveness, Anxiety, HosJlity, Self-consciousness, and Vulnerability.



about 50% variance

Reliability (Alpha and Test-Retest)
Sets an upper limit on the magnitude of correlaJons with external measures
Can be enhanced by aggregaJon
Average out random noise or state-specic variance
HabituaJon is bad
For event-related measures (e.g., responses to aecJvely laden sJmuli or
situaJons, may need to vary the sJmuli)


about 50% variance

Can be enhanced by aggregaJon (increase the number of measurements)
HabituaJon is bad
For event-related measures (e.g., responses to aecJvely laden sJmuli or
situaJons, may need to vary the sJmuli)


about 50% variance

HabituaJon is bad
For event-related measures (e.g., responses to emoJonal sJmuli or situaJons)
may need to vary the sJmuli


about 50% variance

HabituaJon is bad
For event-related measures (e.g., responses to emoJonal images), may need to
vary the sJmuli to prevent habituaJon

Shackman Psyc210 Module04 Measuring 021015

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Shackman Psyc210 Module04 Measuring 021015

Hochgeladen von

Copyright:

Verfügbare Formate

Nuts

& Bolts Plan for Today

Take-home cri4cal thinking ques4ons

Nuts & Bolts Plan for Today

Take-home cri4cal thinking ques4ons

Nuts & Bolts Plan for Today

Take-home cri4cal thinking ques4ons

Nuts & Bolts Plan for Today

Take-home cri4cal thinking ques4ons

Todays Conceptual Roadmap

Todays Conceptual Roadmap

Todays Conceptual Roadmap

Todays Conceptual Roadmap

From Measures to Models

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Whats Factor Analysis

Whats Factor Analysis

Whats Factor Analysis

Whats Factor Analysis

Choices, choices, choices

Choices, choices, choices

CumulaJve: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors

Choices, choices, choices

2nd factor explains

CumulaJve: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors

Choices, choices, choices

Choices, choices, choices

If we retain 4 factors (~2/3 variance)

Limita4ons of Factor Analysis

Subjec4ve, the Analyst Has to Choose

"...each orientaJon is equally acceptable mathemaJcally. But dierent

Limita4ons of Factor Analysis

Subjec4ve, the Analyst Has to Choose

"...each orientaJon is equally acceptable mathemaJcally. But dierent

Limita4ons of Factor Analysis

Subjec4ve, the Analyst Has to Choose

"...each orientaJon is equally acceptable mathemaJcally.

underlying psychology and neurobiology

Limita4ons of Factor Analysis

Subjec4ve, the Analyst Has to Choose

Limita4ons of Factor Analysis

Subjec4ve, the Analyst Has to Choose

Garbage In/Garbage Out

Back to Blocks history lesson

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Students Any concerns with this approach?

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Why 5 Factors? Words, words, words

Jack Blocks Cri4que

Students: What are some potenCal

Jack Blocks Cri4que

Students: What are some potenCal

Jack Blocks Cri4que

Students: What are some potenCal

Jack Blocks Cri4que

Jack Blocks Cri4que

Jack Blocks Cri4que

Jack Blocks Cri4que