Introduction to Validity

What is Validity?

● The best available approximation to the

truth or falsity of a given inference,
proposition, or conclusion
● A set of standards by which research
can be judged
 The Causal Context

Theory

Observation
 The Causal Context

Theory

Cause
construct

Observation
 The Causal Context

Theory

Cause
construct

Observation
 The Causal Context

Theory

Cause cause-effect construct

Effect
construct construct

Observation
 The Causal Context

Theory What you think

Cause cause-effect construct

Effect
construct construct

Observation
 The Causal Context

Theory What you think

Cause Cause-Effect Construct

Effect
construct construct
Operationalize

Program

Observation In this study

 The Causal Context

Theory What you think

Cause Cause-effect Construct

Effect
construct construct
Operationalize Operationalize

Program Observations

Observation In this study

 The Causal Context

Theory What you think

Cause Cause-effect Construct

Effect
construct construct
Operationalize Operationalize

Program Observations
What you do What you see
Observation In this study
 The Causal Context

Theory What you think

Cause Cause-effect Construct

Effect
construct construct
Operationalize Operationalize

Program Program-outcome Relationship

Observations
What you do What you see
Observation What you test
In this study
 Conclusion Validity

Is there a relationship between...

● What you did and what you saw?
● Your program and your observations?

Program Program-outcome Relationship

Observations
What you do What you see
Observation In this study
 Internal Validity

Is the relationship causal between...

• What you did and what you saw?

• Your program and your observations?

Alternative
cause
Alternative
cause

Program Program-outcome Relationship

Observations
What you do What you see
Alternative
cause Alternative
Observation cause In this study
 Construct Validity
Theory What you think

Cause Cause-effect Construct

Effect
construct construct

Can we generalize to the constructs?

Program Program-outcome Relationship

Observations
What you do What you see
Observation
 External Validity

Theory What you think

Cause Cause-effect Construct

Effect
construct construct

Can we generalize to other persons, places, times?

Program program-outcome relationship

Observations Program program-outcome relationship
Observations
What You Do What You See What You Do What You See

Observation Observation

Program program-outcome relationship

Observations Program program-outcome relationship
Observations
What You Do What you see What You Do What You See

Observation Observation
The Validity Questions Are Cumulative...
The Validity Questions Are Cumulative...

Is there a relationship between

In this study the cause and effect?
The Validity Questions Are Cumulative...

In this study Is the relationship causal?

Is there a relationship between

Conclusion the cause and effect?
The Validity Questions Are Cumulative...

Can we generalize to
In theory the constructs?

Internal Is the relationship causal?

Is there a relationship between

Conclusion the cause and effect?
The Validity Questions Are Cumulative...

Can we generalize
In theory to other persons,
places, times?

Construc Can we generalize to

the constructs?
t
Internal Is the relationship causal?
Is there a relationship between
Conclusion the cause and effect?
The Validity Questions are cumulative...

Validity

Can we generalize
External to other persons,
places, times?
Can we generalize to
Construct the constructs?
Internal Is the relationship causal?

Is there a relationship between

Conclusion the cause and effect?
 Threats to Validity

You Want to Make an Inference...

● There is a relationship between
the cause and effect.
● The relationship is causal.
● You can generalize to the
constructs.
● You can generalize to other
persons, places, and times.
 Threats to Validity

How could you be wrong in the inference?

Conclusion Validity

● There is a relationship, but you don’t

see it.
● There is no relationship, but you do
see one.
 Threats to Validity

How could you be wrong in the inference?

Internal Validity

● There is a causal relationship, but you

don’t see it.
● There is no causal relationship, but
you do see one.
 Threats to Validity

How could you be wrong in the inference?

Construct Validity

● You can generalize to constructs, but

you conclude you can’t.
● You can’t generalize to constructs, but
you conclude you can.
 Threats to Validity

How could you be wrong in the inference?

External Validity

● You can generalize to other contexts,

but you conclude you can’t
● You can’t generalize to contexts, but
you conclude you can
 Threats to Validity

● How you can be wrong in making your

inference
● Specific factors that can bias or distort
your conclusions
● A list of common threats to the quality of
your study
● A “checklist” you can use in planning
your study