Sources of Knowledge

The Scientific Method

• 4 Steps
• Observe a phenomenon
• Formulate hypotheses – tentative statements
of cause & effect
• Further observation / experimentation to rule
out alternative explanations
• Refining & retesting of explanations
• Provides a check on the validity of its
conclusions
 Hypothesis Formation
Library Research Based on Lib. Rsh., propose
Learning “what is known” some “new knowledge” Research Design
about the target behavior Determine how to
obtain the data to test
the RH:
The “Research Loop”
Data Collection
• Novel RH:
Choose sample,
Draw Conclusions • Replication measures, method
Decide how your “new • Convergence
knowledge” changes
“what is known” about
the target behavior Data Analysis
Appropriate
statistical analysis
Hypothesis Testing
Based on design properties
and statistical results
Scientific Explanations
• Accepted because they have the
following properties:
• Empirical: based on objective &
systematic observation with the senses
• Rational: follows rules of logic,
consistent with known facts
• Testable: able to be verified, disproved
• Parsimonious: uses fewest assumptions
• General: apply to broad circumstances
• Tentative: admittedly imperfect
• Rigorously evaluated
 Scientific Explanations
• Mechanistic
• Describes the physical components and the
chain of cause and effect through which the
behavior is produced
• Describes how it works (but not why)
• Functional
• Describes something in terms of what is it
does, but not how it happens
• Need both for a full understanding of the
concept
Research Hypotheses
• The whole research process revolves around
hypotheses -- literature reviews to form them,
designs to generate data to be analyzed to test
them, replication and convergence of them, etc.

• A research hypothesis is a tentative explanation

or “guess” about the target behavior
• What you will find when you complete your
research and data analysis
Basic vs. Applied
Research
• Basic research
• Conducted in order to test theories
or empirical positions – basic goal
is to obtain general information
about a phenomenon – not really
interested in applying finding to
real-world situations
• Applied research
• Purpose is to investigate a
problem that occurs in the real
 Sources of Research
Ideas
• Research starts with an interesting
idea
• Experience
• Unsystematic Observation
• You read/see/hear/experience something, and
ask WHY?
• Systematic Observation
• Planned observation of real world behavior
• Previous observations of others (published
research)
• Own research
• Internet research
 Scientific Theories
• Theory - a partially verified
statement of a scientific
relationship that cannot be
directly observed (Martin, 1985)
Sources of Research
Ideas
• Theory
• A developed set of assumptions and
rules about the causes of behavior
• Research can involve applying a theory
to a novel situation
• Testing competing theories about
behavior
 Classifying Theories
• Can be classified along 3 dimensions
• Quantitative vs. qualitative
• Level of description
• Domain
• Quantitative Theory
• Theory expressed in mathematical terms
• Formulas used to explain behavior
• Qualitative Theory
• States relationship between variables in
verbal rather than mathematical terms
 Classifying Theories
Level of description
• Descriptive Theory
• Merely describes the relationships
among variables – it does not explain
the relationship

• Analogical Theory
• Explains the relationships among
variables through analogies to something
that is well-understood
• E.g., Memory as a computer
Classifying Theories
• Fundamental Theory
• Proposes a new structure to
explain the relationships among
variables
• Not only describes behavior, it
explains behavior – not through
analogies but through a new
structure
• Highest level of theory
• E.g., Cognitive dissonance theory
Classifying Theories

The theory’s domain (scope)

• The range of situations to which

the theory may be applied
• E.g., Cognitive dissonance –
applicable to many situations
 A Good Theory
• Ability to Account for Data
• Account for existing data and well-
established facts within its domain
• Explanatory Relevance
• Offer good grounds for believing that
the phenomenon would occur under
specified conditions
• Theory must define logical links
between variables
• Testability
• It must be capable of failing some
empirical test
A Good Theory

• Prediction of Novel Events

• Should predict phenomena that the
theory was not specifically
designed to account for but that
are within its domain
• Parsimony
• Should explain phenomena within
its domain with the fewest
possible assumptions and the
simplest terms possible
Testing Theories

• Confirmational Strategy
• Look for evidence to confirm
predictions from a theory
• Important part of theory testing,
but has limits
• Confirmation does not prove a
theory is correct
• Confirmation may occur when
predictions are too loosely defined
 Testing Theories
• Disconfirmational Strategy
• Using a positive research result to
disconfirm a theory’s predictions
* Should use both strategies
• Begin with confirmational
• Can it explain the phenomena?
• Move to disconfirmational
• Do unexpected outcomes happen?
 Strong Inference
• Theory is tested and modified based on
outcome of research and then tested
again
• Cycle of testing and modification
continues until theory adequately
accounts for behavior
• Several alternative explanations can
be tested with an experiment
• Some ruled out
• New experiment tests remaining
alternatives
• Continue until only one alternative remains
Good Questions
• Not too broad
• Operational definitions – describe
how a variable will be measured
− Limits generalizability
Good Questions
• Ask IMPORTANT questions
• Clarify theoretical or empirical issues
• Support one hypothesis/theory over
another
• Address important practical issues
• Probably unimportant if:
• Answer already firmly established
• Small effects of no theoretical interest
• No reason to believe that 2 variables are
related
Purposes of Literature
Review
• To fully describe the results from
prior research
• What is the “state of the knowledge”?
• To clearly state the purposes of the
study
• Purpose is to address some need
• To clearly state the hypotheses,
which should follow logically from
the literature review
Sources of Research
Info
• Primary vs. Secondary Sources
• A primary source includes a full
report of a research study, including
methodological details
* Primary sources are preferred because
secondary sources may be biased or
author may not have discussed
something you may think is important.
• A secondary source summarizes
information from a primary source
(e.g. review article)
• Exception – meta-analysis
Sources
• Books
• General textbooks or specialized
professional publications
• Anthologies - assemble papers that
an editor feels are important in a
field
• Possibility for editor’s bias
• Most useful in early stages of
literature search
• Should be used with caution - may
not undergo rigorous review; info
Sources Cont.
• Scholarly Journals
• Current research and theoretical
thinking
• Refereed vs. nonrefereed journal
* Prefer refereed sources
• You can evaluate the quality of a
journal by
• Consulting Journals in Psychology
• Consulting the Social Science
Citations Index
• Using the method of authority
Sources Cont.
• Conventions and Professional
Meetings
• Gatherings of researchers to
present findings
• Provide the most up-to-date info
• Advantages of attending
convention
• Information is at the frontiers of
science
• Meet others in your field and
exchange ideas
 Evaluating a Research
Article: The Introduction
• Has relevant research been adequately
reviewed?
• Are assertions supported with the
appropriate citations?
• Are the purposes of the study clearly
stated?
• Are the hypotheses clearly stated, and
do they flow logically from the info in
the introduction?
 Evaluating a Research Article:
The Method Section
• Was the nature of the subject sample
specified?
• Does the design of the study allow an
adequate test of the hypotheses?
• Are there any methodological flaws
that might affect the validity of the
results?
• Is sufficient detail presented to allow
one to replicate the study?
 Evaluating a Research Article:
The Results Section
• Did the statistically significant effects
support or refute the hypotheses?
• Are the differences reported large or
small?
• Were the appropriate statistics used?
• Do the tables, figures, and text match?
 Evaluating a Research Article:
The Discussion Section
• Do the conclusions presented match the
results reported?
• If the author speculates about
implications of results, does he or she
stray too far from the results reported?
• How well do the results mesh with
existing theory and empirical data?
• Does the author point the way to
directions for future research?
 Factors Affecting the Quality
of Research Information
• Statistical Significance
• Journals typically do not publish
findings that do not meet the
minimum .05 level of statistical
significance

• File drawe r p henomenon: Findings

that don’t reach significance at .05
end up in the file drawer, perhaps
masking true effects
 Relationships
• Causal relationships
• One variable influences another
• Unidirectional vs. bidirectional
• Correlational relationships
• Changes in one variable associated with
changes in another variable (variables
covary)
• Can’t say one change causes the other
change
* Our ability to distinguish between the 2 types of
relationships depends on the level of control in
the study
Correlational Research

• Form of nonexperimental
research
• Goals
• Determine whether two variables
covary
• Establish direction, magnitude,
forms of the relationship
• No variables are manipulated
• Instead just observed
 Correlational Research
Descriptive Purposes
• Determine whether a relationship
exists between 2 variables

Predictive Purposes
• Knowledge of value of one variable
can help us predict value of related
variable
• Predic to r v aria ble used to predict
the value of a Criterion variable
 Correlation & Causality
• Correlational research ca nno t be
used to establish caus al
re lat ion ship s among variables –
correlation does not equal causation
• Causality: 3 conditions to say that A
causes B
• A precedes B
• A is related to B
• All alternative explanations have been
ruled out
 Obstacles to Causal
Explanations
• Third variable problem
• An unmeasured variable may account
for changes in both variables
• Both observed variables may vary together
although they are not directly related
• Examples
• Aggressive video games and aggression –
aggressive personality
• Ice cream sales and crime – hot weather
• There are ways to statistically
account for the third variable problem
 Obstacles to Causal
Explanations
• The directionality problem
• When a causal relationship does exist,
it is often hard to know the direction of
the effect
• Does A cause B, B cause A, or both?
• Examples
• Playing aggressive video games and
aggression
• Weight and frequency of exercise
 When to Use Correlational
Research
• Gathering data in the early stages
of research
• Identify possible causal relationships
which can then be tested
experimentally
• Inability to manipulate variables
• Manipulating independent variable
may be impossible or unethical
• Relating naturally occurring
variables
 Experimental Research
• Involves high degree of control over
variables
• Allows us to establish causal
relationships

• An Independent Variable is
manipulated
• A variable whose values are chosen
and set by the experimenter
• Participants must be exposed to at
least two levels of this variable
 Experimental Research
• A Dependent Variable is measured
• The variable whose value you observe (the
outcome)
• The value of the dependent variable depends
on the participants’ behavior

• The most basic experiment consists of

an experimental and a control group
• The exper im ental gr oup receives the
treatment
• The cont rol group does not receive the
treatment
• Assignment to these groups/conditions must
be completely random
 Experimental Research
• Control over extraneous variables
• Extraneous variables = other variables
besides the IV that may affect the DV
• May mask any influence of IV on DV or
may produce chance differences in DV
that are unrelated to IV
• To control these possible effects:
• Hold extraneous variables constant
• Random assignment of participants to
conditions
 Research Settings
• Depends on: costs, convenience,
ethical considerations, research
question

• The laboratory setting

• Anywhere other than where the
behavior actually occurs – a formal
lab, a classroom, etc.
• Affords greatest control over
extraneous variables
 Research Settings
• Simulations
• Attempt to recreate the real world in a lab
setting
• When attempting to control extraneous
variables and interested in generalizing
results to real-world settings
• Realism is an issue
• Mundane real ism : How well does a
simulation mimic the real world event being
simulated
• Experi mental real ism : How engaging
(psychologically) is the simulation for
participants
* Experimental realism is more important than
mundane realism
 Research Settings
• The field setting
• Study conducted in a real world
environment – where the behavior
actually occurs
• Field experiment: Manipulate variables
in the field
• Could be nonexperimental, or contain
all the characteristics of a true
experiment
• Easily generalizable to the real world
• Little control over confounding
 Internal Validity
• Internal validity
• The degree to which your design tests
what it was intended to test
• The degree to which inferences about
whether variations in the independent
variable cause variations in the
dependent variable are warranted
• The extent to which the research
design adequately tests your
hypothesis
 Threats to Internal
Validity
• History
• Events may occur between multiple
observations
• E.g., High profile news story
• Maturation
• Participants may become older or
fatigued
• Repeated Testing
• Taking a pretest can affect results of a
later test
 Threats to Internal
Validity
• Instrumentation
• Changes in instrument calibration or
observers may change results
• Statistical Regression
• Subjects may be selected based on
extreme scores
• Biased Subject Selection
• Subjects may be chosen in a biased
fashion
 Threats to Internal
Validity
• Experimental Mortality
• Differential loss of subjects from
groups in a study may occur

• These threats make it impossible

to tell if results (changes in DV) are
due to changes in the IV
 External Validity
• External validity - the degree to which
results can be accurately generalized
beyond your sample and research setting
• Other participants, tasks, stimuli, settings &
times

• Population: Will the results generalize to

other persons or animals ?
• Will a study of college students generalize to
your target population of “consumers”?
• Will a study of chronically depressed patients
transfer to those who are acutely depressed?
 Components of External
Validity
• Setting: Will the findings apply to
other settings ?
• Will a laboratory study generalize to what
happens in the classroom?
• Will a study in a psychiatric hospital
generalize to a out-patient clinic?
• Societal/Temporal: Will the findings
continue to apply?
• Will a study conducted in 1965 generalize
to today?
• Will a study conducted today still be
useful 10 years from now? … 5 years
from now?
Components of External
Validity
• Task/Stimuli: Will the results
generalize to other tasks or stimuli ?
• Usually the participant is “doing
something” that directly or indirectly
generates the behavior that is being
measured
• What do I learn about “consumer
decision making” from a study that asks
participants to select the best “widgit”?
• Will research using visual illusions
inform us about the perception of
everyday objects ?
Threats to External
Validity
• Degree of control
• Data obtained in highly controlled
lab settings may not generalize to
real-world situations

• Sample used
• Results may apply only to subjects
representing a unique group (i.e.
college students)
Threats to External
Validity
• Reactive effects of
experimental arrangements
• Participants’ knowledge that they
are research subjects may affect
results (demand characteristics)

• Multiple treatment interference

• Exposure to early treatments may
affect responses to later
treatments
 Internal vs. External
Validity
• Trade-off characterization
• Impossible to promote both internal
and external validity within a single
study
• Steps taken to increase internal
validity may decrease external validity
and vice versa
• Must choose which will be emphasized
• Internal validity (control)
• External validity
 Internal vs. External
Validity
• Internal validity may be more
important in basic research,
external validity  in applied
research
• Which is more important depends on
the reasons why the research is being
conducted

• Internal validity an important

precursor
• Without causal interpretability (from