EXPERIMENTAL PSYCHOLOGY AND THE SCIENTIFIC METHOD

Common Sense Psychology

• Nonscientific data gathering that shapes us and our behavior towards others
• But nonscientific data sometimes leaves us in the air..
• Thus, there is a need for more specific approach to understanding and predicting
behavior.

2 Constraints of Common Sense Psychologists

1. Nonscientific Sources of Data
2. Nonscientific Inferences

1. Nonscientific Sources of Data

 Family
 Friends
 Authority
 Media
 Books
o But these are not always valid and accurate

Confirmation Bias
• Once we believe we know something, we tend to overlook instances that might
disconfirm our beliefs, and we seek confirmatory instances of behavior instead.

2. Nonscientific Inference
• When we understand people’s behavior, there is a strong bias to overlook situational data
in favor of data that substantiate trait explanation.

Overconfidence Bias
• Our predictions, guesses, and explanations tend to feel much more correct than they
actually are, and the more data we have available, the more confidence we have in our
judgments about behavior.
THE SCIENTIFIC METHOD: TOOLS OF PSYCHOLOGICAL SCIENCE
SCIENTIFIC EXPLANATION IN PSYCHOLOGICAL SCIENCE

Antecedent Conditions
• Circumstances that come before the event or behavior we want to explain

Comparing Antecedent Conditions

 We compare set of antecedent conditions to explain behavior systematically and
scientifically.
 The treatment does not necessarily mean there’s something to treat. It’s treating subjects
differently.

THE PSYCHOLOGY EXPERIMENT

Psychology Experiment
• Controlled procedure in which at least 2 different treatment conditions are applied to
subjects.
• Subjects behaviors are measured and compared to test a hypothesis about the effects of
those treatment on behavior

Experimental psychology refers to work done by those who apply experimental methods to the
study of behavior and the processes that underlie it. Experimental psychologists employ human
participants and animal subjects to study a great many topics, including (among others) sensation
& perception, memory, cognition, learning, motivation, emotion; developmental
processes, social psychology, and the neural substrates of all of these.

Experimental Designs
• Between Subjects: receive only one kind of treatment
• Within Subjects: present all treatments to each subject and measure the effect of each
treatment after it is presented

Control is most often achieved by:

• Random assignment of subject to different treatment conditions (same characteristics)
• Presenting a treatment condition in an identical manner to all subjects
• Keeping the environment, the procedures, and the measuring instrument constant for all
subjects in the experiment

EXPERIMENTAL RESEARCH STRATEGY

Establishing Cause and Effect

• Cause and effect relationship between the antecedent conditions and the subject’s
behaviors.
• the term experiment is often used as a generic label for any kind of scientific research
• however, a research study must meet specific criteria to qualify as an experiment
 • the goal of experimental strategy is to establish the existence of a cause-effect
relationship between two variables
• in an experiment we are trying to show that changes in one variable are directly
responsible for causing changes in the other variable

Causation and the third-variable problem

• the variables that we want to manipulate and study rarely exist in isolation
• in a real world we are usually exposed to a tangled network of interrelated variables
• it is essential that a researcher separates and insolates the specific variables being studied
• it is possible that a third (unidentified) variable is controlling the two variables and is
responsible for producing the observed relation
• e.g.
• Is the fitness program causing higher productivity or
• some other variable such as being more fit or healthier is influencing both the
tendency to participate in a fitness program and to have higher productivity?

Cause and effect relation and external validity

• to establish a cause-and-effect relation, an experiment must control nature, essentially
creating an “unnatural” situation wherein the two variables being examined are isolated
from the influence of other variable and wherein the exact character of a relationship can
be seen clearly
• just because an experiment takes place in an unnatural environment (i.e. lab) does not
necessarily imply that the results are unnatural
• the goal of any experiment is to reveal the natural underlying mechanisms and
relationships that may be otherwise obscured

Elements of an experiment
• in order to show that there is a cause-effect relationship between two variables we have
two goals:
• it must be shown that changes in one variable cause changes in he other variable
• an experiment must rule out the possibility that the changes are caused by some
other variable
• these two goals lead to two characteristics differentiating experiments and other research
strategies
• manipulation of one variable while measuring a second variable
• control of other, extraneous variables

Manipulation and measurement

• the primary purpose of manipulation is to allow researchers to determine the direction of
a relationship
• e.g. change in the position of the switch vs. change in brightness of the light or
depression and insomnia
• a second purpose of manipulation is to give the researcher command of one of the
variables being studied
• experimenter is the one determining the manipulation
• independent variable
 • is the variable manipulated by the researcher
• the independent variable usually consists of two or more treatment conditions to
which participants are exposed (e.g. changes in room temperature)
• dependent variable
• is the variable that is observed for changes in order to assess the effects of
manipulating the independent variable
• the dependent variable is typically a behavior or a response measured in each
treatment condition (e.g. test performance)
• extraneous variable
• all variables in the study other than the independent and dependent variables
• treatment condition
• a situation or environment characterized by one specific value of the manipulated
variable
• an experiment contains two or more treatment conditions that differ according to
the values of the manipulated variable
• levels
• the different values of the independent variable selected to create and define the
treatment conditions are called the levels of the independent variable

Control and extraneous variables

• the purpose of an experiment is to show that the manipulated variable is responsible for
the changes observed in the dependent variable
• therefore, an experiment must rule out any other possible explanation for the observed
changes – that is, eliminate all confounding variables
• e.g. Pavio’s research on imagery causing better memory with the possible confounding
variable of meaningfulness (or abstractness)

Dealing with extraneous variables

• an experimental researcher must prevent any extraneous variable from becoming a
confounding variable
• this is the basic purpose of control within an experiment
• confounding variable has two important characteristics
• extraneous variable becomes a confounding variable only if it influences the
dependent variable
• a confounding variable must vary systematically with the independent variable
(e.g. IQ and memory)
• There are two standard methods for controlling extraneous variables:
• control by holding constant or matching
• control by randomization

Control by holding constant or matching

• the impact of an extraneous variable on the results can be eliminated by holding it
constant
• e.g. all individuals in the experiment will be observed in the same room, at
the same time of day, by the same researchers, the individuals have the
same IQ etc.
 • the impact of an extraneous variable on the results can be minimized by matching
the levels of the variable across treatment conditions
• e.g. 10 males and 20 females can be assigned to each separate treatment
condition
• in this way we eliminate possible variation across treatments

Control by randomization
• randomization is the use of a random process to help avoid a systematic
relationship between two variables
• the intent of randomization is to disrupt any systematic relation between
extraneous variables and the independent variable and thereby prevent the
extraneous variable from becoming a confounding variable
• e.g. IQ and memory or time of the day
• random assignment is the use of a random process to assign participants to
treatment conditions
• randomization is a powerful tool for controlling extraneous variables; its
primary advantage is that it offers a method for controlling a multitude of
variables simultaneously and does not require specific attention to each
extraneous variable

Control groups
• sometimes a researcher wishes to evaluate only one treatment rather than compare a set
of different treatments; in such case he selects an experimental group and a control group
• the term experimental group refers to the treatment condition in an experiment
• the term control group refers to the no-treatment condition in an experiment
• there are two types of control groups:
1. no-treatment control groups
2. placebo control groups
• no-treatment control group
• is a condition in which the participants do not receive the treatment being
evaluated
• this group provides the standard of normal behavior or baseline against the
treatment condition
• e.g. anxiety treatment group vs. no treatment control group
• placebo control groups
• placebo is a term for a fake medical treatment such as a sugar pill or water
injection
• placebo effect refers to a response by a participant to an inert medication that has
no real effect on the body; the placebo effect occurs simply because the individual
thinks the medication is effective
• e.g. use of inactive drugs or non-specific psychotherapy (therapy with the
therapeutic components removed)

Manipulation checks
• The purpose of the manipulation check is to assess whether the independent variable had
the intended effect on the participant
 • Two ways to check manipulation
• An explicit measure of the independent variable
• E.g. a measure of mood in an mood inducing experiment
• Questionnaire for participants after the experiment whether they perceived and
how they interpreted the manipulation
• E.g. masked priming
• Manipulation checks are especially important in
• Participant manipulations
• E.g. frustration task
• Simulations
• Real world situations in the lab
• Placebo controls
• Whether participants believed that they received a real drug

Increasing external validity: simulation and field studies

• when research seeks cause-and-effect explanations for behavior in real-world situations,
it is essential that the experimental results generalize outside the confines of the
experiment
• two standard techniques to accomplish this:
• simulation
• field studies
• simulation
• is the creation of conditions within an experiment that simulate or closely
duplicate the natural environment in which the behaviors being examined would
normally occur
• e.g. the Stanford prison study or prisoner’s dilemma studies
• field studies
• a field study is an experiment conducted in a place that the participant or subject
perceives as a natural environment
• e.g. a variety of staged emergencies such as a flat tire, a lost wallet, a
collapsed victim

Definition and Criteria of a True Experiment

A true experiment is a type of experimental design and is thought to be the most accurate type of
experimental research. This is because a true experiment supports or refutes a hypothesis using
statistical analysis. A true experiment is also thought to be the only experimental design that can
establish cause and effect relationships. So, what makes a true experiment?

There are three criteria that must be met in a true experiment

1. Control group and experimental group
2. Researcher-manipulated variable
3. Random assignment

Let's look at each of these requirements more closely.

Control Group and Experimental Group
True experiments must have a control group, which is a group of research participants that
resemble the experimental group but do not receive the experimental treatment. The control
group provides a reliable baseline data to which you can compare the experimental results.
The experimental group is the group of research participants who receive the experimental
treatment. True experiments must have at least one control group and one experimental group,
though it is possible to have more than one experimental group.

Researcher-Manipulated Variable
In true experiments, the researcher has to change or manipulate the variable that is hypothesized
to affect the outcome variable that is being studied. The variable that the researcher has control
over is called the independent variable. The independent variable is also called the predictor
variable because it is the presumed cause of the differences in the outcome variable.
The outcome or effect that the research is studying is called the dependent variable. The
dependent variable is also called the outcome/criterion variable because it is the outcome that
the research is studying. The researcher does not manipulate the dependent variable.

Random Assignment
Research participants have to be randomly assigned to the sample groups. In other words, each
research participant must have an equal chance of being assigned to each sample group. Random
assignment is useful in that it assures that the differences in the groups are due to chance.
Research participants have to be randomly assigned to either the control or experimental group.

Advantages
 The results of a true experimental design can be statistically analyzed and so there can be
little argument about the results.
 It is also much easier for other researchers to replicate the experiment and validate the
results.
 For physical sciences working with mainly numerical data, it is much easier
to manipulate one variable, so true experimental design usually gives a yes or no answer.

Disadvantages
 Whilst perfect in principle, there are a number of problems with this type of design. Firstly,
they can be almost too perfect, with the conditions being under complete control and not
being representative of real world conditions.
 For psychologists and behavioral biologists, for example, there can never be any guarantee
that a human or living organism will exhibit ‘normal’ behavior under experimental
conditions.
 True experiments can be too accurate and it is very difficult to obtain a complete rejection or
acceptance of a hypothesis because the standards of proof required are so difficult to reach.
 True experiments are also difficult and expensive to set up. They can also be very
impractical.
 While for some fields, like physics, there are not as many variables so the design is easy, for
social sciences and biological sciences, where variations are not so clearly defined it is
much more difficult to exclude other factors that may be affecting the manipulated variable.
 RESEARCH STRATEGIES AND VALIDITY

Research strategy
• reflects the general approach and goals of a research study
types of research strategies
• Descriptive strategy
• the goal is to describe the state of affairs at the time of the study
• measures variables as they exist naturally
• e.g. 19% of eligible voters participated in the election
• Correlational strategy
• measures two variables, usually as they exist naturally
• the goal of this strategy is to describe a relationship between the two variables
without attempting to explain the cause of the relationship
• e.g. Are students GPA’s related to their parent’s income?
• Nonexperimental strategy
• Answers questions about the relationship between two variables by demonstrating
a difference between two groups or two threatment conditions
• E.g. verbal scores of 6-years old boy and 6-years old girls
• Experimental strategy
• the researcher manipulates one variable (called independent variable) while
observing or measuring a second variable (dependent variable)
• this is the ‘true’ experiment because independent variable is manipulated by the
researcher (e.g. room temperature)
• the goal of experimental strategy is to determine whether a causal relationship
exists between two variables
• Quasi-experimental strategy
• uses a nonmanipulated variable to define groups or conditions (e.g. time or age)
or pre and post threatment
• controls other variables as much as possible
• the goal is to obtain evidence in support of a cause-and-effect relationship
• however, a quasi-experimental strategy can not unambiguously establish a causal
relationship

VALIDITY AND ITS THREATS

• validity is the standard criterion by which researchers judge the quality of research
• in this case the concept of validity applies to an entire research study
• any component of a research study that introduces questions or raises doubts about the
quality of the research process or the accuracy of the research results is a threat to validity

Internal validity
• is concerned with factors within the research study that raise doubts about the results or
the interpretation
• any factor within the study that allows an alternative explanation for the results is a threat
to internal validity
• e.g. example with room temperature and performance
Threats to internal validity
• Extraneous variables
• any variable in a research study other than the two variables being studied (both
systematic and unsystematic)
• unsystematically changing variables are usually not a problem
• confounding variable is an extraneous variable that is allowed to change
systematically along with the two variables being studied (e.g. time of the day in the
temperature-performance study)
• sources of extraneous/confounding variables
• participant variables
• assignment bias – when the process used to assign different participants to
different threatments produces groups of individuals with noticeably different
characteristics (e.g. one group is smarter, more motivated)
• environmental variables
• size of room, time of day, or gender of the experimenter
• measurement variables
• practice effects – prior exposure to a measurement procedure provides
participants with additional skills that produce improved scores (e.g. the same
exam)
• fatigue – prior participation tires the participants so that their scores on
subsequent measurements are lower
In their classic book on experimental research, Campbell and Stanley (1966) identify and discuss
8 types of extraneous variables that can, if not controlled, jeopardize an experiment's internal
validity.
1. History--These are the unique experiences subjects have between the various
measurements done in an experiment. These experiences function like extra, and
unplanned, independent variables. Compounding this, the experiences are likely to vary
across subjects which has a differential effect on the subjects' responses. Studies that take
repeated measures on subjects over time are more likely to be affected by history
variables than those that collect data in shorter time periods, or that do not use repeated
measures.
2. Maturation--These are natural (rather than experimenter imposed) changes that occur as
a result of the normal passage of time. For example, the more time that passes in a study
the more likely subjects are to become tired and bored, more or less motivated as a
function of hunger or thirst, older, etc. As Isaac and Michael (1971) point out, subjects
may perform better or worse on a dependent variable not as a result of the independent
variable but because they are older, more/less motivated, etc.
3. Testing--Many experiments pretest subjects to establish that all the subjects are starting
the study at approximately the same level, etc. A consequence of pretesting
programs/protocols is that they can contaminate/change the subjects' performance on
later tests (e.g., those used as dependent variables) that measure the same domain beyond
any effects caused by the treatment itself.
4. Instrumentation--Changing the measurement methods (or their administration) during a
study affects what is measured. Additionally, if human observers are used, it may be the
judgment of the observer(s) that change over time rather than the subjects' performance.
 5. Statistical Regression--When subjects in a study are selected as participants because
they scored extremely high or extremely low on some measure of performance (e.g., a
test, etc.), retesting of the subjects will almost always produce a different distribution of
scores, and the average for this new distribution will be closer to the population's. For
example, if the chosen subjects all had high scores initially, the group's average on the
retest will be lower (i.e, less extreme) than it was originally. Conversely, if the group's
mean was originally low, their retest mean would be higher.
6. Selection--The subjects in comparison (e.g., the control and experimental) groups should
be functionally equivalent at the beginning of a study. If they are, then observed
differences between the groups, as measured by the performance dependent variable(s),
at the end of the study are more likely to be caused only by the independent variable
instead of organismic ones. If the comparison groups are different from one another at the
beginning of the study the results of the study are biased.
7. Experimental Mortality--Subjects drop out of studies. If one comparison group
experiences a higher level of subject withdrawal/mortality than other groups, then
observed differences between groups become questionable. Were the observed
differences produced by the independent variable or by the different drop out rates?
(Mortality is also a threat when drop out rates are similar across comparison groups but
high.)
8. Selection Interactions--In some studies the selection method interacts with one or more
of the other threats (described above), biasing the study's results.

External validity
• concerns the extent to which the results obtained in a research study hold true outside the
constraints of the study
• Can the results be generalized to other populations, other settings, other measurements?
• e.g. can we generalize results from a well-controlled laboratory situation to the
uncontrolled chaos of the real world?

threats to external validity

• Participants
• characteristics unique to a specific group of participants in a study may limit ability to
generalize the results to individuals with different characteristics
• e.g. college students, volunteer bias, cross-species generalizations
• Features of the study
• characteristics unique to the specific procedures used in a study may limit ability to
generalize the results to situations where other procedures are used
• e.g. novelty effect, reactivity, specifics of the study (masking experiments, lexical
decisions)
• experimenters
• characteristics unique to the specific experimenter conducting the study may limit
ability to generalize the results to situations with a different experimenter
• e.g. experimenter bias, experimenter characteristics
• measurements
• characteristics unique to the specific measurement procedure may limit ability to
generalize the results to situations where a different measurement procedure is used
 • e.g. sensitization (the impact of being assessed), generality across different measures
(heart beat vs. questionnaire), time of measurement

Balancing internal and external validity

• attempts to increase internal validity can reduce external validity (laboratory
experiments)
• research that attempts to gain a high level of external validity will often create a research
environment that closely resembles the outside world
• there tends to be a tradeoff between internal and external validity (if you increase internal
validity, external validity decreases a vice versa)

Increasing Internal and External Validity

In group research, the primary methods used to achieve internal and external validity are
randomization, the use of a research design and statistical analysis that are appropriate to the
types of data collected, and the question(s) the investigator(s) is trying to answer. Single-subject
experimental studies almost always have high internal validity because subjects serve as their
own controls but, as mentioned earlier, are extremely low with respect to external validity.
Single-subject studies acquire external validity through the process of replication and extension
(i.e., repeating the study in different settings, with a different subject, etc.). The results of group
studies are also more acceptable by the scientific community when replicated.

Validity of individual research strategies

• descriptive strategy •
experimental strategy
• high external validity • high internal validity
• low internal validity • low external validity
• Nonexperimental strategy • quasi-experimental strategy
• High external validity • higher internal validity
• Low internal validity than descriptive and
• correlational strategy correlational studies
• high external validity • lower internal validity
• low internal validity than true experiments
• relatively high external
validity
Research strategies, research designs, and research procedures
• Research strategy
• refers to the general approach and goals of the study
• Research design
• general plan for implementing a research strategy (e.g. group versus individual,
same individuals vs. different individuals, number of variables included)
• 3 major design: Qualitative, Quantitative, and Mixed-Method
• Research procedure
• an exact, step-by-step description of a specific research study (exact involvement
of individuals, measurement of variables etc.)
 ALTERNATIVES TO EXPERIMENTATION: NONEXPERIMENTAL DESIGNS

Nonexperimental Approaches
 Used in situations in which an experiment is not practical or desirable.

Nonexperimental Method
 Used to study behaviors in natural settings to explore unique or rare occurrences or to
sample personal information

DESCRIBING RESEARCH ACTIVITIES

 All approaches to research can be described along two major dimensions:
◦ The degree of manipulation in antecedent conditions
◦ The degree of imposition of units

The Degree of Manipulation in Antecedent Conditions

 Varies from low to high
 From letting things happen as they will to setting up carefully controlled conditions
 Ex: Nutrition

The Degree of Imposition of Units

 Refers to the extent to which the researcher constrains, or limits, the responses a subject
may contribute to the data.
 Ex: Study of Teenagers

5 Common Nonexperimental Approaches used by Psychologists

1. Phenomenology
 Description of an individual’s immediate experience
 Rather than looking at behaviors and events that are external to us, we begin with
personal experience as a source of data
 Antecedents are not manipulated, and data may consist of any immediate experience; no
constraints are imposed
 Used by mental philosophers
 Disadvantage of Phenomenology:
 We cannot be sure that the process we are observing in ourselves is not altered in
some way by our attention to it.
 We may not be able to achieve the degree of accuracy and objectivity.
 It will be difficult for others to replicate our experiences and apply scientific criteria
to our findings. Without replication, it cannot be known if others would have the
same experiences.
 Phenomenology cannot be used to understand the causes of behavior. It can describe
but not explain.
 It may lead us into areas of discovery that may lead us to formulate hypotheses, but
experimentation is still required to determine which antecedent conditions produce
the behavior or experience.
2. Case Studies
 Involved no manipulation of antecedent conditions.
 Descriptive record of a single individual’s experiences, or behaviors, or both, kept by an
outside observer.
 Used in clinical, forensic and organizational psychology
 5 major purposes
 They are a source of inferences, hypotheses and theories.
 They are a source for developing therapy techniques.
 They allow the study of rare phenomena.
 They provide exceptions, or counterinstances, to accepted ideas, theories, or
practices.
 They have persuasive motivational value
 Deviant Case Analysis
 An extension of the evaluative case study
 Cases of deviant and normal individuals are compared for significant differences
 Disadvantage of Case Studies
 We cannot be sure the people we are evaluating are representative of the general
population
 We are not able to observe an individual directly all the time, we cannot be sure
that we are aware of all the relevant aspects of that person's life
 Subject or others providing data for case studies might neglect to mention
important information either because they believe it is irrelevant or because they
find it embarrassing
 Case studies frequently rely on retrospective data – data collected in the present
that are based on recollection of past events

3. Field Studies
 Nonexperimental approaches used in the field or in real life settings.
 Antecedent conditions are not manipulated but degree of constraints on responses varies
from study to study
 Participants are composed of groups
 2 Types:
 Naturalistic Observation Studies
 Participant Observation Studies
 Naturalistic Observation Studies
 Technique of observing behaviors as they occur spontaneously in natural setting
 Imposes very few constraints
 Should remain unobtrusive (can use unobtrusive measures)
 Ethology
 Used in animal research
 Study behavior in the wild as well as in captivity
 Disadvantage of Naturalistic Observation Studies:
 We are dealing with specific samples of time that may or may not contain the
behaviors we want to observe
 We might find that behaviors become very different when the subjects know they
are being watched (Reactivity)
  Participant-Observer Studies
 The researcher actually becomes part of the group being studied
 Sometimes this is the only method that can used to study a group – particularly if
the group would not reasonably be expected to cooperate voluntarily with a
research investigation
 Disadvantage of Participant-Observer Studies
◦ There is a possibility that the mere presence of an observer can alter subject’s
behaviors in unknown ways
◦ Sometimes observers in these studies find if difficult to remain objective and
unbiased.
◦ Often, particularly if friendships form, it is hard to remain an objective scientist.

4. Archival Study
 A descriptive research method in which already existing records are reexamined for a
new purpose
 Disadvantage of Archival Study
◦ Causal inferences cannot be supported all the time

5. Qualitative Research
 Relies on words rather than numbers for the data being collected
 It focuses on self-reports, personal narratives and expression of ideas, memories, feelings,
and thoughts,
 Without understanding qualitative methods, one cannot really appreciate the breadth of
research methodology used by contemporary psychological scientists.
 Candidate for paradigm shift
 Disadvantage of Qualitative Research
 Interpretation of data might be influenced by the researcher’s point of view
 The presence of a researcher could influence the way subjects response
 There is always a concern about the accuracy of self-reports as well as the use of
retrospective data
 Internal and External Validity are issued of concern
 ALTERNATIVES TO EXEPERIMENTATION: SURVEYS AND INTERVIEWS

SURVEY RESEARCH
o It is a useful way of obtaining information about people’s opinions, attitudes, preferences,
and behaviors simply by asking. Survey allow us to gather data about experiences,
feelings, thoughts and motives that are hard to observe directly.
o Examples:
1. Written questionnaires
2. Face-to-face interviews
3. Conducted by computer
4. Over the telephone

CONSTRUCTING SURVEYS
o STEPS IN COSTRUCTING SURVEY
1. Map out your research objectives (specific)
Ex.
General
The attitudes of other psychology students toward animal research in psychology
Specific
Animal rights, animal welfare, benefits to humanity, pain and suffering, humane treatment,
euthanasia, different species, animals used for class demonstrations, etc.
2. Design the survey items
a. Closed Questions (Structured Questions): must be answered by one of a limited
number of alternatives.
b. Open-ended Questions (Open Questions): soliciting information about opinions
and feelings by asking the question in such a way that the person must respond
with more than a yes, no, or 1-10 rating.

Content analysis – evaluate and categorize the content of each answer.

TIPS:
o Keep items simple
o Keep people involved
o Use language that is easy to comprehend
o Avoid double-barreled or compound questions

MEASURING RESPONSES
 Open-ended questions – content analysis
 It is just a matter of writing a simply worded questions that requires subjects to
answer in narrative form.
 Closed questions – level of measurement
 The level of measurement is the kind of scale used to measure a response
 Nominal, ordinal, interval and ratio

Scales of measurement
• Scales define the type categories we use in measurement and the selection of a scale has
direct impact on our ability to describe relationships between variables
 • the nominal scale
• simply represents qualitative difference in the variable measured
• can only tell us that a difference exists without the possibility telling the direction
or magnitude of the difference
• e.g. majors in college, race, gender, occupation
• the ordinal scale
• the categories that make up an ordinal scale form an ordered sequence
• can tell us the direction of the difference but not the magnitude
• e.g. coffee cup sizes, socioeconomic class, T-shirt sizes, food preferences
• the interval scale
• categories on an interval scale are organized sequentially, and all categories are
the same size
• we can determine the direction and the magnitude of a difference
• May have an arbitrary zero (convenient point of reference)
• e.g. temperature in Farenheit, time in seconds
• the ratio scale
• consists of equal, ordered categories anchored by a zero point that is not arbitrary
but meaningful (representing absence of a variable
• allows us to determine the direction, the magnitude, and the ratio of the difference
• e.g. reaction time, number of errors on a test

SELECTING LEVELS OF MEASUREMENT

2 Things to Consider:
1. The nature of the variable you are studying
2. How much measurement precision you desire

IMPORTANT CONSIDERATIONS FOR SURVEY ITEMS

A.
1. Get subjects involved right away by asking interesting questions.
2. Begin your survey with questions that people will not mind answering.
First questions must be:
1. Relevant to the central topic
2. Easy to answer
3. Interesting
4. Answerable by most respondents
5. Closed format

B.
1. If you are collecting demographic information, consider placing those questions at the
end of the questionnaire. (People will be more likely to answer demographic items if they
have already invested their time and energy in answering the survey questions up to this
point)
C.
 Make sure your questions are not value laden. (Do not word your questions in ways that
would make a positive (or negative) response seem embarrassing (or undesirable.)
 Be sure to keep the ethical guidelines in mind when you write survey questions.
RESPONSE STYLES
 Response Styles are tendencies to respond to questions or test items in specific ways,
regardless of the content
 Different response styles: willingness to answer, position preferences, and yea-saying and
nay-saying

WILLINGNESS TO ANSWER
 Comes into play whenever questions require specific knowledge about facts or issues.
 When unsure, some people will leave questions blank; others will take a guess.
Ex.
 Tell the subjects....
 To guess if they are not sure of the answer to a question
 That are no right or wrong answers to the questions (if that is true)
 Other researchers say nothing – and hope for the best

POSITION PREFERENCE
 If your questions are multiple-choice, response styles can influence the selection of
answers.
Ex.
 When in doubt about the right answer on a multiple-choice exam, perhaps you always
answer C

YEA-SAYING (Response Acquiescence)

 Yea-sayers apt to agree with a question regardless of its manifest content
NAY-SAYING (Response Deviation)
 Nay-sayers tend to disgaree no matter what they are asked

How can we solve this problem?

 Design questions that force the subject to think more about the answer.
 Once the questions have been designed, they need to be pretested before actual data
collection can begin.

EVALUATING SURVEYS AND SURVEY DATA

 The “goodness” of the survey itself is typically evaluated into two areas: reliability and
validity
 RELIABILITY – is the extent to which the survey is consistent and repeatable
 VALIDITY – usually refers to the extent to which a survey actually measures the
intended topic
 ALTERNATIVES TO EXPERIMENTATION: CORRELATIONAL AND QUASI-
EXPERIMENTAL DESIGNS

CORRELATIONAL DESIGNS

Correlational study
 is one that is designed to determine the correlation, or degree of relationship, between
two traits, behaviors, or events.
 Use correlational studies to explain behaviors that are not yet well understood.
 In the correlational study, the researcher measures events without attempting to alter the
antecedent conditions in any way; the researcher is simply how well the measures go
together.
Simple Correlation
 relationships between pairs of scores from each subject.
 Pearson Product Moment Correlation Coefficient (r) is the most commonly used
procedure for calculating simple correlations with three general possible outcomes: a
positive relationship, a negative relationship, or no relationship.

 When r is computed, three general outcomes are possible:

1. Positive Relationship
 When the computed value of r is positive, there is a positive correlation
(direct relationship)
2. Negative Relationship
 When the computed value of r is negative, there is a negative correlation
(Inverse relationship)
3. No relationship
 r is near zero
 Scatterplots
1. visual representations of the scores
belonging toe ach subject in the study.
 Regression lines or lines of best fit:
1. lined drawn on the scatterplots.

Correlation coefficients are also affected by:

 Range Truncation (an artificial restriction of the range of values of X or Y)
 By restricting the age range, the positive trend becomes very weak.
 Presence of Outliers (extreme scores)
 The presence of outliers can dramatically reduce the size of the correlation
coefficient because it disturbs the general linear trend of the data.

 Correlation does not imply causation. Why? because the causal direction between two
variables cannot be determined by simple correlations.
 Bidirectional causation: Variable #2 may affect variable #1 or the other way
around BUT NOT A CAUSE
 Third variable problem: some third agent are causing two behaviors to appear to
be related.
Coefficient of determination (r2):
 estimates the amount of variability in scores on one variable that can be explained by the
other variable – an estimate of the strength of the relationship between them.

LINEAR REGRESSION ANALYSIS

 Linear Regression Analysis
 uses regression equation to predict scores on one variable from scores on a second
correlated variable.
 The stronger the correlation, the better the prediction, in general.

MULTIPLE CORRELATION AND MULTIPLE REGRESSION

 Multiple Correlation
 tests the relationship of several predictor variables.
 Multiple Regression Analysis
 uses regression equation to predict scores on one variable from scores on sets of
other variables.

CAUSAL MODELING
1. PATH ANALYSIS
- uses beta weights from multiple regression analysis to generate possible direction of
cause and effect from correlated variables
2. CROSS-LEGGED PANEL DESIGN
- measures the same pair of variables at two different points in time; look at patterns of
correlations across time for possible direction of cause and effect.

QUASI-EXPERIMENTAL DESIGNS
QUASI-EXPERIMENTAL DESIGNS
 investigates differences in preexisting groups of subjects; group differences on some
variable may be explored or different treatments given to preexisting groups may be
compared.
 like an experiment, typically involves a comparison of groups or conditions
 however, it uses a nonmanipulated variable to define groups or conditions being
compared such as age, gender or time (before vs. after treatment)
 within the context of quasi-experimental research, the variable that is used to differentiate
the groups of participants or the groups of scores being compared is called the quasi-
independent variable (e.g. age)
 the variable that is measured to obtain a score for each individual is called the dependent
variable (e.g. IQ score)
Types
 EX POST FACTO STUDIES: explores characteristics, behaviors, or effects of naturally
occurring events in preexisting groups of subjects
 NONEQUIVALENT GROUPS: compares the effects of different treatment conditions
on preexisting groups of subjects
 LONGITUDINAL DESIGN: investigate changes across time by measuring behavior of
same group of subjects at different points in time.
  CROSS-SECTIONAL STUDIES: investigates changes across time by comparing
groups of subjects already at different stages at a single point in time
 PRETEST/POSTTEST DESIGN: explores the effects of an event (or treatment) by
comparing behavior before and after the event (or treatment)

NONEXPERIMENTAL RESEARCH STRATEGY

• Very similar to quasi-experimental research strategy
• Major distinction – nonexperimental designs make little or no attempt to minimize threats
to internal validity
• Just like in the quasi-experimental designs there is no real manipulation of variables
• In contrast to quasi-experimental designs there is less rigor in control of extraneous
variables
In what follows we will use the following symbols
• X – represents the treatment
• O – represents an observation or measurement
2 groups of nonexperimental and quasi-experimental designs:
• Between-subjects designs or nonequivalent group designs
• Within-subjects designs or pre-post designs

NONEQUIVALENT GROUP DESIGNS (BETWEEN-SUBJECTS)

• nonequivalent group design is a research study in which the different groups of
participants are formed under circumstances that do not permit the researcher to control
the assignment of individuals to groups because those groups are pre-existing; the groups
of participants are therefore considered nonequivalent
• e.g. a researcher wants to evaluate a teen pregnancy prevention program by comparing
the pregnancy rates in a high school where the program is used with pregnancy rates in a
high school that does not use the program
• nonequivalent group design has a built-in threat to internal validity that precludes an
unambiguous cause-and-effect explanation, i.e. assignment bias
• three examples of nonequivalent group designs
• the differential research design (nonexperimental)
• the posttest-only nonequivalent control group design (nonexperimental)
• the pretest-posttest nonequivalent control group design (quasi-experimental)

the differential research design (nonexperimental)

• simply compares pre-existing groups
• uses a participant characteristic such as gender, race, or personality to automatically
assign participants to groups
• no random assignment of participants to groups
• dependent variable is then measured for each participant to obtain a set of scores within
each group
• the goal of the study is to determine whether the scores for one group are consistently
different from scores in another group
• e.g. mother’s and father’s involvement in the peer relationships of their adolescent
children
 • in differential research, participant differences in one variable are used to create separate
groups, and measurements of the second variable are made within each group
• correlational study treats all the participants as a single group and simply measures the
two variables for each individual

Post-test only nonequivalent control group design (nonexperimental)

• compares two nonequivalent groups of participants
• one group is observed (measured) after receiving a treatment, and the other group is
measured at the same time but receives no treatment
• Does not protect agains assignment bias
• e.g. the teen pregnancy program
X O (treatment group)
O (nonequivalent control group)

Pretest-posttest nonequivalent control group design (quasi-experimental)

• compares two nonequivalent groups
• one group is measured twice – once before a treatment and once after
• Problem of assignment bias even though it is reduced by pre and post measurement
• Potential problems with differential history effects, differential instrumentation,
differential testing effects, differential maturation and differential regression
• the other group is measured at the same two times but does not receive any treatment
O X O (treatment group)
O O (nonequivalent control group)

PRE-POST DESIGNS (WITHIN-SUBJECTS)

• series of observations is made over time on one group of subjects
• Internal validity is threatened by time-related effects
• one-group pretest-posttest design (nonexperimental)
• Each individual in a group is measured once before and once after a treatment
• Does not control for possible extraneous variables possibly causing change over
time
O X O
e.g. evaluating the effectiveness of a new political TV commercial

time series and interrupted time-series designs (quasi-experimental)

• Series of observations for each participant before a treatment and a series of
observations after the treatment
• Eliminates many problems with the pretest-postest design, outside events are a
confound only if they occur simultaneously with the treatment
O O O X O O O
• Time series design – treatment administered by the researcher
• e.g. anger management program for students
• Interrupted time-series design – event or treatment is not created by the researcher
• e.g. legal change

Equivalent time-sample design (quasi-experimental)

 • Consists of a long series of observations during which a treatment is alternately
administered and then withdrawn
OOOXOOONOOOXOOONOOOXOOON…
N – no treatment
• It reduces likelihood of an external event confound to the treatment
• Differences between expected temporary and permanent effects
 EXPERIMENTAL DESIGNS: BETWEEN-SUBJECTS DESIGN

Introduction – Characteristics of between-subject design

There are two basic research designs associated with the experimental research strategy
between-subjects design
• we obtain each of the different groups of scores from a separate group of participants
• e.g. one group of students is assigned to teaching method A and a separate group to
method B
within-subjects design
• different groups of scores are all obtained from the same sample of participants
• e.g. one sample of individuals is given a memory test using a list of one-syllable words,
and then the same set of individuals is tested again using a list of two-syllable words

Characteristics of between-subject design

• the defining characteristic of a between-subjects design is that it compares separate
groups of individuals
• another feature of a between-subjects design is that it allows only one score per
participant (every score represents a separate, unique participant)
• because each score represents a separate participant, a between subjects design is often
called an independent-measures design
• a between-subjects experimental design requires a separate, independent group of
individuals for each treatment condition compared
• individuals are assigned to groups using a procedure that attempts to create equivalent
groups
• the general goal of between-subjects experiment is to determine whether differences exist
between two or more treatment conditions (e.g. a researcher may want to compare two
teaching methods (two treatments) to determine whether one is more effective than the
other)

Advantages and disadvantages of between-subjects designs

Advantages
• each individual score is independent of the other scores
• participant’s score is not influenced by such factors as:
• practice or experience gained in other treatments
• fatigue or boredom from participating in a series of treatments
• contrast effects that result from comparing one treatment to another (e.g. room
temperature)

Disadvantages
• large number of participants (problem with special populations)
• individual differences
• characteristics that differ from one participant to another are called individual
differences
• individual differences can become confounding variables
• individual differences can produce high variability in the scores
Confounding variables in between subjects designs
• individual differences
• participant characteristics differ from one group to another
• e.g. the participants in one group may be older, smarter, taller etc. than the
participants in another group
• environmental variables
• characteristics of the environment differ between groups
• e.g. one group may be tested in a large room and another group in a
smaller room
• Equivalent groups
• in a between-subjects experimental design, the researcher does have control over
the assignment of individuals to groups
• the separate groups must be:
• created equally
• treated equally (except for the treatment conditions)
• composed of equivalent individuals

Limiting confounding by individual differences

• random assignment (randomization)
• a random process is used to assign participants to groups
• matching groups (matched assignment)
• involves assigning individuals to groups so that a specific variable is
balanced or matched across the groups (e.g. IQ)
• holding variables constant
• simply hold the variable constant (e.g. restrict the participants to those
with IQs between 100-110)

Within and between treatments variability

• advantage
• variability between treatments
• it can be increased by increasing differences between conditions (levels)
• disadvantage
• variability within treatments
• it is caused by individual differences
• should be minimized

minimizing variability within treatments

• standardize procedures and treatment setting
• limit individual differences by holding a participant variable constant
• random assignment and matching
• sample size
• using a large sample can help minimize the problems associated with high
variability

Other threats to internal validity of between-subjects designs

• assignment bias
 • groups of participants are different before the treatments
• the group assignment process produces groups with noticeably different
characteristics
• differential attrition
• attrition refers to participant withdrawal from a research study before it is
completed
• differential attrition refers to differences in attrition rates from one group to
another and can threaten the internal validity of a between-subjects experiment
(e.g. effectiveness of a dieting program)
• diffusion or imitation of treatment
• refers to the spread of the treatment effects from the experimental group to the
control group (e.g. new depression therapy)
• compensatory equalization
• occurs when an untreated group learns about the treatment being received by
another group and demands the same or equal treatment (e.g. watching Batman in
violent TV group)
• compensatory rivalry
• occurs when an untreated group learns about the treatment received by another
group and then works extra hard to show that they can perform just as well as the
individuals receiving the special treatment
• resentful demoralization
• opposite of compensatory rivalry
• occurs when an untreated group learns about the treatment received by another
group and is less productive and less motivated because they resent the expected
superiority of the treated group

Applications and statistical analyses of between-subjects designs

• comparing only two groups of participants
• this design is referred to as the single-factor two-group design or simply two
group design
• an independent-measures t test is used to determine whether there is a significant
difference between the means
• comparing means for more than two groups
• e.g. single factor multiple group design may be used and analysis of variance
(ANOVA) would be used for statistical analysis
• adding extra groups to a research study tends to reduce the differences between
groups
 EXPERIMENTAL DESIGNS: WITHIN-SUBJECTS DESIGN

Introduction – Characteristics of within-subjects designs

• a within subjects experimental design compares two or more different treatment
conditions (or compares treatment and control) by observing or measuring the same
group of individuals in all of the treatment conditions being compared
• a within-subjects design looks for differences between treatment conditions within the
same group of participants
• a within subjects design is often called a repeated-measures design because the research
study repeats measurements of the same individuals under different conditions
• it is used in experimental situations comparing different treatment conditions and also to
investigate changes occurring over time

Advantages and disadvantages of within-subjects designs

Advantages of within-subjects designs
• it requires relatively few participants
• it essentially eliminated all of the problems based on individual differences that are the
primary concern of a between-subjects designs
• a within-subjects design has no differences between groups
• each individual serves as his or her own control or baseline
Disadvantages
• time-related problems
• participant attrition
• history
• any outside events that occur during the time that a within-subjects experiment
is being conducted and has an influence on the participants’ scores
• maturation
• any physiological or psychological changes that occur in a participant during the
time a within-subjects experiment is conducted and that can influence the
participants’ scores (e.g. young children)
• instrumentation
• refers to changes in the measurement instrument that occur over time (e.g.
observer changes)
• statistical regression
• or regression toward the mean is a mathematical phenomenon in which extreme
scores (high and low) on one measurement tend to be less extreme on a second
measurement (especially a problem when participants are selected for their
extreme scores)

• order effects
• carryover effects
• changes in behavior or performance that are caused by participation in an earlier
treatment condition
• carryover effects exist whenever one treatment condition produces a change in
the participants that affects their scores in subsequent treatment conditions (e.g.
new skill from treatment 1 can influence results in treatment 2)
 • progressive error
• changes in participant's behavior or performance that are related to experience
over time in a research study but not related to a specific treatment or treatments
(e.g. practice effects and fatigue)

Dealing with time-related threats and order effects

controlling time
• if the different treatment conditions are scheduled over a period of weeks, the chances
greatly increase that the results will be influenced by some outside event (history) or
maturation or change in the measurement instrument
when a within-subjects design is not a good idea
• e.g. comparing two methods of teaching reading to first-grade children (carryover effects)
counterbalancing
• involves changing the order in which treatment conditions are administered from one
participant to another
• the goal is to use every possible order of treatment with an equal number of individuals
participating in each sequence
• the purpose of counterbalancing is to eliminate the potential for confounding by
disrupting any systematic effects from factors related to time or the order of treatments
• e.g. with two treatments one half of the participants begins in treatment 1, then moves to
treatment 2 and the other half begins in treatment 2, then receives treatment1

Applications and statistical analyses of within-subjects designs

• two-treatment designs
• a repeated-measures t test or an analysis of variance can be used to evaluate the
statistical significance of the mean difference
• if the data are measured on an ordinal scale, a Wilcoxon test can be used to
evaluate significant differences
• multiple-treatment designs
• with too many treatment conditions, the distinction between treatments may
become too small to generate significant differences in behavior
• statistical analysis – repeated-measures analysis of variance to test for any
significant differences among the treatment means

Matched-subjects designs
• each individual in one group is matched with a participant in each of the other groups
• the matching is done so that the matched individuals are equivalent with respect to a
variable that the researcher considers to be relevant to the study (e.g. IQ)
• maintains all the advantages of between-subjects and within-subjects designs without the
limitations of either (e.g. eliminates individual differences, time-related factors and order
effects)
 FACTORIAL DESIGNS

Introduction to factorial designs

• in real life variables rarely exist in isolation
• to examine these more complex, real-life situations, researchers often design research
studies that include more than one independent variable (e.g. caffeine and alcohol)
• in an experiment, an independent variable is often called a factor, especially in
experiments that include two or more independent variables
• a research design that includes two or more factors is called a factorial design
• this kind of design is often referred to by the number of its factors, as a two-factor design
or a three-factor design
• a research study with only one independent variable is often called a single-factor design
• each factor is usually denoted by a letter (A, B, C)
• factorial designs use a notation system that identifies both the number of factors and the
number of values or levels that exist for each factor
• e.g. caffeine (3 levels) and alcohol study (2 levels) would be described as 3 x 2
two factor design

Main effects and interactions

• the main differences among the levels of one factor are called the main effect of that
factor
• when the research study is represented as a matrix with one factor defining rows and the
second factor defining the columns, then the mean differences among the rows define the
main effect for one factor, an the mean differences among the columns define the main
effect for the second factor
• an interaction between factors occurs whenever the mean differences between individual
treatment conditions, or cells, are different from what is predicted from the overall main
effect of the factors
• when the effects of one factor depend on the different levels of a second factor, then there
is an interaction between the factors
• when the results of a two-factor study are graphed, the existence of nonparallel lines
(lines that cross or converge) is an indication of an interaction between the two factors

Types of factorial designs

• between-subjects designs
• there is a separate group of participants for each of the treatment conditions
• large number of participants – e.g. 20 participants in each condition for a 2 x 4
design means 160 participants
• within-subjects designs
• single group of individuals participates in all of the separate treatment conditions
• only 20 participants for 2 x 4 factorial design
• mixed-designs (with respect to factors)
• between subjects design can apply to one factor and a within-subjects design is
preferable for a second factor (e.g. mood as between subjects factor and memory
as within subjects factor)
 • a factorial study that combines two different research designs is called a mixed
design
• a common example of a mixed design is a factorial study with one between-
subjects factor and one within-subjects factor
• Experimental and nonexperimental or quasi-experimental research strategies
• a factorial study that combines two different research strategies
• a common example of a mixed design is a factorial study with one experimental
factor and one nonexperimental factor (e.g. gender differences in memory tests)
• Pretest-posttest control group designs
• Quasi-experimental (two factor mixed design)
• One factor between subjects – treatment type
• Second factor within subjects – pre-post test
O X O (treatment group)
O O (control group)
• Experimental version of the same design
ROXO
RO O
Where R symbolizes random assignment of subjects into groups

• Higher-order factorial designs

• Complex designs involving three or more factors
• Example of a three factor design examining two teaching methods (A), boys and
girls (B) and first and second grade classes (C)
• This three factor design can be summarized as 2x2x2
• 2 (two teaching methods) x 2 (boys and girls) x 2 (first and second grade)
 SINGLE-SUBJECT RESEARCH DESIGNS

Introduction to single-subject designs

• single-subject designs or single-case design are research designs that use the results from
single participant or subject to establish the existence of cause-and-effect relationships
• Whereas case studies belong to the group of descriptive research strategies, single-case
studies are experimental
• like time-series research, the single-subject approach typically involves a series of
observations made over time but again single-subject designs are experimental and time-
series designs are quasi-experimental

Evaluating the results from a single-subject study

• the presentation and interpretation of results form a single-subject experiment are based
on a simple graph of the data
• because the results of a single-subject study do not involve any traditional statistical
methods, researchers must rely on the visual inspection of a graph to convey the meaning
of their results

Phases and phase changes

• a phase is a series of observations of the same individual under the same conditions
• when no treatment is being administered, the observations are called baseline
observations; a series of baseline observations is called a baseline phase and is identified
by the letter A
• when a treatment is being administered, the observations are called treatment
observations. A series of treatment observations is called a treatment phase and is
identified by the letter B.
• a consistent level occurs when series of measurements are all approximately the same
magnitude; in a graph, the series of data points cluster around a horizontal line
• a consistent trend occurs when the differences from one measurement to the next are
consistently in the same direction and are approximantely of the same magnitude; in a
graph the series of the data points cluster around a sloping line
• the stability of a set of observations refers to the degree to which the observations show a
pattern of consistent level or consistent trend; stable data may show minor variations
from a perfectly consistent pattern, but the variations should be relatively small and the
linear pattern relatively clear

The ABAB reversal design

• An ABAB design, also known as a reversal design, is a single-subject phase-change
design consisting of four phases: a baseline phase, a treatment phase, a return to baseline
phase, and a second treatment phase
• The goal of the design is to demonstrate that the treatment causes changes in the
participant’s behavior
• Limitations of the ABAB design
• It is not appropriate for evaluating treatments that are expected to have a
permanent or long-lasting effect
• There is also the ethical question of withdrawing a successful treatment
Dismantling or component-analysis design
• This design consists of a series of phases in which each phase adds or subtracts one
component of a complex treatment to determine how each component contributes to the
overall treatment effectiveness
• E.g. reward and punishment for biting behavior in an autistic boy
BC-B-BC-C-BC
BC – combination of the reward and punishment
B – reward only
C – punishment only
• When initial baseline phases correspond to separate participants, the design is called a
multiple-baseline across subjects
• Advantages
• The major advantage of this design is that it eliminates the need for a reversal or
return-to-baseline phase and is therefore well suited for evaluating treatment
effects that are permanent or long-lasting
• Disadvantages
• The results can be compromised by individual differences between participants or
between behaviors (e.g. one participant being more responsive to the treatment
than another)

Strengths and weaknesses of single-subject designs

• Advantages
• Researcher can establish a cause-and-effect relationship between treatment and
behavior using only a single participant
• Flexibility – development of the design depends on participant’s responses
• Disadvantages
• Problem with generalizations since designs use only one participant
• Multiple observations can affect participant’s responses
• Absence of statistical controls and reliance on visual inspection of the data
 SELECTING RESEARCH PARTICIPANTS

Research attempts to answer a general question about a large group of individuals, as opposed to
a specific question about a few, unique individuals. therefore, researchers typically want to
generalize or extend their results beyond the individuals who participate in a study

Population
• is the entire set of individuals of interest to a researcher
• although the entire population usually does not participate in a research study, the results
from the study are generalized to the entire population
Sample
• a set of individuals selected from a population and usually is intended to represent the
population in a research study
Target population
• the entire set of individuals who have the characteristics required by the researcher
• e.g. the whole population of people with eating disorders in the world or the US
Accessible population
• a subset of the target population, consisting of those individuals who are accessible to be
recruited as participants in the study
• e.g. people with eating disorders in local clinics
the sample
• a subset of the accessible population, consisting of those individuals who are selected to
participate in the research study
• 10% of the people with eating disorders in local clinics

• representative samples
• representativeness of a sample
• refers to the extent to which the characteristics of the sample accurately
reflect the characteristics of the population
• a representative sample
• is a sample with the same characteristics as the population
• a biased sample
• is a sample with different characteristics from those of the population
• selection bias
• occurs when participants or subjects are selected in a manner that
increases the probability of obtaining a biased sample
• e.g. a sample from the people in a university parking lot is not
representative of the adult population
SAMPLING BASICS
• sampling
• is the process of selecting individuals for a study
• probability sampling
• the entire population is known, each individual in the population has a
specifiable probability selection and sampling is a random process
• nonprobability sampling
• the population is not completely known, individual probabilities cannot be
known
• we try to avoid bias and maintain representativeness

Probability sampling methods

Simple random sampling

• each individual in the population has an equal and independent chance of being selected
• no individuals are more likely to be chosen than another
• The process of simple random sampling consists of the following steps
• clearly define the population from which you want to select a sample
• list all the members of the population
• using a random process, select individuals from the list
• e.g. children in a class
Systematic sampling
• sample is obtained by selecting every n-th participant for a list containing the total
population, after a random start
• after selecting the first individual it is not random
• e.g. every 5th child in a class
Stratified random sampling
• used when the focus of a research study is on specific subgroups within a population
• Steps in the process
• identify the specific subgroups (or strata) to be included in the sample
• we select equal random samples from each of the pre-identified subgroups
• we combine the subgroup samples into one overall sample
• e.g. selecting 25 men and 25 women from an INDV class
Proportionate stratified random sampling
• used if researchers try to improve the correspondence between a sample and a population
by deliberately structuring the sample so that its composition matches the composition of
a population
• e.g. if there are 750 men and 250 women in the INDV population the sample of 100
participants would contain 75 men and 25 women
Cluster sampling
• is used when there are well-defined clusters (groups) within the population
• e.g. selecting students from 10 classes instead of selecting 300 students one at a time

Nonprobability sampling methods

Convenience sampling
 • the most commonly used sampling method in psychological research
• researchers simply use as participants those individuals who are easy to get (e.g. INDV
students or volunteers)
• two strategies to help correct most of the serious problems associated with convenience
sampling
• researchers try to ensure that their samples are reasonably representative and not
strongly biased
• researchers provide a clear description of how the sample was obtained who the
participants are in their research studies
Quota sampling
• researcher first identifies specific subgroups to be included in the sample and then
establishes quotas for individuals to be selected from each subgroup
• e.g. 30-preschool children, parents respond to an ad, you don’t take the first 30 but
impose a quota of 15 girls and 15 boys
Purposive sampling
 nonrandom samples are selected because the individuals reflect a specific purpose of the
study
 these have set of criteria
Snowball sampling
 A researcher locates one or a few people who fit the sample criterion and asks these
people to locate or lead them to additional individuals.