Sie sind auf Seite 1von 51

# Siska Nopa Tambunan_1031100_National Chiayi University

Instructional Objectives
Describe characteristics of experimental research
Define levels of the independent variable
Explain the concept of control and its importance in experimental research
Distinguish between true control groups and comparison groups in
experimental research
Define internal and external validity
Identify threats to internal validity and strategies for avoiding or
minimizing them
Identify internal validity problems in research proposals and reports
Describe the difference between random selection and random assignment
and show how they are related to internal and external validity

## Siska Nopa Tambunan_1031100_National Chiayi University

Experimental Research
An experiment is a scientific investigation in which the researcher
manipulates one or more independent variables, controls any other
relevant variables, and observes the effect of the manipulations on the
dependent variables.

## Experiment has 3 characteristics:

1. An independent variable is manipulated
2. All other variables that might affect the dependent variable
are held constant
3. The effect of the manipulation of the independent variable on
the dependent variable is observed

## Siska Nopa Tambunan_1031100_National Chiayi University

Characteristics of Experimental
Research

## Control Over Extraneous Variable

Manipulation
Observation and Measurement

## Siska Nopa Tambunan_1031100_National Chiayi University

Control Over Extraneous Variable

## The purpose of control in an experiment is to arrange a

situation in which the effect of a manipulated variable
on a dependent variable can be investigated
An extraneous variable is not related to the purpose of
the study but may affect the dependent variable

Manipulation

## Manipulation of an independent variable

is a deliberate operation performed by the
experimenter

## Levels of the independent variable is the

to the subjects in the experiment

## Siska Nopa Tambunan_1031100_National Chiayi University

Observation & Measurement
After applying the experimental treatment,
the researcher observes to determine if the
hypothesized change has occured.
For the variable that cannot observed
directly, then measurement is used.

## Siska Nopa Tambunan_1031100_National Chiayi University

Experimental Comparison
A group comparison is the process of a
researcher obtaining scores for individuals or
groups on the dependent variable and
comparing the means and variance both within
the group and between the groups.

## Siska Nopa Tambunan_1031100_National Chiayi University

Experimental Design
The experimental design sets up the
conditions required for demonstrating
cause-effect relationships.
Experimental Design differ in number of
independent variable, method of assigning
subjects and differ in how often dependent
variable measured.

## Siska Nopa Tambunan_1031100_National Chiayi University

Validity of Research Designs
There are four types of validity:
- Internal Validity
The validity of inferences about whether the effect of variable A (the
treatment) on variable B (the outcome) reflects a causal relationship
- External Validity
The validity of the inference about whether the cause-effect relationship
holds up with other subjects, settings, and measurement
- Construct Validity
The validity of the inferences about psychological constructs involve in
the subject, settings, treatment and outcome
- Statistical Conclusion Validity
The validity of the inferences about the covariation between treatment
and outcome

## Siska Nopa Tambunan_1031100_National Chiayi University

Threats to Internal Validity
History
Maturation
Testing
Instrumentation
Statistical Regression
Selection Bias
Experimental mortality (attrition)
Selection-maturation interaction
Experimenter Effect
Subject Effect
Diffusion

## Siska Nopa Tambunan_1031100_National Chiayi University

Threats to Internal Validity

## Siska Nopa Tambunan_1031100_National Chiayi University

Dealing With Threats to Internal
Validity
Random Assignment
Randomized Matching
Homogeneous Selection
Building Variables into the Design
Statistical Control
Using Subjects as their Own Control
Controlling Situational Differences

## Siska Nopa Tambunan_1031100_National Chiayi University

Dealing With Threats to Internal
Validity
Random assignment is the process of assigning individuals
at random to groups or to different groups in an experiment.
Randomized matching is the process of identifying one or
more personal characteristics that influence the outcome and
assigning individuals with that characteristic equally to the
experimental and control group.
Homogeneous selection, makes groups reasonably
comparable on an extraneous variable by selecting sample
that are as homogeneous as possible on that variable.
Analyis of covariance (ANCOVA) is a statistical technique
used to control for the effect of an extraneous variable known
to be correlated with the dependent variable.

## Siska Nopa Tambunan_1031100_National Chiayi University

Threats to External Validity
Selection-treatment interaction
(nonrepresentativeness)
Setting-treatment interaction (artificiality)
Pretest-treatment interaction
Subject effects
Experimenter effects

## Siska Nopa Tambunan_1031100_National Chiayi University

Threats to External Validity

## Siska Nopa Tambunan_1031100_National Chiayi University

Dealing With Threats to External
Validity
Randomly sample the target population to select subjects for the
study and then randomly assign them to treatment groups
Identify the relevant characteristics of individuals in the target
population, and determine the impact of these characteristics by
incorporating them into the research study
Control problem from pretest-treatment by choosing a design
that not use pretest
Control the reactive effects by arranging for a second control
group to experience an interesting interaction with the
researchers like experimental group
Replicate the research study in new setting

## Siska Nopa Tambunan_1031100_National Chiayi University

Instructional Objectives
Explain the function of a research design
Explain the relationship between research design and internal validity
Define preexperimantal design and discuss its limitations
Describe the most common randomized experimental designs and
Define factorial design and state its advantages
Distinguish between-groups and within groups experimental designs
Define quasi experimental and discuss its limitations
Describe single-participant experimental research and its uses

## Siska Nopa Tambunan_1031100_National Chiayi University

Experimental Designs Meaning
The general plan for carrying out a study with
an active independent variable (Ary et al,
2014)

## The traditional approach to conducting

quantitative research where you test an idea
(or practice or procedure) to determine
whether it influences an outcome or dependent
variable (Creswell, 2012)

## Siska Nopa Tambunan_1031100_National Chiayi University

Classifying Experimental Designs
Preexperimental Designs
True Experimental Designs
Factorial Designs
Repeated-measures Design
Quasi-Experimental Designs
Time Series Designs
Single Subject

## Siska Nopa Tambunan_1031100_National Chiayi University

Preexperimental Designs
Provide little or no control of extraneous variables.

## There are 2 types:

Design 1: One-Group Pretest-Posttest Design
Design 2 :Static Group Comparison

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 1: One-Group Pretest-Posttest
Design
Conducted in three steps:
1. Administering a pretest to measure the dependent
variable;
2. Applying experimental treatment X to the subjects; and
3. Administering a posttest to measure dependent variable
No control of extraneous variable; history and maturation
No way to assess the effect of the pretest
Design 1: One-Group Pretest-Posttest Design
Pretest Independent Posttest
Y1 X Y2

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 2: Static Group Comparison
The static group comparison uses two or more preexisting
or intact (static) groups, only one of which is exposed to the
experimental treatment
No randomized or matching
No pretest
Design 2: Static Group Comparison
Group Independent Posttest
E X Y2
C - Y2

## Siska Nopa Tambunan_1031100_National Chiayi University

True Experimental Designs
Subjects are randomly assigned to an experimental group and at
least one control group

## There are 5 types :

Design 3: Randomized Subjects, Posttest-Only Control Group
Design
Design 4: Randomized Matched Subjects, Posttest-Only Control
Group Design
Design 5: Randomized Subjects, Pretest-Posttest Control Group
Design
Design 6: Solomon Three-Group Design
Design 7: Solomon Four-Group Design

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 3 : Randomized Subjects, Posttest-Only
Control Group Design

## Design 3 is one of the simplest yet most powerful of all experimental

designs
Randomized and have a control group
Control main effects of history, maturation, regression, and presenting
No pretest
Design 3: Randomized Subjects, Posttest-Only Control Group
Design

E X Y2
C - Y2

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 4 : Randomized Matched Subjects,
Posttest-Only Control Group Design
Similar to Design 3, except it uses a matching technique to form
comparison groups
Matching : Select pairs of individuals with identical or almost
identical characteristics and randomly assign one member of the
matched pair to treatment A and the other to treatment B

E X Y2
C - Y2

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 5: Randomized Subjects, Pretest-
Posttest Control Group Design
Randomly assigns subject to the experimental and control groups
and administers a pretest measuring the dependent variable Y.
To determine the effectiveness of the treatment, compare the two
groups mean scores on the posttest. If no significant result,
compare the average between posttest and pretest (Y2-Y1) scores for
each group.
Design 5: Randomized Subjects, Pretest-Posttest Control Group Design

(R) E X Y2
(R) C - Y2

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 5: Randomized Subjects, Pretest-
Posttest Control Group Design

## Using analysis of covariance (ANCOVA)with

posttest as the dependent variable and pretest as
covariate to control any initial differences on the
pretest.
Threats external validity : pretest-treatment
interaction

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 6: Solomon Three-Group Design

## Identical with design 5, but the advantage is of employing a

second control group labeled C2 that is not pretested but is
exposed to the treatment X
The interactive effect of pretesting and the experimental
treatment
Design 6: Solomon Three-Group Design
Group Pretest Independent Posttest
(R) E Y1 X Y2
(R) C1 Y1 - Y2
(R) C2 - X Y2

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 7 : Solomon Four-Group Design
Extending Design 6 to include one more control group that receives neither
pretest nor treatment
Design 7: Solomon Four-Group Design
Group Pretest Independent Posttest
(R) E Y1 X Y2
(R) C1 Y1 - Y2
(R) C2 - X Y2
(R) C3 - - Y2

## Combination of the randomized subjects pretest-posttest control group design

and randomized subjects posttest-only control group design.
The disadvantages are more time and effort are required to conduct two
experiments simultaneously, problem about number of subjects, difficulty of
statistical analysis

## Siska Nopa Tambunan_1031100_National Chiayi University

Factorial Designs
Researcher manipulates two or more
variables simultaneously in order to study
the independent effect of each variable on
the dependent variable, as well as the
effects caused by interactions among the
several variables.

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 8: Simple Factorial Design
The simplest factorial design is the 2 x 2 read as
2 by 2. this design has two variables (factors)
and each factor has two levels
Variable 1 (X1)
Variable 2 (X2)
Treatment A Treatment B
Level 1 Cell 1 Cell 3
Level 2 Cell 2 Cell 4

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 8: Simple Factorial Design
The effect of one independent variable on the
dependent variable irrespective of the influence of
the other independent variable is referred to as
main effect
Main effect does not mean the most important effect
but rather the effect of one independent variable
(factor) ignoring the other factor
Interaction effect between two independent
variables that is the different effects of one of them
at different levels of the other.

## Siska Nopa Tambunan_1031100_National Chiayi University

Repeated-measures Design
individual in all of the different treatment is called a
within-subjects design.
It is also called repeated-measures design because
the research repeats measurement of the same
indiduals under different treatment conditions

## Siska Nopa Tambunan_1031100_National Chiayi University

Quasi-Experimental Designs
Quasi-Experimental Designs are similar to
randomized experimental designs that they involve
manipulation of an independent variable but differ in
that subjects are not randomly assigned to treatment
groups.

## There are 2 types:

Design 9: Nonrandomized control group, pretest-
posttest design
Design 10: Counterbalanced Design

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 9: Nonrandomized Control Group,
Pretest-posttest Design

E Y1 X Y2
C Y1 - Y2

## One of the most widely used in educational research.

This design does not permit random assignment of
subjects to the experimental and control group.

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 9: Nonrandomized Control
Group, Pretest-posttest Design

## Internal Validity Threats

Interaction of selection and maturation
Interaction of selection and regression
Interaction of selection and instrumentation

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 10: Counterbalanced Design
Used with intact class groups, rotates the groups at
interval during the experimentation
Design 10: Counterbalanced Design
Experimental Treatments
Replication X1 X2
1 Group 1 Group 2
2 Group 2 Group 2

## Siska Nopa Tambunan_1031100_National Chiayi University

Time Series Designs
There are 2 types :
Design 11: One-Group Time-Series Design
Design 12: Control Group Time-Series Design

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 11: One-Group Time-Series Design
The one-group time series is a quasi-experimental design that
involves periodic measurement on one group and the
introduction of an experimental treatment into this time series
of measurements.
Design 11: One-Group Time-Series Design
Y1 Y2 Y3 Y4 X Y5 Y6 Y7 Y8

## The major weakness of design 11 is its failure to control

history: that is, you cannot rule out the possibility that it is not
X but rather some simultaneous event that procedures the
observed change

## Siska Nopa Tambunan_1031100_National Chiayi University

Design 12: Control Group Time-Series Design

## Control group time-series design is an extension of Design

11 to include a control group.
This design overcomes the weakness of Design 11 that is
failure to control history as a source of extraneous variance.

## Design 12: Control Group Time-Series Design

E Y1 Y2 Y3 Y4 X Y5 Y6 Y7 Y8
C Y1 Y2 Y3 Y4 - Y5 Y6 Y7 Y8

## Siska Nopa Tambunan_1031100_National Chiayi University

Single-Subject Experimental Designs

## Single-subject experimental designs share a unique

feature: the sample size is just one, or is composed of a few
participants who are treated as one unit.
Researcher measure the dependent variable repeatedly
during at least two different points in time, when the
treatment is not present and again when a treatment is
present.
Treatment periods: the periods when the treatment is given
Baseline periods: the treatment is not present.

## Siska Nopa Tambunan_1031100_National Chiayi University

Single-Subject Experimental Designs
ABAB Designs
Consist of a period of no treatment, or baseline (A), during which the
behavior of interest is repeatedly measured.

Multiple-Baseline Designs
In this design, observations are made on several participants, different
target behaviors of one or more subjects, or different situations.
There are 3 basic designs: multiple-baseline across-participants
design, multiple-baseline across-behaviors design, and multiple-
baseline across-settings design.

## Siska Nopa Tambunan_1031100_National Chiayi University

Comparison of Single-subject and
Group Designs

## The goal of both designs is to establish as unequivocally as

possible the connection between the manipulation of the
independent variable (treatment) and its effect on the
dependent variable (behavior)
Random assignment of subject eliminates many rival
explanation of differences
Single-participant design controls amount of time which
baseline and treatment.
Single-subject threats to external validity generalizability
of experimental findings

## Siska Nopa Tambunan_1031100_National Chiayi University

Example of Article
True Experimental Designs

## Independent Variable: Multiple Intelligences Approach

Dependent Variable: Students achievement and attitude toward science
Experimental Group: Multiple Intelligences Approach
Research Design: Pretest-Posttest Control Group (Design 5)
Sample 60 students randomly selected from 183 students
Randomly assigned to the experimental and control groups

## Siska Nopa Tambunan_1031100_National Chiayi University

Example of Article
Quasi-Experimental Designs

## Independent Variable: Collaborative Writing Activity using Google Docs

Dependent Variable: Students Writing Abilities
Experimental Group: Collaborative Writing Activity using Google Docs
Control Group: Group face to face
Research Design: Quasi-experimental Design (Design 9)
Sample 80 students (2 sections) cluster sampling from 5,625 students
(123 sections)
No Randomly assigned to the experimental and control groups
Content Validity by three experts
Pretest for both groups

## Siska Nopa Tambunan_1031100_National Chiayi University

Continue

Quasi-Experimental Designs
Threats to internal Validity (analysis result)
Subject Effects (Hawthorne effect):
The students know that their teacher can check who
works less or more throughout the learning process. As
such, student in experimental group were more serious
convention and practices

## Siska Nopa Tambunan_1031100_National Chiayi University

References
Ary, D., Jacobs, L. C., Sorensen, C. K., & Walker, D. A. (2014). Introduction to
Research in Education 9th Edition. Wadsworth, USA: Cengage Learning.
Creswell, J. W. (2012). Educational Research 4th Edition. Boston: Pearson
Education.
Suwantarathip, O., & Wichadee, S. (2014). The effects of collaborative
writing activity using google docs on students writing abilities. The
Turkish Online Journal of Educational Technology, 13(2), 148-156
Kaya, O. N., Dogan, A., Gokcek, N., Kilic, Z., & Kilic, E. (2007). Comparing