by John W. Creswell, Ph.D.

Department of Educational Psychology,

University of Nebraska-Lincoln, and Founding Co-Editor, Journal of
Mixed Methods Research

University of Pretoria, October 21, 2008

Please do not duplicate or use these slides without the express permission of the author.
1
Topics

Introductions
Positioning myself
A definition of mixed methods research
Steps in the process of designing a mixed methods
study

2
Positioning myself

View research as set of interactive components;

not always linear
Focus on rigorous data collection and analysis
Work as an applied research methodogist
Trained in quantitative, self-trained in qualitative,
lst generation mixed methods writer
Serve as a consultant on mixed methods on
projects
Work on projects in an order not in proposal
format; then I reassemble into proper format
A few more thoughts about myself

Published by Sage Publications, Pearson Education (Merrill Educ.)

A Definition of Mixed Methods
Research
How would you combine two types of data?

Qualitative Quantitative Numeric Data

Text Data
23425231123423321
This is a sample of a text file of 11534123144554121
words that might be collected 43351423155221535
on interview transcripts, 13153225132443124
observation fieldnotes, or
optically-scanned documents. 2241554215
Framework for viewing perspectives on
mixed methods

Quantitative Data Qualitative Data

Mixed Methods

Method
Methodology
Paradigm
Perspective Use of mixed methods
in other designs
7
A mixed methods researcher

Collects both quantitative and qualitative data

Mixes them
Mixes them at the same time (concurrently) or one
after the other (sequentially)
Emphasizes both equally or unequally
A definition
Mixed methods research is both a method and methodology for
conducting research that involves collecting, analyzing, and
integrating quantitative and qualitative research in a single study or
a longitudinal program of inquiry.
The purpose of this form of research is that both qualitative and
quantitative research, in combination, provide a better understanding
of a research problem or issue than either research approach alone.

9
Collecting both quantitative and
qualitative data
Quantitative data Qualitative data
Interviews
Instruments
Observations
Checklists
Documents
Records Audio-visual materials

10
Quantitative and qualitative data
analysis

Quantitative analysis Qualitative analysis

Use statistical Use text and
analysis, images,
For description For coding
For comparing For theme
groups development
For relating For relating themes
variables

11
Mixing or linking the data

Converge data:

Qual Results Quan

Connect data:
Qual Quan Results

Embed the data:

Quan data
Qual data
12
Typical situations in which mixed methods is
used

To compare results from quantitative and qualitative

research
To use qualitative research to help explain quantitative
findings
To explore using qualitative research and then to
generalize findings to a large population using
quantitative research
To develop an instrument because none are available
or useful
To augment an experiment with qualitative data
 What is the reason for using mixed methods?

The insufficient argument either quantitative or qualitative may be

insufficient by itself
Multiple angles argument quantitative and qualitative approaches
provide different pictures
The more-evidence-the-better argument combined quantitative and
qualitative provides more evidence
Community of practice argument mixed methods may be the
preferred approach within a scholarly community
Eager-to-learn argument it is the latest methodology
Its intuitive argument it mirrors real life
Designing a Mixed Methods Study

Preliminary considerations
Creating a title
Posing a general question
Listing the types of data collection and analysis
Making explicit your worldview
Identifying your research design
Drawing a figure of your design
Writing a purpose statement
Writing research questions
Completing a research plan
Preliminary considerations (before you begin
to design)

Research problem
Content any topics
Fit the problem to mixed methods (arguments)

Access to both qualitative and quantitative data

Background and resources

Receptive audience

16
Create a working title

Writing the title

Short
Topic
Participants
Include the words Mixed methods
Neutral neither quan or qual

17
Pose the general question to be answered

Write it as a question
Look to see how it is phrased
Make sure that it is specific enough and focused (an
answerable question)
Ask yourself, when I end the study, what question
would like to have answered?

18
 List your types of data collection (a
review)
Quantitative data Qualitative data (open-ended)
(closed-ended) Interviews
Instruments Observations
Documents
Behavioral
Audio-visual materials
checklists
Records

19
Activity List your sources of data

Quantitative Sources of Qualitative Sources of

Data Data

20
List your approach to data analysis (a
review)

Quantitative analysis Qualitative analysis

Use statistical Use text and
analysis, images,
For description For coding
For comparing For theme
groups development
For relating For relating themes
variables Design-type
Design-type

21
Determine your worldview
Postpositivism Constructivism
Determination Understanding
Reductionism Multiple participant meanings
Empirical observation and Social and historical
measurement construction
Theory verification Theory generation

Advocacy/Participatory Pragmatism
Political Consequences of actions
Empowerment issue-oriented Problem-centered
Collaborative Pluralistic
Change-oriented Real-world practice oriented

22
Several stances on philosophy in mixed
methods

One paradigm (pragmatism, transformative) (Tashakkori &

Teddlie, 2003; Mertens, 2003)
Multiple paradigms (dialectic perspective) (Greene, 2007)
Linking paradigms to design features) (Creswell & Plano
Clark, 2007)
Epistemological stance (ontology, epistemology, axiology,
methodology) (Guba & Lincoln, 2005)
Shared beliefs in a research field (Morgan, 2007)

What it is
How it informs your study

23
Make explicit your interpretive lens (theory)

Advocacy lens (feminist, racial, ethnic, disability,

sexual orientation) (Mertens, 2003)
Social science lens (social science theory)

Components:
What is it
Who has used it in your field
How it will shape your study (rephrase your guiding
research question, if a lens applies)

24
State your procedures (methods)

Procedures for handling your qualitative and quantitative

data

Sequence concurrent or sequential or both

Emphasis emphasis on qualitative or quantitative
Sometimes both concurrent and sequential phases are used
Designs may include more than two phases
Think about using a simple, elegant design

25
Parsimonious designs (Creswell & Plano Clark, 2007)
Concurrent Mixed Methods Designs
Triangulation Design

QUAN QUAL
Data & Data &
Interpretation
Results Results

Embedded Design

QUAN Intervention QUAN

Pre-test Post-test
Interpretation
Data & Data &
qual
Results Results
Process

26
 Sequential designs
Explanatory Design
QUAN qual
Data & Data &
Interpretation
Results Results
Following up

Exploratory Design
QUAL quan
Data & Data &
Interpretation
Results Building to
Results

Sequential Embedded Design

Before-
QUAN After-
Intervention
intervention
Trial intervention
qual qual Interpretation

27
 Design options

Design Name Equal priority QUAN emphasis QUAL emphasis

Concurrent, triangulation QUAL+QUAN QUAN+qual QUAL+quan

Concurrent, embedded n/a QUAN(qual) QUAL(quan)

Explanatory, sequential, quan first QUANQUAL QUANqual quanQUAL

Exploratory sequential, qual first QUALQUAN qualQUAN QUALquan

Sequential, embedded n/a (qual) QUAN (quan) QUAL

QUAN (qual) QUAL (quan)

28
Explanatory sequential design
Quantitative Case Selection Qualitative Interpretation
Quantitative
Data Collection Data Analysis Data Analysis based on quan
(quan) (QUAL) ad QUAL results

+
Qualitative
Data Collection

Quantitative Data* Quantitative Analysis Case Selection Qualitative Analysis Interpretation

Number of cigarettes Graphic plot of CES- Selected 5 cases Description of each Why did changes in
CES-D6 D6 scores over time maximally varying case smoking occur?
for each participant Identified critical Identification of life
Qualitative Data* Graphic plot of months in which events occurring
cigarettes/day values smoking varied during critical
Semi-structured
over time for each months where
interviews, audio
participant smoking increased or
recorded and
decreased
transcribed
Thematic analysis of
life events for each
* Data collected 10 times
case
over the course of a
calendar year for 40 Cross-case thematic
participants analysis

Source: Creswell, Plano Clark, Shope, McVea. (in progress)

 Phase I Qualitative Research - Year 1

Qualitative Data Collection Unstructured Interviews -

50 participants
8 observations at the site
16 documents

Qualitative Data Analysis Text Analysis: Using QSR N6

Development of codes and themes

Qualitative Findings for each site

Phase II Quantitative Research - Year 2

Create approximately a 80-item
Quantitative Instrument Development instrument plus demographics

Administer survey to 500 individuals

Quantitative Test of the Instrument
Determine factor structure of items and
conduct reliability analysis for scales

Quantitative Results
Determine how groups differ
using ANOVA test

Exploratory sequential design

 Embedded research design

Experiment
Intervention
QUAN QUAN
Data collection Data collection
Pre-test Post-test

Process collection
and analysis of qualitative
data
(before, during, after trial)
32
Mixing the quan and qual data

Type of Mixing Type of Design Why Mixing Where Mixing

Occurs Occurs in
Research Process
Connecting Sequential One phase builds Between data
on the other analysis (Phase 1)
and data collection
(Phase 2)
Merging Concurrent Bring results After analysis of
together both quan and
qual typically in
discussion
Embedding Sequential or Either building or Either between
Concurrent bringing results phases or in
together discussion after
analysis
33
Methodological issues
Concurrent Designs
Use strategies to explore
contradictory findings
Use parallel questions
Select sub-sample of
quantitative for qualitative
Be sensitive to bias from
one data collection to the
other
Sequential designs
In Explanatory Design,
select qual sub-sample
from quan sample
In Explanatory Design,
consider alternatives for
followup qual sampling
In Exploratory Design,
samples can differ
In Exploratory Instrument
Design, consider qual data
analysis approaches for
developing instrument
34
Write a purpose statement for a
triangulation design
This mixed methods study will address _________________ (overall
content-aim of the study). A triangulation mixed methods design will
be used, and it is a type of design in which different but complementary
data will be collected on the same topic. In this study, _________________
(quantitative instruments) will be used to test the theory of
_____________ (the theory) that predicts that __________________
(independent variables) will influence ________________ (positively,
negatively) the __________ (dependent variables or outcomes) for
____________ (participants) at __________ (the research site).
Concurrent with this data collection, qualitative _______________ (type
of qualitative data, such as interviews) will explore ____________ (the
central phenomenon) for _________________ (participants) at
_____________(site). The reason for collecting both quantitative and
qualitative data are to bring together the strengths of both forms of
research to ___________________ (e.g., compare results, validate results,
corroborate results).

35
Write a purpose statement for
a embedded design
This mixed methods study will address _________________ (overall
content aim of the study). An embedded mixed method design will be
used, and it is a design in which one data set provides a supportive,
secondary role in a study based primarily on the other data set. The
primary purpose of this study will use ________________ (quantitative
instruments) to test the theory of _____________ (the theory) that
predicts that __________________ (independent variables) will
influence ________________ (positively, negatively) the __________
(dependent variables or outcomes) for ____________ (participants) at
__________ (the research site). A secondary purpose will be to gather
qualitative data _______________ (type of qualitative data, such as
interviews) that will explore ____________ (the central phenomenon)
for _________________ (participants) at _____________(site). The reason
for collecting the secondary database is ________________ (e.g., to
address different question, to provide support for the primary
purpose).

36
Write a purpose statement for
an explanatory design
This study will address _______________ (content-aim of the study). An
explanatory mixed methods design will be used, and it will involve
collecting qualitative data after a quantitative phase in order to explain
or follow up on the quantitative data in more depth. In the first
quantitative phase of the study, ______________ instrument data to be
collected from _______________ (participants) at ___________ (research
site) to test _______ (the theory) that explains why ______________
independent variables) relate to the ____________ (dependent
variables). The second qualitative phase will be conducted because
______________ (intent of the qualitative phase). In this exploratory
follow-up, the ______________ (central phenomenon) will be tentatively
explored with ___________ (participants) at _____________ (the research
site). The reason for the exploratory follow-up is to _______________
(e.g., to help explain or build upon initial quantitative results).

37
Write a purpose statement for
an exploratory design
This study addresses _________________ (content-area of the study). The
purpose of this exploratory sequential design will be to __________________
(e.g., develop an test an instrument, generate a taxonomy). The first phase
of the study will be a qualitative exploration of ________________ (the
central phenomenon) by collecting ___________________ (types of data)
from _________________ (participants) at _____________ (the research site).
The second quantitative phase will follow up on the qualitative phase for
the purpose of __________(intent of this followup). In the quantitative
phase, ___________ (instrument data) will be collected from ____________
(participants) at ______________ (research site). Quantitative research
questions/hypotheses will be formulated after the completion of the initial
qualitative phase. The reason for collecting qualitative data initially is that
_____________________ (e.g., instruments are not available, variables are not
known, there is little guiding theory).

38
Qualitative research questions

Qualitative central question

Begin with what or how
Focus on single phenomenon
Use exploratory verbs (discover, understand, explore)
Non-directional language
A general question (allowing participants perspectives
to emerge)

39
Quantitative research questions

Can be hypotheses or questions

State variables independent, dependent, mediating,
covariates
Develop from theory
Use distinct measures for independent and dependent
variables
Order variables from independent to dependent

40
Writing research questions/hypotheses in
mixed methods research

Write qualitative research questions and write quantitative

research questions/hypotheses

Also write a mixed methods research question

Write these questions separately

41
A new type of research question: A mixed
methods question
Three ways to write this question:

Methodologically-focused:

To what extent do the qualitative results confirm the quantitative

results?

Content-focused:

How do the interviews with adolescent boys support the

quantitative results that their self-esteem changes during the
middle school years?

Hybrid of quantitative and qualitative elements:

What results emerge from comparing the exploratory qualitative

data about boys self-esteem with outcome quantitative
instrument data measured on a self-esteem instrument?
Order the topics for your plan

Title
Introduction
(Problem)
Worldview/theoretical lens
Audience
Purpose Statement
Research Questions
(Literature Review)
Methods
Type of Mixed Methods Design (also add definition of mixed methods)
Types of Data Collection
Types of Data Analysis
Sequence/Emphasis/ Mixing Procedures
Figure of procedures
Anticipated methodological issues
Ethical issues anticipated
Validity issues
Researcher resources and skills
References, Appendices

43
Additional resources
Books:

Creswell, J. W., & Plano Clark, V. L. (2007). Designing and

conducting mixed methods research. Thousand Oaks, CA:
Sage.
Creswell, J. W. (2008). Research design: Qualitative,
quantitative, and mixed methods approaches (3rd ed.).
Thousand Oaks, CA: Sage Publications.
Greene, J. C. (2007). Mixed methods in social inquiry. San
Francisco: Jossey-Bass.
Mertens, D. M. (2005). Research methods in education and
psychology: Integrating diversity with quantitative and
qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage
Publications.
Plano Clark, V. L., & Creswell, J. W. (2008). The mixed
methods reader. Thousand Oaks, CA: Sage.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology:
Combining qualitative and quantitative approaches.
Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of
mixed methods in social and behavioral research. Thousand
Oaks, CA: Sage Publications. 44
Additional resources
Articles and Chapters:

Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-
method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2),
195-207.
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003).
Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-
240). Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological
issues in conducting mixed methods research. In M.M. Bergman (Ed.),
Advances in mixed methods research. London: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual
framework for mixed-method evaluation designs. Educational Evaluation and
Policy Analysis, 11 (3), 255-274.
Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods
sequential explanatory design: From theory to practice. Field Methods, 18(1),
3-20.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of
Mixed Methods Research, 1, 48-76.
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological
triangulation. Nursing Research, 40, 120-123.
45
 by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor, Journal of
Mixed Methods Research

University of Pretoria, October 21, 2008

Please do not duplicate or use these slides without the express permission of the author.
46