Beruflich Dokumente
Kultur Dokumente
OVERVIEW OF
CONTEMPORARY ISSUES
IN MIXED METHODS RESEARCH
Objectives
◆ 1
2–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
I
Conceptual Orientations:
Philosophical, Theoretical,
Sociopolitical
II III
Issues Regarding Contemporary
Methods Applications
and Methodology of Mixed Methods
Research
Note: These circles portray the information contained in the three parts of this volume.
4–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
Methods” could have been placed in Part II, easily accepted in some disciplines or specialty
but we put it in Part I because of its argu- areas than others? Why are academic disci-
ment for universal social science research plines reluctant to embrace mixed methods
principles devoid of paradigm considera- (e.g., psychology)? Are mixed methods
tions or schisms between the QUAL and techniques applied similarly across discipli-
QUAN approaches (see also Onwuegbuzie nary lines, or are there differences?
& Leech, 2005). The overlap between conceptual orienta-
We think that the overlaps or interfaces tions and applications of MMR also con-
among the three Handbook sections, as tains some interesting topics. Foremost
depicted in Figure 1.1, are among the most among these are sociopolitical commitments,
valuable characteristics of the organiza- which we characterized earlier as the inter-
tional structure of this volume. The topics action between concerns and problems of
within those overlapping areas are in the the real world and the axiological orienta-
“border land” between conceptual issues tions of researchers.
and methods (Circles I and II), between
methods and applications (Circles II and III),
and between conceptual issues and applica- WHAT IS THE
tions (Circles I and III). As such, these top- STRUCTURE OR “MAP” OF
ics tend to be dynamic and fluid. For MIXED METHODS RESEARCH?
instance, how are conceptual issues differ-
ent from and similar to issues regarding The structure or “map” of MMR emerged
methods and methodology? What does the as an important issue as a result of
overlap between these two sections consist Creswell’s (2009, 2010) recent insightful
of in terms of specific topics? How do con- reflections on the topic. He bases the impor-
ceptual orientations affect the selection of tance of a current map of MMR on a very
methods, or do they? practical consideration: Authors submitting
Authors of three Part I chapters address articles to publications such as the Journal
the overlap between Circles I and II directly of Mixed Methods Research have needed
by demonstrating how conceptual orienta- such a structure “so that they could position
tions are inextricably linked to how MMR their study within the existing discussions”
is conducted (Greene & Hall; Hesse-Biber; ongoing in the field (Creswell, 2009, p. 96).
Mertens et al.). On the other hand, Leech Creswell (2010) compares three perspec-
(Chapter 11), who interviewed four of the tives regarding the current field of MMR
early developers of the field, reported that (Creswell, 2009; Greene, 2008; Tashakkori
two of them did not include “philosophy” & Teddlie, 2003c) that were useful in devel-
(or conceptual orientation) in their defini- oping the general domains in his map of
tions of MMR (Alan Bryman, Janice MMR. He compares each of these three
Morse), while a third (John Creswell) sources in terms of specific issues and ques-
included it in his 2009 interview with tions that were addressed in their perspec-
Leech but not in a definition given 2 tives on MMR. (See Table 2.1 in Creswell,
years earlier (Johnson et al., 2007). The 2010.) The five general domains that
interaction (or lack of it) between concep- Creswell identified are: the essence of mixed
tual and methodological issues in MMR is methods domain, the philosophical domain,
a complex and evolving one, which we the procedures domain, the adoption and
detail later in this chapter. use domain, and the political domain.
Topics in the overlap between methods Creswell (2009) used a similar set of
and applications include issues such as why domains to categorize specific topics (e.g., the
and how mixed methods are differentially use of the QUAL theoretical lens in MMR,
applied across different disciplines. For joint displays of QUAN and QUAL data)
example, why are mixed methods more within the literature. We believe that the
8–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
methods. The alternative to this point of critical theory, the dialectic stance, critical
view, the compatibility thesis, contends that realism, and so forth (e.g., chapters by Biesta;
combining QUAN and QUAL methods is Greene & Hall; Maxwell & Mittapalli;
appropriate in many research settings, Hesse-Biber; Mertens et al.).
denying that such “a wedding of methods We believe that contemporary MMR is
is epistemologically incoherent” (Howe, a kind of “big tent” and that it is both
1988, p. 10). The rejection of the incom- unwise and unnecessary to exclude individ-
mensurability of paradigms thesis7 is a uals from the MMR community because
major point of demarcation between advo- their conceptual orientations are different.
cates of MMR and others advocating purist We agree with Denzin’s (2008) paraphrase
methodological stances. of a theme originally stated by Guba
Methodological eclecticism means that (1990): “A change in paradigmatic pos-
we are free to combine methods and that tures involves a personal odyssey; that is,
we do so by choosing what we believe to be we each have a personal history with our
the best tools for answering our questions. preferred paradigm and this needs to be
We have called this choice of “best” meth- honored” (p. 322). Paradigm pluralism calls
ods for answering research questions for practitioners of mixed methods to honor
“design quality”8 and have included it as an a variety of philosophical or theoretical
essential part of our framework for deter- stances among their colleagues.
mining the inference quality of MMR The third characteristic of contemporary
(Tashakkori & Teddlie, 2008). While we MMR is an emphasis on diversity at all levels
endorse methodological eclecticism, it is of the research enterprise, from the broader,
also important to recognize that: more conceptual dimensions to the narrower,
more empirical ones. This characteristic
1. The best method for any given study extends to issues beyond the aforementioned
in the human sciences may be purely methodological eclecticism and paradigm
QUAL or purely QUAN, rather pluralism. For example, MMR can simulta-
than mixed. neously address a diverse range of confir-
matory and exploratory questions, while
2. Most seemingly purist QUAL or
single-approach studies often address only
QUAN studies might actually include
one or the other. Properly conducted MMR
shades of the other approach (i.e.,
also provides the opportunity for an assort-
studies that may be placed on multi-
ment of divergent conclusions and inferences
ple continua, each including a shade
due to the complexity of the data sources and
of QUAL and QUAN approaches.
analyses involved in the research.
We will discuss this later under the
MMR emerged partially out of triangu-
fourth characteristic of MMR).
lation literature, which has commonly been
3. The terms QUAL and QUAN are associated with the convergence of results.
often proxies for different con- Nevertheless, there is a growing awareness
cepts/attributes across studies (i.e., that an equally important result of com-
QUAN approach might mean dif- bining information from different sources
ferent things in different studies). is divergence or dissimilarity (e.g., Erzberger
& Kelle, 2003; Greene, 2007; Johnson &
The second contemporary characteristic Onwuegbuzie, 2004; Tashakkori &
of MMR is paradigm pluralism, or the belief Teddlie, 2008). This emphasis on divergent
that a variety of paradigms may serve as the results often provides greater insight into
underlying philosophy for the use of mixed complex aspects of a phenomenon, which
methods. A variety of conceptual orientations can then lead to more in-depth investiga-
associated with mixed methods are repre- tion of previously unexplored aspects of
sented in this volume, including pragmatism, that phenomenon.
10–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
The fourth characteristic of contem- (associated with deductive logic) and the
porary MMR is an emphasis on con- context of discovery (associated with induc-
tinua rather than a set of dichotomies tive logic), which has recently been dis-
(e.g., Newman, Ridenour, Newman, & cussed in MMR (e.g., Johnson & Gray,
DeMarco, 2003; Niglas, 2004; Patton, 2010; Hesse-Biber, 2010 [this volume];
1990, 2002; Tashakkori & Teddlie, 2003c). Teddlie & Johnson, 2009a). While practi-
A hallmark of MMR is its replacement of tioners of MMR recognize the logic of jus-
the either-or from the paradigm debates tification as a key part of their research,
with continua that describe a range of they also acknowledge the importance of
options from across the methodological the context of discovery, which involves
spectrum. Johnson and Gray (2010) refer to creative insight possibly leading to new
this antidualistic stance as synechism, which knowledge. This discovery component of
involves replacing binaries with continua. MMR often, but not always, comes from
For example, we have applied what we the emergent themes associated with QUAL
called the QUAL-MIXED-QUAN multidi- data analysis.
mensional continuum to a variety of The sixth characteristic endorsed by
research issues, including statement of many writing in MMR is a focus on the
research questions, designs, data analysis, research question (or research problem) in
and validity or inference quality (Teddlie & determining the methods employed within
Tashakkori, 2009). Niglas (2010 [this vol- any given study (e.g., Bryman, 2006;
ume]) has extended this discussion through Johnson & Onwuegbuzie, 2004; Niglas,
her multidimensional model of research 2010; Tashakkori & Teddlie, 1998). This
methodology, which presents a variety of centrality of the research question was
philosophical and methodological continua initially intended to move researchers
within a multidimensional space and the (particularly novices) beyond intractable
placement of specific research methods philosophical issues (e.g., epistemological,
within that space. ontological) associated with the paradigms
The fifth characteristic of contemporary debate and toward the selection of methods
MMR is an iterative, cyclical approach to that were best suited to investigate phe-
research, which includes both deductive and nomena of interest to them.
inductive logic9 in the same study (e.g., Much has been written about the starting
Krathwohl, 1993, 2004; Tashakkori & point for research in the past decade; that is,
Teddlie, 1998). The cycle of research may be do researchers start with a worldview or
seen as moving from grounded results (facts, conceptual problem, a general purpose for
observations) through inductive logic to gen- conducting research, a research question, or
eral inferences (abstract generalizations or some combination thereof? Newman et al.
theory), then from those general inferences (2003) have argued convincingly that dur-
(or theory) through deductive logic to tenta- ing the past four decades, the research pur-
tive hypotheses or predictions of particular pose has gained in importance relative to the
events/outcomes. Research may start at any research question. We maintain, however,
point in the cycle: Some researchers start that once researchers have decided what
from theories or abstract generalizations they are interested in studying (e.g., what
whereas others start from observations or motivates the study, purpose, personal/
other data points. We believe that all MMR political agenda), the specifics of their
projects go through a full cycle at least once, research questions will determine the choice
regardless of their starting point (e.g., of the best tools to use, which may be
Teddlie & Tashakkori, 2009). QUAL, QUAN, or mixed.
This cyclical approach to research may The seventh characteristic of contempo-
also be conceptualized in terms of the dis- rary MMR is a set of basic “signature”
tinction between the context of justification research designs and analytical processes,
Overview of Contemporary Issues in Mixed Methods–––◆–––11
which are commonly agreed upon, although community are inclined toward generating
they go by different names and diagram- a balance between the excesses exhibited by
matic illustrations. For example, we scholars at either end of the methodological
defined parallel mixed designs (Teddlie & spectrum, while forging a unique MMR
Tashakkori, 2009) as identity. In their survey of Western think-
ing, Johnson and Gray (2010) similarly
a family of MM designs in which mixing depict balance and compromise as one of
occurs in an independent manner either the core principles of MMR, tracing that
simultaneously or with some time characteristic back to several philosophers.
lapse. The QUAL and QUAN strands In a similar vein, Denzin (2008) recapit-
are planned and implemented in order to ulated three of Guba’s (1990) themes regard-
answer related aspects of the same ques- ing paradigms as follows:
tions. (p. 341, italics in original)
• “There needs to be decline in con-
These designs have also been called con- frontationalism by alternative para-
current, simultaneous, and triangulation digm proponents”
designs, but there is much commonality
• “Paths for fruitful dialog between and
across their definitions.
across paradigms need to be explored”
We call these design and analysis processes
“signature” terms because they are unique to • “The three main interpretive commu-
MMR and help set that approach apart from nities . . . must learn how to cooperate
QUAL and QUAN research. Other signature and work with one another.” (p. 322)
design and analysis terms include sequential
mixed designs, conversion mixed designs, We believe that most mixed methods
quantitizing, qualitizing, and inherently researchers are in agreement with these
mixed data analysis. themes, which call for compromise in dia-
While there is general agreement about logues among the three methodological
the existence of these unique MMR design communities.
and analytical processes, there is consider- The ninth characteristic of MMR is
able disagreement about terminology and a reliance on visual representations
definitions, which increase as more complex (e.g., figures, diagrams) and a common nota-
typologies are generated. For example, tional system. MMR designs, data collection
many believe that a complete typology of procedures, and analytical techniques lend
MMR designs is impossible due to the emer- themselves to visual representations, which
gent nature of the QUAL component of the can simplify the complex interrelationships
research and the ability of MMR designs to among elements inherent in those processes
mutate, while others seek agreement on a (e.g., Creswell & Plano Clark, 2007;
basic set of designs for the sake of simplicity Dickinson, 2010 [this volume]; Ivankova,
and pedagogy. This disagreement is another Creswell, & Stick, 2006; Maxwell &
manifestation of the tension between those Loomis, 2003; Niglas, 2010; Onwuegbuzie
who want MMR to become more system- & Combs, 2010 [this volume]; Tashakkori
atic and organized (e.g., Tashakkori, 2009) & Teddlie, 2003c; Teddlie & Tashakkori,
and those who believe we are not ready for 2009). QUAN methodologists sometimes
consensus (e.g., as noted in Leech, 2010). graph experimental designs (e.g., Cook &
The eighth contemporary characteristic Campbell, 1979), but MMR seems particu-
of MMR is a tendency toward balance and larly prone to this form of communication.
compromise that is implicit within the An important characteristic of these dia-
“third methodological community.” MMR grams and figures is their ability to incorpo-
is based on rejecting the either-or of the rate more dimensions as the processes they
incompatibility thesis; therefore, we as a describe evolve.
12–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
Table 1.1 Nine Important Issues or Controversies in Contemporary Mixed Methods Research
BOX 1.1
positions. One of the central ideas links conceptual issues (Circle 1) and issues
in pragmatism is that engagement in of methods (Circle 2) in MMR. In other
philosophical activity should be done in words, the methodology of mixed research
order to address problems, not to build can be characterized as the mediator
systems. (p. 97) between conceptual and methods issues
within the field, or as the point of integra-
Chapter 31 presents further details on tion between the two.10
these alternative conceptual stances draw- Our characterization of the methodol-
ing from various chapters in this volume. ogy of mixed research as the mediator or
point of integration between conceptual
and methods issues highlights the impor-
THE CONCEPTUAL/ tance of delineating the basic principles of
METHODOLOGICAL/ that methodology. What are the method-
METHODS INTERFACE IN MMR ological principles that bind practitioners of
MMR together regardless of differences on
There are many differences among prac- other issues? What are the methodological
titioners of mixed methods, but perhaps the principles of MMR that set us apart as a
most basic one is between those who are community of scholars? At this point in the
conceptually oriented (represented by Circle 1 development of MMR, we believe that at
in Figure 1.1) and those who are methods least two methodological principles set it
oriented (represented by Circle 2). Johnson apart from other approaches, both of which
et al. (2007) and Tashakkori (2006) have were described earlier as general character-
referred to this distinction as that between a istics of MMR.
“top-down” approach, in which research is
driven by the conceptual or philosophical 1. Rejection of the either-or at all levels of
orientation of the researcher, and a “bot- the research process, which leads to method-
tom-up” approach, in which research ques- ological eclecticism (i.e., the researcher as a
tions and methods related to those questions connoisseur of methods). Practitioners of
drive the research process. mixed methods are constantly looking for
While many conceptual and methods other methods to explore a research prob-
issues can be addressed separately, we lem or answer a research question through a
believe that they are linked in a number of synergistic process that Sammons (2010
important ways, which we portray as the [this volume]) refers to as mutual illumina-
overlap or interface between Circles 1 and 2 tion. We believe that MMR in the future
in Figure 1.1. We call this overlap the will feature a more exotic mix of methods as
“conceptual/methodological/methods researchers become more comfortable
interface in MMR” and put it forward as with crossing traditional methodological
an important new issue that has emerged boundaries in answering research questions
explicitly since the publication of the first or furthering our knowledge regarding a
Handbook in 2003. particular research problem. Mixed methods
We defined the methodology of mixed researchers are “shamelessly eclectic” as
research earlier in this chapter as the broad described by Rossman and Wilson (1994),
inquiry logic that guides the selection of and the future of the field should feature
specific methods (represented by Circle 2) increasingly interesting mixtures of methods
and which is informed by conceptual posi- (e.g., mixing geographical information
tions common to mixed methods practi- systems and qualitative software; Fielding &
tioners (represented by Circle 1). We Cisneros-Puebla, 2009). Several authors in
propose that the methodology of mixed this volume describe MMR that inte-
research is the overlap or interface that grates more advanced techniques from
Overview of Contemporary Issues in Mixed Methods–––◆–––17
the QUAL and QUAN approaches, inher- QUAL or QUAN approaches? (see
ently mixed techniques (Teddlie & Chapter 15 by Collins for some
Tashakkori, 2009), and other methods answers to this question)
unique to MMR (e.g., Bazeley; Bergman;
• What are the methodological princi-
Hesse-Biber; and Newman & Ramlo, all in
ples or frameworks for data analysis
this volume).
that distinguish MMR from the tra-
2. Subscription to the iterative, cyclical ditional QUAL or QUAN approaches?
approach to research. Fully integrated (see Chapter 17 by Onwuegbuzie and
MMR mixes top-down deductive and bot- Combs for some answers to this
tom-up inductive processes in the same question)
study, using both confirmatory and • What are the methodological princi-
exploratory research questions in a search ples or frameworks for determining
for relationships between entities, the the quality of inferences that distin-
processes that underlie these relationships, guishes MMR from the traditional
and the context of these occurrences. It QUAL or QUAN approaches? (see
involves as many diverse data collection Chapter 21 by O’Cathain for some
and analysis procedures as the researchers answers to this question)
think appropriate and results in thoroughly
integrated findings and inferences. These We realize that these are difficult ques-
inductively and deductively based findings tions that are confounded by the fact that
and inferences then generate another cycle there are a number of strong voices in the
of research as the phenomenon under study field and that diversity of opinion has
is explored at deeper levels of understand- always been a trademark of MMR.
ing. All truly mixed research studies go Nevertheless, we also believe that our col-
through this full cycle at least once, regard- lective efforts in this Handbook mark the
less of the initial starting point. beginning of the delineation of method-
ological principles for mixed research.
We believe that other methodological
principles of mixed research will emerge as
the field progresses and that a crucial mis- THE RESEARCH QUESTION
sion for the MMR community is to dis- OR RESEARCH PROBLEM IN
cover or generate these principles over the MIXED METHODS RESEARCH
next several years. In putting together the
Handbook, we asked ourselves a series of While the methodological principles
questions about these methodological prin- discussed in the previous section guide the
ciples of, or frameworks for, mixed general conduct of studies employing
research, including the following: MMR, the research question (or research
problem) determines the specific methods
• What are the methodological princi-
(QUAN, QUAL, or MMR) used within
ples or frameworks for research
any given study. The following section
design that distinguishes MMR from
briefly summarizes recent dialogue con-
the traditional QUAL or QUAN
cerning the role of the research question
approaches? (see Chapter 13 by
(or problem) in MMR.
Nastasi, Hitchcock, & Brown for
We initially referred to the “dictatorship
some answers to this question)
of the research question” over a decade
• What are the methodological princi- ago (Tashakkori & Teddlie, 1998) in an
ples or frameworks for sampling that effort to bring the importance of the research
distinguish MMR from the traditional question to the center of the ongoing
18–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
discourse and to move researchers beyond mixed methods have focused on questions
the paradigm debate. Since then, much has about the nature of integration (i.e., how
been written about the importance and the do the findings of the two strands relate to
attributes of MMR questions (Creswell & each other?). Although these questions are
Plano Clark, 2007), the importance of pur- essential, and should be asked during the
pose and political agenda in MMR course of a mixed methods study, we do
(Mertens, 2007; Newman et al., 2003), not consider them research questions.
and the necessity of correspondence between Our rationale for this assertion is that
these elements and the research design, researchers do not conduct research with
data analysis, and inferences (Tashakkori the purpose of finding out if components
& Teddlie, 2008). of a study agree or disagree with, or com-
Currently, there seems to be a pervasive plement, each other (unless the study’s
acknowledgment that a mixed methods main problem is to solve a methodological
project must start with a research question problem by comparing the QUAL and
(or a set of questions) that drives all later QUAN approaches).
stages/components of the project (even A variety of issues remain to be fully
though it might get modified as the explored and discussed in mixed methods
research proceeds). Consequently, the cru- community:
cial question becomes: What shape should
the mixed methods research question take? • the shape/format of the questions
We have always asserted that a mixed (overarching, inquiring about the
methods question is one that clearly calls nature of mixing, and so forth)
for a mixed methods study. In other words,
• general attributes of MMR questions
we have favored an overarching question
(emergent, preplanned, etic, emic,
that potentially requires a structured quan-
exploratory, explanatory, under-
titative approach and an emergent and
standing, etc.)
holistic qualitative type of approach. A
consequence of such a question is that it • components of MMR questions (one
may be broken into subquestions, each overarching question, two separate
requiring a different (QUAL or QUAN) questions, other)
approach to answer.
• functional utility of asking and answer-
Such an umbrella question may lead the
ing MMR questions (i.e., the stated
researcher to any one of the families of
need for using mixed methods), and
mixed designs (parallel, sequential, con-
version, or a combination of these three • consequences of asking and answer-
families, as we discuss later). In some ing MMR questions (e.g., call for
emergent sequential studies, the questions social-political change)
of a later phase develop as a reaction to
the inferences of the previous one. In these We have included a chapter (Plano Clark
designs, the new components are added to & Badiee, Chapter 12) on this issue in this
the initial question, forming an emergent Handbook and will re-examine some of the
umbrella question that incorporates all controversies again in Chapter 31.
aspects of the events or behaviors under
study. This is a necessary augmentation,
making it possible to make integrated, THE LANGUAGE OF MIXED
meta-inferences as answers to these revised METHODS RESEARCH
umbrella questions.
Some discussions of research questions The language of MMR is a broadly
(e.g., Creswell & Plano Clark, 2007) in defined term that we labeled “nomenclature
Overview of Contemporary Issues in Mixed Methods–––◆–––19
and basic definitions” in the first edition variants including, but certainly not lim-
of the Handbook. Language issues in ited to: multimethod research (a histori-
MMR include both the names and defini- cal term not used much now), multiple
tions of the most important concepts in methods, mixed methods, mixed method-
the field. These issues have become pro- ology, mixed research, integrated or inte-
gressively more complex as the number of grative research, blended research, and
terms has increased, and the variations so forth.
(often subtle) of definitions associated Fortunately, there appears to be some
with those terms have multiplied. Language consensus around mixed methods research
is very important in an emergent field as the de facto term due to common usage
such as MMR because the words we use (e.g., the name of this Handbook and of the
to define the field ultimately shape how leading journal in the field). We suspect
we make sense of it (e.g., Creswell, 2010). that this term will endure because it now
We are now at the point of needing greater has the trappings of a brand name, widely
precision in our construction of the lan- disseminated and commonly used through-
guage of MMR. out the social and behavioral sciences.
The following section is divided into As for the definition of MMR, Johnson
two areas: (1) issues in creating a new lan- et al. (2007) presented 19 alternative mean-
guage for MMR and (2) issues in creating ings from leaders in the field, which varied
a common language across methodological considerably in terms of specificity and con-
approaches (QUAN, QUAL, MMR). Taken tent. Their constant comparative analysis of
together, these two subsections address a these definitions resulted in five themes,
basic question: Should we create a new lan- which they then incorporated into a com-
guage for MMR, should we be more inter- posite definition:
ested in creating a common language
across methodological approaches, or Mixed methods research is the type of
should our approach be a combination of research in which a researcher or team
the two? We have seen evidence for both of researchers combines elements of
approaches over the past few years (unique qualitative and quantitative research
MMR language; common language across approaches (e.g., use of qualitative and
the three approaches) which we detail quantitative viewpoints, data collec-
throughout this section. tion, analysis, inference techniques) for
the broad purposes of breadth and
depth of understanding and corrobora-
Issues in Creating a
tion. (Johnson et al., 2007, p. 123)
New Language for MMR
Many practitioners of MMR believe While a reader may disagree with some
we need a language unique to the field, aspects of this definition (e.g., it is too
one that would define and describe those generic or does not include a component of
concepts that differentiate it from QUAL interest to the reader), it is difficult to criti-
or QUAN research. For instance, as cize the process that Johnson and his col-
the field has developed, several authors leagues employed to generate it. This
have labored to identify and define systematic approach for defining terms
exactly what mixed methods research is with multiple meaning in MMR is a valu-
(e.g., Creswell & Plano Clark, 2007; able one, which we discuss again later in
Greene, 2007, 2008; Johnson et al., this chapter.
2007; Tashakkori & Teddlie, 1998, The first step in creating a vocabulary
2003a). There has been continued debate for MMR is to identify the terms to
over what the field should be called, with include in it. It appears that there are at
20–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
least three potential sources for a vocabu- generated that combine elements of the
lary of MMR: QUAN and QUAL research processes.
• Terms that describe particular
• Terms that are in widespread use
research processes indigenous or unique to
throughout the literature, such as the names
MMR, such as fused data analysis
for the signature design and analytical
(Bazeley, 2003) or inherently mixed data
processes (e.g., sequential designs, quanti-
analysis (Teddlie & Tashakkori, 2009).
tizing). Some of these mixed methods
These terms are used to identify MMR
processes have multiple names and defini-
processes that are discovered or generated
tions, thereby requiring procedures such as
by practitioners as they employ mixed
that employed by Johnson et al. (2007) to
methods in their research.
generate composite terms and definitions.
• Blended or amalgamated terms Box 1.2 presents a partial list of unique
describing MMR concepts that are a com- terms related to mixed methods data analy-
bination of QUAL and QUAN terminol- sis that have emerged since the 1990s. The
ogy, such as inference transferability, a emergence of new analytical processes con-
term that subsumes the QUAN terms exter- stitutes one of the most creative areas in
nal validity and generalizability, plus the MMR and often comes from researchers
QUAL term transferability (e.g., working on practical solutions for answer-
Tashakkori & Teddlie, 1998). Such MMR ing their research questions using available
blended terms emerge as typologies are QUAL and QUAN data.
BOX 1.2
The vocabulary of MMR will constantly be proposed and defined, but then dis-
expand as additional blended and indige- carded due to lack of common usage or
nous terms are generated. Some terms will conceptual clarity. The term multimethod,
Overview of Contemporary Issues in Mixed Methods–––◆–––21
for instance, has been largely discarded in consistency in the language of MMR, which
MMR because it connotes a limited type of we as a community of scholars are currently
mixing of methods (i.e., keeping the QUAL constructing. While such precision and con-
and QUAN components largely separated sistency entails hard work, such as that
until the end of the study), which has been expended by Johnson and his colleagues
superseded by approaches that emphasize (2007) in developing their composite defin-
the integration of methods across the entire ition of mixed methods research, we believe
research process. that such work will yield great benefits for
Other terms will survive because they the field. One suggestion11 for accomplish-
find common usage and there is general ing this is the generation of a dictionary of
agreement about what they mean. For MMR terms similar to that developed for
example, the term iterative sequential qualitative inquiry by Schwandt (1997).
mixed analysis has been used (e.g., Teddlie Such a dictionary could go into detail
& Tashakkori, 2009) to describe the analy- regarding the etiology and various meanings
sis of data from a sequential study with associated with MMR terms. Chapter 31
more than two phases (e.g., QUAL→ presents more details on this suggestion and
QUAN→QUAL). Examples of iterative other issues related to the further develop-
sequential mixed analysis are found through- ment of the language of MMR.
out the literature (e.g., Kumagai, Bliss,
Daniels, & Carroll, 2004; Tolman &
Generating a Common Language
Szalacha, 1999) and the concept has been
Across Methodological Approaches
applied specifically to research conducted
over the Internet (Teddlie, Tashakkori, & If there are unique languages for QUAN
Johnson, 2008). The term iterative sequen- research, QUAL research, and MMR, then
tial mixed analysis will most likely become a researchers need to be trilingual to converse
part of the lexicon of MMR, or another across methodological boundaries. Although
more inclusive term will evolve that this trilingualism may be necessary for the
describes the types of analyses associated time being, we believe that a long-term goal
with complex sequential mixed designs. of mixed methods practitioners should be
Glossaries of MMR terms have begun to generate a language that identifies com-
appearing (e.g., Morse & Niehaus, 2010; mon processes across the methodological
Tashakkori & Teddlie, 2003a; Teddlie & approaches. Such a language would encom-
Tashakkori, 2009). The compilation of these pass those processes that are highly similar to
glossaries has revealed a problem that MMR one another across multiple applications.
has faced since its emergence as a separate At this stage in the development of
methodological approach: inconsistency in thought about this language, it is unclear
terminology and definitions (e.g., Bryman, how many common processes there are and
2008). These inconsistencies have included the extent of their similarities. It is clear,
(1) having a number of different definitions however, that many specific methods or
for the same term and (2) having a number techniques are not subsumable (i.e., cannot
of different names for the same concept. For be placed into a broader or more compre-
example, we included a glossary in the first hensive cross-methodological category)
edition of the Handbook with some 150 because they have no equivalent in the
terms, many of which had multiple defini- other languages, or equivalents have not yet
tions (e.g., mixed methods had four different been developed. The search for terms for
meanings) indicating that different authors this common language involves looking for
thought the term was important, yet dis- what Gorard (2010) calls the universal
agreed as to its exact meaning. logic of all research.
As noted in the introduction to this The belief that some limited vocabulary
section, we need greater precision and of common terms is possible stems from the
22–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
rejection of either-or dualisms, which is at knows that cluster analysis employs the same
the heart of MMR. Practitioners of MMR modus operandi as the categorizing process
replace these dualisms with continua that of the constant comparative method: that is,
describe a range of options from one end of maximizing between-group variation and
the methodological spectrum to the other. minimizing within-group variation. Other
Once a set of multidimensional continua examples include: comparing analyses from
has been substituted for the dichotomy, it is one part of a sample with analyses from
possible to look for the commonality that another part of the sample; comparing actual
binds each continuum (dimension, aspect) results with expected results; and contrasting
together. For example, Sandelowski, Voils, components of research design or elements
and Knafl (2009), in discussing the nature to find differences. Recognition of these
of data, concluded that “qualitative and common processes is a step in the direction
quantitative data are not so much different of developing a language that crosses
kinds of data as these data are experiences methodological lines.
formed into, for example, words or
numbers, respectively” (p. 209, italics
added). The commonality that binds the DESIGN ISSUES IN MIXED
dichotomy of QUAL and QUAN data METHODS RESEARCH
together is the “something experienced”
that generated the data in the first place. Design typologies have long been an
We believe that as mixed methods data important feature of MMR, starting with
analysis evolves, “researchers will think of Greene, Caracelli, and Graham (1989)
data less in terms of words or numbers and writing in the field of evaluation and Morse
more in terms of transferable units of infor- (1991) in nursing. The reasons for the
mation that happen to be initially generated importance of MMR design typologies
in one form or the other” (Teddlie & include their role in (1) establishing a com-
Tashakkori, 2009, p. 283). mon language for the field, (2) providing
Practitioners of MMR are in a unique possible blueprints for researchers who
position because their approach to research want to employ MM designs, (3) legitimiz-
allows them to look across diverse method- ing MMR by introducing designs that are
ological applications for the commonalities clearly distinct from those in QUAN or
that bind similar processes together. For QUAL research, and (4) providing useful
example, one of the distinguishing charac- tools for pedagogical purposes (i.e., having
teristics of MMR discussed earlier in this students compare and contrast alternative
chapter is the “iterative, cyclical approach typologies).
to research,” which combines the inductive In the context of these calls for develop-
processes typically associated with QUAL ing mixed methods design typologies or
research and the deductive processes typi- prototypes, a number of frameworks have
cally associated with QUAN research. This been proposed by the community of mixed
cycle of research is a term that could be methods scholars, often with both overlap-
included in a common methodological lan- ping and divergent components and/or
guage because it contains elements associ- different names/labels. For example, we
ated with all three approaches. discussed a signature design type earlier in
We recently (Teddlie & Tashakkori, this chapter, which we called the parallel
2009, p. 282) generated a list of common mixed design (e.g., Teddlie & Tashakkori,
analytical processes used in both QUAL and 2009) and which has had a number of dif-
QUAN research. These processes are cogni- ferent names over time (e.g., concurrent,
tively interchangeable, although one uses simultaneous, triangulation designs). These
numbers and the other employs words as designs have been defined similarly yet have
data. For example, a practitioner of MMR differed on key particulars such as whether
Overview of Contemporary Issues in Mixed Methods–––◆–––23
or not the QUAL and QUAN phases of the complain that there are too many design
study occurred at the same time, or with types, or too many suggestions about how
some time lapse, or both. to plan a mixed methods study. However,
It is apparent that the conceptualization we are confident that over time, useful
of mixed methods designs has undergone and common components of different
substantial changes over the past decade. frameworks will be identified and recon-
For example, our typology of mixed designs ciled by the MMR community, especially
has evolved considerably from the initial by the same group (doctoral students and
version (Tashakkori & Teddlie, 1998) up young scholars) that is currently critical of
through the latest edition (Tashakkori, what members consider to be unnecessary
Brown, & Borghese, 2009; Tashakkori & complexities.
Newman, 2010; Tashakkori & Teddlie, in Perhaps, these differences would be
progress). We discuss particulars of our lat- made more salient if we briefly review three
est framework later in this section. different frameworks for planning and
Recently, some authors have contended implementing mixed methods designs:
that there is an overemphasis on research those of Janice Morse, Jennifer Greene, and
design typologies (e.g., Adamson, 2004; our own. Although other perspectives are
Bazeley, 2009), arguing that other areas equally valuable, we chose these three
(e.g., data analysis) should be stressed because they represent the diversity of ideas
more. Some have suggested a need for a set underlying almost all design frameworks
number of prespecified designs, while and demonstrate many of the ongoing
others contend that MMR design typolo- issues related to MMR designs.
gies can never be exhaustive due to the iter- We discussed Morse’s (1991, 2003,
ative nature of MMR projects (i.e., new 2010) design typology earlier in this chap-
components or strands might be added dur- ter with regard to the common notational
ing the course of a project). This is an system and the complementary strengths
important point; many inexperienced stance. In Morse’s system, the priority of
researchers want a design “menu” from one method over the other is an important
which to select the “correct” one, similar to dimension predetermined before data col-
the menus provided in QUAN research lection starts. Each study has a theoretical
(e.g., Shadish, Cook, & Campbell, 2002). or primary drive (inductive or deductive)
In contrast, researchers using mixed meth- that determines the overall purpose of the
ods are encouraged to continuously re- study, a core component (primary or main
examine the results from one strand of a study), and a supplementary component
study compared to the results from another (which is incomplete by itself and is
and to make changes both in the design and regarded as complementary to the core
data collection procedures accordingly. component). Morse argues that MMR is
Although some find the lack of consensus possible, but that the QUAN and QUAL
regarding the specific number and types of components must be kept as separate as
designs disconcerting, others believe that this possible so that the strengths of each para-
is a healthy sign of the growth of the mixed digmatic position can be realized.
methods community. The ultimate value of In Morse’s system, there is no mixing of
these typologies lies in their ability to provide primary drives. This position is, of course,
researchers with viable design options to quite different from that generally endorsed
choose from and build on (i.e., modify, in the contemporary field of MMR, where
expand, combine) when they are planning or a more thorough mixing of methods is a
implementing their MMR studies. We given. Morse’s (2010) latest version of her
acknowledge the fact that this diversity typology includes the “point of interface”
makes it more difficult to teach and to (where the two components join in either
learn mixed methodology. Students often the data analysis or narrative of the results)
24–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
and contains interesting diagrams of the As noted above, our design typology has
relationships between the core and supple- evolved as MMR has developed over the
mentary components of the research pro- past decade (Tashakkori & Teddlie, 1998,
ject, designated as left and right pathways. 2003c; Teddlie & Tashakkori, 2009). In the
Greene (2007) contends that researchers latest edition of our typology (Tashakkori et
cannot divorce method from “assumptive al., 2009; Tashakkori & Newman, 2010;
frameworks” when designing MMR stud- Tashakkori & Teddlie, in progress), we
ies; therefore, she encourages mixing those have made an effort to simplify it, while also
frameworks in single research studies. Her incorporating as many recent developments
designs are anchored in mixing methods for in the field as possible. We have identified
five basic purposes, which emerged from four families of designs in our typology,
Greene et al. (1989): triangulation, comple- three of which are basic: parallel, sequential,
mentarity, development, initiation, and and conversion. The fourth one, fully inte-
expansion. Caracelli and Greene (1997) dis- grated, is a complex and iterative type that
tinguished between component designs, in potentially includes combinations of the
which the methods are connected or mixed other three. These families are based on
only at the level of inference, and integrated what we call “type of implementation
designs, in which the methods are integrated process”; that is, how does the integration
throughout the course of the study. of the QUAL and QUAN strands actually
Greene (2007) presented two examples occur when conducting a study.
of component designs (convergence, exten- We have subdivided each of the three
sion) and four examples of integrated basic families of designs into three varia-
designs (iteration, blending, nesting or embed- tions based on the data sources: multiple
ding, mixing for reasons of substance or samples, same/subsample, and multilevel
values). These six examples of MMR designs samples/data. In the first variation, QUAL
map onto the five basic purposes for mix- and QUAN data are collected from differ-
ing, with each example aligned with one or ent individuals or are not linked. In the sec-
two of the original purposes. Greene (2007) ond variation, both data types are available
concludes that designing an MMR study for at least some individuals and are linked
does not involve following a formula or in one form or another (this includes the
set of prescriptions, but rather is “an artful conversion of some data to another type).
crafting of the kind of mix that will best ful- In the third, qualitative data are collected at
fill the intended purposes for mixing within one level of a social structure (e.g., parents),
the practical resources and contexts at while quantitative data are collected at
hand” (p. 129). another (e.g., children), and are linked dur-
In our approach to MMR, we have ing analysis and inference.
always treated design as separable from This 3 × 3 combination produces nine
research purpose. That is not to deny the basic design options. The fourth family of
importance of purpose; obviously, if you designs (fully integrated) incorporates multi-
did not have a purpose for doing a study, ple forms of these nine options, often in an
you would not have research questions, and iterative and emergent manner. Increasingly,
you would probably not be conducting MMR studies appear to be using this last
research at all. We think purpose is a com- design family by combining the basic config-
plex, psycho-socio-political concept that urations, often with multiple types/sources
motivates any given research project, and of data.
we believe each individual has a multiplic- We conclude this section by re-iterating
ity of purposes for doing research, ranging a few characteristics of the three typologies
from advancing his or her career to under- we have discussed. All three reflect coherent
standing complex phenomena, to improv- and internally consistent perspectives,
ing society. which remain viable as they have evolved
Overview of Contemporary Issues in Mixed Methods–––◆–––25
framework, or are the criteria that practi- Aside from the research methodology
tioners of MMR have used to create their literature, in cognitive psychology, the term
mixed analysis typologies too divergent for inference has been used in discussions of
a single framework? As Greene (2008) inductive and deductive reasoning that
asked, is “integrated analysis . . . a mixed results in causal and noncausal conclusions
methods methodological area in which in everyday life (i.e., by “everyday pragma-
practice may always take the lead?” (p. 15). tists,” as labeled by Biesta, 2010). For
example, Sternberg (2009) suggests that
3. If an inclusive framework for mixed “one approach to studying inductive rea-
methods data analysis is possible, what soning is to examine causal inferences—
shape will it take? Onwuegbuzie and how people make judgments about whether
Combs (2010) have furthered the discus- something causes something else” (p. 515,
sion by proposing a “meta-framework of bold in original). He also discusses inference
mixed analysis strategies,” which we dis- as a complex process of making conclusions
cuss along with other analysis issues in about relationships (causal or otherwise) in
Chapter 31. everyday life: “The great puzzle of inductive
reasoning is how we manage to infer useful
general principles based on the huge number
ISSUES IN DRAWING INFERENCES of observations of covariation to which we
IN MIXED METHODS RESEARCH are constantly exposed” (Sternberg, 2009,
p. 515). Smith and Kosslyn (2007) present a
Scholars in both the QUAL and QUAN slightly different view of inference which
traditions have used the term inference to links it to “category knowledge” in reason-
denote the process of making sense of the ing and cognition:
results, or the outcomes, of the research
process (i.e., conclusions, constructions, Indeed, the whole point of categorizing
etc.). We initially used the term in an is to allow you to draw inferences,
attempt to differentiate three distinct com- namely, to allow you to derive informa-
ponents of research projects (Tashakkori & tion not explicitly present in a single
Teddlie, 1998): data (as an input to the member of a category but available
process of meaning making in research), because of knowledge of the characteris-
data analysis (as the process of applying a tics of the group or groups to which it
set of tools to summarize the data and link belongs. Once you categorize a perceived
its components), and inference (as the out- entity, many useful inferences can fol-
come of the process of meaning making). low. (p. 149, italics in the original)
These distinctions emerged from the need
to differentiate between standards/audits Our definition of inference has roots in
for assessing quality in research: We called cognitive psychology, philosophy, and
for distinguishing (1) data quality from research methodology. We have defined it
(2) data analysis quality/adequacy from as “a researcher’s construction of the rela-
(3) the quality of conclusions that are made tionships among people, events, and vari-
on the basis of the findings or results. ables as well as his or her construction of
(In Chapter 31, we refer to this as a systems respondents’ perceptions, behaviors, and
approach to assessing the quality of feelings and how these relate to each other
research projects). Although some scholars in a coherent and systematic manner”
still confuse data with results/findings or (Tashakkori & Teddlie, 2003c, p. 692).
with the final outcome of research, there is Although inferences are the most impor-
growing awareness that inferences are tant aspects or outcomes of any study, little
clearly separate from the other two and has been written about their characteristics,
must be explicitly evaluated for quality. the process of making them, and possible
28–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
standards for assessing their quality. An Perhaps the most fundamental step in
interesting and complex question to answer making inferences is to examine each part of
in MMR is: How do we make inferences on a set of data analysis outcomes (results) sep-
the basis of the results of QUAL and QUAN arately and then evaluate how effectively it
analyses of our data? This question is closely answers a research question/purpose set
related to one that has been asked about the forth earlier. These results might be themes
naïve analysis of events and behaviors. obtained from content analysis, numerical
Discussing the process of inference in every- summaries of observed/measured variables,
day human problem solving, Sternberg or complex outcomes of inferential statis-
(2009) asks, “On what basis do people tics. In each case, one might ask: What does
draw inferences? People generally use both this mean? What does this tell me about the
bottom-up strategies and top-down strate- behavior or event under investigation? How
gies for doing so” (p. 519). Bottom-up does this answer my research (specific) ques-
strategies are “based on observing various tion? In MMR, these initial queries are
instances and considering the degree of made from the results of both QUAL and
variability across instances” (p. 519). QUAN data analyses, which are compared
Top-down cognitive strategies, on the other and contrasted on an ongoing basis, then
hand, include “selectively searching for con- integrated to create a more general answer
stancies within many variations, and selec- to each specific research question. After
tively combining concepts and categories” going through this first stage of making
(p. 519). We believe that the process of inferences, one needs to compare and con-
making inferences in research follows a sim- trast the answers to different questions
ilar model, but it is more formal and sys- (actually, aspects of the same overarching
tematic. We will expand this idea in mixed methods question) and to assess con-
Chapter 31, when we refer to mixed meth- ceptual variations and similarities between
ods as a humanistic methodology. them. This is the stage in which the more
How do we make inferences in MMR? abstract/global explanations are found for
We have made an effort to identify possible the events and behaviors.
steps in generating inferences in MMR (see How do we know that our inferences are
Teddlie & Tashakkori, 2009, pp. 289–293). credible or believable, and not merely a
A major part of that process includes keep- function of our imaginations? This question
ing one’s research questions in the fore- has received more attention in the literature
ground because at the most basic level, than the question regarding how to make
inferences are answers to research ques- inferences in MMR. At least three broad
tions. At the most abstract level, inferences types of answers have been offered so
are mini-theories and explanations for far in the literature (Dellinger & Leech,
explaining events and behaviors. From this 2007; Onwuegbuzie & Johnson, 2006;
point of view, inferences fall on a contin- Tashakkori & Teddlie, 2003c). We have
uum from the more specific to the more used social cognition as a model by focus-
general; that is, they include conclusions ing on the similarities between the
that range from the meaning of a specific researcher and the naïve analyst of behav-
event, behavior, or relationship to global iors and events in everyday life (the “every-
explanations of why events, behaviors, or day pragmatist”). In this model, quality of
relationships occur. Obviously, the former inferences is assessed simultaneously by
is more concrete, and the latter is more examining (a) the process of reaching the
abstract. By virtue of being concrete, the results that they are based on (i.e., design
former is more specific to the context in quality, Tashakkori & Teddlie, 2003c) and
which the behaviors or events were (b) the attributes of the conclusions them-
observed, whereas the latter is much less sit- selves (i.e., interpretive rigor). The degree of
uation specific. confidence that one has in a conclusion is
Overview of Contemporary Issues in Mixed Methods–––◆–––29
properly within the QUAN or QUAL tradi- complexity of teaching the numerous design
tions alone. Leech’s (2010) description of typologies that were discussed earlier in this
how the early developers of MMR began to chapter (e.g., Earley, 2007, reported that
combine QUAL and QUAN components in students in his classes counted a total of
their work describes how this sometimes 52 different design possibilities). Several of
happened: Researchers were often trained these pedagogical issues are discussed in this
in traditions that emphasized numerical volume by Christ (Chapter 25), including a
data collection and statistical analysis, detailed description of how he used action
picked up some skills in narrative data col- research to improve his introductory and
lection and thematic analysis as their advanced mixed methods courses.
careers developed (due to their interest in Nevertheless, pedagogy tells only part of
those topics), and then found themselves the story regarding how a researcher
applying all that they knew about research becomes a methodological connoisseur. In
methods in studies of complex social phe- the previous Handbook, we presented three
nomena. In the preface to this volume, we models for what we called professional
also shared with you our own experiences competency and collaboration:
and struggles in this process of learning
MMR through a “bottom-up approach” to • A single researcher develops dual
research. This process of intuitively using a competencies in both QUAL and QUAN
variety of methods and techniques and methods to the point that he or she can
drawing conclusions based on syntheses of conduct “solo” mixed methods investiga-
the various types of evidence available is tions. This dual competency is the ultimate
also described by Gorard (2010). goal for the connoisseur of methods we
In the first Handbook, we described the have been discussing, but critics are skep-
lack of formal training in mixed methods as tical that this is a realistic goal for most
“the failure of pedagogy” and briefly researchers, who do not have the training
described the handful of textbooks that or field experiences to be competent in
covered mixed methods at that time and both QUAL and QUAN methods. We will
the even smaller number of articles that discuss this in more detail in Chapter 31.
addressed pedagogical issues (e.g., Creswell,
• The second model solved the problem
Tashakkori, Jensen, & Shapley, 2003;
of dual competency by proposing a collab-
Tashakkori & Teddlie, 2003b). As detailed
orative team approach to mixed research
throughout this Handbook, there has been
consisting of members with competency in
an explosion in the number of texts devoted
one of the two traditions (i.e., collaborative
to mixed research since that time, and a
teams consisting of one or more qualita-
corresponding upsurge in the number of
tively oriented researchers and one or more
universities offering formal courses in
quantitatively oriented researchers). Such
mixed research as chronicled by Christ
collaborative efforts are not uncommon in
(2009, 2010 [this volume]), Earley (2007),
large-scale studies in the health sciences or
and Niglas (2007).
in studies conducted in complex educa-
Recent articles on pedagogical practice
tional or evaluation settings.
have been quite valuable, such as Earley’s
(2007) account of the 12-step process he • The third model calls for each team
used to develop a syllabus for his MMR member in a mixed study to have a mini-
course and Christ’s (2009) description of the mum level of competency in QUAL and
generation for his students of a research pro- QUAN methods, plus expertise in one or the
posal process with eight interactive features. other (e.g., Shulha & Wilson, 2003; Teddlie
Nevertheless, the first generation of instruc- & Tashakkori, 2003). A problem with
tors of mixed methods courses must still the second approach (teams consisting of
face some problematic areas, including the qualitatively and quantitatively oriented
Overview of Contemporary Issues in Mixed Methods–––◆–––31
researchers) is that without minimum com- skills to research issues from a methodolog-
petency in both types of research, team ical perspective at least slightly different
members may not be able to communicate from their own. For instance, the Jang,
effectively because they lack a “common” McDougall, Pollon, Herbert, and Russell
methodological language (discussed earlier (2008) study of “schools in challenging cir-
in this chapter). We concluded that the third cumstances” quoted one of the graduate
model (minimum competency model) is students involved in the study as follows:
probably prerequisite for the second one (the
team approach) to actually work in practice. My participation in a mixed methods proj-
ect expanded my horizons from research
Lieber and Weisner (2010 [this volume]) methodology as a debate between para-
discuss the value of collaborative teams digms that dealt with “people versus
consisting of colleagues with different numbers” and from an understanding that
training and experiential backgrounds in abstract debates between “either/or” actu-
terms of generating a “respectful environ- ally, and quite compellingly, dialectically
ment” in which team members can struggle resolve into an “and.” (p. 243)
to design and carry out the best mixed
research possible given the context of the This qualitatively oriented graduate
study. They also describe the CHILD pro- researcher had originally been concerned
ject, a longitudinal family and child devel- about how she could contribute to the
opmental study, conducted by a team QUAN part of the study. She commented
consisting of members from the fields of that her “rich” understanding of the QUAL
education, anthropology, psychology, sta- data led her to seek a better understanding
tistics, family studies, and so forth. of the statistical analyses and graphic dis-
Similarly one of the co-editors of this plays, which she discovered to be “full of
volume (Teddlie) participated in a longitu- life.” This novice researcher appears to be
dinal educational effectiveness project in the beginning stages of becoming a
(Louisiana School Effectiveness Study) with methodological connoisseur.
a core team of 11 investigators from educa- Other practical issues presented in this edi-
tion, psychology, statistics, nursing, and tion of the Handbook include funding and
research methods. Five of the team writing mixed methods, both of which are
members were self-identified as mixed discussed by Dahlberg, Wittink, and Gallo
methods practitioners, while three main- (Chapter 30). The Dahlberg et al. approach
tained a primarily QUAN orientation, and to both topics stresses practical considera-
three were primarily QUAL in orientation. tions: they see their mission as providing “the
These varieties of disciplinary/training reader with tangible strategies at the point
backgrounds and research orientations led where the epistemological rubber meets the
to lively group interchanges in which indi- road—to publication and grant funding”
vidual schools were discussed. These dis- (p. 777, this volume). Creswell (2010) pro-
cussions were tape-recorded and were a vides further information on funding oppor-
primary source for six extensive mixed tunities for MMR. These and other practical
methods case studies, which appeared in issues are discussed further in Chapter 31.
Teddlie and Stringfield (1993).
Experiences on such mixed methods
research teams can do much to create and CROSS-DISCIPLINARY AND CROSS-
enhance methodological connoisseurship. CULTURAL APPLICATIONS OF
Researchers become more competent in MIXED METHODS RESEARCH
various research methodologies as they
work collaboratively on projects where Cross-disciplinary and cross-cultural appli-
they see others applying problem-solving cations of MMR were not included as a
32–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
major issue in the first edition of the published in 70 specific fields within
Handbook, but the recent diffusion of broader disciplines, indicating the utility of
mixed research throughout the human sci- MMR across a wide spectrum of academic
ences and across academic communities specialty areas. Third, when looking at
around the world is a topic of growing inter- national origin of the first author of the arti-
est in the field. Much of the dynamic energy cles, researchers from more than 30 coun-
within MMR comes from this expansion tries contributed to the database, with over
into other disciplines and cultures. There are half of those from the United States, another
several interesting trends in this cross- 20% from the United Kingdom, and a sig-
disciplinary and cross-cultural dispersion, nificant number of the remainder from
which we briefly introduce in this section, Canada and Australia (compared to all the
including the wide variance in adoption other countries).
rates of MMR that is apparent within aca- The prevalence rates study conducted by
demic discipline and specialty areas. Alise and Teddlie (in press) compared the
MMR has been rapidly expanding into proportion of articles employing QUAL,
all disciplines in the social and behavioral QUAN, or mixed methods within “elite”
sciences over the past decade, as indicated journals in four disciplines. Education and
by several studies of incidence rates (counts nursing were selected to represent applied
of the absolute number of MMR articles disciplines, while sociology and psychology
published per year) and prevalence rates were chosen to represent “pure” or basic
(the proportion of research studies pub- disciplines using the Biglan (1973) classifi-
lished in a given field that are mixed in cation system. The prevalence rates for
nature). Although several incidence and mixed methods studies was considerably
prevalence rates studies have been pub- higher (16%) in the applied disciplines
lished (e.g., Hart, Smith, Swars, & Smith, compared to the pure or basic disciplines
2009), we briefly review information from (6%). The higher prevalence rates for
two recent analyses (Alise & Teddlie, in MMR in applied fields were expected
press; Ivankova & Kawamura, 2010 [this because MMR originated in areas such as
volume]) as evidence of trends in the cross- nursing, education, and evaluation. The
disciplinary adoption of MMR. prevalence rate for QUAN studies in elite
Ivankova and Kawamura’s Chapter 23 journals in psychology was 93%, with the
documents three interesting trends in the other 7% classified as mixed.
incidence rates of empirical mixed research Incidence and prevalence rates studies
published in several major databases from are crucial at this time for practitioners of
2000 to 2008. First, there was a dramatic mixed methods because they describe how
increase in the number of articles that were MMR techniques are spreading across a
identified as “mixed methods” from only 10 variety of disciplines and how they are
in 2000 to 243 in 2008. This sharp increase evolving as they expand into areas where
was especially noticeable after 2003, when other methodologies have previously domi-
the first edition of the Handbook was pub- nated. A number of interesting questions
lished and the term mixed methods became emerge from information that has accumu-
more widely used. Second, there was a wide lated thus far. What can be done to encour-
variance in the use of mixed methods across age greater use of mixed methods in applied
disciplines, with the health and medical areas where they already used? What
fields accounting for 47% of the total remaining barriers exist to their greater
number of mixed articles published, educa- use? How can mixed methods be intro-
tion accounting for 21%, and the rest of the duced into applied research fields where the
fields accounting for the remaining 32%. QUAN or QUAL tradition is still domi-
Altogether mixed research studies were nant? Chapters 27 and 28, by Sammons
Overview of Contemporary Issues in Mixed Methods–––◆–––33
and by Song and her colleagues, respec- increasingly lively geographic and national
tively, address the last question by dis- diversity. Much writing, research reports,
cussing how mixed methods have been and lively scholarly debates have emerged
successfully introduced into fields of study from the United States, Europe, Canada,
that have been dominated by the traditional Australia, and to some extent, New Zealand
QUAN approach. and Japan. Although scholars from other
How can mixed methods be introduced parts of the world are publishing mixed
into “pure” or basic disciplines such as psy- methods research articles and methodolog-
chology, which has long been dominated by ical papers, the number and scope of these
the QUAN tradition (especially experimental/ writings is still small. We see indications of
quasi-experimental methods)? A promising accelerating growth in trans-cultural mixed
sign for the use of MMR in psychology was methods studies.
the recent publication of an article in One of the advantages of mixed methods
Developmental Psychology on mixing has been its flexibility to use cultural
QUAL and QUAN research (Yoshikawa, knowledge and systematic/anecdotal field
Weisner, Kalil, & Way 2008). Yoshikawa observations as research data/evidence in
and colleagues described research settings different types of research. Use of QUAL
in development science, where mixed meth- observations and cultural/linguistic knowl-
ods might be especially appropriate, includ- edge in interpreting QUAN research
ing studies that explore causal associations and measurement results is not new in
and their mechanisms (for an excellent ear- cultural/cognitive anthropology, cross-
lier review of these applications, see cultural psychology, and related disci-
Waszak & Sines, 2003). plines (for example, see Hambleton,
It is obvious that researchers working Merenda, & Spielberger, 2005; Waszak &
within specific disciplines and fields will Sines, 2003). However, there is a need for a
shape MMR to fit the context within which systematic set of procedures that help in
they work. Ivankova and Kawamura summarizing and presenting both the
(2010) provide insightful descriptions of QUAL and the QUAN results (e.g., QUAL
how researchers in the fields of health and observations and field notes and QUAN
medicine, education, computer science, and questionnaires and structured data). Mixed
social work have applied MMR within methods provide such an impetus while
their fields. As MMR disperses throughout also legitimizing the integration of QUAL
the human sciences, one challenge will be to and QUAN methods, data, and results.
ascertain if practitioners of mixed methods Currently, the developing world is not
can develop and maintain a “core identity” highly visible in publications regarding or
(e.g., a set of commonly understood involving mixed methods. This, however, is
methodological principles) that cuts across not an indication of lack of feasibility or use
disciplinary lines. of mixed methods in these countries. There
While researchers from a few countries are many indications that researchers are tak-
have dominated the academic discourse, ing a bottom-up path to mixed methods in
there is evidence that MMR is attracting many areas of the world by creatively
scholars from a wide variety of national integrating QUAL and QUAN methods/
and cultural backgrounds. For example, approaches (also see our preface to this
the literature review by Ivankova and volume). An examination of cross-cultural
Kawamura (2010) indicated that scholars research books (e.g., Smith, Bond, &
from more than 30 countries generated arti- Cagitcibasi, 2006) provides ample examples
cles employing mixed methods between of integrating cultural knowledge, field notes,
2000 and 2008. In the past decade, the and qualitative observations/interviews in
mixed methods community has enjoyed an interpretation of survey results (or vice versa).
34–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
1. Consider the three general sections of the Handbook. How are topics within those
sections different from and similar to one another? Discuss points of overlap among them.
2. Discuss the importance of developing a “map” of the field of MMR, including specific lines
of inquiry. (You may want to reconsider this question after reading Chapter 2 by John Creswell.)
3. Which of the nine common characteristics presented is the most important in terms of
setting MMR apart from the two traditional approaches to research? Why?
4. Which of the nine issues or controversies currently being debated in MMR do you con-
sider the most important? Why?
5. What is meant by the terms methodological eclecticism and connoisseur of methods
(or methodological connoisseur)?
6. What are two principles of mixed methodology? Describe how they set practitioners of
mixed methods apart from researchers who use QUAL or QUAN methods exclusively.
7. What are some of the issues in developing a language for MMR?
8. Select two of the following topics and write a short essay comparing their importance for
the future of MMR: design issues, analysis issues, issues in drawing inferences.
9. Select two of the following topics and write a brief essay comparing their importance
for the future of MMR: pedagogy, collaborative teams, cross-disciplinary applications, and
cross-cultural applications.
Overview of Contemporary Issues in Mixed Methods–––◆–––35
seems to lean in the direction of QUAN Biglan, A. (1973). The characteristics of subject
research, rather than a language for MMR. We matter in different academic areas. Journal
caution our readers that the way we use the term of Applied Psychology 57(3), 195–203.
inference is not the same as statistical inference, Brannen, J (2005). Mixed methods: The entry of
which is used in a very specific context within qualitative and quantitative approaches
QUAN data analysis. As noted in the text, our into the research process. International
definition of inference is much broader and is Journal of Social Research Methodology,
based on an extensive literature with origins in 8(3), 173–184.
cognitive psychology (social cognition), philoso- Brewer, J., & Hunter, A. (2006). Foundations of
phy, and research methodology, including multimethod research: Synthesizing styles
QUAL research traditions. (2nd ed.). Thousand Oaks, CA: Sage.
Bryman, A. (2006). Paradigm peace and the
implications for quality. International
♦ References Journal of Social Research Methodology
Theory and Practice, 9(2), 111–126.
Bryman, A. (2008). Why do researchers com-
Adamson, J. (2004). [Review of the book bine quantitative and qualitative research?
Handbook of mixed methods in social and In M. M. Bergman (Ed.), Advances in
behavioral research]. International Journal mixed methods research: Theories and
of Epidemiology, 33(6), 1414–1415. applications (pp. 87–100). London: Sage.
Alise, M. A., & Teddlie, C. (in press). A contin- Caracelli, V. J., & Greene, J. C. (1993). Data
uation of the paradigm wars? Prevalence analysis strategies for mixed-method evalu-
rates of methodological approaches across ation designs. Educational Evaluation and
the social/behavioral sciences. Manuscript Policy Analysis, 15(2), 195–207.
accepted for publication in the Journal of Caracelli, V. J., & Greene, J. C. (1997).
Mixed Methods Research. Crafting mixed-method evaluation designs.
Bazeley, P. (2003). Computerized data analy- In J. C. Greene & V. J. Caracelli (Eds.),
sis for mixed methods research. In Advances in mixed-method evaluation: The
A. Tashakkori & C. Teddlie (Eds.), challenges and benefits of integrating diverse
Handbook of mixed methods in social paradigms (New Directions for Evaluation,
and behavioral research (pp. 385–422). No. 74, pp. 19–32). San Francisco:
Thousand Oaks, CA: Sage. Jossey-Bass.
Bazeley, P. (2009). Integrating data analyses in Christ, T. W. (2009). Designing, teaching, and
mixed methods research. Journal of Mixed evaluating two complementary mixed
Methods Research, 3(3), 203–207. methods research courses. Journal of Mixed
Bazeley, P. (2010). Computer assisted integra- Methods Research, 3(4), 292–325.
tion of mixed methods data sources and Christ, T. (2010). Teaching mixed methods and
analysis. In A. Tashakkori & C. Teddlie action research: Pedagogical, practical, and
(Eds.), SAGE handbook of mixed methods evaluative considerations. In A. Tashakkori
in social & behavioral research (2nd ed.). & C. Teddlie (Eds.), SAGE handbook of
Thousand Oaks, CA: Sage. mixed methods in social & behavioral
Bergman, M. M. (Ed.). (2008). Advances in research (2nd ed.). Thousand Oaks,
mixed methods research: Theories and CA: Sage.
applications. London: Sage. Cook, T. D., & Campbell, D. T. (1979).
Biesta, G. (2010). Pragmatism and the philo- Quasiexperimentation: Design and analysis
sophical foundations of mixed methods issues for field settings. Boston: Houghton
research. In A. Tashakkori & C. Teddlie Mifflin.
(Eds.), SAGE handbook of mixed methods Creswell, J. W. (2009). Mapping the field of
in social & behavioral research (2nd ed.). mixed methods research. Journal of Mixed
Thousand Oaks, CA: Sage. Methods Reseach 3(2), 95–108.
Overview of Contemporary Issues in Mixed Methods–––◆–––37
Lincoln, Y. S. (2010). “What a long strange trip control, rigor, and complexity. In
it’s been . . . ”: Twenty-five years of qualita- A. Tashakkori & C. Teddlie (Eds.), SAGE
tive and new paradigm research. Qualitative handbook of mixed methods in social &
Inquiry, 16(1), 3–9. behavioral research (2nd ed.). Thousand
Lincoln, Y. S., & Guba, E. G. (1985). Oaks, CA: Sage.
Naturalistic inquiry. Beverly Hills: Sage. Morse, J., & Niehaus, L. (2010). Mixed method
Lincoln, Y. S., & Guba, E. G. (2000). design: Principles and procedures. Walnut
Paradigmatic controversies, contradictions, Creek, CA: Left Coast Press.
and emerging confluences. In N. K. Denzin Newman, I., Ridenour, C., Newman, C., &
& Y. S. Lincoln (Eds.), Handbook of qual- DeMarco, Jr., G. M. P. (2003). A typology
itative research (2nd ed., pp. 163–188). of research purposes and its relationship to
Thousand Oaks, CA: Sage. mixed methods research. In A. Tashakkori
Maxwell, J., & Loomis, D. (2003). Mixed meth- & C. Teddlie (Eds.), Handbook of mixed
ods design: An alternative approach. In methods in social and behavioral research
A. Tashakkori & C. Teddlie (Eds.), (pp. 167–188). Thousand Oaks, CA: Sage.
Handbook of mixed methods in social Niglas, K. (2004). The combined use of qualita-
and behavioral research (pp. 241–272). tive and quantitative methods in educa-
Thousand Oaks, CA: Sage. tional research. Tallinn, Estonia: Tallinn
Maxwell, J. A., & Mittapalli, K. (2010). Pedagogical University Series, Dissertations
Realism as a stance for mixed method on Social Sciences.
research. In A. Tashakkori & C. Teddlie Niglas, K. (2007). Introducing the qualitative-
(Eds.), SAGE handbook of mixed methods quantitative continuum: An alternative view
in social & behavioral research (2nd ed.). of teaching research methods courses. In
Thousand Oaks, CA: Sage. M. Murtonen, J. Rautopuro, & P. Vaisanen
Mertens, D. M. (2007). Transformative paradigm: (Eds.), Learning and teaching of research meth-
Mixed methods and social justice. Journal of ods at university (pp. 185–203). Turku, Finland:
Mixed Methods Research, 1(3), 212–225. Finnish Educational Research Association.
Mertens, D. M., Bledsoe, K. L., Sullivan, M., & Niglas, K. (2010). The multidimensional model
Wilson, A. (2010). Utilization of mixed of research methodology: An integrated set
methods for transformative purposes. In of continua. In A. Tashakkori & C. Teddlie
A. Tashakkori & C. Teddlie (Eds.), SAGE (Eds.), SAGE handbook of mixed methods
handbook of mixed methods in social & in social & behavioral research (2nd ed.).
behavioral research (2nd ed.). Thousand Thousand Oaks, CA: Sage.
Oaks, CA: Sage. Onwuegbuzie, A., & Combs, J. (2010). Emergent
Morgan, D. (2007). Paradigms lost and pragma- data analysis techniques in mixed methods
tism regained: Methodological implications research: A synthesis. In A. Tashakkori &
of combining qualitative and quantitative C. Teddlie (Eds.), SAGE handbook of mixed
methods. Journal of Mixed Methods methods in social & behavioral research
Research, 1(1), 48–76. (2nd ed.). Thousand Oaks, CA: Sage.
Morse, J. M. (1991). Approaches to qualitative- Onwuegbuzie, A. J., & Johnson, R. B. (2006).
quantitative methodological triangulation. The validity issue in mixed research.
Nursing Research, 40(2), 120–123. Research in the Schools, 13(1), 48–63.
Morse, J. M. (2003). Principles of mixed meth- Onwuegbuzie, A. J., & Leech, N. L. (2005). Taking
ods and multimethod research design. In the “Q” out of research: Teaching research
A. Tashakkori & C. Teddlie (Eds.), methodology courses without the divide
Handbook of mixed methods in social between quantitative and qualitative para-
and behavioral research (pp. 189–208). digms. Quality & Quantity: International
Thousand Oaks, CA: Sage. Journal of Methodology, 39, 267–296.
Morse, J. M. (2010). Procedures and practice Onwuegbuzie, A. J., & Teddlie, C. (2003).
of mixed method design: Maintaining A framework for analyzing data in mixed
40–––◆–––Handbook of Mixed Methods in Social & Behavioral Research
methods research. In A. Tashakkori & Shadish, W., Cook, T., & Campbell, D. (2002).
C. Teddlie (Eds.), Handbook of mixed Experimental and quasi-experimental
methods in social and behavioral research designs for general causal inference. Boston:
(pp. 351–384). Thousand Oaks, CA: Sage. Houghton Mifflin.
Patton, M. Q. (1990). Qualitative research and Shulha, L., & Wilson, R. (2003). Collaborative
evaluation methods (2nd ed.). Thousand mixed methods research. In A.Tashakkori
Oaks, CA: Sage. and C. Teddlie (Eds.), Handbook of mixed
Patton, M. Q. (2002). Qualitative research and methods in social and behavioral research
evaluation methods (3rd ed.). Thousand (pp. 639–670). Thousand Oaks, CA: Sage.
Oaks, CA: Sage. Smith, E. E., & Kosslyn, S. M. (2007). Cognitive
Peirce, C. S. (1974). Collected papers psychology: Mind and brain. Upper Saddle
(C. Hartshore, P. Weiss, & A. Burks, River, NJ: Pearson.
Eds.). Cambridge, MA: The Belknap Smith, P. B., Bond, M. H., & Cagitcibasi, C.
Press of Harvard University Press. (2006). Understanding social psychology
Plano Clark, V. L., & Creswell, J. W. (2008). across cultures: Living and working in a
The mixed methods reader. Thousand changing world. London: Sage.
Oaks, CA: Sage. Sternberg, R. J. (2009). Cognitive psychology.
Ridenour, C. S., & Newman, I. (2008). Mixed Belmont, CA: Wadsworth.
methods research: Exploring the interactive Tashakkori, A. (2006, July). Growing pains?
continuum. Carbondale: Southern Illinois Agreements, disagreements, and new direc-
University Press. tions in conceptualizing mixed methods.
Rossman, G., & Wilson, B. (1985). Numbers Keynote address presented at 2nd annual
and words: Combining quantitative and Mixed Methods Conference, Humerton
qualitative methods in a single large scale School of Health Sciences, Cambridge, UK.
evaluation study. Evaluation Review, 9, Tashakkori, A. (2009). Are we there yet? The state
627–643. of the mixed methods community. Journal of
Rossman, G., & Wilson, B. (1994). Numbers and Mixed Methods Research, 3(4), 287–291.
words revisited: Being “shamelessly eclec- Tashakkori, A., Brown, L. M., & Borghese, P.
tic.” Quality and Quantity, 28, 315–327. (2009). Integrated methods for studying a
Sale, J., Lohfeld, L., & Brazil, K. (2002). systemic conceptualization of stress and
Revisiting the qualitative-quantitative coping. In K. Collins, A. J. Onwuegbuzie,
debate: Implications for mixed-methods & Q. G. Jiao (Eds.), Toward a Broader
research. Quality and Quantity, 36, 43–53. understanding of stress and coping: Mixed
Sammons, P. (2010). The contribution of methods approaches (Research on Stress
mixed methods to recent research on edu- and Coping in Education Series, Volume
cational effectiveness. In A. Tashakkori & V). New Age Publishing.
C. Teddlie (Eds.), SAGE handbook of mixed Tashakkori, A., & Creswell, J. (2008).
methods in social & behavioral research Envisioning the future stewards of the social-
(2nd ed.). Thousand Oaks, CA: Sage. behavioral research enterprise. Journal of
Sandelowski, M. (2000). Combining qualitative Mixed Methods Research, 2(4), 291–295.
and quantitative sampling, data collection, Tashakkori, A., & Newman, I. (2010). Mixed
and analysis techniques in mixed-method methods: Integrating quantitative and quali-
studies. Research in Nursing Health, 23, tative approaches to research. In B. McGaw,
246–255. E. Baker, & P. P. Peterson (Eds.),
Sandelowski, M., Voils, C. I., & Knafl, G. International encyclopedia of education (3rd
(2009). On quantitizing. Journal of Mixed ed.). Oxford, UK: Elsevier.
Methods Research, 3(3), 208–222. Tashakkori, A., & Teddlie, C. (1998). Mixed
Schwandt, T. (1997). Qualitative inquiry: A methodology: Combining the qualitative
dictionary of terms. Thousand Oaks, CA: and quantitative approaches. Thousand
Sage. Oaks, CA: Sage.
Overview of Contemporary Issues in Mixed Methods–––◆–––41
Tashakkori, A., & Teddlie, C. (Eds.), (2003a). 10-year study of school effects. New York:
Handbook of mixed methods in social Teachers College Press.
and behavioral research. Thousand Oaks, Teddlie, C., & Tashakkori, A. (2003). Major
CA: Sage. issues and controversies in the use of mixed
Tashakkori, A., & Teddlie, C. (2003b). Issues methods in the social and behavioral sci-
and dilemmas in teaching research methods ences. In A. Tashakkori & C. Teddlie
courses in social and behavioral sciences: (Eds.), Handbook of mixed methods in
U.S. perspective. International Journal of social and behavioral research (pp. 3–50).
Social Research Methodology, 6, 61–77. Thousand Oaks, CA: Sage.
Tashakkori, A., & Teddlie, C. (2003c). The past Teddlie, C., & Tashakkori, A. (2009). The foun-
and future of mixed methods research: From dations of mixed methods research:
data triangulation to mixed model designs. Integrating quantitative and qualitative
In A. Tashakkori & C. Teddlie (Eds.), techniques in the social and behavioral
Handbook of mixed methods in social Sciences. Thousand Oaks, CA: Sage.
and behavioral research (pp. 671–702). Teddlie, C., & Tashakkori, A. (in press). Mixed
Thousand Oaks, CA: Sage. methods: Contemporary issues in an emerg-
Tashakkori, A., & Teddlie, C. (2008). Quality ing field. In N. K. Denzin & Y. S. Lincoln
of inference in mixed methods research: (Eds.), Handbook of qualitative research
Calling for an integrative framework. In M. (4th ed.). Thousand Oaks, CA: Sage.
M. Bergman (Ed.), Advances in mixed Teddlie, C., Tashakkori, A., & Johnson, B.
methods research: Theories and applica- (2008). Emergent techniques in the gathering
tions (pp. 101–119). London: Sage. and analysis of mixed methods data. In S.
Tashakkori, A., & Teddlie, C. (in progress). A Hesse-Biber & P. Leavy (Eds.), Handbook of
practical guide for planning and conducting emergent methods in social research (pp.
integrated research in human, behavioral, 389–413). New York: Guilford Press.
and social research. New York: Guilford Tolman, D., & Szalacha, L. (1999). Dimensions
Press. of desire: Bridging qualitative and quantita-
Teddlie, C., & Johnson, B. (2009a). Method- tive methods in a study of female sexuality.
ological thought before the twentieth Psychology of Women Quarterly, 23, 7–39.
century. In C. Teddlie & A. Tashakkori Waszak, C., & Sines, M. C. (2003). Mixed
(Eds.), The foundations of mixed methods methods in psychological research. In
research: Integrating quantitative and A. Tashakkori & C. Teddlie (Eds.),
qualitative techniques in the social and Handbook of mixed methods in social and
behavioral sciences (pp. 40–61). Thousand behavioral research (pp. 557–576). Thousand
Oaks, CA: Sage. Oaks, CA: Sage.
Teddlie, C., & Johnson, B. (2009b). Method- Yanchar, S. C., & Williams, D. D. (2006).
ological thought since the twentieth Reconsidering the compatibility thesis and
century. In C. Teddlie & A. Tashakkori eclecticism: Five proposed guidelines for
(Eds.), The foundations of mixed methods method use. Educational Researcher, 35(9),
research: Integrating quantitative and qual- 3–12.
itative techniques in the social and behav- Yoshikawa, H., Weisner, T. D., Kalil, A., & Way,
ioral sciences (pp. 62–82). Thousand Oaks, N. (2008). Mixing qualitative and quantita-
CA: Sage. tive research in developmental science: Uses
Teddlie, C., & Stringfield, S. (1993). Schools and methodological choices. Developmental
make a difference: Lessons learned from a Psychology, 33(7), 344–354.