Data Saturation

Linda L. Barnes
In 1967, sociologists Barney Glaser and Anselm Strauss introduced the concept of “saturation” to
the field of qualitative research. By this they meant the point in data analysis at which no new themes or
information continued to appear. Glaser and Strauss developed the concept to address the challenge of
determining appropriate sample sizes for qualitative studies. How many research participants are called
for, that is, in order to arrive at a comprehensive understanding of an issue?
Qualitative studies involve gathering a range of in-depth answers to semi-structured or open-
ended questions. Sample sizes are rarely calibrated according to any particular formula. In probabilistic
sampling, statistical measures define the necessary number of subjects. In contrast, qualitative and
anthropological interviews and participant observation deliberately defy neat measures. They aim,
instead, for as in-depth a representation as possible of views, attitudes, formulations, and experiences in
connection with a research question.
Qualitative researchers review and analyze their data over the course of a study, instead of
waiting until the completion of data collection. This “processual,” or process-based, approach allows
researchers “to go back and refine questions, develop hypotheses, and pursue emerging avenues of
inquiry in further depth” as an integral and ongoing aspect of data analysis (Pope 114). Far from being a
failing, this emergence of new themes and ideas is, instead, viewed as a methodological strength. It allows
researchers to revisit their hypotheses and conclusions along the way, reviewing and rethinking what the
data appear to indicate.
Data saturation is not necessarily an appropriate method for all kinds of qualitative research. For
example, it tends not to be an analytical aim in 1) case studies, which by definition involve small
numbers; 2) phenomenological research, in which a limited number of individuals are studied in depth,
with an interest in the range of meanings they may assign to particular phenomena; 3) pilot studies, when
the objective of the project is to generate variables for more elaborate research; 4) any other studies in
which the population is, by definition, small or limited (Anon. 2006).
There also may be idiosyncratic responses, sometimes characterized as “outliers,” which are
viewed as desirable insofar as they contribute to shaping the spectrum of possible views. One does not
exclude the outliers, or merge them into the larger categories. Instead, one preserves such apparent
exceptions and attempts to interpret them. One might also look for additional examples of that particular
viewpoint, taking the first example as an indicator of a body of potential data one might otherwise have
overlooked. One is then challenged to theorize and explain the varieties of response. Ultimately, as in
quantitative methods, it is the research question that governs the sample size.
Despite the apparent lack of precision in the definition of data saturation, in practice,
anthropologists and other qualitative researchers have, by and large, had the experience that, through this
process, a point arrives when it is clear that no significantly new additional data are emerging. This holds
true with answers to specific questions, as well as to conceptual issues. Yet rarely is “data saturation”
defined or operationalized with any greater precision than this admittedly impressionistic perception.
Indeed, in a review of studies that drew on this method, Francis et al. found that related definitions were
consistently vague: data saturation referred generically to the point at which no new information, themes,
findings, concepts or problems appear in the data. However, none of these studies provided a widely
accepted or consistent method for determining when an exact moment of data saturation had been
reached. As a result, from a quantitative perspective, what this term means in practice remains imprecise
(Francis et al, 2009).
When data saturation is employed, therefore, it can be tempting for quantitatively oriented
researchers to want to reorient the process according to more explicitly measurable criteria. How many
times, for example, is a given term or concept used? Why not enter such data into a spreadsheet? The
thought is that to do so will enable one to devise a standardized approach and formulate systematic
principles that will allow one to predict and link saturation with a specific sample size. Francis et al., for
example, propose the principle that researchers set an initial analysis sample—a representative set of
interviews, to be determined based on the complexity of the research questions, diversity of the sample,
and pre-specified factors such as age, gender, rurality, ethnicity, etc. (Francis et al. 2009).
In the case of the Somali Oral Health Literacy project, such factors can be drawn from the Sample
Characteristics, with the qualitative team consulting with the larger research team to determine which
characteristics pose the greatest ranges of variety. They may, for example, include age, years in the U.S.,
gender, ethnicity, level of education, ability to speak or read English, and access to dental insurance.
Given that this study involves multiple subgroups of interviewees, each of which must be considered
separately, the research team might determine which leading sample characteristics define these
subgroups. The quantitative team, for example, might suggest which key clusters of characteristics stand
out. This determination might, in turn, then guide some of the recruitment of participants for the
qualitative interviews.
Interviews might then be grouped into these subgroup clusters, providing an additional lens
through which to examine themes, issues, and topics appearing in the qualitative data. Examples would
include ways of discussing the roles of Islam in relation to oral health practices; social resources (e.g.,
family, friends, acquaintances) in connection with accessing care and information about care; uses of
traditional oral health practices; or modifications to practices in other countries and in the United States.
Saturation would be tested, then, in connection with each subgroup, rather than with the larger study
population.
 A second principle proposed by Francis et al. is that, in addition to selecting a tentative number of
transcripts from which one expects to reach saturation (recognizing that one cannot set absolutes), one
should have a method by which to test what one finds. Francis’s team suggests that researchers specify
how many additional interviews be conducted and coded sequentially, in order to test whether or not any
additional topics or themes arising (e.g., three additional interviews). If one could code three additional
interviews in this way, one would have a method by which to establish a “stopping criterion.” They also
recommend the involvement of at least two independent coders, and the reporting of data saturation
methods and findings, such that readers can evaluate the evidence (Francis et al. 2009).
We are utilizing a modified form of these principles. First, the qualitative team prepared a draft
codebook by having each member independently review four transcripts, as part of a first round of
analysis. This set has provided be the initial analysis sample, and is the basis for themes, terms, and
topics, which we have then organized into a detailed topical and thematic outline. We will then test this
outline against two additional transcripts, to evaluate it for completeness, with the expectation that it will
continue to undergo modification.
Other researchers who have attempted a more systematic method to determine how many
interviews tend to be necessary to reach saturation suggest that at least ten to twelve interviews per
subgroup are optimal. We can aim for this number. To test saturation for each subgroup, we might then
adopt a “stopping criterion” of three additional consecutive interviews in which no new topics or themes
arise that would require further modification to the codebook. Therefore, based on the subgroups defined
by the research team as a whole, the qualitative team will make every effort to recruit at least fifteen
interviewees who represent each of the specific subgroups, but will not be limited by this number. (This
suggests, of course, that we might want to aim for roughly five subgroups, if the sample characteristics so
warrant.)
This approach will contribute to a more systematic implementation of saturation. However, it is
also essential to keep in mind that the very nature of qualitative data analysis involves what Marshall
refers to as “a flexible research design and an iterative, cyclical approach to sampling, data collection,
analysis and interpretation,” through which the number of subjects necessary for saturation emerges over
the course of the study (Marshall 1996, 523). The process of qualitative data analysis is, by definition,
more fluid, often impressionistic, and dependent not on accumulating quantitative findings, but rather on
the in-depth interpretation of meanings. To attempt to translate it into quantitatively valid results does not
do justice to the method or, in many cases, to the data itself. It is preferable to employ both strategies at
their best, allowing them to function not only as checks and balances to each other, but also as mutually
illuminating strategies.
References
Anon. 2006. “Criteria for Review of Reports of Qualitative Research.” Journal of Clinical Nursing
October: http://www.blackwellpublishing.com/pdf/JCN_Criteria_Qual_Research.pdf (Accessed
7/2/10).
Francis, Jill J., Marie Johnston, Clare Robertson, Liz Glidewell, Vikki Entwistle, Martin P. Eccles, and
Jeremy M. Grimshaw. 2009. “What is an adequate sample size? Operationalising data saturation for
theory-based interview studies.” Psychology and Health (October) 1–17.
Guest, Greg, Arwen Bunce, Laura Johnson, Greg Guest, Arwen Bunce, Laura Johnson. 2006. “How
Many Interviews Are Enough?: An Experiment With Data Saturation And Variability.” Field
Methods 18(1): 59-82.
Marshall, M. N. 1996. “Sampling for qualitative research.” Family Practice 13: 522-525.
Onwuegbuzie, Anthony J., and Nancy L. Leech. 2005. “The Role of Sampling in Qualitative Research.”
Academic Exchange Quarterly (Fall):
http://findarticles.com/p/articles/mi_hb3325/is_3_9/ai_n29219699/ (Accessed 7/2/2010).
Pope, Catherine, Sue Ziebland, and Nicholas Mays. “Qualitative research in health care: Analysing
qualitative data.” BMJ 320: 114-116.
Wray, Natalie, Milica Markovic, Lenore Manderson. 2007. "Researcher Saturation": The Impact Of Data
Triangulation And Intensive-Research Practices On The Researcher And Qualitative Research
Process.” Qualitative Health Research 17(10): 1392-1402.