British Journal of Psychology (2000), 91, 1±20 Printed in Great Britain 1

# 2000 The British Psychological Society

Objectivity and reliability in qualitative

analysis: Realist, contextualist and radical
constructionist epistemologies

Anna Madill*, Abbie Jordan and Caroline Shirley

School of Psychology, University of Leeds, UK

The eåect of the individual analyst on research ®ndings can create a credibility
problem for qualitative approaches from the perspective of evaluative criteria
utilized in quantitative psychology. This paper explicates the ways in which
objectivity and reliability are understood in qualitative analysis conducted from
within three distinct epistemological frameworks : realism, contextual construct-
ionism, and radical constructionism. It is argued that quality criteria utilized in
quantitative psychology are appropriate to the evaluation of qualitative analysis
only to the extent that it is conducted within a naive or scienti®c realist framework.
The discussion is illustrated with reference to the comparison of two independent
grounded theory analyses of identical material. An implication of this illustration is
to identify the potential to develop a radical constructionist strand of grounded
theory.

Qualitative research is a diverse ®eld situated within a series of debates with

quanti®cation ; e.g. natural vs. arti®cial settings, induction vs. deduction, identifying
cultural patterns vs. seeking scienti®c laws (Hayes, 1997). However, a dominant
theme in qualitative research is the understanding of linguistic meaning within
textual material. The explication of meaning requires a certain level of inference
however, and qualitative approaches can be criticized for the space they aåord the
subjectivity of the researcher. This issue is a pertinent one within psychological
science as psychology has been based traditionally in positivist epistemology and
hence concerned to establish objective and reliable methods of investigation.
It is perhaps fair to say that few psychologists today espouse a strict logical
positivism in which knowledge is understood to be demonstrated through its direct
correspondence with observed events and research methods modelled on those of the
hard sciences. In practice, a range of post-positivist epistemologies are utilized which
recognize an element of interpretation and metaphor in the production of social
scienti®c theories and ®ndings. However, traditional methods of investigation, and
the criteria used to evaluate results, still emphasize objectivity and the importance of
non-biased data collection and analysis. This is demonstrated, for example, in the use
of tools such as coding manuals (providing precise de®nitions which can be
* Requests for reprints should be addressed to Dr Anna Madill, School of Psychology, University of Leeds, Leeds,
LS2 9JT, UK (e-mail : annam! psychology.leeds.ac.uk).
2 Anna Madill et al.
operationalized in a reasonably objective way) and overwhelming reliance on
statistical analyses (which can be applied in the same manner by any competent
researcher). Maximizing the objectivity of research is linked with the production of
reliable ®ndings; a multifaceted concept which refers to the consistency, stability, and
repeatability of results (Brink, 1991). So, even within post-positivist approaches,
reliable results that are not the product of systematic errors can be presumed to tap
phenomena which are reasonably independent of the researcher and instruments of
research.
The possibility of producing objective, reliable knowledge within the social
sciences has been questioned (e.g. Danziger, 1990 ; Harre! & Secord, 1972 ; Manicas,
1987 ; Sherrard, 1998) and alternative, qualitative approaches advocated as relevant
methods of investigation in these ®elds (e.g. Rennie, 1995). However, it is still entirely
appropriate that qualitative research be open to scrutiny and that the credibility of
®ndings rest on more than the authority of the researcher. As Parker (1994 b)
suggests, ` quantitative research preoccupations do need to be taken seriously, and if
qualitative research needs to refuse questions that are habitually posed in the
mainstream it must at least explain why it will not address those questions’ (p. 14).
A major source of misunderstanding, however, is that quantitative and qualitative
researchers often do not share fundamental assumptions like, for instance, what
constitutes ` doing analysis ’ (e.g. scienti®c vs. hermeneutic) or appropriate mode of
explanation (e.g. paradigmatic vs. narrative, see Rennie & Toukmanian, 1992).
A further source of confusion is that qualitative research is not a homogeneous
®eld. There are a number of epistemological positions within which the qualitative
researcher can work and many diåerent methods of analysis. As one way of imposing
order on this diversity, three broad epistemological strands are identi®ed here:
realist, contextual constructionist, and radical constructionist (adapted from
Henwood & Pidgeon (1994) and Parker (1994 b)). In many ways, these epistemologies
can be equated respectively with the paradigms of natural science, human science,
and poststructuralism. Most importantly, these positions carry diåering implications
for the evaluation of research conducted under their auspices ; a feature of qualitative
research too often ignored in reviews of quality criteria. The appropriateness of
transferring the notions of objectivity and reliability to the evaluation of qualitative
research will therefore be explored here while keeping these epistemological
diåerences at the foreground of discussion.
As an exemplar of key issues, our discussion is aided by the examination of
two independent qualitative analyses of identical material. For clarity, two simple
studies were selected in which two researchers (AJ and CS) used grounded theory to
analyse interviews with three relatives of individuals diagnosed as schizophrenic.
This illustrative material is useful in three ways. First, the presentation of two
independent analyses anchors the discussion in an evaluation of the objectivity and
reliability of actual qualitative ®ndings. Secondly, grounded theory analyses are
particularly suitable for exploring such issues as grounded theory is an approach
that can be applied within a realist or contextualist framework. Hence, the
importance of evaluating an analysis by the logic of justi®cation entailed by its stated
epistemological stance (as opposed to the nature of the data or method of analysis per
se) is underlined. Thirdly, an additional bene®t of using this empirical illustration is
 Objectivity and reliability in qualitative analysis 3
that a potential methodological development of applying grounded theory within a
radical constructionist epistemology is identi®ed and linked to issues of objectivity
and reliability in qualitative analysis.

Discussion
Realism : Discovering knowledge
To what degree can I trust the reporter that the research is true ? To what degree can I believe
that all possible occurrences of error were minimised rather than ` built into ’ the design? (Brink,
1991, p. 165)

Three realist epistemologies can be distinguished: naive, scienti®c, and critical. Naive
realism asserts a correspondence theory of truth in which the world is largely
knowable and just as it appears to be. Scienti®c realism adds that, although fallible,
the scienti®c method can tap true representations of the world and is the best mode
of inquiry. Critical realism, on the other hand, contends that ` the way we perceive
facts, particularly in the social realm, depends partly upon our beliefs and
expectations ’ (Bunge, 1993, p. 231). Hence, critical realism admits an inherent
subjectivity in the production of knowledge and, as discussed later, has much in
common with constructionist positions (Watkins, 1994±95).
Some authors have transferred notions such as objectivity and reliability directly,
or with little modi®cation, into the evaluation of qualitative research (e.g. Brink,
1991 ; Kirk & Miller, 1986 ; Miles & Huberman, 1994). It is argued here that this is
achievable only to the extent that a qualitative analysis has been conducted within a
naive or scienti®c realist epistemology. This is so because the ensuing correspondence
theory of truth entails that ®ndings be replicable, at least, to warrant status as
knowledge; that is, if something is true, it will be the same for all observers time and
time again.

Realist triangulation. Within a realist framework, triangulation can be used to assess

the reliability of qualitative analysis. Triangulation refers to the use of multiple
researchers, research methods, sources, or theories in order to assess the consistency
of ®ndings (Flick, 1991 ; Tindall, 1994). Results are understood to be substantiated
where diåerent perspectives converge so that triangulation becomes a process of
mutual con®rmation (Kna¯ & Breitmayer, 1991). The implication is that convergence
provides evidence of accuracy and objectivity.
Our two example qualitative analyses are examined below in the light of this
discussion of objectivity and reliability within realist epistemology. Making use of
the techniques of triangulation, we use the results of two independent grounded
theory analyses of three interviews with relatives of individuals diagnosed as
schizophrenic. The aim of these analyses was to extend our understanding of the
experience of caring for such a family member.

Background to the example analyses. Grounded theory (Glaser & Strauss, 1967) was
developed as a radical, discovery-oriented alternative to quantitative sociology out of
a dissatisfaction with the inability of the latter to capture lived experience. Although
4 Anna Madill et al.
more recent discussion of this approach stresses the interpretative nature of
grounded theory analysis (e.g. Corbin, 1998), the approach can certainly be applied
within a naive or scienti®c realist framework. In particular, the early articulation of
grounded theory utilized the language of realism implying that `the phenomenon
exists `` out there’’, awaiting discovery, like a fossil in a stratum’ (Rennie, 1996, p.
322). This can be understood as a form of ` soft’ positivism which assumes that
research is essentially a process of revealing or discovering pre-existing phenomena
and the relationships between them.
In essence, the process of grounded theory analysis utilized in our example studies
entailed the following steps: (1) a systematic coding of the interview material where
the categories used were not imposed but were derived from the data. Speci®cally,
the ` constant comparative method’ was used whereby the researchers continuously
checked and adjusted derived categories against successive paragraphs of text
(Strauss & Corbin, 1990) ; (2) the memoing of links between categories from the
perspective of a deep emersion in the material ; (3) generation of a justi®able model
using the derived categories and memoed links as a traceable audit trail through the
analysis.

Realist triangulation of the example analyses. A basis for the comparison of the example
analyses can be achieved most economically by presenting the ®nal models which
were produced (Figs 1 and 2). However, in their original reports, AJ and CS
substantiated their categories through full descriptions, quotes of relevant material,
and a frequency table of the number of quotes placed within each category.
Fundamental assumptions about objectivity and reliability as they are understood
within a quantitative framework can be applied with little or no modi®cation in the
evaluation of realist qualitative research. For example, such an analysis will typically
pay attention to representative sampling, generalizability, and inter-rater reliability.
In one form of realist analysis (which may rest on the boundary of qualitative
research), coders are trained to recognize pre-determined categories, or domains, in
textual material. The assumption is that such categories exist in the data and can be
identi®ed in an objective way by observers who know what they are looking for.
This type of analysis lends itself to quantitative assessment of inter-rater reliability.
If reliability is low, it may be assumed that raters were not adequately trained or
insu¬ciently rigorous in their scrutiny of the material. However, a critique of this
approach is that reliability, here, may simply re¯ect consistent error into which the
coders have been trained. This may stem, for example, from forcing the data into
categories favoured by the research culture as opposed to those best ®tting the
material.
On the other hand, as in our example analyses, qualitative researchers can
approach their material with a view to deriving categories from it. Within a realist
framework, such categories are considered to be discovered within the data.
Reliability in this form of realist analysis can be assessed though triangulating the
®ndings of multiple researchers. Traditionally, reliability has been considered in
terms of measurement. However, in the type of qualitative analysis used in our
example analyses, researchers are unlikely to describe all derived categories in exactly
the same way so reliability is properly assessed qualitatively in terms of consistency of
 Ded
S
Treatment services
Ded Blame
Coping S
Resentment of the illness
Des
Sadness
S
Des
Anger/Bitterness
Hopeful
T
T
Des STRUGGLE RESENTMENT
S
Fight to keep going
Peace of mind

Des
Ded
Friends/Partner
Treatment requirements
T T
Des
Ded BURDENED LOSS
Time
Symptoms of schizophrenia S
Des
Job Loneliness
Ded Ded
Objectivity and reliability in qualitative analysis

Des
Needs of other family members Finance
Privacy

Ded Des

Caring for schizophrenic relative Relationship with

schizophrenic relative

Figure 1. Model created by CS. Shaded boxes represent high-order categories.

5
 6

T T
RELATIVES’ EXPERIENCE OF FEELING TOWARDS
SCHIZOPHRENIC INDIVIDUAL MENTAL HEALTH SERVICES

Ded Ded Ded Ded Ded

Responses of family Symptoms of Experiences of Carers’ perceptions Interactions of carers
to SZ individual schizophrenia non-medical of schizophrenic & mental health
organizations/public individual’s experience professionals

Ded
Des Ded Des Ded
T Self-
Emotions Carer’s view of centred Importance of
Practical good relationships
issues T Support mental illness behaviours

Ded Fear of Des

future Des Des
Direct Dissatisfaction
Ded Stigma/Ignorance Anger/Fear
actions
Sadness Des
Des Des Des
Ded
Anna Madill et al.

Des Positive aspects

Carers’ Day-to-day Interactions & change
Instability coping
issues Anger with medical
profession
Ded Ded
Ded Ded Early manifestations Social Des
Family Changes in Des Des Realization of
structure SZ person problem
Perceived causes Genetic
Des Ded
Des Des
Exacerbates Change in Physical health Uncertainties
problems interaction

Figure 2. Model created by AJ. Shaded boxes represent high-order categories.

 Objectivity and reliability in qualitative analysis 7

Table 1. Comparison of categories produced by the two researchers, CS and AJ

Integrated theme CS AJ

Perceived causes Struggle Blame Perceived causes Genetic, Social,

Uncertainties
Support & Struggle Hopeful Positive aspects &
encouragement change, Support
from non-medical
organizations,
Importance of good
relationships
Problems with Struggle} Treatment services Dissatisfaction, SZ’s
treatment services Resentment Anger} Fear, SZ’s
interactions with
medical profession
Emotional toll Struggle Coping, Fight to Instability Emotions Day-to-day
keep going, coping,
Sadness Sadness
Resentment Anger} Bitterness Anger
Loss Peace of mind Fear of future
Understanding Resentment Resentment of the Carer’s view of
mental illness illness mental illness
Eåect on family Burdened Needs of other Changes in family Exacerbates
family members structure problems
Perception of the Burdened Symptoms of Early manifestations,
symptoms of schizophrenia Physical health,
schizophrenia Self-centred
behaviours,
Changes in schizo-
phrenic person, In-
stability, SZ’s re-
alization of problem
Practical burdens Burdened Finance, Treatment Practical issues Direct actions,
requirements, Carers’ issues
Caring for schizo-
phrenic relative
Loss Job, Time, Privacy
Adjusting to a new Loss Relationship with Change in interaction
personality schizophrenic
relative
Isolation Loss Friends} Partner, Stigma} Ignorance
Loneliness

Note. SZ stands for schizophrenic.

meaning (Denzin, 1989 ; Leininger, 1994 ; Stones, 1985). However, as Hill,

Thompson, and Williams (1997) point out, the best way of assessing such consistency
has yet to be determined. Table 1 shows a comparison of categories produced by the
two researchers as a kind of inter-rater reliability appropriate to this form of analysis,
providing evaluation of the similarity of interpretation demonstrated.
The models developed by the two researchers represent a form of theorization or,
at least, conceptualization (Charmaz, 1995) that necessarily goes beyond the data
8 Anna Madill et al.
itself. It seems appropriate therefore, within a realist framework, to concentrate on
objectivity and reliability as these issues relate to the actual categories produced. It
is obvious that AJ and CS present a diåerent number of categories with a variety of
diåerent labels. However, the two analyses can be integrated reasonably well within
10 common themes without overly distorting original meanings. Table 1 also
demonstrates how the diåerences between the two sets of categories are primarily at
the level of analytic detail. For example, the integrated theme ` Practical burdens’ was
analysed in more detail by CS than by AJ. On the other hand, although CS identi®ed
the carers’ burdens associated with the ` Perception of the symptoms of schizo-
phrenia’, AJ provided more detail about speci®c facets of this.

Table 2. Comparison of categories produced by two independent analyses of data in

Helstone et al. (1999) and a separate study on similar material by Stiles et al. (1997)

Helstone et al. (1999)

Helstone & Van

Integrated theme Zuuren (1999) Houtkooper (1995) Stiles et al. (1997)

Predicament Type of Problems Speci®city of

complaints problematic experiences
Developmental Developmental Psychodynamic Ð
psychodynamics history characteristics
Psychological Depth of verbal Introspective Richness of
mindedness exchange inclination understanding
Orientation for Locus of seeking Internality
change solution
Working climate Aåective Development of Ð
resonance rapport

Helstone, Van Zuuren, and Houtkooper (1999) demonstrate similar ®ndings in

their two independent sequential analyses of 36 interviews with intake workers.
Their sequential analysis involved the examination of interview transcripts for
themes central to a research question related to the intake workers’ reasons for
recommending particular treatment options (Wertz & Van Zuuren, 1987). As in the
present study, Helstone et al. found it useful to integrate the two analyses producing
themes capturing the insights of both (see Table 2). The authors considered the
description of themes from Analysis 1 (Helstone & Van Zuuren, 1999) to bear much
similarity to most of those from Analysis 2 (Houtkooper, 1995). Dimensions found
in a study of related material (Stiles, Shankland, Wright, & Field, 1997) is also
included in Table 2. Helstone et al. draw attention to the similarity in themes found
in this separate study as a potential indicator of the trustworthiness, or reliability, of
their ®ndings.
 Objectivity and reliability in qualitative analysis 9
Realist qualitative analysis : Conclusions. The form of comparison conducted by
Helstone et al., and in the present exploration of objectivity and reliability in realist
qualitative analysis, requires a certain amount of interpretative work and is not
rigorous in the sense that a statistical test may be. However, the thematic similarity
between the analyses in these studies should go some way to demonstrating that
qualitative researchers can produce results which are, at least, not wildly idiosyncratic.
Moreover, they show that results can be broadly reproducible given the ¯exibility
and relatively unstructured nature of textual data and our example modes of
qualitative analyses.

Contextual constructionism : Inter-subjective meanings

¼ bringing to public light researcher subjectivities, tells a more complete account of the research
process than is to be found in the customary sanitised versions of scienti®c report-writing ¼
(Pidgeon & Henwood, 1997, p. 270)
Researchers working within a contextualist or a radical constructionist epistemology
are more likely to reject a straightforward transference of criteria such as objectivity
and reliability into the evaluation of their work. In contrast to a naive or scienti®c
realist framework, it is no longer assumed that there is one reality that can be revealed
through the utilization of correct methodology. This position is of particular
relevance to the human sciences where the researcher and subject of research are both
conscious beings interpreting and acting on the world around them within networks
of cultural meaning (e.g. Giorgi, 1995). Relevant to constructionist epistemology,
Stiles (1993) suggests that `objectivity ’ be replaced with a notion of `permeability ’
by which he means `the capacity of theories or interpretations or understandings to
be changed by encounters with observations’ (p. 602).
Contextualism is the position that all knowledge is local, provisional, and situation
dependent (Jaeger & Rosnow, 1988). Hence, this perspective contends that results
will vary according to the context in which the data was collected and analysed. For
example, Pidgeon and Henwood (1997) identify four dimensions which may eåect
the production of knowledge: ` (1) participants’ own understandings, (2) researchers’
interpretations, (3) cultural meaning systems which inform both participants’ and
researchers’ interpretations, and (4) acts of judging particular interpretations as valid
by scienti®c communities ’ (p. 250). By implication, all accounts, whether those of
participants or of researchers, are understood to be imbued with subjectivity and
therefore not prima facie invalidated by con¯icting with alternative perspectives.
However, within a contextualist framework there is a desire to ®nd some kind of
grounding for results. This may be accomplished by an onus on the researcher to
represent the perspectives of participants through basing ®ndings in participants’
actual descriptions (Tindall, 1994). In addition, contextualism may utilize a critical
realist stance which `grounds discursive accounts [¼] in social practices whose
underlying logic and structure can, in principle, be discovered’ (Parker, 1996, p. 4).

Contextualist triangulation. Although particular accounts may not be invalidated by

alternative versions, there is the notion that some accounts may be more persuasive
or valuable than others or merely more relevant to particular research questions.
Hence, evaluation may involve a process of weighing one account against another,
10 Anna Madill et al.
which could include consultation with participants themselves (testimonial validity ;
Stiles, 1993). However, authors such as Wallat and Piazza (1988) advocate the bene®ts
of maintaining truly diverse perspectives. They describe this within the context of
multidisciplinary research which recognizes diåerent areas of expertise but also the
diåerent ways in which research questions emerge and constructs are utilized
between disciplines. This is diåerentiated from interdisciplinary studies which seek
knowledge through consensus. In a similar vein, Fielding and Fielding (1986) argue
that triangulation ` might get a fuller picture, but not a more `` objective ’’ one’ (p. 33)
as the goal of triangulation within a contextualist epistemology is completeness not
convergence. Thus, a particular strength of the contextualist approach to
triangulation is the possibility of retaining truly novel perspectives which may have
been discounted when consensus (and hence probably conventional) understandings
are valued (Tinsley, 1992).

Researcher subjectivity in contextualist analysis. As the name indicates, contextualism is

particularly concerned with the relationship between accounts and the situations in
which they were produced. Findings are considered context speci®c and hence
applicable to a narrow constituency which may be di¬cult to de®ne with precision
(McGuire, 1983). So, there is a strong rationale for requiring researchers to articulate
the perspective from which they approached their material (e.g. Wilkinson, 1988).
This includes such details as gender, ethnicity, age, and other factors which
conceivably inform the audience of the positions from which the researcher writes.
Of particular importance is facilitating reader assessment of the extent to which
researcher and participant share basic cultural assumptions and its aåect on the data
(e.g. the inter-subjective construction of material in a research interview) and analysis
of it. Articulation of researcher perspective may be utilized in a more realist analysis,
although it will ful®l a diåerent function consistent with that epistemology, e.g. to
help researchers lay aside their ` biases ’ (see Hill et al., 1997). In contrast, contextualist
analysis accepts the inevitability of bringing one’s personal and cultural perspectives
to bear on research projects. In fact, the empathy provided by a shared humanity and
common cultural understanding can be an important bridge between researcher and
participant and a valuable analytic resource.

Researcher subjectivity in the example analyses. We have examined how the objectivity
and reliability of our example grounded theory analyses may be evaluated within a
realist framework. Grounded theory may also be applied within a contextualist
epistemology. In fact, Charmaz (1986) argues that the approach is ideally placed to
bridge positivist and interpretative methods. The diåerence is the extent to which
®ndings are considered to be discovered within the data or the result of the
construction of inter-subjective meanings. Simplifying somewhat, the former
perspective is more consistent with Glaser’s recent position (e.g. 1992), the latter
with Strauss and Corbin’s development of grounded theory (e.g. 1990).
One particular aspect of researcher subjectivityÐanalytic styleÐis raised by our
example analyses and is selected for discussion here. As an aspect of researcher
subjectivity, analytic style has received much less critical examination than has, for
example, gender or ethnicity (see Wilkinson & Kitzinger, 1996), however as Miller
 Objectivity and reliability in qualitative analysis 11
et al. (1997) suggest, a researcher will ` be revealing glimpses of his own internal
models and personal constructs’ (p. 208) in the style and content of his or her
analysis. Hence, a discussion of analytic style allows a pragmatic exploration of one
aspect of researcher subjectivity which may particularly resonate with concerns about
the procedural rigour of qualitative analysis.
Although both researchers conducting our example analyses followed the general
procedures of grounded theory, it was revealed during supervisory sessions that
there was a diåerent emphasis in their analytic style. This can be explained through
the notion of serialist and holist approaches which originates in literature on learning
strategies. Pask (1976) identi®es the serialist learner as one who bene®ts from a step-
by-step approach, aided by rules and linear procedures. In contrast, the holist learner
prefers to get the overall picture and, in this way, to understand the relationship
between diverse material through the making of intuitive jumps.
The nature of qualitative analysis, in making sense of and seeing patterns in
complex textual data, arguably requires some facility with a holist style (Smith, Harre! ,
& Langenhove, 1995). This appeared to be the emphasis of CS’s approach in that she
describes identifying her four high-order categories (Loss, Burdened, Struggle, and
Resentment) before developing lower-level categories guided by these themes. In
contrast, AJ developed higher-order themes through clustering together lower-level
categories created during her close inspection of the data. The holist and serialist
styles may also be seen in the structure of the models created by each researcher. CS’s
model incorporates an integrated core in which the four main themes are holistically
interconnected in a cyclical manner. On the other hand, AJ’s model is more
systematically hierarchical.
Miller et al. (1997) provide an alternative conceptualization of analytic style. In
their study, researchers were required to provide summary vignettes which identi®ed
major themes from interview material. Four analytical approaches were identi®ed. In
order of increasing abstraction, but decreasing occurrence, these were : (1)
descriptive (sticking to the facts) ; (2) deductive (drawing conclusions); (3) thematic
(detecting underlying themes) ; and (4) speculative (creative interpretation). It is
interesting to note that many vignettists used a variety of styles although some
appeared to have de®nite preferences. AJ and CS used Miller et al.’s typography to
label each of their categories (noted on Figs 1 and 2). The frequency with which each
type of category appears in the two models is shown in Table 3.

Table 3. Percentage of type of category in the models created by CS and AJ

Descriptive Deductive Thematic Speculative

CS 33.3 % 29 % 16.7 % 21 %
AJ 44.5 % 44.5 % 11 % 0%

Using the two typographies outlined above, the presented models appear to be
predominantly descriptive and deductive (although CS also provides a large minority
of thematic and speculative categories) while AJ steers towards a serialist and CS a
12 Anna Madill et al.
holist style. Within a naive or scienti®c realist epistemology, diåerences in analytic
style would be understood as introducing biases into the research. Hence, an attempt
would be made to eradicate such diåerences through, for example, increasing the
speci®city of procedures. In contrast, in a contextualist framework, it is expected that
researchers will identify diåerent codes depending on, for instance, their training and
research interests (Charmaz, 1995 ; Miller et al., 1997). Articulating the position of the
researcher (and many other features apart from the analytic styles of the two
researchers would have been relevant here) allows the reader to evaluate the
inevitable contribution this makes towards their understanding of the materials
(Marshall, 1986).

Contextualist qualitative analysis : Conclusions. Within a contextualist epistemology, the

goal of triangulation is completeness not convergence. Consistent with this
approach, the two models presented here, which incorporate diåerent emphases in
analytic style, demonstrate how researchers can provide complementary pictures of
a phenomenon. The models are not incompatible but allow us to view the experience
of participants from two diåerent perspectives, both of which are justi®able. AJ’s
model oåers a clearly systematic analysis of the content of the interviews identifying
the participants’ main concerns. CS places the complex psychological impact of
caring for a relative diagnosed as schizophrenic at the centre of her model to which
the diåerent aspects of these participants’ lives are connected. In fact, it is arguable
that these analyses submit to this form of comparison with greater ease than our
search for consistency of meaning required within the context of realism. This
implies that grounded theory sits more comfortably within a contextualist
framework, which is consistent with the contemporary description of the approach
(Rennie, 1998).

Radical constructionism : Challenging the foundations of knowledge

It is the discourses that ` form the objects of which they speak’, and not authors who speak
through the text as if the text were a kind of transparent screen upon which the writer’s intentions
were displayed. (Parker, 1994a, p. 100)
Radical constructionism shares with contextualism the view that ` focus upon
countering sources of `` bias ’’ in data collection and analysis is incompatible with the
realisation that knowledge production involves an essential interconnection between
researcher and researched ’ (Henwood & Pidgeon, 1994, p. 231). Moreover, the
subjects of research (participants and the context of the investigation) are understood
to be more or less constituted by the methodologies used to examine them. However,
contextualism maintains the possibility of grounding research in, for example,
participants’ accounts. In contrast, radical constructionism moves closer to
challenging the notion that there can be any absolute foundations for knowledge.
The radical constructionist position is characterized by a profound distrust of
the idea that language can represent reality. Rather than consider objects to be the
foundation of representations, representations are understood to construct the
objects which then come to populate our world. So, for example, Rose (1989) traces
how the discipline of psychology produces objects of a certain kind, e.g. extroverts,
 Objectivity and reliability in qualitative analysis 13
and then presumes to study them. Hence, knowledge is considered a discursive
construction, questions of absolute truth or falsity are put to one side, and the
spotlight turned toward the ways in which knowledge claims function and are
legitimated within overarching `regimes of truth’, such as the discourses of science.
Critique of objectivity and reliability . Two key features of the discourses of science are
the central themes of this paper : objectivity and reliability. However, both notions
can themselves be critiqued. First, Parker (1994 c) challenges the idea that objectivity
and subjectivity are necessarily counterposed arguing that, in fact, ` objectivity ’ is
dependent upon ` subjectivity ’. For instance, the traditionally objective stance of
the scientist can be viewed as no more than a particular form of detached and
emotionless subjectivity. Second, Collins (1975) shows that what counts as a
successful replication, or other demonstration of the reliability of ®ndings, is
not unproblematic but subject to argument and negotiation between scientists (see
also Mulkay, 1991). Even if consensus is stabilized for a time it is vulnerable
to the charge of representing a shared interpretative framework rather than
ultimate truth (just as groups of psychoanalytic or cognitive therapists may
formulate consensual, but contrasting, understandings of a clinical case, see Stiles,
1997). Hence, assessing radical constructionist research in terms of traditional
notions of objectivity and reliability is inappropriate as both are critiqued as
rhetorical devices within radical constructionist epistemology. This does not mean,
though, that any interpretation of data is as good as another. Quality criteria oåered
within the context of this form of research include internal coherence, deviant case
analysis, and openness of the analysis to reader evaluation (e.g. Potter, 1996).
Alternative quality criteria. Internal coherence is an evaluation of the extent to which
the analysis `hangs together’ or is non-self-contradictory. However, the extent to
which any text can be completely coherent on close inspection has been questioned
within radical constructionist perspectives and forms the basis of deconstructionist
analysis (Derrida, 1978). Moreover, forms of discourse analysis stress the
inconsistency and variability of accounting practices and make much of this in their
analyses (e.g. Potter & Wetherell, 1987). It seems ` inconsistent’, therefore, to
suggest that research should be evaluated on a criterion that, from their own
perspective, is not to be found in other texts. The retort to this point may come from
the acknowledgement that radical constructionist research can be subjected to further
analysis and deconstruction and the avoidance of foundationalist knowledge claims.
So, perhaps internal coherence is to be understood in a gross sense here, as in `no
abhorrent contradictions’.
In order to persuade the reader of the internal coherence of an analysis the
researcher should seek out material which appears to challenge their developing
theory; that is, does it explain exceptions to the rule as well as typical examples ?
Deviant cases may usefully challenge a theory or delimit the context of its
applicability. On the other hand, it may substantiate the theory through participants’
orientation to a break in the expected pattern. This may be evidenced within the data
by participants’ requests for clari®cation, outright puzzlement, embarrassment, etc.
Finally, reader evaluation is the bottom-line appeal of the analysis to its audience.
Has the study contributed to the reader’s understanding of the phenomenon? Does
14 Anna Madill et al.
it facilitate productive action ? The audience for a piece of research will be varied.
There are journal editors and reviewers, other researchers, practitioners, and people
who may be aåected by the implementation of results. Each are likely to view the
research from a diåerent perspective and their evaluation will hold sway at diåerent
stages in the life of a manuscript and the diåerent contexts in which it is read. For
example, if reviewers do not get the sense that they have learnt something from a
study it is unlikely to be published. Ultimately, reader evaluation is a pragmatic
criterion as productive studies should stand the test of time through being widely
acknowledged to have contributed to the development of the ®eld.
Internal coherence, deviant case analysis, and reader evaluation do not guarantee
the reliability of an analysis and research within a radical constructionist framework
does not claim to be perfectly replicable. As Parker (1994 b) states : ` It is certainly
possible to repeat the work that has been described, but that repetition will
necessarily also be a diåerent piece of work’ (p. 11). This is not because analysis is
arbitrary. In the attempt to provide meaningful research, even radical constructionists
must assume that there is structure within human activity and their analyses are
bounded by the limits of inter-subjective understandings. However, underlying
structures and inter-subjective understandings are not ` unalterably given or
irreducible’ (Hemingway, 1995, p. 44) and hence aåord space for interpretation.
Moreover, constructionist research aims to explain not predict. This rejection of
prediction is linked to the nature of the phenomena most amenable to constructionist
analysis, e.g. the expression of human experience within language. Stiles (1993)
argues that such material is non-linear and chaotic in that small changes can produce
increasingly greater eåects. So, for example, a participant’s reply to a question may
aåect the whole course of an interview in a way that would be virtually impossible
to predict, although it follows discoverable patterns consistent with `doing an
interview ’.

The grounding of analysis. The inability, or refusal, of radical constructionists to deliver

secure foundations for knowledge can seem highly dissatisfactory. This may be
linked with suspicion of relativism which seeks to understand the ways in which
forms of knowledge about the world are sustained, and undermined, rather than
appeal to any reality existing outside of human interpretation (Edwards, Ashmore,
& Potter, 1995). Challenges have been made, however, to the idea that radical
perspectives in psychology require a wholly relativist stance. Parker (1996), for
instance, expresses concern about the potential political vacuousness of extreme
relativism and advocates instead a critical realist position as it incorporates an explicit
emancipatory project. This position is realist in positing the existence of social
structures which are independent of human understanding (e.g. hierarchies of
privilege), but relativist in emphasizing the constitutive role of discourse and
allowing a radical scepticism of institutionalized forms of knowledge (Magill, 1994).
Perhaps oåsetting the unpalatability of relativism, radical constructionist research
does stress the importance of providing readers with enough raw data to assess the
adequacy of an analysis. Although rarely expressed as a form of grounding, such a
procedure can be understood as its equivalent. This is particularly evident, for
example, in ethnomethodological forms of discourse analysis. Here, justi®cation for
 Objectivity and reliability in qualitative analysis 15
an interpretation is oåered often through the turn-by-turn analysis of given excerpts
of talk in which participants’ own unfolding understanding of the meaning
constructed within the conversation is the focus of study (e.g. Edwards & Potter,
1992). It is important to note that ` participants’ orientation ’ here is diåerent from
that provided within contextualism. In contextualist research, participants may be
considered the ®nal arbiters of an analysis. Radical constructionists, in contrast, shift
the emphasis to the openness of their analyses to reader evaluation through
providing evidence of participants’ ongoing and changing orientation while under
construction in, for example, the research interview.
In fact, from the perspective of radical constructionism, it seems that contextualism
does not take its principles to their logical conclusion. First, contextualism maintains
that, although an analysis is always partial and subjective, results can be justi®ed to
the extent that they are grounded in the data. This requires a careful, and tenuous,
balance between realist claims that results emerge from the data, with the
constructionist position that analysis is necessarily interpretative (Hammersley, 1989 ;
Henwood & Pidgeon, 1994) (a dilemma which radical constructionism, however,
does not fully escape). Moreover, radical constructionism includes a useful scepticism
against prima facie privileging of participants’ re¯ections on an analysis. An analysis
may be distasteful to participants although justi®able in terms of good research
practice (e.g. suggesting that managers’ decisions may lead to unequal opportunities
for their staå). As Silverman (1989) warns, researchers must `avoid treating the
actor’s point of view as an explanation ’ (p. 218) as it may, for example, be shaped by
unre¯ective, conventionalized understandings (e.g. see Linde, 1983).
Secondly, in seeking to ground analysis, contextualism may sometimes appear to
imply a rei®cation of participant and researcher subjectivity contra its emphasis on
the contextual speci®city of knowledge. For example, in our discussion of analytic
style, articulating the position of the researcher implied that the features selected for
comment `belong to ’ the analyst and have a causal eåect on the outcome of research.
This is to ignore the way in which researchers can approach their work
simultaneously from a variety of positions (e.g. use variable styles, see Miller et al.,
1997). It also obscures the constructive work achieved through providing a
particular description of the researcher and research context (e.g. making the
implication available that the research process has been made transparent). In fact,
discourse analytic research suggests that `confessions’, such as descriptions of the
researcher’s perspective, can be understood as a way of pre-empting and hence
undermining the impact of criticism through the rhetorical management of the
researcher’s stake and interest (Potter, Edwards, & Wetherell, 1993). Contextualism
can therefore be vulnerable to assuming an ` uncomplicated subjectivity ’ (Parker,
1994 c) through expecting that both participants and researchers may articulate their
position fully and neutrally (see also Rennie, 1997).

Subjectivity in radical constructionist analysis. Researchers working within a radical

constructionist framework present various approaches to these issues. One strategy,
that might be referred to as `agnosticism’ or ` blank subjectivity ’ (Parker, 1994 c), is
to be found in ethnomethodological forms of discourse analysis. Such approaches
may explore how subjectivities are constructed within texts but refuse to comment
16 Anna Madill et al.
on subjects as producers of texts. However, this has been criticized as potentially
obscuring the constructive work of the researcher to an even greater extent than
might the contextualist position (Parker, 1994 a ; for contrary position see Potter,
1988).
An alternative strategy is to ground analysis in a decentred or ` complex
subjectivity ’ (Parker, 1994 c). Here, there is an attempt to theorize subjectivity but in
a way which captures the interweaving of self-as-experienced and self-as-constructed-
within-the-symbolic-order. Such perspectives have often incorporated a psycho-
analytic framework which helps to avoid the assumption that participants and
researchers can articulate their own subjectivity as if it were unitary and transparent
(Hollway, 1989 ; Parker, 1994 c). However, this has to be balanced against the
possibility of reifying a particular theory of subjectivity which is culturally and
historically situated.

A radical constructionist grounded theory. Grounded theory appears to be most compatible

with realist and contextualist epistemologies. However, there has been a call to
develop a more radical strand which does not assume that participants’ accounts will
be internally consistent nor imply an essentialized model of subjectivity (Pidgeon &
Henwood, 1997). In particular, qualitative analyses concentrating on the level of
categories and themes have been criticised for ` fracturing the data ’ (Charmaz, 1995,
p. 49). Categorization separates statements from context and therefore loses the
complex and often contradictory ways in which positions can be refuted, undermined,
con®rmed etc. even within the account of a single participant. Analysis of this
process requires a much closer reading of extracts in a style nearer to that of discourse
analysis which documents the reasoning behind the researcher’s interpretation. As in
grounded theory, discourse analysis often commences with a general categorization
of the data as a way of making a bulk of material more manageable (Potter &
Wetherell, 1995). The themes identi®ed in the examples here could therefore present
a starting point for more detailed analysis compatible with a radical constructionist
development of grounded theory.
Further analysis could focus on the macro-discourses shaping the underlying
assumptions of the accounts or, alternatively, explicate the rhetorical devices through
which the participants construct and give meaning to their lives and re¯ect on their
experiences. For example, the model created by CS raises the issue of how relatives
incorporate notions of `hope’ and maintain a ` ®ght to keep going’ alongside
descriptions of the ` struggle’ their situation entails. On the other hand, the
researcher could focus on assumptions about normal and abnormal inter-personal
functioning drawn on within participants’ accounts of losing their former
relationship with their schizophrenic relative and the ways in which they orient to the
metaphors, models, and categories such description entails. The model provided by
AJ might catalyse a more detailed analysis of how carers manage the tension between
placing value on and seeking to maintain good relations with mental health
professionals while, at the same time, describing a dissatisfaction with the service
received. This would raise the analyses from classi®cation or development of theory
to a level of explanation involving the explication of ambiguity (Smith, 1995).
 Objectivity and reliability in qualitative analysis 17

Conclusion
This paper has explicated what objectivity and reliability mean in qualitative analysis
from the perspective of realist, contextualist constructionist, and radical con-
structionist epistemologies and demonstrated how these positions carry diåering
implications for the evaluation of research conducted under their auspices.
Speci®cally, it is argued here that criteria such as objectivity and reliability are
appropriate to the evaluation of qualitative analysis only to the extent that it is
conducted within a naive or scienti®c realist framework.
Using our example analyses as illustrative material, it was shown how realist
qualitative research can demonstrate a level of objectivity and reliability, interpreted
appropriately as `consistency of meaning ’, through triangulating the ®ndings of
independent researchers. A diåerent form of triangulation was discussed in relation
to contextualist analysis which contends that all accounts are imbued with subjectivity
and rejects the search for consensus, per se, through arguing that diverse perspectives
can provide a fuller understanding of social psychological phenomena. The onus on
researchers is to make their relationship to the material clear and to ground analysis
in participants’ own accounts. The example analyses in this study were argued to
provide usefully complementary perspectives on our participants’ experience.
Finally, radical constructionist positions were shown to be sceptical of all
foundationalist knowledge claims and to be critical of the concepts of objectivity and
reliability as rhetorical devices. Analyses are assessed on their own merits, e.g. for
internal coherence, ability to explain deviant cases, and enabling understanding and
productive action. Hence, the comparison of independent analyses of the same
material for the purposes of evaluation is redundant.
The diversity of paradigms available within psychology today make it important
that research be evaluated by the standards entailed by its own logic of justi®cation.
However, as incorporated in our discussion of the three epistemologies identi®ed
here, diåerent positions exist even within these broad themes. Hence, qualitative
researchers have a responsibility to make their epistemological position clear,
conduct their research in a manner consistent with that position, and present their
®ndings in a way that allows them to be evaluated appropriately. This may be
particularly important with approaches such as grounded theory which can be
applied within a realist or contextualist framework and, as demonstrated above, has
the potential to develop a further radical constructionist strand.

Acknowledgements
The authors express their heartfelt thanks to the three participants who so willingly shared their time
and personal experiences. We would also like to thank the following people for their help at various
stages in the development of this paper: Mark Conner, Wendy Hollway, Anne Rees, Carol Sherrard,
Lawrence Smith, and Peter Stratton (University of Leeds, UK), Derek Edwards and Jonathan Potter
(Loughborough University, UK), Constance Fischer (Duquesne University, USA), Karen Henwood
(University of East Anglia, UK), John McLeod (University of Abertay, Dundee, UK), Ian Parker
(Bolton Institute, UK), Bill Stiles (Miami University, USA).
18 Anna Madill et al.
References
Brink, P. J. (1991). Issues of reliability and validity. In J. M. Morse (Ed.), Qualitative nursing research : A
contemporary dialogue (Rev. ed., pp. 164±186). London: Sage.
Bunge, M. (1993). Realism and antirealism in social science. Theory and Decision, 35, 207±235.
Charmaz, K. (1986). Using grounded theory for qualitative analysis, Master Lecture, Sociology
Department, York University, Toronto, 12 May. Cited in K. Charmaz (1995). Grounded theory. In
J. A. Smith, R. Harre! , & L. Van Langenhove (Eds.) Rethinking methods in psychology (pp. 27±49).
London: Sage.
Charmaz, K. (1995). Grounded theory. In J. A. Smith, R. Harre! , & L. Van Langenhove (Eds.),
Rethinking methods in psychology (pp. 27±49). London: Sage.
Collins, H. M. (1975). The seven sexes : A study in the sociology of a phenomenon, or the replication
of experiments in physics. Sociology, 9, 205±224.
Corbin, J. M. (1998). Alternative interpretations : Valid or not? Theory and Psychology, 8, 121±128.
Danziger, K. (1990). Constructing the subject : Historical origins of psychological research. Cambridge:
Cambridge University Press.
Denzin, N. (1989). The research act (3rd ed.). Englewood Cliås : Prentice Hall.
Derrida, J. (1978). Writing and diåerence. London: Routledge and Kegan Paul.
Edwards, D., Ashmore, A., & Potter, J. (1995). Death and furniture : The rhetoric, politics and theology
of bottom line arguments against relativism. History of the Human Sciences, 8, 25±49.
Edwards, D., & Potter, J. (1992). Discursive psychology. London: Sage.
Fielding, N. G., & Fielding, J. L. (1986). Linking data. London: Sage.
Flick, U. (1991). Triangulation revisited : Strategy of validation or alternative ? Journal for the Theory of
Social Behaviour, 22, 175±197.
Giorgi, A. (1995). Phenomenological psychology. In J. A. Smith, R. Harre! , & L. Van Langenhove
(Eds.), Rethinking psychology (pp. 24±42). London: Sage.
Glaser, B. J. (1992). Emergence vs forcing : The basics of grounded theory analysis. Mill Valley, CA : Sociology
Press.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory : Strategies for qualitative research.
New York : Aldine.
Hammersley, M. (1989). The dilemma of qualitative method. London: Routledge.
Harre! , R., & Secord, P. F. (1972). The explanation of social behaviour. Oxford : Blackwell.
Hayes, N. (1997). Qualitative research and research in psychology. In N. Hayes (Ed.), Doing qualitative
analysis in psychology (pp. 1±8). Hove, UK : Psychology Press.
Helstone, F. S., & Van Zuuren, F. J. (1999). Assessment for psychotherapy: A qualitative enquiry into
intake workers’ observations and their consequences for therapy indication. Psychotherapy Research, 8,
248±263.
Helstone, F. S., Van Zuuren, F. J., & Houtkooper, S. (1999). A cross-case comparison of two
independent analyses of intake workers’ descriptions of the process of assessment for psychotherapy.
British Journal of Medical Psychology, 72, 355±369.
Hemingway, J. L. (1995). Leisure studies and interpretive social inquiry. Leisure Studies, 14, 32±47.
Henwood, K., & Pidgeon, N. (1994). Beyond the qualitative paradigm: A framework for introducing
diversity within qualitative psychology. Journal of Community and Applied Social Psychology, 4, 225±238.
Hill, C. E., Thompson, B. J., & Williams, E. N. (1997). A guide to conducting consensual qualitative
research. The Counseling Psychologist , 25, 517±575.
Hollway, W. (1989). Subjectivity and method in psychology : Gender, meaning, and science. London: Sage.
Houtkooper, S. (1995). Intake en indicatiestelli ng [Assessment and treatment assignment in psycho-
therapy]. Unpublished master’s thesis. University of Amsterdam, Dept of Clinical Psychology, The
Netherlands.
Jaeger, M. E., & Rosnow, R. L. (1988). Contextualism and its implications for psychological inquiry.
British Journal of Psychology, 79, 63±75.
Kirk, J. L., & Miller, M. (1986). Reliability and validity in qualitative research. London: Sage.
Kna¯, K. A., & Breitmayer, B. J. (1991). Triangulation in qualitative research : Issues of conceptual
clarity and purpose. In J. M. Morse (Ed.), Qualitative nursing research : A contemporary dialogue (Rev. ed.,
pp. 226±239). London: Sage.
 Objectivity and reliability in qualitative analysis 19
Leininger, M. (1994). Evaluation criteria and critique of qualitative research studies. In J. M. Morse
(Ed.), Critical issues in qualitative research methods (pp. 95±115). London: Sage.
Linde, C. (1983). The creation of coherence in life stories. Norwood, NJ : Ablex.
Magill, K. (1994). Against critical realism. Capital and Class, 5, 113±136.
Manicas, P. T. (1987). A history and philosophy of the social sciences. Oxford : Blackwell.
Marshall, J. (1986). Exploring the experiences of women managers: Towards rigour in qualitative
methods. In S. Wilkinson (Ed.), Feminist social psychology : Developing theory and practice (pp. 193±209).
Milton Keynes: Open University Press.
McGuire, W. J. (1983). A contextual theory of knowledge: Its implications for innovation and reform
in psychological research. Advances in Experimental Social Psychology, 16, 1±47.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis : A sourcebook of new methods (2nd ed.)
London: Sage.
Miller, T., Velleman, R., Rigby, K., Orford, J., Tod, A., Copello, A., & Bennett, G. (1997). The use
of vignettes in the analysis of interview data: Relatives of people with drug problems. In N. Hayes
(Ed.), Doing qualitative analysis in psychology (pp. 201±225). Hove: Psychology Press.
Mulkay, M. (1991). Sociology of science : A sociological pilgrimage. Milton Keynes: Open University Press.
Parker, I. (1994a). Discourse analysis. In P. Banister, E. Burman, I. Parker, M. Taylor, & C. Tindall
(Eds.), Qualitative methods in psychology : A research guide (pp. 92±107). Buckingham: Open University
Press.
Parker, I. (1994 b). Qualitative research. In P. Banister, E. Burman, I. Parker, M. Taylor, & C. Tindall
(Eds.), Qualitative methods in psychology: A research guide (pp. 1±16). Buckingham: Open University
Press.
Parker, I. (1994c). Re¯exive research and the grounding of analysis : Social psychology and the psy-
complex. Journal of Community and Applied Social Psychology, 4, 239±252.
Parker, I. (1996, April). Social constructionism , discourse, and realism. Day conference, Discourse Unit,
Manchester Metropolitan University.
Pask, G. (1976). Learning styles and strategies. British Journal of Educational Psychology, 46, 4±11.
Pidgeon, N., & Henwood, K. (1997). Using grounded theory in psychological research. In N. Hayes
(Ed.), Doing qualitative analysis in psychology (pp. 245±273). Hove: Psychology Press.
Potter, J. (1988). What is re¯exive about discourse analysis ? The case of reading readings. In
S. Woolgar (Ed.), Knowledge and re¯exivity : New frontiers in the sociology of knowledge (pp. 37±54).
London: Sage.
Potter, J. (1996). Discourse analysis and constructionist approaches: Theoretical background. In
J. T. E. Richardson (Ed.), Handbook of qualitative research methods for psychology and the social sciences
(pp. 125±140). Leicester : British Psychological Society.
Potter, J., Edwards, D., & Wetherell, M. (1993). A model of discourse in action. American Behavioral
Scientist, 36, 383±401.
Potter, J., & Wetherell, M. (1987). Discourse and social psychology: Beyond attitudes and behaviour. London:
Sage.
Potter, J., & Wetherell, M. (1995). Discourse analysis. In J. A. Smith, R. Harre! , & L. Van Langenhove
(Eds.), Rethinking methods in psychology (pp. 80±92). London: Sage.
Rennie, D. L. (1995). On the rhetorics of social science : Let’s not con¯ate natural science and human
science. Humanistic Psychologist, 23, 321±332.
Rennie, D. L. (1996). Fifteen years of doing qualitative research on psychotherapy. British Journal of
Guidance and Counselling, 24, 317±327.
Rennie, D. L. (1997). Commentary on ` Clients ’ perceptions of treatment for depression : I and II ’.
Psychotherapy Research, 6, 262±268.
Rennie, D. L. (1998). Grounded theory methodology: The pressing need for a coherent logic of
justi®cation. Theory and Psychology, 8, 101±119.
Rennie, D. L., & Toukmanian, S. G. (1992). Explanation in psychotherapy process research. In S. G.
Toukmanian & D. L. Rennie (Eds.), Psychotherapy process research : Paradigmatic and narrative approaches
(pp. 234±251). London: Sage.
Rose, N. (1989). Governing the soul. London: Routledge.
Sherrard, C. (1998). Social dimensions of research. The Psychologist, 10, 486±487.
20 Anna Madill et al.
Silverman, D. (1989). Six rules of qualitative research : A post-romantic argument. Symbolic Interaction,
12, 215±230.
Smith, J. A. (1995). Semi-structured interviewing and qualitative analysis. In J. A. Smith, R. Harre! , &
L. Van Langenhove (Eds.), Rethinking methods in psychology (pp. 9±26). London: Sage.
Smith, J. A., Harre! , R., & Van Langenhove,L. (1995). Introduction. In J. A. Smith, R. Harre! , & L. Van
Langenhove (Eds.), Rethinking psychology (pp. 1±9). London: Sage.
Stiles, W. B. (1993). Quality control in qualitative research. Clinical Psychology Review, 13, 593±618.
Stiles, W. B. (1997). Consensual qualitative research : Some cautions. The Counseling Psychologist, 25,
586±598.
Stiles, W. B., Shankland, M. C., Wright, J., & Field, S. D. (1997). Dimensions of clients’ initial
presentation of problems in psychotherapy: The early assimilation research scale. Psychotherapy
Research, 7, 155±171.
Stones, C. R. (1985). Qualitative research : A viable psychological alternative. The Psychological Record,
35, 63±75.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research : Grounded theory procedures and techniques.
Newbury Park, CA : Sage.
Tindall, C. (1994). Issues of evaluation. In P. Banister, E. Burman, I. Parker, M. Taylor, & C. Tindall
(Eds.), Qualitative methods in psychology : A research guide (pp. 142±159). Buckingham: Open University
Press.
Tinsley, H. E. A. (1992). Am I the ®fth horseman of the apocalypse? Comment on Galassi, Crace,
Martin, James and Wallace (1992) and comment on research concerning expectations about
counseling. Journal of Counseling Psychology, 39, 59±65.
Wallat, C., & Piazza, C. (1988). The classroom and beyond: Issues in the analysis of multiple studies
of communicative competence. In J. L. Green & J. O. Harker (Eds.), Multiple perspective analysis of
classroom discourse (pp. 309±341). Norwood, NJ : Ablex.
Watkins, J. M. (1994±95). A postmodern critical theory of research use. Knowledge and Policy, 7, 55±77.
Wertz, F. J., & Van Zuuren, F. J. (1987). Qualitative research : Educational considerations. In F. J. Van
Zuuren, F. J. Wertz, & B. Mook (Eds.), Advances in qualitative psychology : Themes and variations.
Berwyn : Swets North America.
Wilkinson, S. (1988). The role of re¯exivity in feminist psychology. Women’s Studies International Forum,
11, 493±502.
Wilkinson, S., & Kitzinger, C. (Eds.) (1996). Representing the other : A feminism and psychology reader.
London: Sage.

Received 13 August 1998; revised version received 9 April 1999