06 Byng (bc/t) 30/3/05 3:26 pm Page 69

Evaluation
Copyright 2005
SAGE Publications (London,
Thousand Oaks and New Delhi)
DOI: 10.1177/1356389005053198
Vol 11(1): 6993

Using Realistic Evaluation to Evaluate

a Practice-level Intervention to
Improve Primary Healthcare for
Patients with Long-term Mental Illness
RICHARD BYNG
Department of General Practice and Primary Care, Kings College London, UK

I A N N O R M A N A N D S A L LY R E D F E R N
School of Nursing and Midwifery, Kings College London, UK

Mental Health Link a facilitated programme aimed to develop systems

within primary care and links with specialists to improve care for patients
with long-term mental illness. A process evaluation based on Pawson and
Tilleys Realistic Evaluation complemented a randomized controlled trial. This
article describes the method developed for this realistic evaluation, the
mechanisms behind the integration of linked specialist workers and
discusses practical and theoretical issues arising from the use of the realistic
evaluation framework as a way of explaining the results of trials and service
development. Retrospective interviews identified the important outcomes
and were used to construct ContextMechanismOutcome configurations.
The 12 case studies represented what had happened. A second-level
analysis using analytic induction developed middle range theories designed
to be of value to those developing care elsewhere. The intervention was
successful in stimulating productive joint working, through case discussions,
but often failed to ensure a review of progress.

K E Y WO R D S : long-term mental illness; process evaluation; realistic

evaluation; shared care

Introduction
In recent years there has been an increasing interest in theory-based formative
and summative evaluation particularly in educational and criminal justice
research (Connell and Kubisch, 1998; Pawson and Tilley, 1997). Chen and Rossi
(1983) were among the first to advocate investigating whether the theoretical
assumptions underpinning a programme are responsible for changes that may
occur. On the other hand, the success of randomized controlled trials (RCTs) as

69
06 Byng (bc/t) 30/3/05 3:26 pm Page 70

Evaluation 11(1)
a means of determining the effectiveness of medicines has led to health services
research being dominated by a preference for trials focusing on whether or not
an outcome was achieved. In recognition of the complexity of healthcare
delivery, the UK Medical Research Council produced guidance on the develop-
ment and evaluation of complex interventions (Campbell et al., 2000). While
providing excellent advice on how to do cluster-based RCTs, it does not discuss
the need for a variety of approaches to achieving useful formative, process or
summative evaluations of complex interventions. In spite of this preference for
trials, qualitative methodologies have become more acceptable to healthcare
academics and practitioners. They were advocated primarily for descriptive
work, but more recently their place in evaluation has been recognized in a Health
Technology Assessment review (Murphy et al., 2001).
The Mental Health Link (MHL) programme was designed to improve the care
of patients with long-term mental illness and had been subjected to a random-
ized controlled trial. This article explores how a relatively low-cost evaluation,
using qualitative methods and Pawson and Tilleys realistic evaluation (RE)
framework (1997) can both help explain the results of the trial and provide gener-
alizable conclusions about service development of relevance to practitioners and
policy makers.

Background

Primary care and long-term mental illness Patients with long-term mental
illness suffer considerable disability and their care is high on the policy agenda of
the UK National Health Service (DOH, 1999). Care for this group of patients is
not highly developed within primary healthcare (Kendrick et al., 1994) and there
is also little evidence to support specific components of service development
(Burns and Kendrick, 1997; Kendrick et al., 1995).
In the UK, primary care is based around multi-disciplinary general practices
led normally by doctors; each practice may have between 1500 and 16,000
patients registered. Specialist mental-health services include hospital care and
community mental-health teams. There are a number of recommendations for a
range of developments in practice systems to support chronic disease manage-
ment and for joint working between the two systems within the National Health
Service (DOH, 1999; Joint Royal College Working Group, 1993; Burns and
Kendrick, 1997). The complexity of mental-health service provision does not
lend itself to simple uni-dimensional interventions because components such as
systems for recalling patients and placing link workers with practices are poten-
tially interdependent (Grimshaw et al., 1995). MHL was developed in response
to this (Byng and Single, 1999).

The Mental Health Link intervention An iterative process was used to develop
the MHL intervention and the conceptual framework underpinning the model of
shared care (Byng and Jones, 2004). There is a clear theory outlining how a
variety of service developments could bring about improved care and ultimately
better health. The direct components of the intervention included the facilitation,

70
06 Byng (bc/t) 30/3/05 3:26 pm Page 71

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
a handbook and payment; indirect components included the different types of
service development encouraged. A facilitator works with a small group of pro-
fessionals from each general practice and associated community mental-health
team. They examine local interests and needs and then work through a series of
options for shared care, paying specific attention to the role of the linked specialist
worker, methods of communication, documentation of responsibilities and plan-
ning the systems within the general practice to support shared care and chronic
disease management. The intervention explicitly avoids a standard model; it recog-
nizes that the differing needs, structures and interests of practices and community
mental-health teams will require different activities to achieve progress towards
the shared outcomes of improved physical and mental health. The facilitator is also
available to support the practice in the development of systems. The practices were
paid up to 2000 for staff to attend the meetings and develop systems.

The randomized controlled trial Twenty three practices were recruited for entry
into the trial which gave mixed results and no definitive conclusions about the
effectiveness or cost-effectiveness of the intervention. There was a mean direct
cost per patient of only 63 over the three-year period for the addition of MHL.
While there were statistically significant improvements in service development
and practitioner satisfaction, these were not translated into improvements to pro-
cesses of physical and mental healthcare as documented in the notes. On the other
hand, lower patients relapse rates were seen for intervention patients (Byng et
al., 2004). This could have been the result of improved informal shared care
developed through improved working of specialist professionals with general
practices. In order to make sense of these findings it was necessary to understand
the active components and mechanisms behind the intervention as a whole.

Developing the Realistic Evaluation Method

Aim of the Realistic Evaluation
There was a need to understand, rather than simply describe, both the contents
of the black box of the MHL intervention and the process of normal service
development.
The first aim was to provide a sufficiently detailed description and analysis of
the causal origins of key intermediate service developments, such as registers and
link working and also of improvements in care itself.
The second aim was to carry out a further analysis of these findings in order
to provide some insight in the form of middle range theories (Pawson and
Tilley, 1997) about how the intervention worked and, as guidance to others,
about how similar interventions or service developments might be of value; in
other words, to provide some generalizable findings.

Theoretical Considerations

A critical realist approach The research was concerned with real services and
real people. The intervention was based on a belief that the systems involved are

71
06 Byng (bc/t) 30/3/05 3:26 pm Page 72

Evaluation 11(1)
complex and holistic in the sense of involving layers of reality and that these
layers of interest are real if not always tangible. We wanted to understand the
links between components of these layers of social reality and were less interested
in a descriptive account of practitioners feelings and perceptions, which a post-
modern approach might have sought.
Critical realism as espoused by Bhaskar (1998) and others has a universal
theory of causation based on generative principles. It is the temporal conjunc-
tion of causative powers, which brings about the regularities seen in societies, or
the transformation of such societies. The potential mechanisms of causation
residing in both actors and society are real and present even when not active, and
when actualized may or may not be observable (empirical). Whether or not an
outcome or regularity occurs is determined by the interplay of positive and
countervailing mechanisms. Critical realists recognize the importance of both
individual agency and the influence of the structures and culture of society.
Critical realism therefore has a philosophical stance in keeping with the study
of very real but complex and interacting phenomena involving individuals and
society. It provided a basis to help describe how and why a complex social inter-
vention did or didnt work. Pawson and Tilley have developed a framework for
evaluating social programmes based on a realist perspective (1997). This
promised to facilitate the description of how different layers of social reality
interacted in the presence of an intervention, as a means of evaluating the inter-
vention. Phenomena such as individual thoughts and actions, team culture, inter-
agency working, financial incentives and policy might all have a part to play in
the development or otherwise of improved systems of care.
The generative theory of causation, in which the objects under consideration
undergo a transformation, and where temporal conjunction or interaction of the
various causal powers is crucial, is a fundamental element of critical realism. In
experiments all other potential causes are equalized across groups but not often
measured; only the impact of the intervention the black box is assessed. In
statistical modelling the influence of potential mechanisms (the independent
variables in regression analysis) is estimated, but only when interaction terms are
introduced is conjunction considered. In complex (social) situations the number
of interaction terms that would be required to make up the whole story would
be too many to deal with.

Pragmatic Considerations
Operational and structural context and constraints influenced the choice of
method. The MHL intervention operated at the level of the general practitioner
(GP) practice. More macro considerations, such as policy and specialist provider
culture, as well as more micro phenomena, such as individual practitioners and
patients beliefs, were likely to affect the working of the intervention; but the
intervention was primarily about systems within the practice and relationships
between the practice and other organizations.
The principal actors were the practitioners within the practice unit and the
associated community mental-health team. Their thoughts and actions occurred
over a considerable period of time and in many settings before the outcomes of

72
06 Byng (bc/t) 30/3/05 3:26 pm Page 73

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
service development and improvements in care became apparent. It would not
be sufficient to study the process of the initial meetings.
This need, to treat the practices as the main unit over a period of time, led to
a decision to carry out a multiple case study based on interviews. Case studies
have been advocated as suitable means of determining the value of interventions
in complex circumstances (Keen and Packwood, 1995). Pawson and Tilleys
Realistic Evaluation would provide the framework for developing theories as to
how the mechanisms within the intervention were dependent on the local
contexts in which they operated.

Multiple Case Studies and Analytic Induction

The proposed programme could be viewed as a set of cases (the GP/community
mental-health team pairs). Yin (1984) provides a framework for multiple case
study evaluation and, like Pawson and Tilley, emphasizes hypothesis building and
testing using analytic rather than statistical generalization.
There is perhaps most agreement that case studies can be used to illustrate
processes and theories. There is also limited agreement that they can be used to
help build theories in the early stage of conjecture (Gomm et al., 2000). The
project aimed to look across cases to create generalizations in the form of middle
range theory, which would provide insights that would add to those of an experi-
enced manager or practitioner.
Case studies have been used in order to look for causes for over 50 years.
Znaniecki and others advocated case studies as the best way of definitively deter-
mining cause and effect; they used the method of analytic induction (Robinson,
1951). In its pure deterministic form, it is probably only applicable in straight-
forward and probably unreal circumstances. Cressey and Lindesmith developed
the method by studying negative cases in order to determine whether causes were
sufficient as well as necessary (Robinson, 1951). The theory was either modified
or rejected on the basis of cases that did not fit. The method has been criticized
for claiming universality while using limited sampling methods. Later adapta-
tions and additions to analytic induction have resulted in procedures that recog-
nize that the resultant models, hypotheses or theories are probabilistic rather
than deterministic.
Reframed, in the terms of an RE, postulated relationships between contexts
and mechanisms and outcomes would be likely to be of importance. The middle
range conjectured ContextMechanismOutcome (CMO) configurations
(CCMOCs) would be a form of guidance based on detailed analysis but would
not provide definitive predictions.
The previous paragraphs have outlined the rationale for the methodological
basis of a process evaluation aiming to both explain the results of the trial and
provide further important information for those involved in similar ventures.
Figure 1 shows the relationship between this and the other components of the
whole research programme. A formative evaluation was used to develop the
intervention (Byng and Jones, 2004), a summative evaluation was needed to
determine whether it had an impact at various stages of change and the process
evaluation was needed to understand how the intervention worked.

73
06 Byng (bc/t) 30/3/05 3:26 pm Page 74

Evaluation 11(1)

Outcomes assessed for

Formative
patients and practices
development of
before the intervention
the intervention

Randomization
by practice

Process
evaluation
based on RE,
MHL
drawing data
Control Intervention Intervention
from context,
intervention
and outcomes

Outcomes measured for practices

and patients at follow up

Analysis of RCT
Synthesis of RCT and controlling for
RE to explain how and baseline, to determine
why the intervention quantitative impact of
brought about any the intervention
measured change

Figure 1. Placing the Realistic Evaluation in the Context of the Formative

Development of the Intervention and the Controlled Trial

74
06 Byng (bc/t) 30/3/05 3:26 pm Page 75

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
Methodology
The design was a multiple case study based on retrospective interviews with key
informants. The RE framework underpinned the data collection and analysis.
Analysis initially focused on constructing case studies of practicecommunity
mental-heath team pairs. The second-level analysis used a modification of
analytic induction.

Constructing a Sample
A purposive sample of the 12 practicecommunity mental-health team pairs was
constructed with the objective of achieving conceptual power rather than popu-
lation representativeness. The 12 intervention practices were divided equally
between substantial and low engagement with the MHL intervention; the
substantial six were selected automatically, and three of the low were selected at
random. Those three of the eleven control practices self-reporting significant
development (according to the Service Development Questionnaire in the trial)
were also selected.

Selecting interviewees For each case, participants who had recent experience of
primary mental-healthcare service development were approached for interviews.
This ensured the participants were information-rich. A combination of nurses,
doctors and managers were interviewed. Where practical, interviews were carried
out in groups except when it was known that that there were strong differences
of opinion. Others were interviewed if it became apparent that they had valuable
information.
A total of 31 individual and group interviews were completed for the 12 case
studies. Respondents included 21 GPs, 8 community mental-health workers, 7
practice managers, 4 mental-health managers, 3 practice nurses, 2 psychiatrists,
a practice counsellor and a facilitator. Three individuals provided information
about two practicecommunity mental-health team pairs.

First Stage of Analysis: Development of Individual Case Studies

Identifying potential contexts, mechanisms and outcomes The first stage of the
inquiry involved generating ideas about which contextual factors were likely to
be important, considering potential mechanisms and deciding which service
development outcomes should be the focus of inquiry. These were derived from
the literature review, the initial development of the intervention, the notes made
by facilitators about each case and from discussions between the facilitators and
the research team.
The outcomes consisted of three overlapping groups:
those achieved and not achieved according to the Service Development
Questionnaire;
those the community mental-health team and practice had aimed for;
any additional unexpected positive or negative outcomes identified during
interviews.

75
06 Byng (bc/t) 30/3/05 3:26 pm Page 76

Evaluation 11(1)
The outcomes included components such as the presence of a disease register
or improved informal communication. The former is an example of a more
concrete structural outcome which itself has a series of possible discrete identifi-
able components. The latter is an example of a more subjective relationship-
based outcome, which had a range of associated indicators. Box 1 shows the list
of outcomes chosen to be the subject of inquiry for each pair.

Using interviews to create prototype CMO configurations Written information

on the explicit aims of the study was sent prior to interviews. The interviewer
familiarized him/herself with the case. Interviews took about 5090 minutes and
were tape-recorded. Interviewees were encouraged to talk openly and were given
permission to discuss failures and problems as well as successes. The first part
of the interview was taken up in agreeing successful, failed and unexpected out-
comes of service development. Discussion then focused on developing prototype
CMO configurations for each outcome. It included closer questioning to elicit
details of contexts or mechanisms or to elicit further mechanisms and contexts.
One of the two interviewers (RB) at times took on the role of facilitator and
also added his own perceptions about the case in question. Achieving a consen-
sus about causality was encouraged, but not forced. Rarely the interviewers were
privy to sensitive information about the case in question from previous inter-
views and this information was not shared.
At times the participants would talk more abstractly about what might have
occurred had things been different. These discussions were also encouraged with
an eye to early versions of more generalizable CCMOCs to be developed in the
second-level analysis.

Post-interview analysis Tapes were transcribed and then checked by the inter-
viewer. The transcribed interviews were then entered in the Atlas/ti software
(Scientific Software Development, 1997). Codes, based on the CMO notation,
were assigned to each relevant piece of interview data. Outcomes were coded as
higher-level outcomes or as more specific indicators of these higher-level out-
comes; for example increased telephone contact by the link worker was an
indicator of improved informal communication. Mechanisms could be positive,
absent or negative. Memos were written throughout the coding process to record
emerging conceptual links and other observations about the data. The prototype

Box 1. Intermediate Service Development Outcomes Investigated for Each Case

Placement or allocation of a link worker for the practice

Development of active integrated link working
Development of a disease register for long-term mental illness
Development of a database of information about each patient with long-term mental
illness
Improvements in physical healthcare provision
Improvements in mental healthcare provision

76
06 Byng (bc/t) 30/3/05 3:26 pm Page 77

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
CMO configurations derived during the interviews were developed by examining
the data from all the interviews for each pair. These were drawn as diagrams in
the network function of the software (Scientific Software Development, 1997)
and a commentary was written explaining the diagrams for each higher-level
outcome. These made up each written case study.

Second-level Analysis: Generalization across Cases

The next stage aimed to generalize from the specifics of individual cases to
develop middle range theories. The analysis used a systematic approach to
determine CCMOCs, which represent how the MHL intervention and other
mechanisms worked across the cases. They are also reframed as hypotheses, or
conjectured middle range theories potentially of use to practitioners and policy
makers in the future. This transition from case specific CMO to cross case
CCMOC and then on to middle range theory represents the shift to more gener-
alizable theory, and corresponds to the cumulation discussed by Pawson and
Tilley (1997). This move from the specifics of individual cases to more abstract
theory is depicted in Figure 2, informed by Figure 5.1 in Realistic Evaluation
(Pawson and Tilley, 1997).

Middle range General statement derived from the CCMOC

conjectured theory of interest to others beyond the MHL site

Conjectured theory CCMOC representing how a mechanism

about how the MHL of the MHL intervention was contingent
intervention affected on certain contexts. Helps explain results
outcomes across the of trial and feeds back to improve
trial site intervention

Proposed explanation
about how a specific CMO relating to one outcome in one case
outcome came about

Individual quote from Coded as context, mechanism or

interview outcome

Figure 2. The Transformation of Individual Items of Data into Generalizable Theory

77
06 Byng (bc/t) 30/3/05 3:26 pm Page 78

Evaluation 11(1)
Stage I: Data reduction The detailed procedure is depicted in Figure 3. The
specific outcome and mechanism codes were grouped to form high- and inter-
mediate-level codes. Individual lower-level codes could come under more than

Stage I: Data reduction

Construct a reduced dataset of contexts,
mechanisms and outcomes using intermediate-
and low-level codes in matrix form.
Select an intermediate outcome of interest (O).

From the reduced dataset select the mechanism

(M) that is most often associated with this
outcome.
(The combination is an MO dyad.)

Stage II: The study of positive cases

Identify other positive case studies for the MO
and identify consistently occurring contexts or
additional interacting mechanisms.
Add context to create a CCMOC (it may consist Check for face
of two mechanisms rather than a context). validity against
Continue until all potential contexts and original transcripts
mechanisms have been assessed. and case studies to
ensure that there is
good reason to
Stage III: The study of negative cases believe that the
Look through negative (with respect to outcome identified context
of interest) case studies for contexts or additional or mechanism is
negative mechanisms that might explain failure contingent.
of the outcome.
Add to the prototype CCMOC/develop negative
CCMOC/reject CCMOC.
Continue until all potential contexts and
additional mechanisms have been assessed.

Select next most common MO to restart

procedure.

Select next intermediate outcome of interest and

repeat procedure.

Figure 3. The Procedure for Second-Level Analysis

78
06 Byng (bc/t) 30/3/05 3:26 pm Page 79

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
one intermediate-level code. The same phenomenon could be coded as an
outcome or context, or as an outcome or a mechanism. For example a disease
register was an outcome of service development and could then act as a mechan-
ism for improving care.
Three high-level outcome codes were created: integration of linked specialist
workers, development of systems in primary care and improvements in care.
The codes for mechanisms included those that were specific to the MHL inter-
vention; those components of service development which had been encouraged
as part of the MHL intervention but which could have occurred in its absence;
and a miscellaneous group.
To facilitate the analysis, the coded data were reduced, via reference to the
CMOs in the case studies, to a tabular form for each higher-level outcome under
consideration. This matrix allowed a visual overview of cases, mechanism,
outcomes and contexts.

Stage II: The study of positive cases For each active mechanism identified as
being associated with a positive outcome other cases with a positive outcome
were examined. Theoretically, single mechanisms could have a 100 percent
association with a positive outcome. Positive cases were examined to look for con-
texts and additional mechanisms that were associated with the MO being
studied. Finding such associations provided evidence that the mechanism was
contingent on the context or additional mechanism. This led to the development
of the CCMOC. Cases with positive outcomes where the mechanism was not
operational were taken as evidence that the mechanism being studied was not
necessary.

Stage III: The study of negative cases The study of negative cases was used
firstly to adjust emerging CCMOCs and secondly to develop CCMOCs based on
failed outcomes; thirdly as a reason to abandon the CCMOC development.
Comparison of negative and positive cases provided evidence for adjusting
CCMOCs. The association of the mechanism in question with a negative
outcome proved that the mechanism alone was not sufficient.
For failed outcomes, prohibitive contexts and negative mechanisms were
identified that were associated with the failed outcome. In the individual case
studies, absent mechanisms had been identified; in this secondary analysis the
absence of a specified mechanism in a case study could contribute to develop-
ment of a CCMOC based on a positive outcome. In some situations these
positive CCMOCs were constructed entirely on evidence from case studies with
failed outcomes and missing mechanisms or negative contexts. These
CCMOCs represent what might be, as described by Schofield (1990).
Frequently, the CCMOCs could not be developed beyond prototype form and
were not included in the results. In some cases potential contingent mechanisms
or contexts could not be identified to explain why a mechanism was associated
with an outcome in some situations but not others. This could be due to the
paucity of data regarding potential contingent contexts or due to inconsistency
of the data and lack of clear associations.

79
06 Byng (bc/t) 30/3/05 3:26 pm Page 80

Evaluation 11(1)
We referred back to the detailed case studies and original transcripts, to ensure
the CCMOC retained validity with respect to the interview data. Furthermore
the memos detailing possible CCMOCs made during the analysis of the original
transcripts were also used as starting points for the analytic induction process.
These latter two explicitly subjective judgements made by the researchers
complemented the logical analytic induction (AI) procedure. This inductive
process was continued by examining all nine or twelve cases until all the
CCMOCs were developed or discontinued.
If active mechanisms were associated with both positive and negative outcomes
and no additional mechanisms or contexts were identified which explained the
difference in outcome then the development of the CCMOC was also ended.

Results
Results are presented below in order to illustrate the process of analysis and to
provide examples of how both parts of this process contribute to new under-
standing. The roles of the MHL intervention and mechanisms of normal service
development in bringing about integration of the link worker are examined. This
article does not present a detailed analysis of the development of systems within
primary care or of improvements in mental and physical care.
The nine cases that had received the intervention showed considerable hetero-
geneity in their contexts, activities and outcomes. There were small practices with
little interest in mental health and large practices with emerging primary care-
based mental-health teams; in some practices the facilitators had spent consider-
able time supporting motivated practitioners, while in others the facilitators and
practices only worked together briefly. In only two case studies were there signifi-
cant discrepancies between respondents.
In the first section, one case study illustrates the extent of the detail and depth
reached, as well as the use made of the theory-building capabilities of the
software (Scientific Software Development, 1997). The diagrams (Figures 4 and
5) are copied directly from the software and a legend (Table 1) is provided for
interpretation of the shorthand notations.
The case studies represent an understanding of what happened in the past. The
second-level analysis distilled this understanding across cases into conjectured
theory, which may be of use to others in the future. The second section is
primarily a worked example of the analytical induction procedure.

Worked Example I: Development of a Case Study Integration of a

Link Worker

Background Case F includes a medium-sized GP practice in an inner-city

setting with a high prevalence of mental illness. The practice has six GP partners
and a relationship with one community mental-health team.
The case study included a joint interview with the GP who had led on the project
and the practice manager who had not been involved. Other interviews were held
individually with the link worker and a second GP with an interest in mental health.

80
06 Byng (bc/t) 30/3/05 3:26 pm Page 81

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
Integration of the link worker into the primary healthcare team The placement
of the link worker into the team was stimulated by the MHL project. The MHL
facilitation meetings provided an arena for eliciting ideas and discussing the role
of a link worker and how it might improve the already good relationships. In the
context of being actively supportive of link working and their good relationships
with the practice, the community mental-health team manager agreed to try out
an aligned case load model, with the link worker preferentially taking on cases
from the practice, as recommended by the MHL project.
The integration (Figure 4) was embodied by and also enhanced through
positive feedback by regular attendance at Primary Health Care Trust (PHCT)

M(i) - LW acting as conduit M(i) - LW/PBCPN discusses

M(i) - LW from ABT for communication with patients with problems
is allocated practice care co-ordinator identified by practice
patients preferentially

M(i) - LW promotes M(i) - LW liaises one to one with

M(i) - LW allocated and maintains informal PHCT member as problems
protected time contact with practice arise from CMHT perspective

C(c) - support of
CMHT/Trust managers
for link working
C(i) - LW/PBCPN has
extensive experience
C(prac) - helpful
reception staff
C(prac) - practice
encourages joint working
C(prac) - practice
prioritizes mental health
=> =>
M(prac) - practice invite M(i) - LW/PBCPN follows
LW to attend meetings up identified problems
and gatherings
M
M => M =>
M
M
M(i) - mutual trust enhances
C communication between
M clinicians and within teams
C
M =>
M
C positive feedback loop
=>
C O - integration
Oi - relationship of
of LW/PBCPN ind
C trust established
into PHCT

C ind
C(prac) - regular Oi - improved rate
PHCT meetings neg ind ind ind and speed of
informing each other

Ofi - failure to develop

an informal relationship Oi - community nurses Ou - primary care
between practice meet/communicate influences specialist
nurses and link nurses with link worker decisions

Notes:
ABT: Assessment and Brief Treatment (team)
CMHT: Community Mental-Health Team
LW: Link Worker
PBCPN: Practice-based Community Psychiatric Nurse
PHCT: Primary Health Care Trust

Figure 4. Integration of Link Worker into Primary Care Team

81
06 Byng (bc/t) 30/3/05 3:26 pm Page 82

Evaluation 11(1)
Table 1. Legend for Figures 4 and 5
Notation for Coding Labels for Links
C(c) community mental-health team context C context contributing to
C(gp) context relating to general practitioner M mechanism contributing to
C(i) interface context Ab-M absent mechanism so no
C(prac) practice context contribution to
M(i) mechanism acting at interface Cneg negative context contributes
M(prac) mechanism acting at practice to failure to bring about
MM missing mechanism an outcome
MM(iMHL) missing mechanism related to Mental ind indicator of
Health Link intervention neg ind indicator not achieved
O higher-level outcome => leads to
Ofi explicit absence of indicator of outcome
Oi indicator of higher-level outcome
Ou unexpected outcome

meetings, the establishment of trust, improved speed and rate of informal

communication. All those interviewed viewed the link working as positive and
saw the link worker as having become part of the PHCT.
The practice welcomed the link worker who was invited to meetings; the
community mental-health team provided protected time to attend the meetings.
The integration occurred in the context of good prior relationships between the
community mental-health team and practice, a high prevalence of long-term
mental illness, the practice overtly prioritizing mental health and also having a
GP with experience of working as a psychiatrist.
The main mechanisms of integration, in terms of developing trusting relation-
ships, were attendance twice monthly at practice meetings, sometimes for only
part of the meeting, and availability for one-to-one case discussions before and
after meetings often by phone. Individual patients of concern were brought up
by practice members; after discussion the link worker followed up the problems
with the community mental-health team. The link worker would discuss and
provide welcome feedback on new referrals to the service. The link worker would
act as a conduit for communication between the GPs and care co-ordinators.
These mechanisms were contingent on the dedication and flexibility of the link
worker who was able to respond to the needs of primary care. They were also
reliant on the link worker being supported in the work and being given dedicated
time by her manager.
The only aspect of integration not achieved was the development of a relation-
ship with practice nurses who may not even have known about the link worker
(Interview with general practitioner 7). This was in the context of the nurses not
attending the PHCT meetings. There had been no review of progress of inte-
gration by the MHL facilitator; and the practice had not found a way of inte-
grating practice nurses more fully into the PHCT, despite recognizing this as a
more general problem for their team (see Figure 5).

82
06 Byng (bc/t) 30/3/05 3:26 pm Page 83

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness

M(i) - LW/PBCPN attends MM - no mechanism

regular PHCT meetings to promote alternative
model for building
relationship between
C(prac) - practice nurses practice nurses and LW
give depot injections

C(prac) - regular
M MM(iMHL) - lack
PHCT meetings
of monitoring of
Ab-M plans and
C
reviewing situation
C(prac) - nurses do not C
attend PHCT meetings
C-neg Ab-M

C(gp) - GP concerned that

proactive health promotion Ofi - failure to develop an
could harm relationship C-neg informal relationship between
with patient practice nurses and LWs

Notes:
LW: Link Worker
PBCPN: Practice-based Community Psychiatric Nurse
PHCT: Primary Health Care Trust

Figure 5. Failure of Link Working with Practice Nurses

Results of the Second-level Analysis: the Integration of Link Workers

The second-level analysis was centred around the analytical induction procedure
but effectively commenced during the construction of the case studies. The
impact of the intervention was variable, although in a minority of practices there
were very positive stories about link working. The RE interviews showed that
for most practicecommunity mental-health team dyads some level of shared
activity occurred between the link worker and the practice following the MHL
meetings, but this did not always last more than a few months or meetings.
Furthermore apparently strong link working relationships also ended.
The integration, as well as being embodied by improved trust and informal
communication, was represented by the continuation of the shared activities. The
data reduction exercise showed that the most prominent mechanism was the
discussion of individual clinical cases jointly by the link worker and the primary
care clinicians. The case discussions took many forms, including informal tele-
phone contact, planned review of patients with psychosis and discussion of
concerns which had not reached crisis level. Other activities, included in the
range of options recommended by the facilitators, such as attending practice or

83
06 Byng (bc/t) 30/3/05 3:26 pm Page 84

Evaluation 11(1)
PHCT meetings, providing training and helping develop systems, were less
common.
The shared activities were both immediate outcomes of the MHL intervention,
but also mechanisms of the MHL intervention, in that they were intermediates
in the path towards integration of the link workers. Furthermore they are also
generic mechanisms and outcomes of normal service development in the absence
of the MHL intervention.
Perhaps the other most important qualitative observation was the circularity
and positive feedback; when shared activities were deemed worthwhile this
naturally led to increased trust and an increased willingness to participate
further. The shared activities led to, represented and were the result of
integration. The opposite was also true; disappointment and lack of trust led to
negative feedback and less joint working.
The next stage in the analysis was to determine on which contexts, or with
which additional mechanisms, the key mechanisms of the MHL intervention
were contingent. At this point the analytical induction procedure came into play.

Worked Example II: The Procedure to Develop CCMOCs about

Integration of the Link Worker
The following worked examples demonstrate the application of the procedure,
described above, employed to develop middle range theory from the highly
specific coding making up the individual case studies.

How did integration come about? Stage I: Data Reduction. A variety of specific
low-level codes were used in the individual case studies to represent the concept
of integration. The matrix for integration included the extent of integration, key
MHL mechanisms, the joint activities and lastly the possible relevant contingent
contexts. Examination of the matrix showed the intermediate-level mechanism
most associated with the positive outcome of integrated link worker was joint
discussion of individual cases. This mechanism was therefore chosen, in associ-
ation with the outcome of integration, as the first MO dyad to be introduced
to the AI process as a means of determining how and why integration might come
about.
Stage II: The Study of Positive Cases. There were five intervention and two
control cases in which full or partial integration of a linked specialist worker was
achieved. The examination of contexts associated with the MO did not reveal any
surprises. The practices were all described by the community mental-health team
and by themselves as having an enhanced interest in mental health. They were
described as being interested in collaborative work. The link workers that had
been fully integrated were all described as being both flexible and experienced.
The analytic induction process so far showed there was an association between
joint case discussions, encouraged by the MHL programme in intervention prac-
tices, but occurring sporadically elsewhere, and integration into the primary care
team. Incorporating the contexts led to the formation of a prototype CCMOC
representing what happened in the MHL intervention. The CCMOCs are
presented in structured tables (see Tables 2, 3 and 4 below) as described in

84
06 Byng (bc/t) 30/3/05 3:26 pm Page 85

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
Realistic Evaluation (Pawson and Tilley, 1997). Those mechanisms related to the
MHL intervention are marked with an *.
A further mechanism, problem solving by the linked worker, was associated
with more satisfaction and a belief that care had been substantially improved in
all five cases where ongoing case discussions occurred. The transcripts suggested
an association between the link workers personality traits and problem solving
and not between problem solving and the MHL intervention (see Table 2).
There were no positive cases of integration in which the discussion of indi-
vidual cases did not occur. This was evidence to suggest that case discussions
were necessary (or at least very important) for integration of the link worker.
Stage III: The Study of Negative Cases. Negative cases included those receiv-
ing the MHL intervention, but in which integration had not occurred. In four
practicecommunity mental-health team pairs, despite initial case discussions
about individual patients, integration was not achieved. In one case discussions
ceased but trust and improved informal communication continued.
Three cases added support to the hypotheses developed during examination
of the positive cases. In two cases the link workers were not highly motivated to
do link work. In two the practice GPs were not highly motivated to collaborate
nor to improve mental healthcare.

Table 2. Integration Further Development of Trust

Context Mechanisms Outcome
Sufficient practice belief in joint Shared discussion of Integration further development
working and need to improve individual cases* of trust and informal
mental healthcare + communication
Flexible and experienced link Problem solving by
worker the link worker

Table 3. Unproductive Joint Working

Context Mechanism Outcome
Experienced link worker and Poor analysis of the situation by MHL Unproductive
motivated primary care team facilitator* joint working
+ +
Critical mass of belief in joint (Poor decision making about role of
working link worker by joint working group*)

Table 4. Integration by Matching Link Worker to Practice

Context Mechanism Outcome
Community mental-health team Selection of link worker following joint Integration
manager willing to deploy working group (assessment of
link workers according to practice needs)
practice need

85
06 Byng (bc/t) 30/3/05 3:26 pm Page 86

Evaluation 11(1)
In three cases the joint discussions did not continue and were not considered
to have been useful. Either the wrong cases had been brought to discussion or
the wrong clinicians had been there to discuss the cases.
In one case, the practice (which was served by two community mental-health
teams) and link worker from the continuing care team had too little in common
to warrant regular meetings. In another practice the lead GP had arranged to
have meetings with the link worker and practice nurses without the other
partners, in order to protect their workload; the other GPs therefore never had
the chance to discuss difficult cases and develop a relationship with the link
worker. In both these practices the decision made during the facilitation about
who the link worker should meet and which patients should be discussed had not
been appropriate. In each of these cases there had been a poor decision about
what the link working would consist of.
These negative cases added one word to the CCMOC developed above. The
case discussions had to be perceived as productive. They also highlighted the
importance of a good analysis of the situation one of the core components of
the MHL facilitation. This led firstly to the minor adaptation of the CCMOC
above by inserting the word productive in front of shared discussion. It also
led to the formation of a second CCMOC (see Table 3) based around the MHL
mechanism of analysis of the situation.
The CCMOCs in Tables 2 and 3 aim to represent what occurred across the
MHL trial sites. They are reframed below into statements in italics encapsulat-
ing the most important findings of use to managers and practitioners elsewhere.

Productive discussion of individual clinical cases and problem solving by a

linked specialist mental-health worker leads to integration into the primary
care team.
Failure of an intervention to facilitate analysis of the situation can lead to
plans and shared working experience which are unproductive and do not
lead to the positive feedback required to ensure continued joint working.

Less well developed and abandoned CCMOCs Sometimes hypotheses devel-

oped by researchers during the project, which appeared to be supported by
common sense, remained at an early stage of development; for example, with no
specific contingent context identified. In other situations they had to be aban-
doned and revised.
The MHL intervention recommended the community mental-health teams
appoint a link worker prior to the joint working group (JWG) meetings. It had
been anticipated that this would lead more readily to productive joint working.
Contrary to expectation, in two of the practices with high levels of integration
the link workers had been appointed after the JWG meetings. This led to the
abandonment of the theory (CCMOC) that prior appointment would be helpful
in any circumstance and to the untested hypothesis that the community mental-
health team managers who attended the JWG might have selected link workers
based on the needs of the practice. This could be formulated as an alternative
CCMOC, as shown in Table 4.

86
06 Byng (bc/t) 30/3/05 3:26 pm Page 87

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
Summary of impact of the MHL intervention and service development as
normal on link working The case studies provided evidence about both the
intervention and how services develop naturally. Put together the results suggest
that the MHL intervention acted directly through the facilitation to:
stimulate allocation of link workers, where this had not occurred fortu-
itously;
stimulate effective joint working in some practices;
stimulate ineffective joint work in other practices (this appears to occur
when the facilitation process failed to bring about a full analysis of the
needs for care and systems of care).
As well as acting as a stimulus for change the MHL intervention also provided
prompts for action and skills to achieve action, raised awareness and negotiated
solutions to disagreement. However the MHL intervention often failed to review
both effective and ineffective joint work adequately. This appears to have been
the major failing of the intervention. Its monitoring and performance manage-
ment function was not sufficiently developed for the adverse contexts in which
it was operating.
The second-level analysis also shed light on the naturally occurring activities
and mechanisms important to the development of link working:
effective joint working, nearly always involving discussion of cases;
continuation or further joint working occurred, as a result of positive
feedback, when the joint work was deemed to be productive.

Discussion
The Utility of the Results
The main aim of this article has been to describe the development of and
procedure for a method combining RE and analytic induction. Before discussing
this in detail it is worth examining the results and their utility. The results
described have focused on the achievement of integration of the linked specialist
workers into the primary care team. As summarized in the preceding paragraph,
the results of the second-level analysis shed light both on the mechanisms under-
pinning the intervention and also on those mechanisms operating in service
development as normal. The results include three different levels of theory, as
depicted in Figure 3.
The lowest level, in terms of theoretical hierarchy, is that of the individual case
study which represents what occurred in the past. Each explains how various
activities, policies, attributes and systems led to or did not lead to certain prede-
fined and unexpected outcomes. In this evaluation, these case studies are
primarily a staging post towards the second-level analysis. However they do
provide a description of the specific, which is of value to grass roots practitioners,
because they help place the context of the evaluation; they are also of value to
those working at policy level who may be less aware of the impact of the messy
and complex world of practice. There is a further potential value of the individual

87
06 Byng (bc/t) 30/3/05 3:26 pm Page 88

Evaluation 11(1)
case studies; the research process and results together appeared to the inter-
viewers to stimulate reflection amongst the interviewees, those responsible for
development at the practice and mental-health team level. We would suggest that
this could be formalized, using a similar interview method, in an action research-
type evaluation, by linking the reflection and results to the next stage of planning
for the individual practices.
Secondly the CCMOCs portray what occurred across all the cases. This
provides a basis for understanding the results of the trial as well as for what might
lead to positive outcomes in the absence of an intervention. Had the MHL
programme been ongoing these cross-case CCMOCs would have provided useful
formative information for improving the programme as a whole.
The results of the third theoretical level, which we suggest can be classified as
middle range theory, are an extrapolation of the second, reformulated to repre-
sent conjecture about what might be rather than what was. These results are
designed to be of value to those outside the immediate setting of the trial, provid-
ing evidence about the sort of activities that, in different contexts, might lead to
positive outcomes.

Methodological Critique
There is no predefined protocol for critiquing the methodology, based on the RE
framework; but developed for this project, there is guidance for qualitative
research (Mays and Pope, 2000). This section starts, however, in a more reflec-
tive way, by considering how the use of the RE framework worked in combi-
nation with analytic induction applied to multiple case studies.

Uses of the contextmechanismoutcome configuration While carrying out the

interviews it became obvious that for each outcome being studied there were
many potential ways of constructing CMOs. The first issue was the multiplicity of
contexts and mechanisms potentially involved in bringing about an outcome. This
was very different from the diagrams, so dominant in Realistic Evaluation, where
single mechanisms and contexts were brought together to illustrate the theoreti-
cal framework (Pawson and Tilley, 1997). One of the reasons for this was the
explicit need for the intervention to adapt to local circumstances in its pursuit of
common goals. This heterogeneity made the search for recurring CMOs more
daunting but also more important.
Sometimes the big picture was depicted by multiple Cs and Ms. This gave
a useful holistic picture, but then the components could be separated into more
specific simpler CMO configurations where the mechanism(s) were more
specifically contingent on each of the contexts depicted. This emphasis was a
more faithful application of RE as a theory. And yet the more holistic picture
also appeared to be of value.
Possibly the most important variation from the basic CMO structure was the
positive or negative feedback loop, where an outcome feeds back to interact with
the original mechanism. Positive feedback loops were described in the results,
but negative feedback was also observed. For example, when a practitioner strug-
gled alone in a GP practice to introduce more pro-active care (M), without

88
06 Byng (bc/t) 30/3/05 3:26 pm Page 89

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
enrolling the support of her colleagues (C), a failed attempt at introducing
change (O) led her to be more reluctant to try again in a different way or to even
to mention the project, with its negative associations, to colleagues. It had started
to become what we have termed a heart sink project.
The feedback loops have much in common with systems thinking and complex-
ity science (Plsek and Greenhalgh, 2001) where local interactions and positive
feedback are often the basis of emergence and account for important and at times
unexpected outcomes. RE does not explicitly discuss the importance of these
interactions between mechanisms or feedback loops, whereas in the original
realist writings of Bhaskar (1998) and others they are seen as fundamental to
emergence.
Another pattern was that of one CMO configuration being dependent on a
prior CMO configuration. For example the practice-based register of patients
with long-term mental illness was an outcome of interest and then became a
context in which various mechanisms could act to bring about improved reviews
of care. Thus complex causal networks started to emerge within some of the case
studies with O(i) becoming C(ii) (see Figure 6).
A similar variation was the CMO configuration within a CMO configuration.
The previous example was based on a temporal difference. It could be argued
that each mechanism or context involving social structures, human action and
thought is itself made up of many CMO configurations when the psychological,
then physiological and biochemical processes are considered. For example the
mechanism of discussing clinical cases is an activity which was both promoted by
(and therefore part of) and also arose from the MHL intervention. To a policy
maker, the MHL programme as a whole could be seen as a mechanism; local
managers may find it useful that case discussions appear to be an important
mechanism which can be reproduced in other circumstances. However the very
activity of discussing cases could be conceived of being made up of other CMOs,
simultaneously.
This leads us to another issue which those familiar with RE might be
concerned with. Can a visible activity such as a case discussion be considered a
mechanism, when surely the aim of RE is to surface mechanisms? Our response
is that it depends on the level of social reality that one is engaged in. For the
local manager, the case discussion could be something they were unaware of, and
therefore in need of surfacing; whereas for the research team it was firstly an
outcome, as a type of shared activity and then a mechanism for both integrating
linked workers and for improving care of patients. Our inquiry in this instance

MiMi
Mi Mii

Ci Oi=>Cii Oii

Figure 6. Series of ContextMechanismOutcome Configurations

89
06 Byng (bc/t) 30/3/05 3:26 pm Page 90

Evaluation 11(1)
was not only about surfacing mechanisms but also about identifying the context
on which previously identified mechanism and outcome was contingent.
The problem in both the interviews and analysis is one of determining how
deep to investigate these layers of social and physical reality. In the example just
given, the CMOs at the physiological and even psychological level are not of
relevance to the research question at hand.
A further, initially troubling phenomenon, was that at times it was hard to
decide whether something was a mechanism or a context. For example, was a
link worker reviewing a register of patients to determine those who needed
follow up a mechanism to ensure follow up was organized, or a context which,
together with the mechanism of the patients key worker bringing them to
primary care, ensured they were seen for physical review? Or was it a CMO
configuration in itself with the contexts being having an active link worker and
having a workable register, the mechanism being the series of actions required
to actually go through and check the notes of those on the register and the conse-
quent outcome being the identification of those needing review?
We concluded that it could be all three depending on the stage and level of
stratification of the process from which it was being viewed. This ambiguity
caused problems when attempting to depict the CMO configurations, but in
terms of understanding the causal pathways it was less relevant. Again the philo-
sophical basis of critical realism helped resolve this issue. Whereas Pawson and
Tilley tend to focus interest on one mechanism and search for contingent
contexts, Bhaskar and Archer talk more of additional mechanisms, which may
be active or countervailing (Archer, 1998; Bhaskar, 1998). As long as the CMO
notation was seen as a methodological technique, with the split between context
and mechanism as pragmatic rather than theoretical, there was less of a problem.
What was important was that the search for CMO configurations improved our
understanding of what had happened.

Interviews and case studies The most important methodological problem was
the retrospective nature of the interviews carried out up to two years following
the initial facilitation. Recall of events may be poor and distorted by time and
other events. We found that the respondents were relatively good at recalling
particularly good or bad outcomes, but were less able to recall, for example, the
nuances of how a facilitator had worked. While the discussion of negative out-
comes had highlighted missing mechanisms, such as the paucity of the review or
evaluation function, recall by interviewees of how much negotiating solutions,
encouraging decisions and analysing situations went on was at times poor.
Recording the JWG meetings and using discourse analysis might have been a
better method for determining the extent and value of these important mechan-
isms. A series of interviews, carried out during the trial, would have been required
to better determine the ongoing work of facilitators. This however would have
almost certainly augmented or distorted the intervention and would not have
been acceptable in a trial context, unless the RE, as an evaluative mechanism,
had been included as part of the intervention.
A further potential problem was the reluctance both of the interviewers and

90
06 Byng (bc/t) 30/3/05 3:26 pm Page 91

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
interviewees to discuss the programmes shortcomings. Interviewees might not
want to offend the researchers, and the researchers might be reluctant to bring
up both potential successes and failures. One of the interviewers (RB) was also
the originator of the programme, potentially exacerbating this issue further.
These problems were addressed by explicitly stating that we knew there had been
both successes and failures and that the main value of the interview was to
improve the programme. There were also advantages in involving the person
who knew most about the facilitation in the interviews. In the event, despite this
threat to validity, the transcripts did reveal critical dialogue about failures as well
as successes; these contributed considerably by providing negative cases for the
analytic induction procedure.
Further issues relevant to qualitative research are sample representativeness
and whether saturation was reached. The sample was constructed to provide
information from a diversity of perspectives; however the last interviews were
still revealing new codes and themes so it is unlikely that saturation was reached.

Combining realistic evaluation with analytic induction To our knowledge this is

the first time RE has been combined with a formal analytic induction procedure.
The procedure includes the study of negative cases, as developed by later pro-
ponents. The negative cases were crucial to the development of theories about
what may be (Schofield, 1990). It is explicitly not designed to develop hypoth-
eses about certainties; it is probabilistic rather than deterministic.
The number of cases was small, and the number of variables large, leading to
many potential CCMOCs being abandoned at an early stage or being considered
as very early potential theories. The explicit value given to a qualitative interpre-
tation of the raw data, as well as the reduced data set, in the formation of the
CCMOC partly compensated for the small numbers. Some commentators may
believe that this combination of paradigms is unsatisfactory, whereas for us it was
a pragmatic solution to some of our problems. Inevitably it leads the second-level
analysis open to accusations of bias. We hope that the rigour with which we have
critiqued the MHL programme will limit those considerations.

Conclusion
This article provides an account of how RE was operationalized to understand
the results of a trial and how the intervention worked. To our knowledge this is
the first attempt to combine RE with a formal analytic induction procedure.
Despite the methodological limitations, the study has also led to a clearer under-
standing of RE as a practical research approach. The lessons learnt will be of
value to those evaluating service interventions in the UK National Health
Service (NHS) and elsewhere.

Acknowledgements
We are grateful for the time given by the managers and practitioners in the study. Richard
Byng is currently an NHS Primary Care Researcher Development Award holder and this
provided time for data collection and analysis.

91
06 Byng (bc/t) 30/3/05 3:26 pm Page 92

Evaluation 11(1)
Funding for the intervention as part of the controlled trial was provided by NHS
Primary Secondary Interface R&D programme based at the London Region (Formerly
North Thames Region).

References
Archer, M. (1998) Realism and Morphogenesis, in M. Archer, R. Bhaskar, A. Collier,
T. Lawson and A. Norrie (eds) Critical Realism: Essential Readings, pp. 35681.
London: Routledge.
Bhaskar, R. (1998) Philosophy and Scientific Realism, in M. Archer, R. Bhaskar, A.
Collier, T. Lawson and A. Norrie (eds) Critical Realism: Essential Readings, pp. 1647.
London: Routledge.
Burns, T. and T. Kendrick (1997) The Primary Care of Patients with Schizophrenia: a
Search for Good Practice, British Journal of General Practice 47(421): 51520.
Byng, R. and R. Jones (2004) Mental Health Link: the Development and Formative
Evaluation of a Complex Intervention to Improve Shared Care for Patients with Long-
term Mental Illness, Journal of Evaluation in Clinical Practice 10(1): 2736.
Byng, R. and H. Single (1999) Developing Primary Care for Patients with Long Term
Mental Illness Your Guide to Improving Services. London: Kings Fund.
Byng, R., R. Jones, M. Leese, B. Hamilton, P. McCrone and T. Craig (2004) Exploratory
Cluster Randomised Controlled Trial of Shared Care Development for Long-term
Mental Illness, British Journal of General Practice 54(501): 25966.
Campbell, M., R. Fitzpatrick, A. Haines, A. L. Kinmonth, P. Sandercock, D. Spiegelhalter
and P. Tyrer (2000) Framework for Design and Evaluation of Complex Interventions
to Improve Health, British Medical Journal 321(16 Sep.): 6946.
Chen, H. and P. Rossi (1983) Evaluating with Sense: the Theory-driven Approach,
Evaluation Review 7(3): 283302.
Connell, J. P. and A. C. Kubisch (1998) Applying a Theory of Change Approach to the
Evaluation of Comprehensive Community Initiatives: Progress, Prospects, Problems,
in K. Fullbright-Anderson, A. C. Kubisch and J. P. Connell (eds) New Approaches to
Evaluating Community Initiatives, Volume 2 Theory Measurement and Analysis.
Washington, DC: The Aspen Institute.
Department of Health (DOH) (1999) National Service Framework for Mental Health.
Modern Standards and Service Models. London: Department of Health.
Gomm, R., M. Hammersley and P. Foster (2000) Case Study Method. London: Sage.
Grimshaw, J., N. Freemantle, P. Langhorne and F. Song (1995) Complexity and System-
atic Reviews: Report to the U.S. Congress, Office of Technology Assessment. Aberdeen:
Health Services Research Unit, University of Aberdeen.
Joint Royal College Working Group (1993) Shared Care of Patients with Mental Health
Problems. Occasional Paper 60. London: Royal College of General Practitioners.
Keen, J. and T. Packwood (1995) Qualitative Research: Case Study Evaluation, British
Medical Journal 31(7002): 444.
Kendrick, T., T. Burns, P. Freeling and B. Sibbald (1994) Provision of Care to General
Practice Patients with Disabling Long-term Mental Illness: a Survey in 16 Practices,
British Journal of General Practice 44(384): 3015.
Kendrick, T., T. Burns and P. Freeling (1995) Randomised Controlled Trial of Teaching
General Practitioners to Carry Out Structured Assessments of their Long Term
Mentally Ill Patients, British Medical Journal 311(6997): 938.
Mays, N. and C. Pope (2000) Qualitative Research in Health Care. Assessing Quality in
Qualitative Research, British Medical Journal 320(7226): 502.

92
06 Byng (bc/t) 30/3/05 3:26 pm Page 93

Byng et al.: Using Realistic Evaluation of Shared Care for Mental Illness
Murphy, E., R. Dingwall, D. Greatbatch, S. Parker and P. Watson (2001) Qualitative
Research Methods in Health Technology Assessment: a Review of the Literature,
Health Technology Assessment 2(16): iiiix.
Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: Sage.
Plsek, P. E. and T. Greenhalgh (2001) Complexity Science: the Challenge of Complexity
in Health Care, British Medical Journal 323(7313): 6258.
Robinson, W. (1951) The Logical Structure of Analytic Induction, American Sociological
Review 16(6): 81218.
Schofield, J. W. (1990) Increasing the Generalisability of Qualitative Research, in E. W.
Eisner and A. Peshkin (eds) Qualitative Inquiry in Education: The Continuing Debate.
New York: Teachers College Press.
Scientific Software Development (1997) Atlas/ti Text Interpretation, Text Management
and Theory Building. Scientific Software Development.
Yin, R. K. (1984) Case Study Research: Design and Methods, 2nd edn. London: Sage.

R I C H A R D B Y N G is a lecturer at the Department of General Practice and

Primary Care, Kings College London, UK. He is also a general practitioner in
Plymouth. His interests are primary mental healthcare and evaluating innovation
in healthcare. Please address correspondence to: Department of General Practice
and Primary Care, GKT School of Medicine, 5 Lambeth Walk, London SE11 6SP,
UK. [email: richard.byng@kcl.ac.uk]

I A N N O R M A N is Professor of Nursing and Interdisciplinary Care, School of

Nursing and Midwifery, Kings College London, UK. His interests include
interdisciplinary working and mental health nursing.
[email: ian.j.norman@kcl.ac.uk]

S A L LY R E D F E R N is Emeritus Professor, Nursing Research Unit, School of

Nursing and Midwifery, Kings College London, UK. She has interests in old age
nursing and evaluation. [email: sally.redfern@kcl.ac.uk]

93