UK EVALUATION SOCIETY

MIDLANDS REGIONAL
NETWORK

June 2018 Edition Your Network

We are an inclusive network, supporting
Welcome to our second network update. We have had a lively evaluators and sharing evaluation
past three months with the UKES Annual Conference in May and practice in the Midlands.
our evaluation exchange in March. Summaries of both of these
Please share news of this network and
events are contained in this update.
this update with others who may be
interested. We would also be delighted
We would like to thank those of you who joined us at to hear from potential network
our evaluation exchange on Realist Evaluation in March 2018. contributors.
Special thanks go to Winning Moves who hosted and sponsored
The Network Convenor is Karl King of
refreshments at this event. It was great to see new and old faces
Winning Moves and can be contacted
at this event. Our next event will be an Evaluation Exchange on
at karlk@winningmoves.com
the 17th July 2018, more information can be found on page 6 or Contributions for network updates can
http://bit.ly/MRNJuly. be sent to Tracey
Wond t.wond@derby.ac.uk

More about your network can be found

Join our LinkedIn group! on the UKES website, here.

Keen to make it easier for Network members to connect

with each other, we have set up a LinkedIn Group, which What's Inside...
you can join here: http://bit.ly/UKESMRNLinkedIn
Join our LinkedIn group - p.1
We will continue to share updates via email, but the
LinkedIn Group will provide an opportunity for members Summary of our March evaluation
and interested parties to interact with each other in exchange - p.2
between our events. Please feel free to post topical Our next event - p.3
articles of relevance to members, opportunities to tender,
jobs, questions and evaluation-related musings as you GDPR Notice - p.3
please.
Summary from national UKES
conference - p.4-5
We look forward to connecting with you more frequently!
Job opportunity - p.6

JUNE 2018  NETWORK UPDATE  1

NETWORK UPDATE JUNE 2018 

EVENT SUMMARY
LET'S GET REAL!
AN EVALUATION EXCHANGE ON REALIST EVALUATION
Our first event of 2018 was an evaluation exchange on Realist Evaluation, facilitated by our
Network Convener Karl King (Winning Moves). The session kicked off with an opportunity for
the nine Network members in attendance to discuss why they were attending the event and
what they hoped to get out of their attendance. The majority of the group had some
knowledge and understanding of the philosophy and principles of Realist Evaluation, but had
limited to no experience of conducting realist evaluations, so were interested to find out more
about how these could be put in to practice.

Karl started by giving a brief overview of realist evaluation, discussing the underlying
philosophy, key principles and types of evaluation question it is intended to answer, before
discussing some of the key insights he had drawn from practical experience of working on
realist evaluations at Winning Moves. This led to a group discussion of how to conceptualise
mechanisms in realist evaluation, ways in which theories could/should be constructed,
presented and tested, and how realist evaluations can be explained to those who may not be
familiar with the philosophy, principles and associated jargon.

Feedback received immediately following the event was positive, with everyone in attendance
saying they had found it interesting and useful. The session closed with an open discussion of
themes and topics for future evaluation exchanges. These were noted by the steering group
for inclusion in future events.

We are aware that some of you with an interest in realist evaluation were unable to attend the
event in March. If you were interested in the event, but were unable to attend, please get in
touch with karlk@winningmoves.com. If there is sufficient interest in running further events on
realist evaluation / putting the philosophy/principles into practice, we will seek to organise a
follow-up event on this at a later date.

2
NETWORK UPDATE JUNE 2018 

OUR NEXT NETWORK EVENT

EVALUATION EXCHANGE
Over to you! An opportunity to discuss current/topical evaluation
challenges with other evaluators/members of the Network

4.00pm to 6.00pm (registration from 3:50pm)

Tuesday 17th July 2018
Baskerville House, 2 Centenary Square, Birmingham B1 2ND (next to the library)
Book via: http://bit.ly/MRNJuly
The UKES Midlands Regional Network is pleased to invite you to our second evaluation exchange of
2018. This time it’s over to you!

Responses to our joining survey suggest that there are several topical evaluation challenges that people
would like to discuss with other members of the Network. These were many and wide-ranging, but
included:

• Stakeholder engagement, and managing stakeholder expectations

Evaluation Events
• Responding to changes in services mid-evaluation
• Procuring robust, rigorous evaluation in the context of limited timescales/budgets
• Recruiting evaluation professionals
• Building evaluation capacity

Ahead of our next meeting, we would like those intending to attend to propose specific items/themes
they would like to discuss. These can include themes listed above, or any other topical evaluation-related
challenges you are grappling with at the moment where you’d appreciate opportunity to discuss with
other members of the Network. Please send your suggestions for topics/themes for discussion to Karl
King (karlk@winningmoves.com) no later than Friday 6th July 2018 to allow opportunity for the agenda
to be constructed and finalised ahead of the exchange.

If we don’t receive enough suggestions – we’ll refer to themes raised by respondents to the joining survey
in finalising the agenda.

If we receive more suggestions than we have time to cover, we’ll organise another evaluation exchange
later in the year to cover these – so please get your suggestions in as soon as you can to avoid
disappointment.

GDPR

All members responding to our joining survey were asked to confirm their preferences regarding
contact in our joining survey. However, we have issued an email requesting that all Network members
opt-in to continue to receive Network updates. If you are reading this and have yet to opt-in (or opt-
out!), please do so at your next convenience.

If you are receiving emails from the Network and no longer wish to do so for any reason, please contact
Karl karlk@winningmoves.com, or any other member of the Network Steering Group, who will remove
you from our mailing list. Similarly, if you would like to update your email address (e.g. because you
have moved to a different organisation since registering) please get in touch.
3
NETWORK UPDATE
EVENT SUMMARY
UKES ANNUAL EVALUATION CONFERENCE 2018
The UK Evaluation Society’s evaluation
conference provides an annual opportunity for
evaluators, evaluation commissioners and users
of evidence to network, learn about current
thinking and discuss/debate topical issues in
evaluation. Those attending the conference this
year were treated to an action-packed
programme, with six parallel sessions of
presentations stimulating wide-ranging
discussion about the quality of evaluations and
evidence from them. We were also treated to
three excellent keynote presentations – from
Professor Ian Boyd, Dr Zenda Ofir and Professor
Annette Boaz.
Hamayoon Sultan and Karl King from the UKES
MRN steering group share five things they took
away from the conference:
1. Have we reached a ceasefire in the methods
war in evaluation? The presentations and
discussions at the conference this year certainly
seem to suggest this is the case. There appears to
be much greater acceptance in the evaluation
community that there are ‘horses for courses’ and
an intrinsic need for mixed-methods
methodologies to deliver high quality evidence.
The quality of evaluations cannot be assessed
solely with reference to the method employed,
but rather the extent to which they meet, in a
timely manner, the learning needs of the
commissioners.
JUNE 2018 
2. Are there ‘evaluation ideas that must die’ to
make more and faster progress in tackling big
societal, economic and environmental
challenges? Zenda Ofir’s keynote presentation
challenged delegates to think more critically
about how we do evaluation, warning of
evaluation ideas that could be holding us back
from enabling and accelerating positive
transformative change. Inspired by John
Brockman’s book, This Idea Must Die, Zenda
suggested a number of ‘evaluation ideas that
must die’ – e.g. the idea that dealing with
complexity is simply a case of experimentation
and adaptive management; and the idea that
conventional theories about development are
appropriate/superior to alternative ways of
thinking. It was argued that developing
economic prosperity does not need strong
institutions, but that institutions and markets co-
evolve with changing contexts. Zenda also
highlighted a number of ideas that are already
dying as the need to better deal with complexity
in evaluation is acknowledged.
3. Evaluation should be designed to contribute
to the wisdom of society, not just to meet the
immediate needs and interests of
stakeholders. Points made by Annette Boaz,
Zenda Ofir and other speakers highlighted a need
to increase the contribution evaluation evidence
makes to the wisdom of society.
cont'd...
4
NETWORK UPDATE
EVENT SUMMARY (CONTINUED)
UKES ANNUAL EVALUATION CONFERENCE 2018
... Evaluators, it was argued, must be voices of
conscience. Designing evaluations to meet
stakeholders’ immediate needs and interests can
often encourage linear models of thinking; but
understanding how to bring about transformative
change will more often require systems thinking,
i.e. for evaluators to engage with the system as a
whole (Best and Holmes, 2010) and immerse
themselves in the bigger picture. This is about
shifting ways of thinking, but also ways of doing
evaluation. For example, relatively short, pre-
written interviews can fail to achieve contextual
understanding that immersion into people’s lives
(e.g. through ethnography) can find. As Zenda
put it in her presentation when referring to work in
international development – it’s not about your
project, it’s about their country.
4. Adaptive Management vs Flexible
Management. A hot topic at the conference,
Adaptive Management is seen as a key tool in
managing complexity. However, it is important to
distinguish truly adaptive management from
merely flexible management. Building in ‘interim
review’ points creates a degree of flexibility, but
genuinely adaptive management requires those
delivering the project/programme to be
empowered and sufficiently free from constraints,
to adapt the project/programme as the need is
identified. Genuinely adaptive management also
allows evaluators to look beyond progress
against plans, and to explore various social,
political, economic, and technological factors that
are at play in any given intervention. Importantly,
there is also a need to embed evaluative thinking
and learning within programme management to
achieve adaptive management in practice.
References:
Best, A. & Holmes, B. (2010) Systems thinking, knowledge and action: towards better models and methods, Evidence & Policy,
6(2), pp.145-59 [Available at: http://www.stmichaelshospital.com/pdf/crich/hef-best-holmes.pdf]
JUNE 2018 
Genuinely adaptive management also requires
evaluators to develop a suite of tailored real-time
products and processes, to present learning in a
timely and accessible manner.
5. Independent evaluator vs learning partner.
Several presentations talked about different ways
of characterising the role of the evaluator,
including Rob Lloyd (Itad) who described his
journey from independent evaluator to learning
partner. Clear from the presentations of several
speakers was the need to get involved as early as
possible, understand stakeholders needs,
establish trust and deliver useful evidence. Dr.
Dee Jupp (Palladium) also pointed out that, at all
levels, there is cognitive overload, time pressures
and risk avoidance. In such conditions,
evaluation methods must be appropriate and
help people to learn. To achieve change,
evaluations must appeal to people’s minds AND
hearts. We know this is the case, of course, but
it’s easy to forget/neglect when designing or
reporting on evaluations.
The conference also incorporated the UKES
Annual General Meeting, which included
discussion of progress in the development of a
new value proposition to UKES members and to
society. We’ll keep you in the loop as associated
actions to enhance UKES’s offer are progressed.
The above however is but a snapshot of the
valuable learning on offer at the conference, and
we therefore highly encourage people to attend
next year!
5
NETWORK UPDATE JUNE 2018 

JOB OPPORTUNITY
RESEARCH AND EVALUATION CONSULTANT: BIRMINGHAM
WINNING MOVES
(£32,000-40,000 PER ANNUM (DEPENDENT ON EXPERIENCE) PLUS BENEFITS)

Winning Moves is a niche consultancy that specialises in providing evidence-based insights

helping organisations to make more effective decisions and achieve better outcomes. We do this
through a unique portfolio of services including insightful research and evaluation, and development
and deployment of intelligent digital products. We are experts in our field and are the trusted advisers
and critical friends to a range of high profile, international clients across the public, private and third
sectors including numerous international, central and local government bodies as well as the United
Nations. Our research and evaluation work is wide-ranging; quant to qual (and anywhere in between),
process evaluation to impact evaluation, routine to strategically important. Our work enables our
clients to continue making a positive impact on the societies they are aiming to improve, regulate and
support. As individuals we are innovative thinkers and problem solvers, driven by a passion to deliver
exceptional results. We think differently to deliver better solutions, faster. And we are looking for
someone like us to join our team.

Evaluation Events
Our Consultant will confidently design and deliver:
- Insightful research – using a wide range of methodologies to answer research questions including
establishing awareness, exploring markets and behaviours and helping clients to understand their
customer’s experience and satisfaction;
- Robust evaluation – deploying incisive and robust solutions to help our clients make informed,
evidence-based decisions by exploring what works, for whom and under what circumstances, and the
difference they have made.

For more information please visit: http://www.winningmoves.com/latest/recruitment-opportunity-

could-you-be-our-next-consultant

Conference Presentation
Dr Tracey Wond, Steering group member and Head of Research (Business, Law and
Social Sciences - University of Derby), summarises her conference presentation at
the International Research Society for Public Management (IRSPM) in April 2018.

This conference presentation highlighted the disconnect between evaluation use literature and public
policy discourse on evidence use. Western evaluation theorists have long highlighted typologies and
examples of mis-use, mis-evaluation and non-use (Alkin, 1990; Christie and Alkin, 1999; Patton, 2005).
Yet, there is little to no evidence that suggests such theoretical and conceptual developments has been
adopted by public policy theorists. The conference presentation explored this disconnect further, as well
as exploring primary data which characterized evaluation/evidence mis-/non-/use.

The full presentation can be found at: http://bit.ly/IRSPMTW