Constructing your impact model

running the exercise

Introduction

Materials

A group exercise to construct a model for

your project, initiative or service lies at the
heart of the impact evaluation model (IEM).
This guidance outlines how to run such an
exercise in one hour and create a basic
model. An hour is enough time to create
the key elements of a model, although
there may still be a need to make additional
refinements afterwards.

Post-it notes and pens

Prompt questions from the left-hand side of the
model (see additional tools section)
Space on flip chart/wall/brown paper to
separate inputs, outputs, intermediate outcomes
and final outcomes
The diagram below illustrates how this should be laid
out and the additional tools section illustrates other
ways of organising the model for the exercise.

Model layout

Inputs
- What are the key design features of our project,
initiative or service?
- Who is this project, initiative or service targeted at?
- What quantity of the service, activity or
intervention do we aim to deliver?

Stage
Inputs

Outputs
- What are the levels of quality in our project,
initiative or service that we want to achieve?
- What do we want users and providers to think
about this project, initiative or service?
- How do we want our target audience to respond?

Intermediate outcomes
- What aspects of users or participants
a) knowledge or skills
b) behaviour or
c) attitudes
do we want to change through their interaction
with the service?

Final outcomes
- Which final outcomes or community outcomes do
we hope to impact upon?

Outputs

Intermediate
outcomes
Final
outcomes

Model
Evidence
components sources

Using the IEM at different delivery stages

If the project, initiative or
service is still at the planning
stage, the team may wish to
start with final outcomes and
work back to inputs.

Plan
If the team is looking back at a
completed project, initiative or
service, then it may wish to
work downward from inputs to
final outcomes.

Review

Precursors
This guidance assumes that the group conducting the
exercise has already agreed and understood the language
used in the IEM or agreed some alternative language
(see the additional tools section if this is not the case).
To run the exercise, you must have an exercise leader
the person leading the groups discussion and a
problem owner. The problem owner could be the person
who needs the data or the one responsible for follow-up
activities. They will be the final decision-maker if there
are any points the group cannot agree on, otherwise
everyone participates equally.

Process
Step one: set ground rules and decide on
the order in which to complete the model
(five minutes)
It is a good idea to set some ground rules at the start of
the exercise these could include:
no suggestion is a bad suggestion everyone has their
own perspective on the project, initiative or service
nive everyone airtime listen to one another
there is no right answer only what the group
judges to be sensible
if something gets excluded from the model, it
doesnt mean its not important.
The boxes in the model can be completed in any order,
although we suggest the decision is based on where the
project, initiative or service is in terms of its delivery
(see diagram above).

Do
If the project, initiative or
service is already being
delivered, then the team may
want to look at final outcomes
and inputs and then fill in
outputs and intermediate
outcomes from there.

Step two: get people talking about the model

(10 minutes)
It is critical that everyone participates in the exercise
right from the start. To do this, begin with a brainstorm
about what people feel should go into the different
boxes (the additional tools section contains some
prompt questions to get you started). The exercise
leader can front the discussion and write down on
Post-its what people say, placing these on the model
themselves, or each individual can complete and attach
their own Post-its.
At this stage there are no wrong or right answers and no
ones contribution, or where they put the Post-its,
should be challenged. However, it can be useful to guide
people in how to write their Post-its. Post-its should:

Describe a positive end state and not just

restate the target
Not: A 0.5 per cent drop in NEET rate
Rather: All 16- to 18-year-olds engaging in learning

Have a clear who

Not: Classes well attended
Rather: Non-English-speaking parents attend classes

Generally (but not always) include a verb and

should avoid vague adjectives
Not: Parents confident
Rather: Parents engaging confidently in their childrens
maths learning
A useful test of any Post-it is whether there is a clearly
identifiable and undesirable opposite if not, then the
Post-it may not be specific enough.

Constructing your impact model 2

Step three: populate the model (20 minutes)

There is now a collection of Post-its on the model. The
exercise leader takes the group through each of the
models middle column of boxes in the order that was
agreed initially. For each box, the exercise leader is
looking to answer the following questions:
Is everyone clear on what each Post-it means?
Do the Post-its in the box answer the prompt
questions to the left of the box? Are any
others needed?
Are some of the Post-its in the wrong box should
some be moved up or down?
Do some Post-its refer to important goals or
elements that are not the focus of this project,
initiative or service? Do they need to
be removed?
It is important to emphasise that there often isnt a
right or wrong answer for where a Post-it should be
placed it is a judgment. The exercise leaders job is to
try to achieve a group consensus around this decision
(and use the problem owner if this is not possible).

Step four: prioritise the elements (10 minutes)

The resource available to collect evidence after the
model has been completed should help to determine
the complexity of the model, but, in general, it is wise
to have no more than four things at each stage. The
exercise leader must re-emphasise the need for
simplicity and that if something is removed from the
model it does not mean it is not important.
The exercise leader can either hold a group discussion
about what should remain in the model or they can ask
participants to dot vote. In the latter, each participant
has six votes for the things they think must stay
somewhere in the model and they mark these votes
by drawing a dot on each relevant Post-it. The Post-its
with the greatest number of votes are then retained and
the others are moved to one side.

Step five: identify potential evidence sources

(10 minutes)
The exercise leader should give participants a couple of
minutes to write down potential evidence sources that
already exist and to stick these in the right-hand boxes
of the model.
They should lead a group discussion about any potential
new evidence sources or data collection needed.
Participants should be encouraged to think creatively
about information gathering, particularly in terms of:
who the data is collected from try to avoid the
usual suspects who are always asked for
information or those who may be biased in
their response, and
When and how the data is collected particularly
avoid jumping to paper-based surveys if there may
be more natural gatherings or activities when
information could be collected.

Step six: agree the indicators and success levels

(time dependent)
Indicators are the specific questions or responses that
the project leader will gather from the evidence sources.
The following example illustrates this:
Evidence source: Interviews with parents
Indicator: parents level of agreement with the
statement I feel engaged in my childs learning
Success level: 75 per cent of parents strongly agree
that they feel engaged in their childrens learning
These indicators and success levels are likely to feature
in the discussion of evidence sources above so the time
needed may depend on this earlier discussion. The
indicators and levels needed for success may need to
be completed by the project manager afterwards and
circulated to the group.

Step seven: sense check the model (five minutes)

The exercise leader needs to help the group ensure the
model can work. For example, for each intermediate
outcome it should be possible to identify an equivalent
input or output that will help achieve it. So there should
be nothing in any of the boxes that does not have
something that refers to it, or is relevant to it, in the
boxes above and below.

The final step is for the exercise leader to get

participants to step back from the model and check that
its contents make sense. There are some questions in
the additional tools section to help you do this, but a
key test is to ask participants whether, using the model
in front of them, they think they could present a twominute synopsis on what they are doing (the project,
initiative or service), how they think it will work and
what they hope it will achieve.

Constructing your impact model 3

Additional tools
1. Layouts for impact models
Page one of this guidance illustrates how the boxes in the model can be arranged. However, it is quite possible to lay
out similar boxes in different ways on flip charts or brown paper. The following photos illustrate ways to do this.

Alternative model layout for flip charts

Alternative model layout for brown paper

Constructing your impact model 4

2. Language used in the model

The following diagram outlines the meanings that the Training and Development Agency for Schools (TDA) suggests
for the terms used in the IEM.

Inputs

Outputs

Intermediate
outcomes

Final outcomes

Meaning

Rule of thumb

A brief description of a projects,

initiatives or services key components,
its resources, what the planned activities
are and who is it for

Inputs are usually something within

your control that you planned to do,
designed for, or put into a project,
initiative or service

A direct product of what is delivered at

the input stage. It must be related to the
actual quality of delivery and have a
particular emphasis on how users have
engaged with and immediately
responded to the project, initiative
or service

Outputs are usually something that looks

at the project, initiative or service through
the eyes of the user or the intended
audience. This could still be some sort
of planned activity for example, how
tailored a service was but if so this should
be focused on user-centred activities

How you want your service users to be

influenced by the project, initiative or
service, particularly in terms of the
changes in knowledge/skills, attitudes or
behaviour you hope to see

Intermediate outcomes are generally

about the population who have come
into contact with a project, initiative
or service

The social phenomena that you are

looking to address within the community
as a whole

Final outcomes are likely to be at a level

where they would appear as an indicator
among the 200 National Indicators for
Local Area Agreements (LAAs)

Outline of model language

If teams do not use the language above, they do need to agree on a common set of terms to use when constructing
their models. The diagram on the following page illustrates a number of terms that other models may use, which are
often equivalent to the terms used in the TDA model.

Constructing your impact model 5

Related terms used and other frameworks

Inputs

Cost, resources, activities, process, efficiency, overhead, unit costs, staffing ration,
participation

Outputs

Efficiency, staff turnover, staff morale, waiting time, waiting list, accident rate,
customer satisfaction, service quality, product, total quality management (TQM),
staffing ratio, participation, results

Intermediate
outcomes

Staff turnover, staff morale, cost/benefit, client results, client outcomes, value
added, service outcome, direct impact

Final outcomes

Community outcome, impact, value-added, social return on investment (SROI),

well-being, so what, social capital

A way to help the group understand the terms being used in the model and come to a consensus on their use, is
to run a quick exercise to construct a model based on a daily experience. We suggest that the exercise leader
starts with a model in mind and completes a number of Post-its for its main components. When these are shown
to the group, they should not be organised in any particular order.
The group exercise is to organise the components into inputs, outputs, intermediate outcomes and final
outcomes. When constructing the main model, the exercise leader should push the participants to use the rules
of thumb that feature in this guidance, to help them come to a common position. However, as with a real model,
there may be some components that require a judgement about where they should be positioned. The important
aspect is that the group can come to a working consensus on where different components should sit in the
model and take this consensus into the creation of their real model.
The diagram on the following page illustrates how some unsorted components of a model for a dinner party
could be sorted into the impact model categories.

Constructing your impact model 6

Dinner party exercise

Suggested Post-its
Seating plan

I have friends when

I need them

Recipes for three

courses

2.5kg leg of lamb

My guests hunger
is satisfied

Six good friends

attend my dinner

My friends value
my friendship

My guests invite
me back to dinner
My guests get on
with each other

A nice bottle of
red wine

Guests enjoy the

food

I see my guests
more frequently

Some of my guests
contact one
another

I have more friends

Suggested alignment to the model

Inputs

Recipes for three

courses

Seating plan

2.5kg leg of lamb

A nice bottle of
red wine

Outputs

Guests enjoy the

food

My guests hunger
is satisfied

My guests get on
with each other

Six good friends

attend my dinner

Intermediate
outcomes

My guests invite
me back to dinner

Some of my guests
contact one
another

I see my guests
more frequently

Final outcomes

I have friends when

I need them

I have more friends

My friends value
my friendship

Constructing your impact model 7

3. Prompt questions to get people talking

The following are some questions you may want to use to generate conversation at the beginning of the exercise:

Why are we doing this project, initiative or service?

Who is the customer for this piece of work?
What will people think, feel, say or do as a result of this work?
Does anyone have any strong views about what works with this kind of project, initiative or service?
Why are we delivering the project, initiative or service in this way?
What will the future look like if we are outrageously successful?

Training and Development Agency for Schools

151 Buckingham Palace Road, London, SW1W 9SZ
TDA switchboard: t 0870 4960 123
Publications: t 0845 6060 323 e publications@tda.gov.uk

www.tda.gov.uk
TDA 2009