Monitoring and

Evaluation
ECO-UNESCOs
experience
Elaine Nevin
National Director
ECO-UNESCO

NYCI Conference 2011

Who we are?
A National
Environmental Education
and Youth Organisation.
Est. in 1986

environmental education
and environmental work
with
young people.
Affiliated to the World
Federation of UNESCO
Clubs, Centres and
Associations (WFUCA)

Our Mission
To channel young
peoples passion into
environmental
protection and
conservation and to
promote the personal
development of
young people.

Our Aims
Promote environmental awareness, knowledge
and understanding in young people
Promote environmental protection &
conservation
Promote the personal development of young
people
Promote UNESCO ideals

What we do
Environmental Youth
programmes
ECO-UNESCO Clubs
Young Environmentalist Awards
Programme
Environmental Workshops
Youth Forums, events and activities
ECO Choices programme
Youth for Sustainable Development
programme

Education and Training

programmes
Short Trainings Teachers, Youth
Leaders & Community Worker
FETAC accredited centre
Sustainable Development L5
ECO-Community Development L3
Educational Publications
Various interactive
resources
Representation
& Advocacy
(Print/Media)

What we do

The Scale of our Work

Over 9500 people annually
National Basis / YEA all-Ireland programme
A team of 10 staff members
2 government work placements at any one time
support 6 graduate interns
over 100 volunteers

What we want to know

M&E

How many people/organisations/

schools are accessing our programmes and
services at any one time and over a period?

What are the demographics of these people

- age, gender, geographical spread etc?

What is the retention rate of our

programmes (for longer term direct contact
programmes) do we have a high fall off
rate?

What we want to know

What have people gained from participating in

an ECO-UNESCO workshop, programme or
training? - knowledge, skills developed, attitudes,
behaviour change, how much have they enjoyed
it etc.
What is the impact of our programme on the
participant?
(impact from a range of perspectives due to the
nature of our work)

What we want to know

Are we delivering programmes that address a

need?

As an organisation are we achieving our goals,

targets and expected outcomes?

Are we delivering quality programmes?

Why do we monitor &

evaluate?

Out of our own desire to develop quality

programmes; review, learn, modify & improve

Out of our own need to ensure our delivery to

greater numbers of young people

To ensure we are doing the job the way we

should be doing it, with the people we should
we doing it with, achieving the results
addressing the needs of young people with an
impact on our participants

Are we responding to needs of young people?

Is our M & E funder driven?

Some of our tools of Monitoring

& Evaluation

Performance Management System Planning,

developing, monitoring and appraisal
Before and after questionnaires on knowledge,
attitude and behavior (baseline/endline)- for
programmes

Analysis of programme participant feedback on

evaluation forms

Specific programme evaluations e.g. YEA evaluations

Other tools
Youth worker self-evaluation forms post youth
programme/training /workshops
people counter to count numbers of people visiting
the Greenhouse etc.

Samples of some of our tools of

Monitoring & Evaluation
PLANNING (Qualitative/Quantitative)
measure
Annual Implementation Plan

MONITORING (Quantitative/Quantitative)
Direct Participant Analysis Information
Direct Participant Feedback on Learning & on Satisfaction
Evaluation forms as part of programmes e.g. YEA
Through YEA/Clubs/YSD, analyse what, how many and
the impact of environmental action projects;

EVALUATION
Reviews carried out
Impact Assessment

SAMPLE 1: PLANNING - Annual

Implementation Plan Organisational Plan
for a year
Programme - Strategic Aims
Objective

Actions

Projected
Output

KPIs (Key

Outcome

Performance

Indicator)

Review on a quarterly basis

Target
Group

Time
Frame/
Deadline

Result /
(Progress)

SAMPLE 2: MONITORING -Participant Analysis

Demographics etc.

Age range, gender breakdown & geographical spread of

participants

<10
DATE

18-21

22-24

>25

NEW
PARTICIPANTS

CONTACT
HOURS

TOTAL
PARTICIPANTS

M
Mth

15-17

NAME
OF
EVENT

PROG
CODE

Day

10-14

Year

Monthly/annual programme -MONITORING

SAMPLE 3: MONITORING - Participant Feedback

on Satisfaction & Learning
ECO-UNESCO measures the following:
Content relevance
Process (quality of facilitation, presentation and group
work/activities)
Resource materials
Learning
We analyse reactions and comments to enhance quality of future work

46% 44% 8% 2% 0% -

Excellent
Very Good
Good
Fair
Poor
3

SAMPLE 3: MONITORING - Participant Feedback on

learning
Youth
Leader
Course
Participants
carry out Training
Self Assessment
of Learning in terms of knowledge, skills &
attitudes before & after each activity/ programme

Self-assessment of Your understanding,

knowledge
and Skills Related to:

Before Course

Q1. What Environmental Education (Education

for Sustainable Development) is

After Course

Q2. How to integrate interactive

Environmental Education activities into your
daily work

Q3. How to encourage critical thinking in

young people through Environmental
Education

Q4. How to foster Environmental Awareness

in Young People through Environmental
Education

Q5. How to carry out an Environmental Action

Project (Energy Audit) with Young People

1 = No understanding or skills 3 = Some understanding or skills

5 = A lot of understanding or skills

SAMPLE 4: Impact Assessment - PILOT INITIATIVE 20102011

In 2010, we wanted to look at how we could measure the impact of our

programmes; reviewed and decided to implement an Impact Assessment
mechanism

Impact Assessment Workshops (3 to 6 months post delivery)

Workshop used to measure two of our programmes:
Youth for Sustainable Development Peer Education programme
FETAC Level 5 Sustainable Development programme

The method/workshop measures in two ways:

In relation to what use and in what ways the learning has been
applied in practice by the participant/learners themselves after
completing the ECO-UNESCO programme
In a broader context how much the ECO-UNESCO programme has
impacted on them and is of use to society in general and on their
communities?

ECO-UNESCO took the role of facilitator

Former participants become the impact assessors

Sample Impact Assessment Grid

Sample Impact Assessment Stories

Story 1
Name: Frida Besong
Thanks to my participation in ECO-UNESCOs FETAC Level 5 Sustainable Development
course, my interest was aroused on Sustainable Development issues and I finally wrote
my M.A thesis on International cooperation regarding the management of water in Lake
Chad and other natural resources in the Lake Chad Basin. At this stage I have applied
for a P.H.D In Education for Sustainable Development and I am waiting for the outcome
Story 2
Name: Julia Lane Barnes
My update since I finished ECO-UNESCOs FETAC Level 5 Sustainable Development
course is that I am involved with working with a local interfaith youth group with an
emphasis on the common theme of the inner spiritual garden. We use lots of the
methodology that I learned doing ECO-UNESCOs FETAC Level 5 Sustainable
Development course. Our long term aim is to make a link between the restoration of a
local walled garden and going to Mount Sinai with the Makhad Trust to help mend the
walled gardens of the Bedouim.
Story 3
Name: Helen
I helped organize the Greenhouse gig fashion show (recycled materials), which was
about organization there was loads of organization in it. And it was about making
people aware about climate change & development, so that was Peer Education.
(about 50 to 60 Peers)

Challenges

Time - Small organisation with limited capacity want to do the

job and make sure we are doing the job well but need to balance
this with getting the job done can be a challenge.

Buy in from all involved - depending on people to collect data

and information and to input it into a system (Salesforce CRM)

Making sure what you are monitoring and evaluating is what you
need to be monitoring and evaluating using the information you
get
M & E needs to focus on Quality as much as Quantity.

We gradually implemented the M & E system but it is under

constant review
Specialist organisation with diverse funding streams with diverse
requirements in terms of outcomes and outputs

Future plans2011
onwards

Review current M & E system ensure it is useful

and informing our work in the way it should be
Ensure it is in line with and collecting necessary
information for the implementation of National
Quality Standards Framework - use this as a tool.

For further information please contact

ECO-UNESCO
'the greenhouse'

17 St., Andrew St.,

Dublin 2
T: 00 353 (0) 1 6625491
F: 00 353 (0) 1 6625493
E: info@ecounesco.ie
W: www.ecounesco.ie

ECO-UNESCO Conserving the Environment; Empowering Young People

ECO-UNESCO
The Greenhouse, 17 Saint Andrew Street, Dublin 2,
tel +353 1 6625491
www.ecounesco.ie
email: info@ecounesco.ie