COFISA Closing Conference

Measurement and
Evaluation: Learning in Real
Time

Peter Greenwood
Non-Zero-Sum Development

18th February 2010

Overview

„ ME&L framework
… LogFrame
… Stakeholder interviews
… Innovation scoreboard

„ Learning

2
Monitoring and measuring for
learning
„ COFISA: systemic, holistic, breaking new
ground (experimental)
„ Building linkages, networks: individuals and
institutions
„ Stakeholders are part of the target system
„ Innovation system ÍÎ Knowledge society

3
 Qualitative
Wide-ranging

Stakeholder Interviews – Reflect the

COFISA perspectives of a range of
Monitoring, stakeholders on key COFISA issues :
• Objectives
Evaluation • Linkages and Alignment
and Learning • Governance
• Implementation
Framework • Achievements
• Miscellaneous

COFISA
Monitoring,
Evaluation
and Learning
Quantitative and Analytical
Quantitative Outcome and Impact-oriented
Output and Outcome-oriented
Objective, data-based approach to
Logical Framework: measure outcomes. Analytical
• Overall Objective and Purpose assessment of innovation-related
• Specific Objectives indicators at a national and provincial
level:
• Results
• Baseline study
• Indicators (activity-level)
• Innovation Scoreboard
Defined: 4 Specific Objectives, 13 • Summative assessment of indicators
Results; 72 Activity-level Indicators.

4
Assessing the LogFrame

5
Innovation scoreboard
„ Available indicators not adequate to assess
COFISA:
… Data insufficiently disaggregated to identify COFISA’s target
populations
… Lags in the likely impacts of the COFISA programme would
mean that the assessment timeframe was not adequate to
capture the results
„ Potential micro-level surveys could address
partially some of these short-comings from
available data

6
Innovation scoreboard
„ Scoreboard dimensions
… Knowledge Demand Indicators (19)
… Knowledge Mobilization Indicators (8)
… Knowledge Input Indicators (10)
… Knowledge Output Indicators (4)
… Knowledge Flow Indicators (3)

„ Baseline
„ Gaps in data

7
Example Scoreboard Indicator:
Knowledge and technology intensity of
manufactured goods

8
Stakeholder interviews
„ Stakeholder inputs on a wide range of issues
„ Interview set usually focused on an aspect of
COFISA
„ Face-to-face or by telephone
„ Flexible interview framework
„ Qualitative, subjective feedback based on
perceptions
„ Insight into dynamics, inhibitors, hurdles
(actionable inputs where possible)

9
Interview framework
1. Objectives
2. Linkages and alignment
3. Governance
4. Implementation
5. Achievements
6. Other

10
Inception
„ Put governance structures into place
„ Adequate, formal inception phase
„ Build the Core Management Team
„ Ensure continuity of thought leadership

11
Alignment and linkages
„ Formulate and articulate a clear position
and message concerning ‘alignment’

„ Good linkages are needed with all relevant

government departments or ministries

12
Stakeholder management and
awareness raising

„ Catalytic interventions such as COFISA

need to be mainstreamed into the
sponsoring department’s or ministry’s
activities

13
Resources, workplan and
administration
„ Adequate time and energy should be allocated
to allow staff, both individually and as a team
(and especially local staff), to undertake learning
activities
„ Special programmes such as COFISA require
customised administrative procedures

14
Making a contribution
„ Specific projects and activities: initiation
and contribution to a range of such
initiatives
„ Facilitating and enabling role: promoting
engagement and collaboration between
practitioners and institutions

15
Some of the value add
„ Identifying and filling the gaps
„ Starting to break ‘silo thinking’
„ Triple-helix (multi-helix?) collaboration
„ Introducing key concepts and the
language of innovation
„ World-class expertise with fresh
perspectives

16
Scope for raising the level
„ To what extent has COFISA changed and
enhanced thinking and activities within DST?
„ Inadequate physical presence in the Eastern
Cape
„ More attention needed on innovation hygiene
factors

17
Continuation
„ Unique role: facilitating networking and
collaboration
„ An agency:
… Distinctfrom government (honest broker)
… With legitimacy and credibility
… Agile, passionate, expert

„ How will the impact of COFISA be

assessed?

18