Breakouts on Intellectual Capital: Impact Assessment

Wednesday 14
th
November
2:30pm

Session reporter: Victor Kuo

Summary of the content of the session:

The SROI Network (Jeremy Nicholls)

Social Finance UK (Jane Newman)
Newman was a former lawyer at an international law firm and ran the office in Shanghai.
Social Finance UK created Social Impact Bonds (SIB). SIB brings together three key parties: an
outcomes funder (government, DFI) that wants the outcomes but doesnt bear the entire
risk, 2) an NGO or group of NGOs around a specific program, and 3) investors providing
upfront finance to deliver the outcome. If successful, theyll be rewarded. If not, theyll carry
the risk. SIB articulated the social value, the investable value, around a concept. We
differentiate SIB as project finance versus investing in an organization directly.
On impact measurement: an example is reducing recidivism in Petersburg, England.
Measurement triggers payment. It is not optional to measure; its essential. We collect
data across the project and monitor intensively. This helps us work out where programs
are working less well. We build in feedback loops so we can change the program as we go
along.
Another example is early childhood education. In this case, the key learning is that its often
a selection of studies that allow us to claim something is valid, but you have to be careful
and look at the studies individual. You have to understand how the evidence has been put
together.

The Happiness Foundation (Mihyun Cho)
Happiness Foundation is the corporate foundation of the SK Group which has companies in
telecommunications and energy. Its philosophy is to maintain sustainability and happiness
through business. It works through the CSR activities. The foundation was established in
2006. We choose social enterprises to do our CSR work. We have an MBA program for social
entrepreneur partnerships with government and social impact investing. We used to give
grants to social enterprises. We use SROI but do not take into account the investment part in
order to keep it simple. The process sounds simple but its time consuming and exhaustive,
because most dont have the data or the management tools. Through the process of
assessment, they have to make up the figures. It takes a lot of time. They have to think


about their financial value. They have a fuller realization of the methods and long term
goals, so its kind of a capability building process.

Kopernik (Toshihiro Nakamura)
I formerly worked for the UN and wanted to seek a different approach. Measuring impact is
important. I need to know it, and its like an obsession. We distributed simple technologies
for the poor (e.g. solar, clean cook stoves, agriculture products). We also try to connect
with the last-mile communities. After they used products, we wanted to see what changed.
Are they spending less time collecting firewood? Less smoke in the house? We conducted
interviews, collected data, and did reporting. It was a time consuming process, so we
experimented with technologies such as SMS with simple mobile phones. We asked Are
you using the stove? We got an above 50% response rate which was promising. Other
technologies, like sensors on the top of the stoves and APPS to analyze the data to populate
the database, were also used (Impact Tracker Technologies). We received support from
Rockefeller and Asia Community Ventures to catalog this work.

Questions from the Audience:

Audience member 1:
How do you measure impact of qualitative characteristics?
How do you measure different projects (arts, education)? And for projects that will have
effects 10-20 years later?
For Randomized Control Trials, in China, it is hard to find the control group. How do you find
the control group?
Can you evaluate across projects in different sectors?
Can we move to something that is the same? Process? External process? There is a growing
convergence around principles.
In Asia, are there grants or training for entrepreneurs to conduct monitoring and impact
assessment?
Its complex and expensive. *Jane+ Its just one person at the computer who is smart and
knows what hes doing. You have to start somewhere and build up to something thats
realistic set of measurements.
[To the SK Group] Did anyone ask you for your marketing return? Branding return?
Specifically, budget planning for the SK Group? [Mihyun] We have to go through the
decision-making process for the group. We have to go through a decision-making process,
just like any other venture investment group.
[Synergy Indonesia.] How do you support social enterprises wanting to scale up to reach
investment readiness? How do you support those in a growing stage and ready for long-term
investments? [Mihyun] We do the assessments for start ups but the data trend is there for
older organizations, and not there for younger. [Jane] We select organizations that are


suitable to achieve our indicators. There has been outrage in the UK recently about
measuring indicators. There is a North Europe approach vs. South Europe approach. What
we concluded is that measurement is important and you need a structure, but it doesnt
matter what that structure is.
Do you pay for the development of an evaluation system for that organization? Is that part
of the funding? Do you specifically designate funds to develop an evaluation for that
organization? [Mihyun] We do the evaluation within our organization or collaborate with a
third party for due diligence to help with objectivity. Its not a difficult system. After we
specify the indicators and the figures for calculation, we explain the methods to the
company. The company can keep using it after the project. In the beginning the cost is on us.
Its an online system for input. [Mihyun colleague] Many want the assessment, and they
learn in the process that their activities are not aligned with the outcomes. So they
discontinue some activities. We just to improve, not prove. It is very time consuming, and
they cant afford it. Realistically, they are surviving. But we ask them to collect the data to
calculate the SROI.
How should I convince the management to do impact assessment? And how do I convince
them to embrace qualitative measures? So far we only measure in the quantitative? [Jane]
For fund-raising, qualitative has strong value. Embed that in your grant proposal. Findings
can be part of a compelling argument.
My question is about measuring outputs and outcomes. For example, with water filters and
bio gas, we want to get to questions like how many were distributed? But we also want to
ask what were the reductions of diarrhea in kids? What side of the spectrum is practical?
Monitoring the outputs levels? [Jane] We design programs around outcomes, but we also
collect lots of data about outputs. [Nakamura] We gather outputs, and then select 20% of
projects to measure outcomes. We try to look at measures slightly above the output level to
see if its making a change in the community. [Mihyun] We focus on outputs but hope to see
the outcomes.



Feedback/Take-Aways for the AVPN:
In positive cases, measurement is key. It is not optional to measure; its essential. For Social
Impact Bonds, the program operators collect data across the project and monitor intensively.
This helps work out where programs are working less well. Feedback loops are built in so
program managers can change the program as they go along.
Doing some kind of evaluation forces funded group to think about their financial value. They
have a fuller realization of their methods and long term goals, so its kind of a capability
building process.


Impact assessment is a time consuming process, so one funder experimented with
technologies such as SMS with simple mobile phones to gather and analyze data. There are
creative ways to minimize the burden.