Terms of Reference for End of Project Evaluation

ADA and PHASE Nepal

August 2018

1 - Background information
PHASE Nepal, the project holder (“grantee”), is a Non Governmental Organization registered
with the District Administration Office Kathmandu. PHASE works for the people in remote
hilly and mountainous districts of Nepal to provide primary health care services and to
improve access to education and livelihood opportunities.

The Austrian Development Agency (ADA), the operational unit of Austrian Development
Cooperation, concentrates its activities on key regions in Africa, Asia, South-Eastern and
Eastern Europe as well as the Caribbean. Besides cooperating with the governments of
priority countries, ADA also promotes initiatives of civil-society and private-sector
organisations in developing countries.
With funding from the Austrian Development Agency, PHASE Nepal has been implementing
the “Integrated Programme for the health of women and children in far Western Nepal”
from January 2016 in Mugu and Bajura Districts in Western Nepal. The programme originally
contributed to the Framework Program 2016-2018 “Health Care for Mountain Farmers in
Nepal” led by EcoHimal Austria, but from October 2016 it is funded by ADA as a stand-alone
project implemented by PHASE Nepal.
The project goal is to “Improve health service, access to education and nutrition status for
women and children in Far Western Nepal”.
As part of the agreement, the project holder is responsible for appointing an external
evaluator to evaluate the project outputs and outcomes compared with those set in the
project log-frame to express an independent professional opinion on the success of the
project and whether the funds granted to the Project have been effectively applied for their
intended purpose.

2 - Project Description
Project Goal: Improving health service, access to education and nutrition status for women
and children in far Western Nepal
Overall objective (outcome):
Better access to health care and better health and nutrition status in the target population,
particularly mothers and children under 5 years of age

Outcome indicators:
1. Increased % of mothers having health care during childbirth (Attended delivery)
2. Increased % of mothers attending for antenatal care 4 times (ANC 4)
3. Decreased % of underweight children under 5 years (U5UW)

Specific objectives and indicators:

1. improved access to health services
1.1. increased number of patients accessing primary care services
1.2. increased number of women receiving antenatal and postnatal care
1.3. increased number of women receiving skilled care during childbirth
1.4. increased number of young children receiving vaccinations and growth
monitoring
2. improved access to nutritious foods
2.1. higher number of children eating leafy green vegetables more than twice a week
2.2. higher number of farmers producing vegetables
2.3. higher number of women rearing chickens and producing eggs
2.4. higher number of improved breed goats
3. improved access to education and health information for adult women
3.1. increased number of women able to read a simple sentence in Nepali
3.2. number of women taking part in health awareness sessions > 5,000
3.3. Increased % of women aware of key messages about child health and nutrition

Expected results:
1. 16,700 people living in target area have access to good standard of primary care
services, aligned with government policies
1.1. approx. 3,500 women of reproductive age have access to family planning,
maternal health care including skilled birth attendance and reproductive health
care
1.2. approx. 2,500 children under 5 years of age have access to better child health
care and weight monitoring
2. 1200 people in target area grow more diverse and nutritious food (figure includes co-
funded project)
3. 640 women in target area have attended adult literacy classes and are able to read
and write
Project Summary:
The overall main objective is to improve the health and livelihoods of the target population,
particularly women and children. This is achieved through improving access to high quality
primary healthcare services, nutritious foods and literacy and health education.

Through a long term continuous presence of project staff in the area, who support and
revitalise government services, people, especially women, have been be made aware of their
basic right to health care services.

At the same time, livelihoods have been strengthened by small scale agricultural skills
building programmes which also included the provision of vegetable seeds and breeding
animals. The main objective of this activity was to increase amount and variety of agricultural
production, mostly to improve nutrition but to some extent this also opened opportunities
for beneficiaries to sell some of their stock and produce locally.

The education pillar of the intervention concentrated on women’s literacy, involving the
literacy groups into the livelihoods programme. Educating women at least to basic literacy
level helped empower them and increases the likelihood that their children attend and finish
school. PHASE has also been conducting other education activities in Bajura, specifically
teacher training, which added to the effectiveness of the intervention.

PHASE programmes are based on the assumption that poor access to health care, lack of
education and food insecurity are all aspects of a wider complex context of poverty and that
they can be addressed in an integrated way. Interventions in the three different sectors are
synergistic and a combined approach where one sectoral intervention builds on the
achievements of the other is much more effective than if each separate sector had been
addressed on its own.

Region: This project is implemented in 2 locations (formerly VDCs) in Bajura (Rugin, Bichaya)
and 5 in Mugu (Dhainakot, Jima, Ruga, Natarphu, Photu). The project was phased-out from
Kotdanda in 2017 and started in a new location, Jima.

Project duration: 1stJanuary 2016 to 31st December 2018

Project Budget: The total budget for the project is € 755,633,00 for three years starting
from 1st January 2016 and ending on 31st December 2018. (This includes the original
framework project budget agreed with EcoHimal Austria)

3 –Report commissioners
This evaluation report is commissioned by PHASE Nepal (Executive Director: Dr Jiban Karki)
subject to approval by ADA (desk officer: Ms Sylvia Hinger).

4 - Purpose of the independent final evaluation

The purpose of the evaluation is to assess the progress made towards achievement of the
specific objectives of the project, its outcome and impact on the lives of beneficiaries that
include the families, communities and governments. It will also measure the level of
community and other stakeholder participation. It will also identify the intended and
unintended outcomes, best practices, lessons learned as well as challenges arising from
project execution. In addition, the evaluation will present conclusions and
recommendations for learning and future intervention.

The independent final evaluation report will be used to inform ADA’s understanding of the
grantee’s performance at the project level and will support learning from the project
outcomes to inform further work in the area.

5– Principles of the evaluation

The Evaluation must be in keeping with the OECD (DAC) quality criteria for evaluations:

Ø Independence
The planning and execution of the evaluation must assure the greatest possible objectivity
and impartiality.
Ø Credibility
The evaluation team must be able to draw on the required methodological and subject
related knowledge as well associated skills. The methods applied in the evaluation and its
findings must be presented clearly in the report.
Ø Participation
Evaluation is designed as a process: External expertise and assessment is combined with a
critical discussion by the project stakeholders and target population to gain new
perspectives and reach agreement on future work.
Ø Transparency
The subject, purpose, scope, addressees, evaluation questions, methods, schedule,
qualifications of the evaluation team, reporting and coordination must be clearly defined in
the terms of reference for the evaluation
Ø Utility
The evaluation findings should be useful for all stakeholders. Steps must be taken to ensure
that they are implemented by the policy and operational decision-makers.
All evaluations include the international quality criteria of relevance, efficiency,
effectiveness, impact and sustainability.

6– Specific objectives of the evaluation

The evaluation has two objectives:
a) To independently verify (and supplement where necessary), the grantees’ record of
achievement as reported through Annual Reports and defined in the project
logframe;
b) To assess the extent to which the project was good value for money, which includes
considering:
• How well the project aligns with ADA’s goals of supporting the delivery of the
SDGs.
• Whether the project met its objectives
• How well the project applied value for money principles of effectiveness,
economy, efficiency in relation to delivery of its outcome
• What has happened because of the project that wouldn’t have otherwise
happened

7 - Evaluation questions
The evaluation should answer the following questions:
Relevance
• To what extent did the project support achievement towards the SDGs?
• To what extent did the project target and reach the poor and marginalised?
• To what extent did the project mainstream gender equality in the design and delivery of
activities (and or other relevant excluded groups)?
• How well did the project respond to the evolving needs of target beneficiaries,?

Efficiency
• To what extent did the project deliver results on time and on budget against agreed
plans?
• To what extent did the project understand cost drivers and manage these in relation to
performance requirements?

Effectiveness
• To what extent are the results that are reported a fair and accurate record of
achievement?
• To what extent has the project delivered results that reflect value for money? To include
but not limited to:
§ How well the project applied value for money principles of effectiveness, economy,
efficiency in relation to delivery of its outcome;
§ What has happened because of the project that wouldn’t have otherwise happened
• To what extent has the project used learning to improve delivery?
• What are the key drivers and barriers affecting the delivery of results for the project?

Impact
• To what extent and how has the project built the capacity of civil society?
• How many people’s lives have been positively affected by the project? – such as:
o farmers producing vegetables and raising chickens
o women accessing safe delivery services and literacy classes
o children being better fed
• To what extent and how has the project affected people in ways that were not originally
intended?

Leverage and Sustainability

• To what extent has the project leveraged additional resources (financial and in-kind)
from other sources?
• What effect has this had on the scale, delivery or sustainability of activities?
• To what extent is there evidence that the benefits delivered by the project will be
sustained after the project ends?

8 - Evaluation methods
Specific details around the methods that are the most appropriate for assessing project
performance will be decided together with PHASE Nepal at the start of the evaluation.
Evaluation methods should be rigorous yet proportionate and appropriate to the context,
data should be triangulated so that findings are as robust as possible.

The research should be based on a contribution-based approach which results in a

‘plausible’ account of the difference that ADA’s funding has made to the impact of grantees.
This approach should be informed by a wide range of evidence sources that are brought
together to produce a ‘plausible’ assessment of the ‘contribution’ of grantees to higher level
outcomes and impacts.
The evaluator should apply a mixed-methods approach for assessing project performance.
This would combine qualitative data to provide an explanation of ‘why’ and ‘how’ the
project has achieved the type and scale of results that are quantitatively observed.

Relevant primary and secondary research may include:

• Review of existing documents and review of context and poverty trends including the
project proposal and other documents, annual and quarterly reports, monitoring and
evaluation reports.
• Field observation of the targeted community
• Household survey using survey questionnaires
• Interviews and workshops with PHASE key staff, local authorities and local groups
• Focus Group Discussions involving primary project participants, other social groups and
key stakeholders.
• Report writing and debriefing with project and management teams
• Scoping reviews of secondary studies and sources, measuring impact where possible and
proportionate through comparison groups and other quantitative methods
• Verifying reported data through back checking and quality control assessments.

List of partners/stakeholders to be involved/consulted in the evaluation include

• Government bodies (DCC, Health, Agriculture, Livestock and Education offices at the
municipal level)
• Community group s
• Participants
• PHASE Nepal staff (management as well as field staff)

Materials to review will include:

• ADA Guide to Project Evaluations if any
• The original application for funding and updated documents
• MOU with ADA for funding
• Project logframes (original and revised)
• Monitoring data
• Monitoring systems
• Annual reports and comments provided if any
• Other evaluation reports undertaken by the grantee
• Financial information / information on resources spent
• Information on synergies / collaboration with other actors
• Published material (e.g. To demonstrate sharing of learning with others)

9 - Deliverables and timeframe

We will be able to provide space in the PHASE Nepal office in Thimi (between Kathmandu
and Bhaktapur) for the consultant to work.

Suggested Time Frame

• By 30th September 2018 – Contract awarded
• By 30th October 2018 – desk study, research design, planning and preparation (5 days)
• By 15th December 2018 – Research and 15 days’ field visit to Far West Nepal + 9 days for
travel
• By 31stDecember 2018 – Initial draft report due (15 days for data analysis and report
writing)
• End of January 2019– Final report due (5 days for finalization and submission)
The following should be delivered:
• Inception report with detailed evaluation plan – by 30th October
• First draft report – by 31st December
• Final evaluation report – by 31st January 2019
• Submission of hard and electronic copies of materials, data collected / analyzed and
other evaluation documents.
• A summary power point presentation highlighting main findings and recommendations

The independent final evaluation report needs to be a substantial document that

(a) Answers all the elements of the Terms of Reference (ToR)
(b) Provides findings and conclusions that are based on robust and transparent evidence
(c) Where necessary supplements the grantee’s own data with independent research.
The main body of the report must be limited to 40 pages (exclude annexes). One of the
annexes should consist of a table which summarises the findings according to the OECD-DAC
criteria.

To ensure consistency across evaluation reports, the following structure should be used
for reporting:
Executive Summary
Introduction
• Purpose of the evaluation
• Organisation context
• Logic and assumptions of the evaluation
• Overview of ADA funded activities
Evaluation Methodology
• Evaluation plan
• Proposed research methods and tools.
• Strengths and weaknesses of selected design and research methods
• Summary of problems and issues encountered
Findings
• Overall Results
• Assessment of accuracy of reported results
• Relevance
• Effectiveness
• Efficiency
• Sustainability
• Impact

Conclusions
• Summary of achievements
• Overall impact and value for money of project activities

Lessons learnt (where relevant)

• Project level - management, design, implementation
• Policy level
• Sector level

Recommendations

Annexes (such as)

• Independent final evaluation terms of reference
• Evaluation research schedule
• Evaluation framework
• Data collection tools
• List of people consulted
• List of supporting documentary information
• Tables, charts and photos.
• Details of the evaluation team
• Grantee management response to report findings and recommendations

10 - Required Qualifications
• A minimum of five years’ experience in programme/project evaluation in an
international development context.
• Experience of results-based monitoring and evaluation
• Ability to design and plan the evaluation approaches and research methodologies,
including quantitative and qualitative research methods.
• Qualifications in Public Health or Nutrition of primary researcher or at least one team
member
• Relevant subject matter knowledge and experience given the aims and objectives of the
project and the context in which it is being delivered
• Significant experience of working in Nepal
• Excellent writing skills
• Fluency in English and Nepali
11 - Terms of payment
PHASE Nepal shall pay the consultant 90% of the agreed total fee upon submission of the
draft report and the final 10% on submission and acceptance of the final evaluation report.

12 – Tenders
Tenders should include and will be assessed on
• Method statement – description of activities to deliver requested services; 30-40%
• Project team’s and organisation’s experience and relevant expertise; 15%
• Project and resource plan; 15%
• Quality assurance and risk; 10%
• Price 20% - 30%
We will invite the top 3 bids for presentation for selection.

The budget of approx. NPR 400,000 is inclusive of all costs covering team member costs,
research costs and any other costs associated with the completion of the work. Costs of
travel to and from the project area will be provided by PHASE Nepal, but the local travel and
other costs within Kathmandu if any should be borne by the consultant team themselves.
PHASE Nepal team will offer support with the project visit to the far West. The project sites
are up to 4 days walk away from the nearest road head or the airport in Eastern part of
Bajura and Southern part of Mugu districts.

Applications from qualified firms/individuals should be submitted by 31stAugust 2017 to

info@phasenepal.org