RESULT BASED MANAGEMENT (RBM) FRAMEWORK FOR 11TH FY PLAN

Royal Institute of Management with Gross National Happiness Commission Secretariat Based on GNHC slides

Why RBM?
Pressure to be Efficient and Effective People want better services Politicians want to meet the needs of people (get elected again) RAA and MoF want more effective use of resources Market responds well to efficiency and effectiveness Donors value both efficiency and effectiveness Public wants to perform better

Public Sector Performance

Efficiency and Effectiveness of Public Sector Organizations is seen to depict the development stage of a country Public Sector Capacity and Performance is objective of development

Focus is on Performance/Results
Why does this organization exist? What would be lost if it did not exist? Who does it serve? What is it supposed to deliver for them?

Key: Measurement
Not easy to measure Improvement is not possible without measurement If you cant measure it you cant manage it?

What do you measure?

Inputs/activities
Measured by expenditure and/or extent of activity How much did we spend on this training?

Outputs
By the extent of goods/services delivered and the inputs to output ratio How many trainings organized? How many officials attended?

Effects/Impacts and sustainability

What difference does this make? Is performance improved?

Sowhat is RBM?
RBM is a management strategy focusing on performance and achievement of outcomes and impacts.

Defining Concepts - Results

The RBM Results-Focus Process (UNFPA 2001):
1. Identify and analyze the problem/policy priority to be addressed; 2. Identify the expected results to be achieved based on a problem analysis & risk assessment; 3. Design activities that will generate expected results within resource constraints;

4. Monitor progress and adjust activities as needed to ensure achievement of results;

5. Report on the results achieved.

So, what is RBM?

Focus on tangible results Value for money concept Performance rather than budget driven operations

RBM
Facilitates systematic thinking of three basic questions
What is our Goal: Are we doing the right thing? How will we reach that Goal: Are we doing it right? and How do we know whether we have achieved our goal or that we are doing it right: How do we know?

Project is asses on

Quality
(Standard, Rules and Regulations)

Budget
(money)

Time
(Schedule, Timetable)

The Origins.
Peter Drucker and MBO in the 60s 70s, evolved to Logical Framework 80s, adopted by UK and New Zealand 90s, adopted by USA and OECD countries 90s , became an aspect of New Public Management Now, increasingly used to show good results

The Evolution of RBM

Focus on Performance not Compliance
Did projects spend their budgets and comply with rules and procedures

Did projects achieve their objectives and deliver results

Focus on Impacts and Outcomes not Inputs, Activities and Outputs

How many trees were planted and people trained?
What was the development result e.g. Improved ecosystem health/Change in management practice
13

RBM KEY CONCEPTS AND TOOLS CONSTRAINTS (Assumptions and Risks)

Indicators Indicators Indicators

Inputs

Activity

Output

Outcome

Impacts

ENABLERS (Assumptions and Risk)

14

KEY CONCEPTS
Results Logic Model
Logic Model is the structured, causal sequence of inputs, activities and outputs required to achieve desired outcomes and impacts of a development intervention (programme, project).

Situational Analysis

Strategies / Issues

I N P U T S

15

Understanding Results Logic

16

Defining Concepts Results

video

Defining Concepts - Results

Defining result in RBM:
a describable or measurable change in state that is derived from a cause and effect relationship. This means that a result is a change that can be observed, described and measured in some way, and for which the cause can be identified (UNFPA 2001: 4).

Defining Concepts - Results

RBMs Cause & Effect transformation process:

Results

Transformation Process

Activities Transformation Process

Inputs

Defining Concepts - Results

Levels of RBM Results: The Results Chain
Output: A short-term result that is the immediate, visible and tangible consequence of a completed activity or set of activities within a project or programme.
Outcome: A medium-term result that is the logical consequence of achieving a combination of outputs. Outcomes usually manifest at the end of a project or programme. Impact: A long-term result that is the logical consequence of achieving project or programme outcomes. Impacts are usually evident years after project or programme completion. As a result, it is important to note that other factors outside of the project or programme also contribute to achieving the impact.

Exercise
Results???

Defining Concepts - Results

Impact

Outcomes
Outputs

Activities

Transformation Process
Inputs

Transformation Process

KEY CONCEPTS

Inputs
Are the financial, human and physical resources that make a program or project possible. Links results logic to budgeting

Examples of Inputs
Expertise, Equipment, Facilities, Supplies and Services like travel, phone and accommodation
23

KEY CONCEPTS Activities

Describes the essential work of the program or project what you DO

Should be summarized in short statements.

Examples of Activities
Construction or Installation; Curriculum Design; Training and Coaching; Immunization
24

KEY CONCEPTS

Outputs
Short term a change that is observable as you complete the activity. You have substantial control and influence. You can assign resources. Involves very specific groups of people. Contributes to the realization of outcomes.

Examples of Outputs
Service centre with facilities constructed Managers trained in change management Gomphu-Panbang Highway constructed
25

KEY CONCEPTS Outcomes SKRAs

Medium term change observable at or shortly after the completion of the program or project. You have influence, but not control. Cannot assign resources directly. Usually involves larger group of people Are fewer in number -usually 2 to 3; each likely result from several Outputs.

Examples of Outcomes
Service centres fully occupied with employment generating businesses Improved performance of x institution Increased mobility of the nearby communities
26

KEY CONCEPTS

Impact NKRAs
Long term a picture of a preferred future; Is observable well after the program or project is complete. You can only make a contribution and have an indirect influence. The change usually involves an even wider group of people

Examples of Impacts
Household incomes increase within areas served by service centres; Improved Governance; Reduced incidence of rural poverty;
27

DIFFERENCE BETWEEN OUTPUT, OUTCOME AND IMPACT Parameters

Output
Immediate, upon completion of an activity

Outcome
Upon completion of a project or program

Impact
Beyond project/program cycle

Time
Control/
Influence Sphere of Influence

Direct and substantial

Specific - people directly involved in an activity

Indirect control and Diminishing influence

Extends to those not directly involved in an activity

Minimum control and Indirect influence

Wider extends to a Dzo. or country wide

Resource Direct Control Allocation

Indirect control

No control

Results Chain Example

IMPACT: Efficient, direct dial telephone service within the region and with the rest of the nation OUTCOME: Improved telecommunication infrastructure system in various localities in the target Region.
OUTPUT: Telephone exchange systems in target Region
OUTPUT: Increased capacity of staff in operations & maintenance of telecom equipment

OUTPUT: Efficient telephone service revenue recovery system

Results Chain Example

IMPACT: Effective and transparent public sector that responds to citizens needs

OUTCOME: Increased management skills among trained public sector professionals

OUTCOME: Training institution has capacity to provide on-going training in the long term

OUTPUT: Curriculum and training materials developed

OUTPUT: Increased capacity of trainers to deliver public sector training

OUTPUT: Completed construction of training centre

 Results what huge difference do we want to make?

Impact what differences can we make that will contribute to the huge one? Outcome what needs to happen to achieve the impact? (Will be more than one)

The Questions

Output what little steps will contribute to each outcome? (Will be more than one)
Indicators how will we measure success at each level? (Different indicators for each level) Measurement - what exactly will you measure? Activities/Strategies/Interventions - what are we going to do?

Quiz 1
Group work

Development Planning and Implementation

Government Issues Planning Guidelines Five Year planning NKRAs SKRAs KPIs Key Interventions

Ministries/Auto-Agencies/LGs prepare FYPs

Alignment to national priorities

Alignment of AWPs and Budgets to FYP program outcomes

Annual Work Planning & Budgeting

Preparation/Implementation of Annual Work Plans based on the Approved FYP

Plan Monitoring & progress Reporting

Weekly/Quarterly/Bi-Annual/Annual Monitoring

Physical and Financial Progress Reporting

SITUATION ANALYSIS
IMAGE OF REALITY
CAUSE AND EFFECT RELATIONS

Food Shortages

Production on Hills decreasing

Rice production decreasing

Erosion of Hills Slopes

Destruction of harvest

Irrigation network Irregular supply not functioning of inputs

Defining Concepts - Results

Criticisms of RBMs results focus:
RBM is just another fad Ill just wait until the next project management paradigm comes along Our culture focuses on today, not on future expected results At our organization, management practices are driven by budgets. Its too difficult to re-orient our entire organization to focus on results as that would appear as a threat to senior managers.

Defining Concepts - Results

1. What are the challenges or constraints to using a results-based approach within the context of the Bhutanese public sector? (maximum 3)

2. How might a results-based approach enhance the management practices within the Bhutanese public sector? (maximum 3)

OUTPUT vs OUTCOME vs IMPACT

(Splash and Ripple Example)

Inputs

Activities

Outputs Outcome
39

What do you see?

Indicators.
Indicators represent what is measured to determine whether or not, and to what extent, results are being achieved. (AUCC 2006).

INDICATOR EXAMPLES
Result:
Microfinance institutions (MFIs) have expanded lending capacity

Indicator: Percentage increase in number of loans provided by MFIs

INDICATOR EXAMPLES
Result:
A functional immunization program contributes to reduced mortality from immunizable diseases.
Indicator: Percentage decrease in mortality rates from immunizable diseases.

INDICATOR EXAMPLES
Result:
Health centres provide appropriate reproductive health services to members of the community.

Indicator:
User satisfaction with reproductive health services provide by centre.

INDICATOR EXAMPLES
Result:
Improved telecommunications system in target rural region.

Indicators:
- % increase in rural people in target region with telephones - Level of satisfaction of users of telecom service.

Enhanced accountability and transparency in the use of public resources

46

Two Types
Quantitative indicators:

Measure quantity with a numeric value e.g % of, # of

Qualitative indicators: Measure peoples attitudes, opinions or

judgements
It is good to use a combination of qualitative and quantitative indicators where possible.

INDICATOR
Indicators:... quantitative or qualitative variable
that allows the verification of changes produced by a development intervention relative to what was planned

Types of Indicator:
Quantitative Indicator expressed in numbers, percentages or ratio
of: E.g. Three numbers of service centers constructed/ 50% of the EX level officials trained in change management

Qualitative Indicator seeks to measure quality, based on perception, opinion and levels of satisfaction; E.g. Citizens perception of service delivery/ Public perception of air pollution in Thimphu
48

INDICATOR
Types of Indicator: Proxy Indicator Used when results cannot be
measured directly either because the data for direct indicator is not available or not feasible to collect at regular intervals. Example type roofing materials or televisions as a proxy measure of increased household income

Process Indicator measures the performance

of key processes. Used for process oriented results. Enactment of an Act;

E.g. Review of related act completed by May 2012; First stakeholder meeting completed by June 2012. 49

Identifying Indicators
1. For each result, ask: What will be observed if this result is achieved?

2. Develop a list of both qualitative and quantitative indicators for each result.
3. Select indicator(s) for each result that best fulfill the following criteria: Specific is the indicator clear on what is being measured? Measurable is it objectively verifiable? Attainable is it realistic and cost effective? Relevant does it capture the essence of the result? Timely can it be measured in a timely manner? 4. Once indicators have been selected, baseline data must be collected.

CHARACTERISTICS OF A GOOD INDICATOR

The CREAM of Good Performance

A good performance indicator must be:

Clear Relevant Economic Adequate
(Precise and unambiguous) (Appropriate to subject at hand) (Available at reasonable cost) (Must provide a sufficient basis to assess performance)

Monitorable (Must be amenable to independent validation)

51

RBM KEY CONCEPTS AND TOOLS CONSTRAINTS (Assumptions and Risks) Indicators Indicators Indicators

Inputs

Activity

Output

Outcome

Impacts

ENABLERS (Assumptions and Risk)

52

What Distinguishes Results from its Indicator

Indicators not same as the results

-Farmers trained on new production technology; Nos. of farmers trained on production management;

Each result should have atleast one indicator

Indicators assigned only to the results Output indicator, Outcome indicator and Impact Indicator
Good Indicator should always have a credible Baseline, realistic Target and reliable Data sources
53

Means of Verification (MOV)

The MOV is made up of two components:

1. The source of indicator information e.g. people, reports, maps

2. The method for collecting information from the source. E.g. interviews, report monitoring, field visits, focus groups

Examples
Result:
Microfinance institutions (MFIs) have expanded lending capacity
Indicator:

Percentage increase in number of loans provided by

MFIs

MOV: Interviews with MFI managers

Examples
Result: A functional immunization program contributes to reduced mortality from immunizable diseases.
Indicator: Percentage decrease in mortality rates from immunizable diseases. MOV: Ministry of Health yearly statistical report

Assumptions describe the necessary conditions that must exist if the cause-effect relationships between levels of results are to behave as expected. (CIDA 2000)
IMPACT ASSUMPTIONS

OUTCOMES

ASSUMPTIONS

OUTPUTS

ASSUMPTIONS

Risk Assessment Strategy

Step 1: Identify assumptions at each results level (output, outcome, impact).

Step 2: For each assumption, rate the level of risk that the assumption will not hold true (low, medium, high)

Step 3: Identify risk management strategies that will lessen the likelihood that assumptions will not hold. Start with risks ranked as high.

Results-Based Planning
Activities/Interventions/Strategies

Results

Impacts
Outcomes

Measurement

Outputs

Indicators

Another View Results Pyramid

Vision

Impact)
Impact(s) Impact(s) Outcomes Outcomes
Impact(s)

Outputs...Outputs...Outputs...Outputs

Activities/Interventions/Strategies

Outputs...Outputs...Outputs...Outputs Outputs...Outputs...Outputs...Outputs

Measurement
Vision Indicators Impact(s)
Indicators

Outcomes Outcomes
Activities/Interventions/Strategies
Indicators Outputs...Outputs...Outputs...Outputs

Activities/Interventions/Strategies

Plan Down
Vision

Bigtime Result Impact(s)

Implement Up

Outcomes Outcomes Outcomes

Outputs...Outputs...Outputs...Outputs

Activities/Interventions/Strategies

Bigtime Results
Impact(s)
Move from bigger to smaller chunks

Outcomes Outputs
Activities

Move from smaller (more) to bigger (fewer) chunks

Plan down

Implement Up

BASELINE A Baseline is Information (quantitative or qualitative) that provides data at the beginning of, or just prior to, the planning period. The baseline is used to:
- Learn about recent levels and patterns of performance on the indicator; and to - Gauge subsequent policy, program, or project performance
64

Building Baseline Information

Who will Data collect Frequency Data Collection data of Cost of Indicator Source Method Collection collection Difficulty to collect Who will analyze & report data

1 2 3

65

Data Collection Methods

Key informant interviews Conversation with concerned individuals Participant Observation Focus Group Interviews Panel Surveys

One-Time Survey

Community Interviews

Direct observation

Census

Reviews of official records (MIS and admin data) Field visits

Field experiments Questionnaires

Informal/Less Structured Methods

More Structured/Formal Methods

66

TARGETS

Targets
are the quantifiable levels of the indicators that a country or organization wants to achieve at a given point in time
-

For Example

Increase Agricultural exports by 20% in the next three years over the baseline

67

SETTING PERFORMANCE TARGETS

Baseline Indicator Level

Desired Level of Improvement

Assumes a finite and expected level of inputs, activities, and outputs

Target Performance
Desired level of performance to be reached within a specific time

For Example: Increase Rice production (MT) by 20% in the 10th FYP Baseline (85000)

20% of 85000 (17000)

102000 (85000+17000)
68

Factors to Consider When Selecting Targets

Clear understanding of baseline starting point

(e.g. average of last 3 years, last year, average trend, etc.) Funding and level of personnel resources expected Amount of outside resources expected to supplement the programs resources; Institutional capacity Political concerns

69

Additional Considerations

Only one target is desirable for each indicator

A target does not have to be one single numerical value; it can be a range Take your baseline seriously; Consider previous performance It takes time to observe the effects of improvements; therefore, be realistic when setting targets
70

RESULTS FRAMEWORK
RESULTS INDICATOR
Percentage of malnourished children under 5 yrs

BASELINE TARGET
35 20

Enhanced Food and Nutrition Security

Percentage of children under 5 yrs stunted

33

20

Helps us to complete matrix - becomes your results framework! Define results and provides a plan to track and measure progress and achievement of these stated results
71

Monitoring System
Each Result should be backed by the strong monitoring system

Indicator Baseline

Target

Data Collection Strategy

Data Analysis

Reporting Plan

72

IN ESSENCE
Conducting a Readiness Assessment Selecting Key Indicators to Monitor Results Planning for Improvement Selecting Results Targets

Evaluations

Using Your Findings

4
Baseline Data on Indicators Where Are We Today?

6
Monitoring for Results

8
Reporting Your Findings

10

Agreeing on Results to Monitor and Evaluate

Sustaining the M&E System Within Your Organization

RELEVANCE TO THE 11TH FYP

RBM Introduced in the 10th FYP (2008 2013; Operationalized through the Planning Monitoring and System (PlaMS) web based system; Take forward the lessons and experiences gained to the 11th FYP;

Important input towards implementing the Government Performance Management System (GPMS);
74

FACILITATE THE PREPARATION OF THE 11TH FYP

PROGRAMME PROFILE
Administrative Unit: Programme:
Contribution to National & International Goals & Targets
Classification Categories Goals Targets/Indicators

Sector: Plan Phase:

Table 1: Programme Results Matrix Results Levels Indicators Baselines Plan Targets

Table 2: Programme Activities

Programme Outputs Programme Activities Start Month/Year Finish Month/Year Lead Implementing Agency Collaborating Agencies

Page 1 of 4

Printed on 22/7/2008 75

FACILITATE THE PREPARATION OF THE 11TH FYP

Table 3: Programme Budget Summary
Estimated Budget (Nu. in Million) Programme Activities Recurrent Capital Total Funding Types Funding Agencies Remarks

Total

Table 4. Monitoring & Evaluation Plan

Reporting Frequencies & Responsibilities Results Levels Indicators Baselines Targets Data Sources Frequency Reporting Responsibility Report to

Page 2 of 4

76

Now lets get going and develop this place of ours!