8

Quality of Supply and

Services


Benchmarking is simply about making comparisons with other
organizations and then learning the lessons that those comparisons throw
up.
(Source : The European Benchmarking Code of Conduct)
Benchmarking is the continuous process of measuring products, services
and practices against the toughest competitors or those companies
recognised as industry leaders (best in class).
(Source : The Xerox Corporation)
There can be innumerable definitions of benchmarking but in broader sense
benchmarking means: One has to improve through learning process and this
learning comes from others. It means that essentially it involves sharing
information for learning from other and adoption of best practices for
improvement and to bring changes in performance.
The benchmarking process though extensively involves comparisons of
performance but one has to keep in mind that it is simply not a competitor
analysis, only. One has to be constantly vigilant, alert about the external
environment and adopt the changes as per requirement to remain in
competition because things are changing very fast in the present times.
We can say that benchmarking leads to superior performance through
constant search and adoption for industry best practices.
Benchmarking is a very important management tool irrespective of the nature
of industry, management style or ownership. In private sector it is used to gain
a competitive edge and in public sector it is used as a powerful tool for
improving and delivering modern public services. In private sector the
emphasis is to excel in competitive environment and be a front-runner so
benchmarking is used as a tool to seek innovation outside the industry
paradigm crossing all the boundaries. In public sector, the widespread and
systematic use of benchmarking helps in improving the performance and can
assist individual and organizational learning. We can very well see that
benchmarking is a tool to be adopted by everyone irrespective of nature of
business, type of industry, management control, etc. It is being recognised as
a valuable tool for external learning for everyone. In the subsequent
subsections we will learn different aspects of benchmarking.
7.1.1 Concept
What is Benchmarking?
We have already discussed some definitions of Benchmarking but at this
juncture it is important to understand the basic concept of Benchmarking and
7.1 INTRODUCTION

Benchmarking
means improving
through learning
process and this learning
comes from others.

9


Benchmarking
its purpose. We have seen that in broader sense Benchmarking is comparison
of performance. It can take many forms but usually it consists of giving scores
to various aspects of product, service or process and then comparing these
scores with the leaders scores. The scores are given against certain
parameters or identified areas, which need improvement (Harper, 2005).
Basically Benchmarking is a practical Management tool for improving
performance by learning from best practices and the processes by which they
are achieved. In short, it is a way of learning and making the most of other
peoples experience and expertise and avoiding the duplication of efforts,
rather avoiding reinventing the wheel (www.dgroups.org).
The purpose of Benchmarking is to help improve the effectiveness of process,
product quality and delivery of services. In all it helps the organisations to
keep themselves on a pedestal and compare themselves with their
competitors in turn helping them to improve. As a result the organisations can
be more creative and innovative in their efforts and can develop a competitive
edge over their competitors. Benchmarking can be considered as a strategic
tool for building competitive advantage. The main objective of Benchmarking
is to understand and evaluate the current position of an organisation, identify
the areas, which needs improvement and work on the ways and means of
performance improvement.
Let us now see different types of Benchmarking to understand the concept
in a better manner.
7.1.2 Types of Benchmarking
Benchmarking can be of different types. It depends on the organizations to
see, which type of benchmarking is the most useful for them. The different
types of benchmarking can broadly be classified as follows:
1. Strategic Benchmarking
2. Performance or competitive Benchmarking
3. Process Benchmarking
4. Functional Benchmarking
5. Internal Benchmarking
6. External Benchmarking
7. International Benchmarking
The description of different types is given in Table 7.1.
The type of
benchmarking
varies with
organization to
organization.

10

Quality of Supply and
Services
Table 7.1 : Types of Benchmarking
Types Description Purpose
Strategic
Benchmarking
Organisations need to improve overall
performance by examining the long-term
strategies and general approaches that have
enabled high-performers to succeed. It involves
considering high level aspects such as core
competencies, developing new products and
services and improving capabilities for dealing
with changes in the external environment.
Changes resulting from this type of
benchmarking may be difficult to implement and
take a long time to materialize.
Re-aligning business
strategies that have become
inappropriate.
Performance
or Competitive
Benchmarking
Organisations consider their position in relation
to performance characteristics of key products
and services. Benchmarking partners are drawn
from the same sector.
Assessing relative level of
performance in key areas or
activities in comparison with
leaders in the same sector
and finding ways of closing
gaps in performance.
Process
Benchmarking
Focuses on improving specific critical processes
and operations. Benchmarking partners are
sought from best practice organisations that
perform similar work or deliver similar services.
This type of benchmarking often results in short
term benefits.
Achieving improvements in
key processes to obtain
quick benefits.
Functional
Benchmarking
Organisations look to benchmark with partners
drawn from different sectors or areas of activity
to find ways of improving similar functions or
work processes. This sort of benchmarking can
lead to innovation and dramatic improvements.
Improving activities of
services for which
counterparts do not exist.
Internal
Benchmarking
Involves benchmarking organisations or
operations from within the same organisation
(e.g. business units in different countries). The
main advantages of internal benchmarking are
that access to sensitive data and information is
easier; standardized data is often readily
available; and, usually less time and resources
are needed.
Several business units within
the same organisation
exemplify good practice and
management want to spread
this expertise quickly,
throughout the organisation.
External
Benchmarking
Involves analysing outside organisation that are
known to be best in class. External
benchmarking provides opportunities of learning
from those who are at the leading edge. This
usually takes long time for the results to come.
Where examples of good
practices can be found in
other organisations and there
is a lack of good practices
within internal business units.
(Source : Adapted from www.tutor2u.net/business/strategy/benchmarking.htm)

11


Benchmarking
7.1.3 Performance Benchmarking
Benchmarking is mainly related to assess the performance level of
organizations or individuals. Performance Benchmarking specifies this
concept. In this type the performance information collection is generally
expressed in measurable, quantifiable units and making comparisons with
other compatible organizations. The performance benchmark parameters are
generally measured for a longer duration spread over a period of two to three
years for effective monitoring and for witnessing the improvement.
It is also known as competitive Benchmarking. The main aim of this type of
Benchmarking is to assess the performance levels in key areas or activities in
comparison with the market leaders and finding ways to improve the
performance and innovate. This type of benchmarking is very important
keeping in mind the strategic perspective as it helps in building a competitive
advantage.
Performance Benchmarking answers the most important questions:
What are the most important performance yardsticks?
Where do we rank, compared with others?
Value of Benchmarking
Benchmarking is not only a concept but it is a proven tool adopted by
management very successfully. Benchmarking not only improves the
performance but also brings the value addition. The value addition can
be in the form of the following:
Helps identify and facilitate sharing of key performance factors for peer
organizations.
Evaluates performance measures and goal-setting as it relates to key
stakeholders like customers, investors, regulators, government and others.
Encourages collaboration on the development of industry performance
measurements.
While organization can benefit from a single quality practice, the impact of
moving from Conformance thinking to performance thinking yields
dramatic business improvements.
7.1.4 Features of Benchmarking
The main features of benchmarking are as follows:
It is a managerial tool.
It is a process of continuous learning and this learning comes from others
through exchange of knowledge experiences.
It is to identify best practices and compare with current performance.
It is to add the value to utility performance.
Performance
benchmarking is
also known as
competitive benchmarking.
The main aim of this type of
benchmarking is to assess
the performance levels in
key areas and find ways to
improve the performance.

12

Quality of Supply and
Services
It is a practical approach in which step changes in performance can be
achieved by replicating the best practices already undertaken by someone
else. It provides better understanding of the `big picture and gaining a
broader perspective of the interplay of the enablers that facilitate the
implementation of good practice.
It gives impetus for seeking new ways of doing things which comes
through learning only. So it provides opportunities for staff to learn new
skills and be involved in the transformation process from the outset.
It promotes a culture that is receptive to fresh approaches and ideas and
creates an atmosphere of awareness.
It gives the greater confidence to staff in developing and applying new
approaches because it is already tested and applied somewhere else.
It helps to improve quality, productivity and assist to identify priorities for
improvement.
It allows a wide range of performance indicators to be studied and
performance shortfall to be clearly seen.
It is for standardization.
It is for validation of data.
It is to enhance performance of utility through data collection and analysis.
The last and most important feature is to satisfy customers and to develop
interest and motivation within the organization with an urge to improve
performance.











Usually benchmarking is treated as a structured process and it is best
provided by the development of a step by step model. The important part
here is that a proper structure provides the ways and means of conducting
benchmarking but it should not be formulated in such a manner so as to
7.2 PROCESS OF BENCHMARKING

Identify the most viable type of benchmarking measurement technique for
the organization you are working in and frame a simple, paper based
benchmarking for your organization.









SAQ 1

13


Benchmarking
create complex situations. The basic structure of any benchmarking process
consists of the following five steps. These steps can then be modified
according to the needs of the organisations.
1. Element: This involves identifying the product, service or processes one
wishes to benchmark.
2. Aspect: This step involves identifying the aspects of the product, service
or process, which one wishes to benchmark (e.g. if a product is
benchmarked with the market leader of the similar kind of product, then
the same aspects can be used to benchmark).
3. Scale: This uses the identified scale, e.g. if comparison is made with the
leader, then the same scale of performance measurement can be used.
4. Score: This involves allocating score to each aspect, within the scale. For
example, the rating agencies making a scale and giving score against
each aspect.
5. Benchmark: This is the final step where the comparison is done between
the scores of each aspect with that of the leader.
Let us now see an illustration, which will give an idea as to how the process of
benchmarking works. This simple model can be applied anywhere depending
on the requirement of the organization.
Illustration 7.1
Suppose different aspects related to operations are to be benchmarked with
that of the leader. Simple table can be used to compare the results.
Others Yours Gap Target
Staff 3.7 3.7 0.0 3.0
Supplier 4.0 3.5 0.5 3.0
Logistics 4.1 4.0 0.1 4.5
IT Systems 3.6 2.7 0.9 3.3
Applications 3.7 3.5 0.2 4.5
Data 3.6 2.0 1.6 3.0
Communications 3.4 3.5 -0.1 3.0
Facilities 3.6 2.5 1.1 3.0
(Source : Adapted from Harper, 2005, Electrum Media Limited)
The five basic
steps involved in
a benchmarking
process are
Element
Aspect
Scale
Score
Benchmark

14

Quality of Supply and
Services
The aspects which includes staff, suppliers, etc. in the illustration can be
measured keeping in mind set standards, which is to be benchmarked or
whatever may be viable depending on the need. These can also be
represented with the help of charts or with graphical representations and the
operational risk can be measured.
We can see that the Gap (Refer illustration) are identified. These gaps can
then be filled by reformulating certain strategies of the organisation and then
the specified goal(s) of the organization can be met keeping in mind the
target score.
This was an overview of the benchmarking process. In the similar manner we
can have different models of bench marking. Any type of benchmarking
process model should provide an adequate framework for the successful
planning and execution of benchmarking exercise (Matters & Evans, 2007).
In the subsequent subsection we will learn a six-step benchmarking
model. We do have a five step model also but here we will only see the
six-step model as it seems to be best suited for all types of organizations.
For more information on five-step/stage model one can refer to
www.benchmarking.com.au.

Application of Benchmarking
Benchmarking can be used for different purposes. It can be for:
1. Identifying and managing risk in the financial sector.
2. 360 Degree appraisals.
3. Benchmarking training for communications.
4. Benchmarking in IT Company.
5. Benchmarking for identifying and merging quality improvement.
One of the most popular benchmarking tool is a 360 degree feedback form,
which is used for performance appraisal. A sample of the 360 degree
feedback form is given below to help us understand the concept of
benchmarking and how it is applied.
Activity 1
Suppose you want to benchmark the financial risk of your organization or
the organization you are familiar with, with that of the leaders. Formulate a
simple benchmarking model to compare the results with that of leaders.

15


Benchmarking











Problem Solving, Decisions, Analytical Skills
1 2 3 4 5 N/A
Analyzing problems:
Successfully
separates facts from
opinions when
identifying key issues
contributing to a
problem

Resolving problems:
Solves complex
problems successfully
after analyzing
possible solutions.

Decision impact:
Makes high-quality
decisions that
positively impact the
work of members in
own group

Comments :


(Source : Adapted from: Harper, 2005. Electrum Multimedia Limited.)
360 Degree Feedback Form
Feedback Your relationship
for (Name) to them
Instructions:
Using the following parameters/aspects, select a number that you feel best
describes the individuals behaviour for each of the competencies below:
Performance Key:
1. Inconsistently demonstrates expected behaviours
2. Usually demonstrates expected behaviours
3. Consistently demonstrates expected behaviours
4. Consistently demonstrates behaviours above expected levels
5. Consistently demonstrates behaviours significantly above expected levels.
N/A I cannot provide a number for the individual.

16

Quality of Supply and
Services



7.2.1 Six Step Process Model
There are many process models in benchmarking ranging from Four-Steps to
Twelve-Steps. Most of them are keyed to the Total Quality Management
(TQM), Deming Cycle of Continuous Improvement, PDCA cycle i.e. Plan, do,
Check and Act. The most important aspect is that one must first understand in
detail ones own process before setting out on a benchmarking expedition.
The simplest process model recommended by GOAL
1
/QPC
2
and APQC has
six steps as shown in Table 7.2.
Table 7.2: The Six Step Process Model in Benchmarking
Step 1 Step 2 Step 3 Step 4 Step 5
Plan Conduct
Research
Select Process/
Programmes,
Partners
Collect and Share
Information
Analyze, Adapt
and Improve
Obtain
managements
commitment
Inform staff of
the plan
Choose a
benchmarking
team with right
mix of skills
Determine
what you want
to benchmark
and why
Determine
what
information is
available in
house
Decide on how
much
information
you still
require and
how this will
be obtained
Find out
more on the
activity you
want to
benchmark:
Check
information
in libraries
journals,
etc.
Establish a list
(not more than 6)
of organizations
well known in the
area (e.g.
publishers of good
newsletter,
organizations best
known for training,
user friendly
website, etc.)
From this list
choose three
potential partners
Contact them and
explain to them
organisation
objectives
Choose a partner
and agree on a
framework 4
Establish an
action and agree
on contact person
Collect and
exchange
information (this
is not a one-off
action; you may
need to visit the
organization at
least three times
during the
process in order
to have a full
understanding of
how things work)
Summarize and
analyze the data
collected.
Compare the
analysis with
baseline data
and try to identify
the gaps.
Summarize your
findings
Develop
strategies on how
to improve ones
own process
Submit findings
to management
and try to obtain
support for
changes
Feedback:
Discuss the
findings at a
general staff
meeting and
agree on an
action plan

Step 6
Continue the process of Improvement
The key implementation strategy is to choose solutions to benchmark findings that also contain an
element of continuous improvement.
Organizational learning: With the experience gained, organize training for staff
Monitoring: Implement strategy and monitor improvement
Take stock and review processes: make a check list, identify what worked and what didnt
Finally, now that you have become an expert, complete the Deming Cycle of continuous
improvement by planning for the next benchmarking exercise and use lessons learned to improve
your new model
1 GOAL: Growth opportunity alliance of Greater Lawrence
2 QPC: Quality, Performance and competitiveness
(Source : Adapted from www.dgroups.org/group/leap/impact/docs/p1.benchmarking).
Activity 2
Try to formulate a similar kind of feedback form for your organization. You
can develop a scale with different aspects.


17


Benchmarking
7.2.2 Advantages of Benchmarking
Benchmarking has proved to be one of the most effective tools for bringing
about quantum-leaps in performance. The main advantages of benchmarking
are as follows:
It is an effective wake-up call which helps to work as an agent for change.
It is a practical approach in which changes in performance can be
achieved by replicating the best practices already undertaken by someone
else. It provides better understanding of the big picture and gaining a
broader perspective of the interplay of the enablers that facilitate the
implementation of good practice.
It gives impetus of seeking new ways of doing things which comes through
learning only. So it provides opportunities for staff to learn new skills and
be involved in the transformation process from the outset.
It promotes a culture that is receptive to fresh approaches and ideas and
creates an atmosphere of awareness.
It gives the greater confidence to staff in developing and applying new
approaches because it is already tested and applied somewhere else.
It helps to improve quality, productivity and assist to identify priorities for
improvement.
It allows a wide range of performance indicators to be studied and
performance shortfall to be clearly seen.
7.2.3 Disadvantages of Benchmarking
There are certain disadvantages of benchmarking but they are very few.
The advantages outweigh the disadvantages, but it is important to know
and consider the possible disadvantages so that corrective measures can
be planned and implemented to minimize the impact of disadvantages. The
few common disadvantages are as follows:
Difficulties getting agreement on what indicators are to be used.
Difficulties in defining the data.
Gives only limited information about how to correct performance shortfalls.
The projects can be completed quickly but the results may take much
longer time.
Highly ambitious goals may lead to unsuccessful benchmarking process.

18

Quality of Supply and
Services
Many organizations avoid the process lest their weaknesses be exposed
to their competitors.
Cultural differences may, lead to some difficulties in applying best
practices as happening MNCs.
7.2.4 Mistakes to be Avoided
It is important to avoid some common mistakes to achieve the desired
targeted benefits from the benchmarking exercise:
Benchmarking should not be just for the sake of doing.
Not defining the data items correctly and consistently and not collecting the
data accurately limits its effectiveness.
Not spending enough time in the beginning to select right framework of
performance indicators and subsequently spending too much time on one
part of the process at the cost of other key parts.
Focusing entirely on comparisons of performance measures and ignoring
the processes and activities for improvement.
Relying on pre-existing benchmarks instead of modifying/adapting
according to organizational needs.














Suppose you are asked to benchmark yourself against a subset of peer
group for providing a relevant comparison. Explain how you would go
about the whole process and the limitations you will incur while going
through this process.
.
.
.
.
.
.
.
SAQ 2


19


Benchmarking

Critical Success Factors (CSFs) are basically those that play an important role
in the success of an organisation. We must know that it is very important for
organisations which really matter in the success of that organization. CSFs as
they are popularly known are the essential areas of activity that must be
performed well if one is to achieve the mission, objectives or goals for ones
organisation or project (www.mindtools.com) If one is able to identify the
CSFs one can easily create a common reference point, which will in turn help
direct and measure the success of ones organization or project. This
becomes important as everyone in the organization will know what exactly
is important and will work in ones own way in reaching the goal. D. Ronald
Daniel in the 1960s floated the idea of CSFs and the credit of making it
popular goes to John F.Rockart of MITs Sloan School of Management. This
has now became an important tool for implementing strategies in the
organisation.
Definition
According to Rockhart CSF is defined as The limited number of areas in
which results, if they are satisfactory, will ensure successful competitive
performance for the organization. They are the few key areas where things
must go right for the business to flourish. If results in these areas are not
adequate, the organizations efforts for the period will be less than desired.
(Source : www.mindtools.com)
He also concluded that CSFs are areas of activity that should receive
constant and careful attention from the management.
The main focus of CSFs is on the most important areas and to get into the
details of what is to be achieved and how it is to be achieved.
Let us take a hypothetical example to understand the concept. We will take
the example of an organization dealing in distribution of power. Figure 7.1
shows the elements involved in strategic implementation.
CSFs
Create successful relationship with
suppliers.
Customer relationship management.
Secure financial plans for expansion.




Figure 7.1 : Relationship of CSF with Mission and Goals
Mission

To become the
number one
power house
in the city
Goals:
Gain market share locally by a defined margin.
24 hrs customer service.
Sustain customer satisfaction rate to 98%.
7.3 CRITICAL SUCCESS FACTORS
Critical Success
Factors (CSFs)
are an important
tool for implementing
strategies in the
organization.

20

Quality of Supply and
Services
We must understand here that a limited number of CSFs should be used to
avoid complications.
The stepwise process of identifying the CSFs:
Identifying the CSFs and implementing it as a strategic tool involves five step
process. These steps are as follows:
Step 1: Establishing the mission and strategic goals possible.
Step 2: Identifying the CSFs to achieve the set goal.
Step 3: Evaluating the list of possible CSFs to absolute essentials.
These are the actual CSFs.
Step 4: Monitoring and measuring each CSF.
Step 5: Continuous monitoring and re-evaluation of CSFs to ensure
that the right track is being followed to achieve the set goals.
Role of CSFs in Performance Benchmarking
As we have discussed that CSFs are an essential part in ensuring the success
of a product or a process. It is true that they play an important role in
evaluating the performance of an individual or an organization. CSFs are an
integral part of the benchmarking process as they help in understanding and
identifying the areas, which need more importance and the areas which need
less importance. This in turn helps in avoiding duplication of efforts and in turn
helping in saving the time. When we identify the gaps in the benchmarking
process while comparing the results, the role of CSFs come into play. These
gaps can be covered up by identifying the CSFs and working on them to
achieve the competitive edge. It can well be said that identifying the CSFs and
communicating them with everyone will help in keeping the project or process
on track and in turn will help in achieving the common goals of the
organisation.
To harness and get the benefits of any initiative, it is important to define and
adhere to certain basic success factors. In benchmarking exercise also, there
are certain critical success factors and it is important to ensure that:
Benchmarking is a management tool so senior managers should not only
support it but should be committed to continuous improvements.
The benchmarking exercise should be done by teams, preferably cross
functional team.
The team should have a clear picture of their organisations performance
and subsequently the team should approach others for comparisons.
The teams should have the right skills and competencies and enabling
environment so as to have access to training, advice and guidance over
the course of exercise.
The objectives of the exercise should be clearly defining in the beginning.
CSF is an
essential part in
ensuring the
success of a product or a
process.

21


Benchmarking
The scope of the work of benchmarking exercise should be clearly defined
in the beginning keeping in the mind the objectives, available resources
along with the experience and skill level of human resources and time
available.










Benchmarking is not a fad but has become the basic requirement of any
organization. It is a continuous process and requires efforts on part of the
organization on a continuous basis. The major key to a successful
benchmarking is doing a benchmarking on a regular basis so as to compare
results over time. This will help in developing a trend and identifying the gaps,
which over the time can be improved upon.
As we know by now that benchmarking helps in evaluating the performance
on an individual basis as well as on the organizational basis. It is important to
note that performance benchmarking should be accompanied with other types
of benchmarking to give a holistic performance based evaluation technique. It
has been seen that paper based manual benchmarking system is quite
tedious and time consuming. Therefore, now-a-days organizations are going
in for automated benchmarking systems, like 360 degree feedback forms,
which require less efforts and can be applied effectively and can produce
return on investments within a short span.
It is important to understand that there are certain critical factors, which are
required for making benchmark successful. These can be summarized as
follows:
Choosing the right things to be benchmarked;
Involving the right people at the right time;
Having a common understanding of the whole process involved;
Understanding the situation prevalent at the time of benchmarking;
Choosing suitable partners;
Managing the improvements.
What is the role of critical success factors (CSFs) in performance
evaluation?
.
.
.
.
SAQ 3

7.4 SUMMARY

The key to
successful
benchmarking is
performing it on a regular
basis.

22

Quality of Supply and
Services
The major success factor is to remember that benchmarking is a continuous
improvement strategy and a process of change management. To conclude it
can be said that any activity can be benchmarked but the crux of the success
lies in learning the lessons in the benchmarking activity, which in turn should
become the source for continuous improvement and innovation in the process
of benchmarking.
To better understand the concept of benchmarking, role playing exercise and
a case study is given in Appendix A and Appendix B.


1. What do you understand by benchmarking? Explain giving examples.
2. Explain the different types of benchmarking with the help of examples.
3. What are the different steps involved in identifying the CSFs? Discuss with
the help of an illustration.
4. Distinguish between different types of benchmarking giving real world
examples.


1. Role Playing Exercise: Customer-utility-vendor-NGO-SERC interface
through public hearing to evolve a consensus on the modalities and
quantum of penalties to be imposed on the utility for non-performance.
(Source : DRUM training program : USAID Ministry of Power).
2. Survival of the fittest: Measuring performance in a changing business
environment. Mike Kennerley, Andy Neoly and Chris Adams
http://www.emeraldinsight.com/insight/viewPDF.jsp?Filename=html/
Output/Published/EmeraldFullTextArticle/Pdf/2670070405.pdf
(Source : DRUM training program : USAID Ministry of Power).
Links to the articles referred:
1. Harper, Stuart. November 2005. Benchmarking and Risk
www.electrum.co.uk/services/papers/benchmarking.pdf
2. 23
rd
November 2007, Strategy-Benchmarking
www.tutor2u.net/business/strategy/benchmarking.htm
3. Matters, Margaret & Evans, Anne. 23
rd
Sept. 2007. The nuts and bolts of
benchmarking http://www.benchmarkingplus.com.au
4. Maboguije-Visser, Lola. 25
th
August 2004. Benchmarking
www.dgroups.org/group/leap/impact/docs/p1.benchmarking.25 AUGUST
2004. doc?015=no.
5. Critical success factors 23
rd
September 2007.
www.mindtools.com/papers/article/newLDR-80.htm
7.5 TERMINAL QUESTIONS
5
REFERENCES


23


Benchmarking
Appendix A


Role Playing Exercise: Customer Utility Vendor NGO SERC interface
through public hearing to evolve a consensus on the modalities and quantum
of penalties to be imposed on the utility for non-performance.
Current Situation
The Vikas Pradesh Electricity Regulatory Commission (VPERC) of Vikas
Pradesh had three years back asked all the DISCOMs to take initiative and
implement time bound plan to improve the power distribution system and
quality of electricity supply service to the customer. It had told the utilities at
that time that after three years (i.e. today) a scheme would be finalized for
imposition of penalties on them if they dont adhere to the quality norms.
In order to achieve this, the VPDISCOM had taken necessary steps to
improve the quality of electricity supply over the last three years and has
improved the services substantially and expects to achieve the expectations of
the customer and the SERC.
SERC has called for a scheme from the utilities for imposing the penalties on
them. It intends to circulate them for public consumption and sought the
reaction with the intention to conduct a public hearing before passing any
orders on the penalty scheme.
The SERC has stipulated the following steps:
1. Submission of penalty scheme by the utilities;
2. SERC to seek comments from the customers by publicizing the scheme;
3. Fixing a date for hearing;
4. Conducting the public hearing and passing the orders.
The factors to be considered in the penalty scheme are:
1. Power supply voltage;
2. Number of power supply interruptions;
3. Duration of power supply interruptions;
4. Time taken to rectify/correct the complaint to the satisfaction of the
customer;
5. Methodology and procedure for the resolution of any dispute over the
penalty;
6. Any other item to be considered in the penalty scheme.
ROLE PLAYING EXERCISE


24

Quality of Supply and
Services
Participants
The participants will be evenly divided into the following five teams:
Team 1 : Customer Groups, Local Leaders, Local Governments (CGs)
Team 2 : NGOs
Team 3 : DISCOMs
Team 4 : Vikas Pradesh Electricity Regulatory Commission (VPERC)
The Role-Playing Exercise
Team 3 (DISCOMs) would evolve, and develop a scheme for penalizing
themselves on slippage or non-conformation to the quality of supply. The
scheme would set a standard of quality service as a basis for imposing the
penalty. Team 3 would submit the proposal to the Team 4 (VPERC), which in
turn publicizes, and invites the comments from Team 1 and Tem 2 comprising
the other stakeholders. All assumptions are up for grabs. Team 1 (RCGs) and
Team 2 (NGOs) would register their differences and agreements on the
proposal. The Team 5 (VPERC) will call all other teams for a public hearing
seek views of all the stakeholders on the proposed penalty scheme. Team 5
(VPERC) would make a final decision, which will be followed by a short debate
among all the teams about the Commissions decision.
The participants will be divided into various teams and each team will be
assigned an identity of an organization. Each team will need to select a team
leader who will act as the spokesperson for the team. At the end of the role-
playing exercise, the spokesperson for each team will make a brief
presentation on its position. The facilitators will provide any clarifications and
assist the teams without taking any sides.
(Source : DRUM Training program : Performance benchmarking and quality of
USAID-Ministry of Power, Supply and Service).

25


Benchmarking
Appendix B




The design and use of performance measurement systems has received
considerable attention in recent years. Many organizations have redesigned
their measurement systems to ensure that they reflect their current
environment and strategies. But how to maintain them over time? Increasingly,
the environment in which organizations compete is dynamic and rapidly
changing, requiring constantly charging strategies and operations that reflect
these changing circumstances. Despite this, few organizations appear to have
the internal culture and systematic processes in place to manage their
performance measurement systems in order to ensure that they continue to
reflect their environment and strategies. This article presents case study
research that investigates what actions organizations can take to ensure that
their measurement systems change over time.
Business environments, like physical environments, change over time.
Indeed, in recent times, it seems as though the pace of change has
accelerated exponentially. To survive and prosper in this rapidly changing
climate, organizations must be ready to adapt. One of the most important
items on senior executives change agenda needs to be keeping the
organizations performance measures and measurement systems relevant to
todays business needs. Failure to do so will invariably lead to a diminution of
the companys reputation; to customer defections; and to breakdowns in
relations with other key stakeholders not least, the deterioration of investor
confidence in management. Some retailers, energy companies, auditors,
telecommunications equipment suppliers and investment banks have all
recently learned this the hard way. Indeed several have already become
extinct.
Managing with Measures
A companys performance measurement system need to be a living entity, not
a fossilized relic of some past intent nor, more commonly but arguably worse,
some monstrous hybrid of past and present requirements. Todays
performance measurement systems must reflect the wants and needs of all
the organizations stakeholders (its investors, customers, employees,
suppliers, regulators and communities); what the organization want and needs
from its stakeholders on a reciprocal basis; and the strategies, processes and
capabilities the organization is implementing to satisfy these sets of wants and
needs. Inevitability these wants and needs, and the strategies required to
CASE STUDY SURVIVAL OF THE FITTEST:
MEASURING PERFORMANCE IN A CHANGING
BUSINESS ENVIRONMENT


26

Quality of Supply and
Services
respond to them will change over time. In accordance the measurement
system needs to be realigned to reflect those changes.
By the term performance measurement system, we mean three inter-related
elements: singular measures that quantify the impact of specific actions; a set
of measure that combine to assess the performance of an organization as a
whole; and a supporting infrastructure that enables data to be acquired,
collated, sorted, analysed, interpreted and disseminated for management use.
And the word use is key here; if managers do not use the data, why is it still
being measured and distributed to them?
But what factors affect the way an organizations measurement system
changes over time? Perhaps surprisingly, until now very little research has
been done in this particular field. Much has been written about frameworks to
aid the design of appropriate measures (such as the performance prism, the
balanced scorecard, and so on) and the overhaul of measurement systems
that are over-dependent on financial data, but not much about maintaining
them over the long haul. However, recent research carried out at Cranfield
School of Managements Centre for Business Performance and funded by the
Engineering and Physical Sciences Research Council, has sought to close
that gap in our knowledge.
This article summarizes the findings of this study, based on seven case
examples undertaken to identify the key factors that affect the way that
performance measurement systems change over time, and an in-depth case
study carried out to further investigate the path-finding observations that this
research revealed.
The seven companies used to develop the research into the way in which
performance measures change within organizations represented a broad
cross-section of UK industry:
Phase 1 Initial Case Examples
1. A transport infrastructure maintenance company;
2. A supplier of IT services;
3. A retail and commercial stationer;
4. A global package delivery courier;
5. An energy generation and supply utility;
6. A food packaging manufacturer; and
7. A printing equipment manufacturer.

27


Benchmarking
The focus of this work was to identify the factor, which both encourage and
inhibit the introduction of new measures, the modification of existing
measures, and the deletion of obsolete measures.
The conclusions from this first phase of the research identified four critical
factors that were most influential as enablers of and barriers to successful
performance measurement systems. These were:
Culture: The existence of a measurement culture within the organization
ensuring that the value of measurement, and so the importance of maintaining
relevant and appropriate measures, is appreciated.
Process: The existence of a process for reviewing, modifying and deploying
measures.
People: The availability of the required skills to use, reflect on, modify and
deploy measures.
System: The availability of flexible information technology that enable the
collection, analysis and reporting of appropriate data.
In addition, it was found that modifications to performance measurement
systems were sometimes triggered by changes within the individual businesss
working environment, either externally (e.g. competitor actions or regulatory
requirements) or internally (e.g. company ownership or management
changes). Table 7.3 illustrates the component factor within each of these
categories that form both barriers to and enablers of the evolution of good
performance measurement systems.
Evolution Framework
The case example data, collected through the seven companies, also
demonstrates that there are a number of stages through which a performance
measurement system goes in order to change or evolve. The data clearly
shows that the active use of the performance measurement system is a
pre-requisite to any evolution. This requires that the performance
measurement system be used to manage the business so that the importance
of the measures is demonstrated throughout the organization and to highlight
whether the measurement system is providing the appropriate information
as and when required.
Given the availability and constructive use of the performance measurement
system, there are three subsequent steps to effective evolution. These are:

28

Quality of Supply and
Services
Table 7.3 : Barriers to and Enablers of Measures Evolution
Critical
Factors
Barriers to Measures
Evolution
Enablers of Measures Evolution
Culture Management inertia towards
measures due to other
priorities;
Ad hoc approach to
measurement;
Measures not aligned to
strategy;
Actions not aligned to
measures;
Lack of management concern
for non-investor stakeholders.
Senior management
sponsorship;
Consistent communication of
multi-dimensional
performance to staff;
Open and honest application
measures;
No Blame/No game
environment;
Integration and alignment of
reward systems;
Process Lack of proactive multi-
dimensional performance
review process;
Poor measures selection
approach;
Lack of data analysis and
insights;
Insufficient measure ownership
delegation;
Ownership of cross-functional
measures not addressed.
Integration of measures with
strategy development;
Integration of measures with
process redesign;
Inclusion of-non-financial
measures in business
performance reviews;
Formal measures review
process conducted at regular
intervals.
People Lack of manager/ supervisor
training in managing with
measures;
Shortage of data analysis skills
and specialist resource;
Shortage of expert IT data
extraction programme staff;
High staff turnover.
Provision of appropriate
performance measurement
resource;
Investment in measures
usage and analysis skills-
building;
Inclusion of appliance of
measures in employee
performance review;
Community of measures
users who make improvement
suggestions.
System Inflexible legacy systems;
Poorly or partially implemented
ERP systems;
Difficult to tailor off-the-shell
performance reporting software;
Poor use of graphical
representation;
Excess of raw data.
Investment in IT hardware and
software;
Data mining/ warehousing
capability;
Readily customizable
information systems;
Internal systems development
adaptation capability.
External/
Internal
Triggers
Change in regulatory/ legislative requirements;
Changes in competitive environment;
Changes in company ownership;
Changes in management;
Changes in technology.


29


Benchmarking
Reflection on the existing performance measurement system to identify
where it is no longer appropriate and where enhancement needs to be
made.
Modification of the performance measurement system to ensure
alignment to the organizations new circumstances and/or strategic intent.
Deployment of the modified performance measurement system so that it
can be used to manage the performance of the organization.


















Figure 7.2 : Framework of Factors Affecting the Evolution of Performance Measurement Systems




External Triggers
Reflect
Deploy
Modify
Use
E
x
t
e
r
n
a
l

T
r
i
g
g
e
r
s

Performance Measurement System:
Individual measures.
The set of measures.
Supporting infrastructure that
enables data to be acquired,
collated, sorted, analysed,
interpreted and disseminated.
P
r
e
-
r
e
q
u
i
s
i
t
e
s

I
n
t
e
r
n
a
l

T
r
i
g
g
e
r
s



30

Quality of Supply and
Services
Table 7.4 : Evolutionary Capabilities Maturity Model
Reflect Ad hoc Basic Emerging Managed Excellence

The current set
of measures is
accepted and
the need to
change
measures is not
recognized.
Those
responsible
for
measurement
appreciate the
need to reflect
on measures
and change
them.
Managers
appreciate
the need to
reflect on
and change
measures.
Those using
the measures
appreciate the
need to reflect
on and
change them.
Constant review
of
appropriateness
of measures is
embedded in the
culture of the
organization.

No process of
reflection on
existing
measures.
Reflection on
measures
happens, but
it is an ad hoc
activity.
Reflection
on
measures is
scheduled
and linked
to strategy
reviews.
Reflection on
measures is
scheduled and
linked to
strategy
reviews.
Clear criteria
exist.
Reflection on
measures
against clear
criteria is an
automatic and
on going
process.

No resources
are dedicated
to review of
measures.
Those
responsible
for
measurement
are
encouraged to
reflect on the
measures.
Managers
are
encouraged
to reflect on
the
measures.
Those using
the measures
are
encourages to
reflect on
them.
Dedicated
resource exists,
with
responsibility for
ensuring that
reflection takes
place.
MODIFY:

The current set
of measures is
a accepted and
the need to
modify
measures is not
recognized.
Those
responsible of
measurement
appreciate the
need to reflect
on measures
and then
modify them.
Managers
appreciate
the need to
reflect on
measures
and then
modify
them.
Those using
the measures
appreciate the
need to reflect
on measures
and then
modify them.
Constant
reflection and
modification of
measures is
embedded in
the culture of
the
organisation.

No clear
process exists
to modify and
redesign
measures.
Modification of
measures
requires a
significant
project driven
by senior
management.
Modification
of measures
requires a
project that
can be
supported
by internal
resources.
A recurring
process is in
place that
enables
measures to be
modified.
It is easy to
modify
measures as
and when
necessary. All
modifications
are
implemented
quickly.

The skills
required to
modify the
measures are
not available
within the
organization.
Only senior
management
have the skills
and
knowledge to
design and
modify
measures.
Manageme
nt at all
levels have
the skills
and
knowledge
to design
and modify
measures.
Measurement
resources are
available to
support users
in defining their
own measures.
Employees at
all levels have
the skills and
knowledge to
design and
modify
measures.
P
r
o
c
e
s
s

C
u
l
t
u
r
e

C
u
l
t
u
r
e

P
r
o
c
e
s
s

P
e
o
p
l
e

P
e
o
p
l
e


31


Benchmarking
DEPLOY:

Individuals are
resistant to
change and
stand in the
way of the
deployment.
Considerable
management
time is required
to overcome
barriers to the
change of
measures.
Action is taken
to encourage
acceptance of
new measures
by all of those
involved in
measurement.
Measures
can easily be
successfully
deployed
throughout
the
organization.
Deployment
of appropriate
modified
measures is
effortlessly
accepted
throughout
the
organization.

There is no
process in
place to deploy
modified
measures.
A performance
measurement
introduction
process has to
be initiated in
order to deploy
measures.
An
implementatio
n project is
required to
deploy
modified
measures.
Individual
departments
can deploy
modified
measures as
and when
required.
All owners of
measures can
deploy
modified
measures as
and when
required.

No resources
are dedicated
to deployment
of modified
performance
measures.
External
support and
senior
management
attention is
required to
deploy
measures.
Management
have the skills
to implement
modified
measures
when they are
defined.
Management
time or
measurement
personnel are
dedicated to
the
deployment
of measures
Users at all
organizational
levels have
the ability,
authority to
deploy
measures.

Systems are
not flexible and
do not allow
modification of
measures
easily.
Changing data
collection and
reporting tools
requires major
systems
development.
Internal
systems
development is
required to
reconfigure
data collection
and reporting
tools.
Management
and
operational
personnel
have the
ability and
authority to
modify
systems.
Systems are
ultimately
flexible. Users
at all levels
have the
ability and
authority to
modify them.
Of these three steps, reflection is clearly the most critical. Without this,
nothing changes or, at least, only randomly. During the reflection stage,
each of the constituent elements of the performance measurement system
should be critically appraised and reviewed to ensure that they remain
appropriate to the requirements of the organization today. Reflection on the
singular measures is to identify whether the measures in place are measuring
performance in the right way. Reflection on the set of performance measures
is to identify whether the processes and systems are in place to ensure that
data is collected, analysed and distributed effectively and efficiently. A number
of audit approaches and self-assessment aids are available to assist with
these vital tasks.
Audit of a measurement systems ability to change over time
The case examples data demonstrate the presence of an evolutionary cycle
reflect, modify, deploy, use with the active and effective use of measurement
as a prerequisite for evolution. External and Internal factors can and often do
C
u
l
t
u
r
e

P
r
o
c
e
s
s

P
e
o
p
l
e

S
y
s
t
e
m
s


32

Quality of Supply and
Services
disrupt normal evolution patterns too, acting as a trigger for reflection on
what is measured.
The data also identifies that for the primary components of the cycle to be
executed, and for the performance measurement system to proactively evolve,
the enablers of evolution grouped under the headings cultural process,
people and systems must be in evidence within the evolutionary cycle.
Consequently, for organizations to effectively manage the evolution of their
performance measurement system, they must consider the capabilities
required as they relate to each of these stages in the form of a maturity model,
which acts as a useful assessment tool.
The representation of the essential capabilities in this way enables
organizations to evaluate their ability to evolve on a five-point scale and
identify the actions they need to take in order to accelerate improvement in
how they manage with measures.
Phase 2 The Audit in Practice
In order to investigate the concepts identified in the case examples and apply
the audit in practice, an in-depth case study was undertaken to investigate
how the measurement system within an electrical wholesale company
changed over time. The company, which we will call Electrical plc, currently
has over 100 branches throughout the UK.
Changes in Electric Plcs performance measurement system over time
The initial measurement system used to evaluate Electricals performance was
designed to meet its requirements in the early 1990s. At that time, the
company had fewer branches and had a strategy of rapid growth in branch
coverage that drove sales expansion and short-term profitability. To support
this strategy, a branch profit and loss measurement system was implemented.
Each branch manager was responsible for the branches net profit and was
awarded a bonus related to profitability achieved. This measurement system
significantly contributed to the development of an entrepreneurial cultural and
was quite appropriate for the company then.
By the late 1990s, however, Electrical had increased its market share
considerably and now had more branches. The firms executives identified that
the entrepreneurial cultural it had actively encouraged through its performance
measurement system also resulted in competition between branches that
prompt sub-optional behaviour. As a result, the companys strategy was
changed to focus on inter-branch co-operation, the sharing of knowledge and
acting as one company with a network of branches rather than as independent
branches competing with each other.
Clearly, the performance measurement system would have to change too and
so a companywide balanced scorecard was implemented using an external
facilitator. The branch bonus system was then aligned to it. The revised
measurement and aligned bonus system contributed to a change of culture
within the organization. Branch staff now gave greater focus to customer
service, while increasing inter-branch co-operation and sharing. This change

33


Benchmarking
in behaviour has given Electrical a competitive advantage as it focuses on
customer service that enables higher profit margin, rather than just focusing
on sales as is the norm within its industry.
The Systems Change Capability
The changes in Electricals performance measurement system demonstrate
the importance of ensuring that measurement systems remain appropriate to a
companys environment and strategy. Having changed its measurement
system to reflect its revised strategy, Electricals attention has now turned to
how they can ensure that the measurement and bonus systems remain
aligned to the firms environment and strategy. The companys executives
recognize that they are trading in a highly competitive marketplace and that
strategies and measures will have to change more frequently in the future if
competitiveness is to be maintained.
An audit of the measurement systems ability to change (evolutionary
capabilities), as outlined above, is an excellent way to address this problem.
In Electricals case, discussion around the audits findings highlighted the
actions required to ensure that their measurement system continues to
change appropriately over time as the companys circumstances change.
Process
In order to maintain evolution, Electrical have now implemented a 3-stage
process to review measures and the mechanisms by which data is collected
and reported. This includes quarterly review workshops to reflect on the
measures used and provide a forum to suggest new or modified measures for
introduction. A new process is also planned to encourage and facilitate
feedback from all branch managers as measures are used. This should
ensure that measures remain relevant to all organizational levels.
People
An external facilitator brought many of the skills required to modify and deploy
the balanced scorecard measurement system. Three directors are engaged in
the steering committee that manages the implementation and use of the
performance measurement system. This ensures that sufficient attention is
paid to its effectiveness. A dedicated balanced scorecard manager has been
employed to support the use and evolution of the performance measurement
system so that the skills required to manage and modify all elements of the
measurement system are maintained within the organization, reducing the
need for external support. An internal education program is planned to make
employees throughout the organization aware of how they can use the
measurement system effectively and contribute to its evolution.
Systems
The information systems used to collect, analyze and disseminate the
performance measurement data were developed internally. The skills and
resources that will enable maintenance and further development of these
systems are retained within the organization.

34

Quality of Supply and
Services
Cultural
Effort was put into developing a favourable culture for using the performance
measurement system. Significant resources were applied to communication
throughout the organization to obtain buy-in at all levels. Furthermore,
considerable attention has been placed subsequently on demonstrating that
action is based on the data collected and that feedback regarding the use and
relevance of the measures is actively sought. These actions encourage the
participation of employees in the measurement process and demonstrate that
their feedback and input will be acted upon. The radical change in attitudes
and behaviours at branch level speaks for itself.
The data collected from Electrical plc supports the findings of the first phase of
the research. It demonstrates the evolutionary cycle at work and that the
capabilities required to manage the way performance measurement system
change must be considered in order to maintain the relevance of the system in
the future.
Conclusions
Many organizations have spent considerable amounts of time and resources
redesigning their measurement systems to make them relevant to their current
circumstances. New measurement systems help organizations influence
behaviour and achieve strategic objectives. However as if these new
measurement systems fail to change or evolve as the organizations
circumstances and strategies change these new measurement systems will
become as inappropriate as the entirely financially focused traditional
measurement systems are currently considered. If organizations are to avoid
the need for another expensive overhaul of their measurement system it is
essential that the necessary capabilities are developed to enable evolution
and incremental change over time.
This research provides an understanding of how measurement systems can
be managed over time so that a dynamic and relevant set of performance
measures can be maintained, reflecting an organizations changing
requirements. It identifies and defines a complex combination of factors that
affect measures evolution and presents a framework that provides an
understanding of how this evolution can be managed. This then enables
organizations to cope with the changing environment they operate in and
modify their performance measurement system accordingly. The paleontology
museum will have to wait while these fit organizations survive to compete
another day.
(Source : DRUM Training Program : Performance Benchmarking and Quality of Supply and
Service, USAID Ministry of Power).