You are on page 1of 10

Water Financing Partnership Facility

RETA 6498: Knowledge and Innovation Support for ADB’s Water Financing
Program

PILOT AND DEMONSTRATION ACTIVITY


Final Report

March 2017

FIJ: Pilot Implementation of


AquaRating in the Water Authority of
Fiji

Prepared by Water Authority of Fiji (WAF)

The Pilot and Demonstration Activity Report is a document of the proposer. The views expressed herein do not
necessarily represent those of ADB’s Board of Directors, Management, or staff, and may be preliminary in
nature. In preparing any country program or strategy, financing any project, or by making any designation of or
reference to a particular territory or geographic area in this document, the Asian Development Bank does not
intend to make any judgments as to the legal or other status of any territory or area.
PILOT AND DEMONSTRATION ACTIVITY (PDA)

“Pilot Implementation of AquaRating in the Water Authority of Fiji”

PDA Completion Report

A. Introduction:

1. This report summarizes the activities, findings and outcomes of the pilot
implementation of the AquaRating tool in the Water Authority of Fiji (WAF) and puts forward
recommendations on the way forward.

B. Background:

2. The pilot implementation of AquaRating in WAF under the ADB’s Facility for Pilot and
Demonstration Activity (PDA) was carried out by the International Water Association (IWA)
through its AquaRating Team1, This pilot implementation represents the first application of the
tool in the Asia Pacific Region.

3. AquaRating is a new and unique rating system for water and wastewater utilities
developed as an initiative of the Inter-American Development Bank (IDB), in close partnership
with IWA. It is a voluntary and universal system that offers a comprehensive and objective
rating of the water and wastewater service provision.

4. The objectives of the pilot were to:

 test the applicability and usefulness of AquaRating in assessing and improving the
performance of water and wastewater service providers in the Asia-Pacific region;

 test the feasibility and usefulness of carrying out a comprehensive water and
wastewater service provision characterization using AquaRating; and

 test the potential for using AquaRating as a support utilities to design, execute and
monitor investment programs.

5. The scope of this pilot was limited to self-assessment and did not include the auditing
part.

6. This pilot implementation involved: (i) characterization of an urban water and


wastewater system management, and (ii) scoping and definition of performance improvement
actions.

1
Composed of Francisco Cubillo and Raimon Puigjaner
-
7. During implementation, a new AquaRating evaluation modality was defined and
applied to provide a reliable enough assessment as a baseline to define performance
improvement actions, without needing to run the AquaRating audit procedures.

8. In general, the pilot application of AquaRaring in WAF represents a comprehensive


case study with sufficient evidence to scale up the use of AquaRating for assessing and
improving utility performance in the Asia-Pacific Region.

WAF AquaRating implementation scope

C. WAF AquaRating implementation Activities and Results


9. The selection of WAF as pilot utility has proven to be an optimal choice considering the
utility’s general characteristics, and specially the commitment shown by the management2 and
staff3.
10. The WAF AquaRating implementation has been executed in 5 phases: 3 workshops
and 2 extended self-assessment exercises (the latter was exclusively carried out by WAF’s
staff).
 Note: the implementation of AquaRating in other utilities/contexts does not
necessarily mean replicating this 5 phases-execution model. Each case would be
duly analyzed to properly define an optimal application model based on needs and
expectations.

2
Led by WAF’s Chief Operating Officer, Mr. Opetaia Rava
3
Led by Manasa Tusulu
Phase 1 (January 2016)

11. The implementation started in late January 2016 through an inception workshop
facilitated by the IWA AquaRating Team to train WAF’s staff (internal implementation team,
management executives and chief operating officer) on how to apply the tool. This workshop
was also participated in by ADB staff4.

 As one of the main conclusions of this first workshop, and to better capture the
essence and use of AquaRating, WAF, ADB and the AquaRating Team agreed to
initially limit the scope of assessment of the AquaRating application in WAF to the
company’s Suva-Nausori system performance in 2015.

Phase 2 (February to May 2016)

12. The inception workshop was followed by the setting up of WAF’s internal
implementation team composed of staff responsible for providing information in each of the 8
assessment areas that make up the system, and led by WAF’s Strategic Planning Department
acting as coordination unit responsible for the overall self-assessment exercise. This
self-assessment exercise was performed between February and May 2016, and represented
the incorporation to the system of a significant amount of information demanded by a good
amount of AquaRating evaluation elements. A good practice that WAF adopted carried out
during the information collection and provision process was the identification and recording of
performance gaps and action requirements for short term and feasible operational
improvement (as correction measures for the gaps detected), as per the AquaRating
guidelines, both at the Suva-Nausori and the national systems.

Phase 3 (June 2016)

13. The self-assessment was followed by a second workshop put together by the
AquaRating Team and performed in June 2016 (and participated in by the ADB5), to review:

 the information provided to respond to each of the 80 AquaRating evaluation


elements self-assessed by WAF (also called validation process).

 the performance gaps and short term and feasible action requirements based on
the detection and recording exercise (described in the previous bullet point) carried
out by WAF’s internal implementation team.

14. As a general summary, the following were the conclusions from the second
workshop’s review process:

 WAF was still able to collect and provide some additional information regarding the
32 AquaRating evaluation elements left void during the self-assessment phase.

4
Alan Baird and Alain Morel
5
Alan Baird
 WAF’s internal AquaRating implementation team and steering committee (the
latter in charge of reviewing the results of the application of AquaRating in WAF)
showed a highly remarkable commitment to this performance assessment and
improvement exercise.

 From the utility’s perspective the results of the implementation of AquaRating in


WAF were: (i) very much focused on developing an operational and feasible
improvement action list, mainly for a short term horizon and were aligned with the
2017-2019 WAF Strategic Plan development.

 From the ADB’s perspective, the results of the implementation of AquaRating in


WAF were aligned with the new ADB-Fiji-WAF investment project/loan
development.

 WAF’s internal AquaRating implementation team performed an honest


self-assessment exercise, including frequent performance underestimations.

WAF AquaRating implementation benchmark (1)

Areas benchmark – WAF reviewed implementation vs pull of similar conditions


implementations

Areas' ratings (net ratings)


100

90

80

70

60

50 Net average

40 WAF net

30

20

10

0
SQ PE OE ME FS AS CG ES

19

Phase 4 (June 2016 to February 2017)

15. Following the second workshop, WAF’s internal AquaRating implementation team
used the July 2016-February 2017 period to collect and provide the additional information
needed to finalize the AquaRating self-assessment process. In addition, the team completed
the identification and recording of performance gaps and action requirements for short term
and feasible operational improvement, producing an action list document. Upon completion of
both activities, the AquaRating Team reviewed and analyzed the new information uploaded in
the system, as well as the action list document, in preparation for the March 2017 third and
final workshop.

Phase 5 (March 2017)

16. The third and final workshop was organized and delivered by the AquaRating Team in
March 2017, this time to:

 review/validate the additional information provided to respond to each of the


remaining 32 AquaRating evaluation elements self-assessed by WAF.

 run a consistency check on the overall information provided by WAF in the system
to ensure enough reliability as a baseline to define performance improvement
actions, without needing to run the AquaRating audit procedures. This exercise
reviews with different criteria and methods every variable and assessment element
uploaded to the system to identify potential self-evaluation mistakes or
misunderstandings, presented as an inconsistencies list outcome. Upon
completion of validation process, , the inconsistencies list was duly discussed and
jointly analyzed with WAF staff. Some amendments were carried out, triggering
some changes in WAF’s self-assessment, but overall the consistency check
provided significant reliability to the WAF self-assessment final results. Should the
AquaRating audit process had been implemented, it would only have checked the
truthfulness of the evidences provided and its correlation with the self-assessment,
skipping any sort of interpretation and reflection exercise performed with WAF staff
(the AquaRating audit process includes checking field equipment and
installations).

 run an in-depth benchmarking exercise comparing the WAF validated


self-assessment + consistency analysis’ results with those of a cluster of worldwide
utilities self-assessed in similar conditions. The exercise showed that WAF has
lower ratings in Operational Efficiency (OE), Financial Sustainability (FS) and
Environmental Sustainability (ES), but better scores in Access to Service (AS) and
Corporate Governance (CG). It is to be noted that these results represent the
application of AquaRating to the Suva Nausori area - scope of assessment and
rating (not to the whole country), which is the one with the best coverage in Fiji.

 review WAF’s final performance gaps and short term and feasible action
requirements detection and recording exercise.

 run a WAF AquaRating present and projected results progression in 3 different


scenarios for the 2015 – 2017 period based on WAF’s action list (developed as a
product of the AquaRating exercise).

 reflect on a WAF AquaRating mid-long term projected results progression. A rough


AquaRating performance-score improvement 2017-2020 projection based on the
assumption of undertaking certain actions and investments was carried out,
boosting the rating above 50.
WAF AquaRating implementation progress (4)

Rating

42.11 Forecasted rating as


45.79
per WAF’s actions list
41.87

2015 2017 2020


31

WAF AquaRating implementation progress (5)

Rating

42.11
45.79 Action plan
41.87

2015 2017 2020


32

D. WAF AquaRating implementation conclusions

17. Overall, the following were the conclusions from the WAF AquaRating implementation
by the end of March 2017:

 AquaRating has proven to be applicable in utilities in the Asia Pacific Region.


 AquaRating has proven to be useful to WAF to assess its performance -
representing a source of comprehensive evaluation and analysis, and a source of
motivation for the company to develop and implement an improvement plan based
on validated data.

 WAF’s staff has proven to be able to solve all AquaRating application challenges
by completing the comprehensive exercise with guidance from the IWA Aquarating
team.

 WAF has experienced a culture shift in relation to the need of establishing and
documenting protocols.

 AquaRating has proven to be useful to WAF to identify short to medium term and
feasible performance improvement actions.

 The AquaRating validated self-assessment + consistency analysis implementation


modality has proven to be adequate for WAF.

 WAF AquaRating implementation’s initial results have proven to be comparable to


those obtained by other similar utilities of the world.

 WAF has appetite to use AquaRating in a continued manner to assess and


compare its performance year by year (and at a national level too), as well as to
engage into an Asia Pacific Region AquaRating collective implementation and
benchmarking exercise (showcasing its application case).

 It is foreseen that AquaRating is a useful tool for WAF to further develop integral
action plans (governance, management, strategic and investment plans). WAF
has shown interest in this possibility.

 It would be advisable for WAF and ADB to assess the company’s 2017
performance and extend the use of AquaRating to the whole country and other
Asia Pacific Region utilities.

E. Outcomes for ADB

18. The following outcomes have been realized from this pilot implementation:

 Testing the applicability and usefulness of AquaRating in assessing and improving


the performance of water and wastewater services providers in the Asia Pacific
Region.

 Testing a simplified evaluation modality of AquaRating: the AquaRating validated


self-assessment + consistency analysis.

 Testing the appropriateness of AquaRating as a baseline to build integral


governance, management, strategic & investment plans.
 Assessing the potential use and benefits of AquaRating in other Asia Pacific
Region utilities/contexts, for instance: (i) running a regional benchmark exercise
and (ii) implementing AquaRating in bigger utilities.

F. Outcomes for IWA

19. The pilot implementation of the AquaRating in WAF has brought some suggestions to
IWA on the way how AquaRating can be implemented in the future, essentially:

 to look into ways to lessen reliance on heavy documentation during the verification
process. Currently the system relies heavily on documentation, understandable but
may not be complied with by many. AquaRating can pave the way for standardizing
many practices through proper but less burdensome documentation;

 to consider developing a lighter version of the system that will make it more
adaptable to smaller utilities in the Asia-Pacific region; and

 to look into how the number of experts doing the verification process can be
increased if AquaRating is to be rolled out to more utilities later. At the moment the
verification process rests on very few individuals.

G. WAF (and beyond) AquaRating implementation potential next steps

20. The following are the potential next steps that ADB and WAF may want to consider to
maximize the results of the pilot:

For WAF:

 Communicate the WAF AquaRating implementation exercise’s characteristics,


conclusions and benefits.

 Allow/facilitate WAF’s participation in the IWA Water and Development Congress &
Exhibition 2017 (Argentina).

 Run the WAF 2017 AquaRating implementation at the Suva-Nausori system level
and compare its results with the 2015 exercise’s.

 Run the WAF AquaRating implementation at the national level.

 Use AquaRating to further develop the company’s operational improvement action


list.

 Build an integral action plan (governance, management, strategy and


investment plan) based on this AquaRating implementation results.
 Together with the ADB, put together an Asia Pacific Region AquaRating collective
implementation and benchmarking exercise showcasing WAF’s experience.

For ADB:

 Test AquaRating in a bigger scale (bigger cities).

 Test AquaRating in a utility managing the service of a whole small nation (at a
national level, not segmented like in WAF’s case).

 Test AquaRating in different cities (or countries) of a well-defined geographical


area/region, and perform any kind of benchmarking exercise among different
utilities with some similarities or common factors.

 Use AquaRating in early stages when designing action plans for utilities
(governance, management, strategy & investment plans).

 Use AquaRating for analysing the correlation/correspondence of utilities


management practices’ maturity with their performance (AquaRating consistency
analysis methodology, which provides information necessary reliability levels).

Annexes for reference

- Phase 1: agenda, presentations and conclusions documents.


- Phase 3: agenda and conclusions documents.
- Phase 5: agenda and conclusions documents.