Environmental Science & Policy 54 (2015) 142–151

Contents lists available at ScienceDirect

Environmental Science & Policy

journal homepage: www.elsevier.com/locate/envsci

Measuring the sustainability of urban water services

Rui Cunha Marques a,*, Nuno Ferreira da Cruz b, João Pires c
a
CESUR – Center for Urban and Regional Systems, CEris, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisbon, Portugal
b
LSE Cities, London School of Economics and Political Science, Houghton Street, London WC2A 2AE, United Kingdom
c
Catholic University of Lisbon, Travessa Palma, 1649-023 Lisbon, Portugal

A R T I C L E I N F O A B S T R A C T

Article history: This paper discusses the concept of ‘sustainable water services’ and suggests a multicriteria method to
Received 15 April 2015 assess it. Although conceptual discussions around this notion are often confined to the triple bottom line
Received in revised form 2 July 2015 (TBL) classification, it seems that the TBL approach does not provide the suitable framework to measure
Accepted 3 July 2015
water services sustainability. It is argued that assets (or technical) and governance aspects are also
Available online 18 July 2015
indispensable dimensions. After revisiting the concept in broader terms, several criteria and metrics are
suggested to operationalize and quantify the sustainability level of urban water services. To aggregate
Keywords:
the numerous aspects that are relevant in this scope a multicriteria decision analysis approach is
Drinking water
proposed. Furthermore, to illustrate the real-world application of the method, a multicriteria model
Indicators
Multi-criteria decision analysis applicable to the case of Portugal was developed and calibrated with the input of a decision-maker with
Sustainability extensive experience in the sector. With the suggested framework it is possible to assess the global
Wastewater sustainability level of the water services (e.g. of each utility) and also the performance in each particular
dimension (‘social’, ‘environmental’, ‘economic’, ‘governance’ and ‘assets’).
ß 2015 Elsevier Ltd. All rights reserved.

1. Introduction Taking into account the theoretical and practical difficulties,

Although some recent efforts may be found in the literature (e.g.
van der Steen, 2011; van Leeuwen et al., 2011), there is no widely
accepted or established method to assess the sustainability level of
urban water services (UWS). In fact, sustainability assessments face
many obstacles. At the outset, the very definition of sustainability
poses significant conceptual challenges. How can one operationalize
such a wide-ranging notion? However, although there are many
challenges towards achieving the sustainable urban water cycle
(Brown et al., 2009), this pursuit is a major societal objective which
therefore makes measurement important.
Sustainability is usually associated with the triple bottom line
(TBL) framework, composed of social, environmental and econom-
ic dimensions or principles (Thornton et al., 2007). These
dimensions find correspondence in the ‘people, planet and profit’
phrase of Shell or to the ‘folk, place and work’ of the planner Patrick
Geddes in his definition of sustainability. The question is whether
the TBL framework is suitable to deal with UWS sustainability. We,
as some other authors, believe that the TBL approach is not
sufficient in this regard (ASCE and UNESCO, 1998 or Ashley et al.,
2003).
* Corresponding author.
E-mail address: rui.marques@tecnico.ulisboa.pt (R.C. Marques).
this paper proposes a framework and a method to assess the
sustainability of UWS. To accomplish this, the research strategy
encompasses three major steps: (1) the dimensions and objectives
of UWS sustainability are defined along with the respective
assessment criteria; (2) the quantitative and qualitative perfor-
mance metrics are identified for each criterion; (3) a multicriteria
decision analysis (MCDA) model is structured taking into account
the Portuguese context (for illustrative purposes). Most of the
conceptual work reported in this study (mainly corresponding to
the abovementioned steps 1 and 2) was developed within the
TRansitions to the Urban Water Services of Tomorrow (TRUST)
research project. The current operationalization of the multi-
criteria model (step 3) was carried out through a participatory
process involving an expert who acted as the decision-maker in the
modelling process (a former Director of EPAL,1 member of the
board of ERSAR2 and current Executive Director of PPA3).
1
EPAL (Empresa Portuguesa das Águas Livres) is a major water utility serving the
region of Lisbon (the capital of Portugal).
2
ERSAR (The Water and Waste Regulation Authority) is the Portuguese sector-
specific regulator.
3
PPA (Portuguese Water Partnership) is an association that ‘aims to develop
synergies and maximize potential for the development of the water sector in the
world, promoting the construction and consolidation of alliances and partnerships
between national institutions and all nations engaged in sustainable water use and
enhancement of water resources.’

http://dx.doi.org/10.1016/j.envsci.2015.07.003
1462-9011/ß 2015 Elsevier Ltd. All rights reserved.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151
After this brief introduction the second section reviews some of
the previous efforts to assess the sustainability level of urban water
services. In the third section we discuss the concept of
sustainability and argue for the addition of the ‘assets’ and
‘governance’ dimensions to the TBL framework. The fourth section
demonstrates the suitability of MCDA modelling regarding the
objective of assessing water services sustainability whereas the
fifth section presents a real application of the procedure. Finally,
the sixth section concludes the paper.
2. Sustainability assessments of urban water services
Sustainability is a trendy concept that is used in different
contexts and with several aims. Sustainable practices are currently
an objective that most organizations pursue; however, the
question is often to know about what is not sustainable rather
than what is sustainable (AWWARF and CSIRO, 2007). Considering
the various stakeholders, the uncertainties, the numerous possible
metrics and the trade-offs between them, it is difficult to know
how to become sustainable and to measure to what extent a
particular sector or activity is sustainable.
Nowadays, the success of a water (and wastewater) utility
depends on the provision of drinking water supply or the collection
of wastewater with the adequate quality levels but it also needs to
take into account the impact of its actions and decisions on people,
places and related resources, both in the short and long-term.
Therefore, along with the economic performance, water utilities
are now implementing new approaches to evaluate themselves.
Some of these approaches are directly related to sustainability
assessment through the TBL lenses, centred on financial, environ-
mental and social performances (although other sustainability
scorecards embrace further dimensions).
The Swedish Foundation for Strategic Environment Research is
responsible for one of the first studies referred to in the literature.
The project adopted the metabolism model and ‘health and
hygiene’, ‘social and cultural’, ‘environmental’, ‘economic’, ‘func-
tional’, and ‘technical’ categories where the criteria adopted for
assessing UWS sustainability were arranged according to their
influence on the water and wastewater systems (Hellström et al.,
2000). At that time the authors concluded that the tools to evaluate
the socio-cultural and functional aspects were not suitable. The
criteria used in the water sector were also analysed by Balkema
et al. (2002) in 15 publications to compare technologies relating to
sustainability and concluded that adequate attention was not
given to the socio and cultural aspects.
A set of sustainability indicators was developed by Lundin
(2003) to assess progress of water and wastewater systems as far
as sustainability is concerned. Those indicators comprise environ-
mental and technical factors and their choice took into account a
number of principles. The International Hydrological Programme
alongside the American Society of Civil Engineers (ASCE and
UNESCO, 1998) defined criteria and set guidelines for the
sustainability of water resource systems. By including economic,
environmental, ecological, social and physical objectives the
authors acknowledged the multidisciplinary, multi-objective and
multi-participatory characteristics of water resource sustainability.
In view of the need for new actions to achieve sustainable
management of the urban water systems, Bertrand-Krajewski et al.
(2000) came across methodological issues related to definition of
objectives, modelling, decision-making tools, metrology and
multidisciplinarity. According to the authors these are essential
conditions to develop the knowledge on the indicators and criteria
used in the methodologies of sustainability assessment. Two
integrated approaches were followed: one related to time and
space scales and the other in the scope of multidisciplinarity. Three
groups of methodological problems that get in the way of a suitable
143
assessment of the indicators and criteria were identified by these
authors, particularly the clear definition of objectives and
operational objectives, the concerted and multidisciplinary mea-
surements and the quality of the metrology.
Mitchell (2006) studied the concept of ‘integrated urban water
management’ (IUWM) as a central pillar of sustainability. A full
process of urban water services, whose components of an
integrated ‘physical system’ are the drinking water supply and
the wastewater collection systems, also comprises the organiza-
tional framework and the surrounding environment. It was shown
that it was possible to successfully put into effect the IUWM
approach, while being technically reasonable and acceptable to
stakeholders. Some of the available IUWM tools that must be
combined to result in the integrated total system solutions
required by urban communities are water recycling, water
efficiency programmes, and water sensitive stormwater manage-
ment. According to this author, the IUWM approach considers that:
(a) all components of the water cycle make part of an integrated
system; (b) all dimensions of sustainability are well-adjusted; (c)
all stakeholders are involved, including all water users; (d) all
water uses are addressed and (e) all idiosyncrasies of the local
context are taken into account.
A typology of five organizational development phases in
sustainable urban water management (including the project,
outsider, growth, insider, and integrated development phases) was
proposed by Brown (2008) as a heuristic model and/or a capacity
benchmarking tool for stakeholders. The results proved that the
political institutionalization of environmental matters along with
the commitment to local leadership and organizational learning
‘are key corporate attributes for enabling sustainable manage-
ment’.
In the UK, the industry umbrella in the water sector built a set of
25 indicators to measure the water utilities progress concerning
environmental sustainability (Water UK, 2000). Since then this
organization has been computing a set of sustainable water
indicators yearly (see Water UK, 2010). Afterwards, a research
project known as SWARD produced a sustainable water services
procedural guide (Ashley et al., 2003, 2004). The aim of this tool is
to help water utilities to make sustainable decisions. It makes a
distinction between principles, criteria and indicators of sustain-
ability and sorts the criteria (or dimensions) into economic,
environmental, social and technical and each one of these into
performance indicators.
In the U.S., Monsma et al. (2009) put emphasis on the role of
integrated water resources management to improve the sustain-
ability in water systems. The sustainable path for the U.S. water
infrastructure found support in a set of 20 performance metrics
which includes the following components: affordability, advanced
procurement and project delivery methods, asset management,
climate change mitigations and adaptation, conservation and
water efficiency, costs of development, energy management,
environmental impacts, full cost pricing, good governance,
modernized plant operations, network optimization, public
outreach and stakeholder investment, regulatory optimization,
security and emergency preparedness, stewardship, transparency,
watershed and regional optimization, workforce management and
research, and technological and managerial innovation.
Recently, an indicator approach called City Blueprint was
developed by the KWR Watercycle Research Institute and Deltares
to assess the sustainability of the urban water cycle. The City
Blueprint includes elements of water footprints, urban metabo-
lism, ecosystem services and indicators (van Leeuwen et al., 2011,
2012; van Leeuwen and Chandy, 2013). The research uses
24 indicators from eight broad categories, namely (1) water
security following the water footprint approach as developed by
Hoekstra (2003), (2) water quality, which includes surface and
144 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151
groundwater, (3) drinking water, (4) sanitation, (5) infrastructure,
(6) climate robustness, (7) biodiversity and attractiveness and (8)
governance. This approach has been applied in different Dutch
cities by means of an interactive multi-stakeholder approach
which includes relevant stakeholders. The results are disclosed in
‘spider web’ graphs (equivalent to a scorecard). Meanwhile van
Leeuwen and his TRUST colleagues applied this methodology in
other cities (van Leeuwen and Chandy, 2013). Because the
indicators are not aggregated, this tool does not allow for
estimating the overall sustainability level of UWS.
The ISO 24.500 also expresses a concern about the sustainability
of water and wastewater services. The ISO 24.500 series makes
reference to the ‘sustainability of water and wastewater utilities’
and to a ‘sustainable development of the community’, and provides
the definition of performance indicators for service improvement.
Therefore, the utilities must ‘address sustainable development, i.e.
the ability for the community to grow and prosper within the
environmental, infrastructural and economic resources available,
without limiting the use of those resources by future generations.’
Furthermore, the ISO 24.500 argues that the utilities should (a)
contribute to and implement sustainable water resources man-
agement policies and practices, (b) contribute to development
planning and resource allocation through consultation, provision
of information and analysis in conjunction with appropriate
institutions, (c) contribute to public health and safety, and (d)
implement information and education of the community on these
topics, particularly on the efficient use of water and pollution
prevention.
Finally, a comprehensive sustainability reporting framework
produced by the Global Reporting Initiative (GRI) became
important for the water utilities worldwide since it allows utilities
to manage their impact on sustainability development, showing
them how to exert positive change on the state of the economy,
and on environmental and social conditions. This reporting
framework was also well accepted due to its capability to measure,
track, and improve utilities’ performance on specific issues; and,
therefore, to diminish potential business risks (GRI, 2011).
Furthermore, GRI helps the utilities and other organizations to
manage and promote transparency and accountability because all
the information is freely available.
Although sustainability scorecards were rapidly adopted
among water utilities and UWS in some countries, examples of
this type of assessment are still scarce. The Australian utilities are
notable exceptions with a pioneering position in this scope. For
example, the Sydney Water Company uses the ‘sustainability
scorecard’ which includes social, economic and environmental
concerns among its objectives. The yearly published sustainability
scorecard comprises the company’s sustainability performance
measurement. It measures performance through sustainability
indicators and defines objectives for the metrics determined. The
sustainability scorecard encompasses summary statements and
progress ratings based on management evaluation of Sydney
Water’s performance against its sustainability indicators and
includes a set of performance indicators aimed at serving
customers (three indicators), maintaining a water efficient city
(4), providing clean and safe drinking water (2), contributing to
clean beaches, oceans, rivers and harbours (4), optimizing resource
use (5), developing a safe, capable and committed workforce (5),
and being an economically efficient business (4). Furthermore, in
New South Wales (NSW) the NSW Office of Water and IPART
(water regulator) use a TBL approach for each water utility so as to
provide a balanced view of the long-term sustainability. In the TBL
report there is a summary of the utilities’ compliance with the
requirements of the ‘best-practice guidelines and its performance
for over 50 key performance indicators together with the state-
wide medians.’
Other relevant case-studies are found in the U.S. (e.g. Seattle
and San Francisco Public Utilities), the UK (e.g. the above-
mentioned Water UK efforts), Canada (e.g. the cities of Hamilton
and Toronto), Singapore (Public Utilities Board), Philippines (e.g.
Manila Water), Israel (Tel-Aviv-Yafo urban water system) and in
some European countries (e.g. EPAL in Portugal). All these case-
studies were considered while developing the conceptual model
presented in the following sections (dimensions, objectives,
criteria and metrics). However, it comes as no surprise that
sustainability is not yet a priority in most developing countries.
3. The concept of sustainability
In the late 20th century the prevailing paradigm of develop-
ment was questioned. People became aware of the relationship
between the development of the economy and environmental
preservation and understood that only if environment protection
and social inclusion were addressed could economic progress be
sustained. These ideas became associated with the concept of
sustainable development. There is a wide range of definitions and
meanings of sustainability for different people. The definition of
‘Our Common Future’, also known as the Brundtland Report, from
the United Nations World Commission on Environment and
Development (WCED, 1987, p. 43) is one of the most used. The
report defines sustainability as the ‘development that meets the
needs of current generations without compromising the ability of
future generations to meet their needs and aspirations’. In a
somewhat different way, the International Union for the Conser-
vation of Nature defines sustainability as the ‘development that
improves the quality of human life while living within the carrying
capacity of supporting ecosystems’ (IUCN, 1991, p. 10). Although
the latter definition deals explicitly with the environment, both of
them focus on intergenerational equity.
In the scope of UWS, sustainability was firstly associated with
‘those water resource systems designed and managed to fully
contribute to the objectives of society, now and in the future, while
maintaining their ecological, environmental and hydrological
integrity’ (ASCE and UNESCO, 1998). Although the ‘objectives of
society’ may also include the economic and social aspects, this
definition clearly emphasizes the environmental dimension of
UWS. Indeed, UWS have made significant progress over time. A few
decades ago, water quantity, drinking quality and adequate
pressure were sufficient requirements for a suitable drinking
water service. Although necessary, today these aspects are far from
being sufficient requirements. UWS now must be efficient,
effective and customer-responsive. In addition, water utilities
recognize their role of organizations with a corporate social
responsibility that therefore must invest on the community.
The impact that the use of water resources for drinking water
might have on the water footprint can be small (Berrittella et al.,
2007, 2008a). Nevertheless, serious consequences for the daily life
of residential and industry customers might come up from other
water uses (e.g. agriculture or industry), water pollution or even
the lack of drinking water supply. Thus, the environment issues
related to the UWS are not the only ones that matter and they are
also very relevant in other dimensions. They must be considered in
an integrated way looking for the IUWM and the life-cycle costs of
the systems (Kluge, 2007). The many actors and stakeholders
belonging to several areas with various objectives and interests
make governance issues (e.g. participation and transparency) very
important in this scope as well. Moreover, the increasing cost of
water (and wastewater) services, the level of investment that is
necessary and the need to reflect the costs into the polluter and
user pay principles, along with the economic and social dimensions
are becoming more and more fundamental issues. UWS are central
for the social and economic cohesion of society. Not only does the
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 145

Table 1
Dimensions related to sustainability identified in the literature.

Authors Dimensions/Criteria

Spangenberg and Bonniot (1998) Economic; environmental; social; human rights; society
Hellström et al. (2000) Health and hygiene; social and cultural; environmental; economic; functional; technical
Hiessl et al. (2000) Social; economic; ecologic
UNCSD (2001) Environment; social; economic; institutional
Balkema et al. (2002) Economic; environmental; technical; social; cultural
Foxon et al. (2002) Economic; ecological; social; technical
GRI (2006) Economic; environment; social; human rights; society
Van Leeuwen et al. (2011) Water security; water quality; drinking water; sanitation; infrastructure; climate robustness; biodiversity
and attractiveness; and governance
Adapted from Marques (2012).

population wish to have sound and transparent drinking water
services at affordable prices but the customers also need to
feel the value for money spent. The awareness that the
sustainability of communities and cities (the so-called ‘city of
the future’) calls for the sustainability of urban infrastructure
and particularly of UWS is also increasing. Water services are a
crucial element of the livability and sustainability of cities
(Binney et al., 2010).
As mentioned above, the sustainability concept is often
confined to the TBL dimensions which can be taken as aggregated
objectives concerning a particular sector that should be developed.
Some authors call ‘criteria’ to these dimensions which find
correspondence in the ‘set of factors that may be used to assess
which of a range of options offers the greatest contribution to
achieving sustainability objectives’ (Ashley et al., 2004). But more
than discussing terminology issues the main question is whether
the TBL approach is the most suitable to deal with UWS
sustainability. Many authors, such as Ashley et al. (2004), disagree
with this notion and propose the technical, economic, environ-
mental and social dimensions. Murray et al. (2009) suggest the
economic, ecological, social, technical and human health dimen-
sions and Sahely et al. (2005) sort them out into environment,
economic, engineering and social dimensions. Also the study of
ASCE and UNESCO (1998) suggests the socio-economic, environ-
mental, public health and management dimensions as the major
ones. Table 1 illustrates the UWS sustainability dimensions
suggested by other authors in the literature (adapted from
Marques, 2012).
The ‘social’ dimension of UWS sustainability should take into
account the access to the services, the satisfaction of the users’
needs and expectations, the public acceptance and the relevant
role in the community of these services (Fleming, 2008). The
‘environmental’ dimension is related to the impact of UWS on
living and non-living natural systems and includes the optimiza-
tion of the use of water, energy and materials and the minimization
of the downstream negative impacts. The ‘economic’ dimension
encompasses all the objectives related to economic and financial
matters (such as the full cost recovery, da Cruz and Marques,
2013a).
However, some other crucial aspects of UWS sustainability are
still missing in this framework. The aspects related to the ‘assets’
sustainability are clearly out of the TBL dimensions and the ones
concerning ‘governance’, despite being included in the three
dimensions of the TBL, are also not specifically detailed. The ‘asset’
dimension is associated with the system of physical infrastructure
and might hold aspects related to the performance of the system,
its durability, reliability, flexibility and adaptability (Ashley et al.,
2002, 2003). Governance is related to transparency, the rules of the
game, the respect for those rules by the various stakeholders, their
participation in the decision-making process (particularly the
customers), the existence and alignment of city planning with the
UWS, the effectiveness and efficiency of the decisions and the
quality of the accountability and adjustment mechanisms (da Cruz
and Marques, 2013b).
Fig. 1 highlights the concept of UWS sustainability and its
dimensions (Marques, 2012; Brattebø et al., 2013). If one assumes
that the TBL is the skeleton of UWS, a common area will correspond
to the ‘assets’ and ‘governance’ dimensions (without which the
objectives of the TBL dimensions will not be reached). Hence,
sustainable development in UWS needs these dimensions to be
considered as instrumental to achieve sustainability.
Taking into account the literature review and the expertise of a
panel of experts (from the TRUST research project), the dimen-
sions, objectives and criteria of UWS sustainability were defined as
shown in Table 2 (adapted from Brattebø et al., 2013). The metrics
that will allow to measure performance in each criterion were
deliberately not defined in detail by the TRUST team (Marques,
2012). The suitable qualitative or quantitative descriptors should
be sought with the aid of the relevant decision-makers in each city,
region or country (depending on the scope of the assessment)
through structured participatory methods. However, some sugges-
tions or examples of how the criteria can be operationalized were
also developed (Marques and van Leeuwen, 2012). In sum, in the
current study UWS sustainability encompasses five dimensions
that may be further divided in 14 objectives. These objectives are
achieved if certain targets are met for several criteria.
4. Multicriteria modelling
4.1. Methodological approach
MCDA modelling is especially suitable to assess the sustain-
ability of UWS. Sound multicriteria models are able to consider the
many aspects of sustainability and, perhaps more importantly, the
distinctiveness of each UWS with its own group of stakeholders,
their preferences and objectives and its particular local (regional or
Fig. 1. Dimensions of sustainability.
Source: Brattebø et al. (2013).
146 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151

Table 2
Dimensions, objectives and criteria of the UWS sustainability.

Dimension Objectives Criteria

Social (a) Access to urban water services (a1) Physical service accessibility
(a2) Economic service accessibility
(b) Effectively satisfy the current users’ needs and expectations (b1) Quality of service
(b2) Drinking water quality
(c) Acceptance and awareness of UWCS (c1) Willingness to pay
(c2) Complaining
(c3) Acceptance of new sources of water
(d) Relevant role in community (d1) Social responsibility
(d2) Work conditions
Environment (e) Optimize the use of water, energy and materials (e1) Efficient use of water
(e2) Energy use
(e3) Material use
(e4) Final uses of efficiency
(f) Minimize downstream negative impacts (f1) Pollution prevention
(f2) Pollution control
Economic (g) Ensure economic sustainability of the UWCS (g1) Investment
(g2) Efficiency
(g3) Leverage
(g4) Liquidity
Governance (h) Public participation (h1) Participation initiatives
(i) Transparency (i1) Availability of information and documents
(i2) Accessible information and written documents
(i3) Public disclosure
(j) Accountability (j1) Individual mechanisms of accountability
(j2) Collective mechanisms of accountability
(k) Clearness, steadiness and measurability of the UWCS policies (k1) Clearness of policies defined ex-ante
(k2) Change of policies
(k3) Implementation of policies
(l) Existence and alignment of city planning (l1) Corporate planning
(l2) City planning
(l3) Water resources planning
Infrastructure (m) Performance (m1) Failures
(n) Robustness (n1) Flexibility
(n2) Adaptability
(n3) Reliability
Adapted from Brattebø et al. (2013).

national) conditions. Indeed, MCDA models have been commonly
used in the water sector since the pioneering research on UWS
sustainability (e.g. Ashley et al., 1999; Icke et al., 1999; Srinivasa-
Raju et al., 2000). Moreover, they have been used in other
applications in this sector (see, e.g. Hajkowicz and Collins, 2007;
Huang et al., 2011) as well as in others related to the environmental
issues (Berrittella et al., 2008b). The current study proposes a
MCDA approach to evaluate the sustainability of UWS that involves
the consideration of sustainability attributes (and the respective
metrics) for the abovementioned five dimensions of UWS
sustainability. MCDA allows for aggregating the performances in
all of those attributes by applying weighting techniques that reflect
the legitimate stakeholders’ preferences.
Using a multicriteria model to assess UWS sustainability has
several advantages, for instance (da Cruz and Marques, 2013b): (1)
all types of attributes (criteria) and respective metrics (qualitative
or quantitative) can be included; (2) the several objectives, criteria,
scores and weighting coefficients are unambiguous and transpar-
ent, allowing for open and informed discussion; (3) the modelling
process is participatory and traceable, which facilitates communi-
cating the results and auditing and reviewing the model; (4) it is
possible to calculate partial (e.g. in a single dimension of
sustainability) and global (taking into account the all criteria
from all dimensions) scores, which increases the informative value
of the model and can be useful for policy-making (Larson et al.,
2009); (5) the procedures to ascribe scores and weights are
theoretically robust.
After the designation of the legitimate decision-maker (or
decision-making group) who will provide his/her judgments
regarding the relative contribution of scoring in each criterion
for overall sustainability, a simple additive aggregation model can
be used to calculate the sustainability score of each UWS (see
Eq. (1)).
X
n X
jSðui Þ ¼ C j  S j ðu j Þ with C j > 0 and Cj
j
j¼1

S j ð‘good’ j Þ ¼ 100
¼ 1 and (1)
S j ð‘neutral’ j Þ ¼ 0
Where, S(ui) is the global sustainability score of UWS ui, cj is the
weighting coefficient of criterion j, Sj(ui) is the local score of UWS ui
considering criterion j, and ‘good’j and ‘neutral’j are the reference
levels of performance on criterion j (corresponding to what is
considered by the decision-maker as being a good practice and
satisfactory or acceptable in terms of sustainability, respectively).
For each modelling problem, the input of the suitable decision-
maker will be crucial to determine and calibrate each one of these
parameters.
4.2. Criteria set
Assessing UWS sustainability is a problem that involves the
consideration of several features (e.g. dimension, objectives,
attributes, consequences, constraints. . .). To use an additive
evaluation model such as the one represented by Eq. (1), the
criteria that represent the fundamental points of view (or
attributes) must respect certain properties (da Cruz and Marques,
2013b). Above all, the criteria set must be exhaustive. That is,
considering that the assessment model should be as simple and
concise as possible (the goal is to transform a complex problem in a
simpler problem), it must also be complete, in the sense that all key
aspects concerning UWS sustainability ought to be included.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 147

Furthermore, the criteria must be preferentially independent (i.e.
the performance in one criterion must not influence or be
contingent upon the performance in any other criterion). Including
different criteria that ultimately will ‘measure’ the same phenom-
ena should also be avoided since redundancy in the model may
lead to the overvaluation of some aspects.
As we have seen, a panel of experts from the TRUST research
project proposed the criteria set shown in Table 2 to operationalize
and measure UWS sustainability. In a real-world development of a
MCDA model these criteria may be adapted taking into account the
opinion of the decision maker (see Section 5). The final criteria are
usually represented in graphs known as ‘value trees’ to organize
the various attributes according to a logical framework (Fig. 2
shows the value tree of our illustrative model).
4.3. Performance descriptors
Each criterion will have one performance descriptor that
operationalizes it. A performance descriptor is an ordered set of
plausible impact levels for a criterion (i.e. an indicator, performance

Fig. 2. Dimensions, objectives and criteria of UWCS sustainability (M-MACBETH software).

148 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151
categories, etc.) that allows us to measure the degree to which the
objectives are being attained (da Cruz and Marques, 2013a).
Descriptors can be quantitative or qualitative. Either way they
must respect some properties to correctly describe the performance
of the UWS in each criterion, namely they must describe the
performance of the UWS in each criterion as objectively as possible
and preserve the independence of the criteria.
4.4. Scoring functions
In MCDA models there are two evaluation stages. For instance,
in our case: (a) the local (or partial) assessment of UWS
sustainability in each criterion and (b) the global assessment of
UWCS sustainability taking into account all criteria (i.e. the
aggregation of the partial scores). During the first stage a scoring
function will have to be determined for each criterion. Indeed, with
the descriptors an analyst is able to measure the performance or
impact of the UWS in each criterion (e.g. in Euros, number of
customers, qualitative level or any other unit, depending on the
selected descriptor). But he/she will need a scoring function to
transform the impacts into scores (cardinal values in predefined
scales). These scores represent the attractiveness of the impacts
according to the preferences of the decision-maker. The shape of
the scoring functions is important because for some criteria the
marginal increase in the score may vary with the impact ranges
(i.e. a linear relationship between performance and score may not
always be admissible). There are several methods available for
estimating non-linear scoring functions, for example, ‘direct-rating’
or the ‘bisection method’ (von Winterfeldt and Edwards, 1986).
4.5. Weighting coefficients
Weighting coefficients are ‘scaling constants’ that transform
partial scores (in each criterion) in global scores. Assuming that
weights reflect the intrinsic ‘importance’ of the criteria for the
overall score is ‘the most common critical mistake’ of MCDA
modelling (Keeney, 1992). In fact, weighting coefficients represent
the importance that the decision-maker gives to the swings in
performance (e.g. from ‘neutral’ to ‘good’) in each criterion. In other
words, the ‘importance’ is contingent upon the performance range
in each criterion.
In compensatory models (such as the one shown in Eq. (1)),
weights operationalize the notion of ‘trade-off’ (i.e. what perfor-
mance the decision-maker considers to be necessary to attain in
one criterion to compensate a poor performance in another
criterion). Once again, there are several methods available for
weight elicitation such as ‘swing weighting’ or the ‘trade-off’
procedure (Greco et al., 2010). There are also non-numerical
techniques such as the MACBETH approach (Bana e Costa et al.,
2003) with which it is possible to determine the weights by asking
the decision-maker to carry out pairwise comparisons through
qualitative judgments of the differences in preference of certain
reference profiles (see Section 5).
5. Real application: a working model for Portugal
5.1. Taking into account the preferences of the decision-maker
To illustrate the application of the MCDA model, we discussed
the criteria and the descriptors of the sustainability dimensions
with a decision-maker with extensive experience in UWS
management in Portugal and abroad (the third author of this
study). These structured discussions were held in five decisions
meetings (one meeting per sustainability dimension where in the
last meeting the procedure that allows for the normalization of all
the weights was also carried out). Each criterion and respective
performance descriptor suggested by the conceptual work
developed within the TRUST R&D project was thoroughly analysed
by the decision-maker. Several adaptations were necessary so that
the MCDA model would adhere to the preferences of the decision-
maker and therefore some criteria were deleted and some new
items were also created. The final criteria are shown in Fig. 2. The
respective performance descriptors are available in the Appendix
to this paper.
After pinpointing the fundamental criteria (the basis of any
additive evaluation model) we asked the decision-maker to define
the upper (‘good’) and lower (‘neutral’) reference levels in terms of
performance for each criterion (see Table 3). This operationalizes the
idea of a ‘good performance’ (the level above which are the
sustainability best practices) and a ‘neutral performance’ (the level
below which performance is perceived to be insufficient or
substandard). As a simplification (and to expedite the decision
meetings), this model assumes a linear relationship between
performance and score (i.e. linear scoring functions). This simplifi-
cation was discussed with the decision-maker who concluded to be
perfectly reasonable to assume liner scoring functions considering
the performance descriptors used in the model.
5.2. Computing the weights
As we have seen, weights must be computed taking into
account the criteria impact scales. The weighting coefficient of a
criterion reflects the increase in the global score associated with
a swing in that criterion between the ‘neutral’ level and the
‘good’ level (da Cruz and Marques, 2013a,b). To compute the
weights through a transparent and participatory procedure the
MACBETH approach was adopted (Bana e Costa et al., 2003). This
approach uses pairwise comparisons of performance swings in
the criteria (i.e. qualitative judgments made by the decision-
maker when comparing two criteria at a time) to determine the
weights. To model the differences in preference the decision-
maker may use the seven MACBETH categories for the
judgments, namely: ‘no’, ‘very weak’, ‘weak’, ‘moderate’, ‘strong’,
‘very strong’ or ‘extreme’ difference. The following protocol was
used in each decision meetings (i.e. for each UWS sustainability
dimension):
Consider a number of ‘hypothetical’ UWS (one UWS for each
criterion) each one of them ‘neutral’ in all criteria except in one for
which the performance is ‘good’ (all alternatives would have a
‘good’ performance in a different criterion).
Rank these ‘hypothetical’ UWS according in order of preference
(from the more sustainable to the least sustainable).
Compare these UWS in terms of preference by providing
qualitative judgments using the seven MACBETH categories.
Fig. 3 displays the results of these protocols for the ‘assets’
dimension (the procedure was repeated for every dimension). As
the matrix below shows, there is no need to make all possible
pairwise comparisons. During the decision meetings only conse-
cutive criteria was compared. The M-MACBETH assists the
modelling process and automatically provides the weights that
are coherent with the judgments of the decision-maker (through
linear programming algorithm, see Bana e Costa et al., 2003).
The matrix of judgments displayed in Fig. 3 allows for
computing the intra-weights of the ‘asset dimension’. To
compute the global weights (so one can determine de global
sustainability score of a given UWS) it was necessary to
normalize all the intra-weights. This was accomplished by
selecting one criterion from each of the five dimensions (the
criteria with the highest intra-weights were arbitrarily chosen)
and repeating the protocol described above (see Fig. 4). With this
final matrix of judgments we were able to normalize all weights
through linear transformations.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151 149

Table 3
‘Neutral’ and ‘Good’ reference levels according to the decision-maker’s preferences.

Code Criteria ‘Good’ reference level ‘Neutral’ reference level

A1 Physical service accessibility Level II Level IIII

A2 Economic service accessibility 1.5% 3.0%
A3 Quality of service Level II Level IIII
A4 Drinking water quality 99.0% 97.5%
A5 Willingness to pay (WTP) Level II Level IIII
A6 Complaining 1 3
A7 Acceptance of new sources of water 90.0% 80.0%
A8 Social responsibility 0.50% 0.25%
A9 Work conditions Level II Level IIII
B1 Efficient use of water Level II Level IIII
B2 Energy use Level II Level IIII
B3 Material use 70% 50%
B4 Final uses of efficiency Level II Level IIII
B5 Pollution prevention and control Level II Level IIII
C1 Investment Level II Level IIII
C2 Efficiency Level II Level IIII
C3 Leverage 2.5 4.0
C4 Liquidity 1.0 0.5
D1 Participation initiatives Level III Level IV
D2 Existence of information and its documentation Level II Level IV
D3 Access to information Level II Level IV
D4 Individual mechanisms of accountability Level II Level IV
D5 Collective mechanisms of accountability Level II Level IV
D6 Clearness and stability of policies Level II Level IV
D7 Strategy and alignment with city planning Level II Level IIII
E1 Failures Level II Level IIII
E2 Flexibility Level II Level IIII
E3 Adaptability Level II Level IIII
E4 Reliability Level II Level IIII

Fig. 3. Matrix of judgments for the ‘assets’ dimension (M-MACBETH software).

Fig. 4. Matrix of judgments for the hierarchical model (M-MACBETH software).

5.3. Results and discussion
Fig. 5 shows the results (i.e. the weights) for all the criteria
according to preferences of the decision-maker. With the criteria
set (see Fig. 2 and Table 3), the performance descriptors (see
Appendix) and having all the weights (Fig. 5), it is possible to
determine the overall sustainability level any UWS (e.g. a utility)
and also how it is faring in each dimension (provided that the data
for the selected metrics are available).
The definition used for the establishment of the ‘good’ and
‘neutral’ levels of reference in each criterion associates an
empirical meaning to the global score attained by any UWS.
150 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151
Fig. 5. Weights of the UWCS sustainability criteria.
Indeed, in face of a sustainability score attained through the model
described above, one is immediately able to categorize the UWS as
follows:
 Highly sustainable UWS – if it has a score above 100 (i.e. more
sustainable than a UWS that has a performance equal to the
‘good’ level in all criteria).
 Sustainable UWS – if it has a score between 0 and 100.
 Unsustainable UWS – if it has a score below 0 (i.e. less sustainable
than a UWS that has a performance equal to the ‘neutral’ level in
all criteria).
Finally, it should be noted that to carry out a global evaluation
with an additive aggregation model (Eq. (1)) the legitimate
decision-maker has to accept the ‘compensatory’ assumption. If,
for some reason, a low performance in a given criterion should
automatically mean that the global sustainability score cannot be
above a certain threshold (irrespective of the actual performance in
all the other criteria), then the aggregation of partial scores
through a typical compensatory model is not admissible. The ‘veto
power’ of some criteria would require the use of non-compensa-
tory models to perform a global evaluation. However, these models
entail some methodological drawbacks and reduce the acceptabil-
ity and informative value of the results (due to the ‘black box’
effect).
6. Conclusion
This study proposes a UWS sustainability scorecard based on a
MCDA model. It reviews the concept of UWS sustainability as well
as previous efforts regarding the operationalization and measure-
ment of this concept. The scorecard is based on five dimensions
(the TBL plus the ‘assets’ and ‘governance’ aspects), 14 objectives of
UWS sustainability, their criteria and performance metrics. Since it
involves a participatory and intuitive procedure, the MACBETH
approach was used in the MCDA modelling illustration to compute
the weights in order to aggregate the several criteria. While
applying this weighting procedure it is possible to take into
account the local conditions and reflect the preferences of the
stakeholders in the model.
The paper contributes to the literature in three different
ways. First, the conceptual argument around the need to include
the ‘assets’ and ‘governance’ dimensions along with the classical
‘environmental’, ‘economic’ and ‘social’ concerns when addres-
sing UWS sustainability is a relevant innovation. Indeed, as
pointed out in the specialized literature, these two overarching
aspects are instrumental to achieve sustainability in the water
sector and not adequately enclosed in the TBL framework.
Second, the suggested criteria and the performance descriptors
provided in the Appendix may be valuable sources for academics
and practitioners willing to undertake their own sustainability
assessments. And third, with a practical example of how a model
was structured considering the Portuguese reality (although to
increase the legitimacy of the model the decision meetings
should have involved more stakeholders with responsibilities in
the water sector and even customer representatives and other
actors), the current research provides a useful roadmap for
MCDA modelling through participatory procedures. Possible
follow-up research might involve the calibration of this model
and its application to a group of Portuguese water and
wastewater utilities.
Acknowledgements
The first part of this work was carried out within the EU
Research Project TRUST (Transitions to the Urban Water Services of
Tomorrow). We would like to thank the TRUST team for their
useful discussions. The European Commission is acknowledged for
funding TRUST in the 7th Framework Programme under Grant
Agreement No. 265122.
Appendix A. Supplementary data
Supplementary data associated with this article can be found, in
the online version, at http://dx.doi.org/10.1016/j.envsci.2015.07.
003.
References
ASCE and UNESCO, 1998. Sustainability Criteria for Water Resource Systems.
ASCE Task Committee on Sustainability Criteria and the Working Group of
UNESCO/IHP IV Project M-4.3, Reston, Virginia, USA.
Ashley, D., Butler, P., Pearson, P., Jowit, P., Moir, J., 2003. Sustainability decision
making for the UK water industry. Proc. Inst. Civ. Eng. Eng. Sustain. 156 (12),
41–49.
Ashley, D., Souter, D., Butler, P., Davies, J., Dunkerley, J., Hendry, S., 1999.
Assessment of the sustainability of alternatives for the disposal of domestic
sanitary waste. Water Sci. Technol. 39 (5), 251–258.
Ashley, D., Blackwood, D., Butler, P., Davies, J., Jowit, P., Smith, H., 2002.
Sustainability criteria for decision support in the UK water industry. J.
Environ. Plan. Manag. 45 (2), 285–301.
Ashley, R., Blackwood, D., Butler, D., Jowitt, P., 2004. Sustainable Water Services:
A Procedural Guide. IWA Publishing, London.
AWWARF AND CSIRO, 2007. Triple Bottom Line Reporting of Sustainable Water
Utility Performance. AWWA Research Foundation and CSIRO Land and
Water, Australia Capital Territory.
Balkema, A., Preisig, H., Otterpohl, R., Lambert, F., 2002. Indicators for the
sustainability assessment of wastewater treatment systems. Urban Water 4
(2), 153–161.
Bana e Costa, C., De Corte, J., Vansnick, J., 2003. MACBETH. Working Paper
LSEOR, no. 56. London School of Economics and Political Science,
London.
Berrittella, M., Katrin Rehdanz, K., Roson, R., Tolb, R., 2008a. The economic
impact of water taxes: a computable general equilibrium analysis with an
international data set. Water Policy 10 (3), 259–271.
Berrittella, M., Certa, A., Enea, M., Pietro, Z., 2008b. Transport policy and climate
change: how to decide when experts disagree. Environ. Sci. Policy 11 (4),
307–314.
 R.C. Marques et al. / Environmental Science & Policy 54 (2015) 142–151
Berrittella, M., Hoekstra, A., Rehdanz, K., Roson, R., Richard, T., 2007. The
economic impact of restricted water supply: a computable general
equilibrium analysis. Water Res. 41 (8), 1799–1813.
Bertrand-Krajewski, J., Barraud, S., Chocat, B., 2000. Need for improved
methodologies and measurements for sustainable management of urban
water systems. Environ. Impact Assess. 20 (3), 323–331.
Binney, P., Donald, A., Elmer, V., Ewert, J., Philips, O., Skinner, R., Young, R., 2010.
IWA cities of the future program. In: Spatial Planning and Institutional
Reform Conclusions from the World Water Congress, September.
Brattebø, H., Alegre, H., Cabrera, E., Marques, R., Hein, A., Cruz, C., 2013. A
Master Framework for UWCS Sustainability. TRUST Project.
Brown, R., 2008. Local institutional development and organizational change for
advancing sustainable urban water futures. Environ. Manag. 41 (2), 221–233.
Brown, R., Keath, N., Wong, T., 2009. Urban water management in cities:
historical, current and future regimes. Water Sci. Technol. 59 (5), 47–855.
da Cruz, N.F., Marques, R., 2013a. A multi-criteria model to determine the
sustainability level of water services. Water Asset Manag. Int. 9 (3), 16–20.
da Cruz, N.F., Marques, R., 2013b. New development: the challenges of designing
municipal governance indicators. Public Money Manag. 33 (3), 209–212.
Fleming, N., 2008. Understanding ‘what’s really going on’ as a basis for
transforming thinking, action and our cities. In: Enviro 08 Australasia’s
Environmental & Sustainability Conference & Exhibition, Melbourne,
Australia.
Foxon, T., Mcilkenny, D., Gilmour, C., Oltean-Dumbrava, N., Souter, R., Ashley, D.,
Butler, P., Pearson, P., Jowit, P., Moir, J., 2002. Sustainability criteria for
decision support in the UK water industry. J. Environ. Plan. Manag. 45 (2),
285–301.
Greco, S., Ehrgott, M., Figueira, J.R., 2010. Trends in Multiple Criteria Decision
Analysis. Springer, Berlin.
GRI, 2006. Sustainable Reporting Guidelines. Global Reporting Initiative,
Amsterdam.
GRI, 2011. Sustainability Reporting Guidelines. Global Reporting Initiative,
Amsterdam.
Hajkowicz, S., Collins, K., 2007. A review of multiple criteria analysis for water
resource planning and management. Water Resour. Manag. 21 (9), 1553–1566.
Hellström, D., Jeppsson, U., Kärrman, E., 2000. A framework for systems analysis
of sustainable urban water management. Environ. Impact Assess. 20 (3),
311–321.
Hiessl, H., Walz, R., Toussaint, D., 2000. Design and Sustainability Assessment of
Scenarios of Urban Water Infrastructure Systems. Published Report.
Fraunhofer Institute Systems and Innovation Research, Breslauer Str.,
Karlsruhe.
Hoekstra, A., 2003. Virtual Water Trade: Proceedings of the International Expert
Meeting on Virtual Water Trade. Value of Water Research Report Series No.
12. UNESCO-IHE.
Huang, I.B., Keisler, J., Linkov, I., 2011. Multi-criteria decision analysis in
environmental sciences: ten years of applications and trends. Sci. Total
Environ. 409 (19), 3578–3594.
Icke, J., Boomen, I., Aalderink, R., 1999. A cost-sustainability analysis of urban
water management. Water Sci. Technol. 39 (5), 211–218.
IUCN, 1991. Caring for the Earth: A Strategy for Sustainable Living. Earthscan,
London.
Keeney, R., 1992. Value-Focused Thinking: A Path to Creative Decision Making.
Harvard University Press, Cambridge, MA.
Kluge, T., 2007. Integrated water resources management (IWRM) as a key for
sustainable development. In: Cogoy, M., Steininger, K.W. (Eds.), The
151
Economics of Global Environmental Change. International Cooperation for
Sustainability. New Horizons in Environmental Economics. Edward Elgar,
Cheltenham (UK), Northampton (MA), pp. 134–154.
Larson, K.L., Whiteb, D.D., Gobera, P., Harlanc, S., Wutichc, A., 2009. Divergent
perspectives on water resource sustainability in a public-policy-science
context. Environ. Sci. Policy 12 (7), 1012–1023.
Lundin, M., 2003. Indicators for Measuring the Sustainability of Urban Water
Systems: A Life Cycle Approach. (Ph.D. thesis)Chalmers University of
Technology.
Marques, R., 2012. MCDA approach for the UWCS sustainability scorecards.
TRUST Internal Report.
Marques, R., van Leeuwen, K., 2012. Current State of Sustainability of Urban
Water Cycle Services. TRUST Project.
Mitchell, V., 2006. Applying integrated urban water management concepts: a
review of Australian experience. Environ. Manag. 37 (5), 589–605.
Monsma, D., Nelson, R., Bolger, R., 2009. Sustainable Water Systems: Step One –
Redefining the Nation’s Infrastructure Challenge. A Report of the Aspen
Institute’s Dialogue on Sustainable Water Infrastructure in the US,
Washington, DC.
Murray, A., Ray, I., Nelson, K., 2009. An innovative sustainability assessment for
urban wastewater infrastructure and its application in Chengdu, China. J.
Environ. Manag. 90 (11), 3553–3560.
Sahely, H., Kennedy, C., Adams, B., 2005. Developing sustainability criteria for
urban infrastructure systems. Can. J. Civil Eng. 32 (1), 72–85.
Spangenberg, J., Bonniot, O., 1998. Developing sustainability criteria for urban
infrastructure systemsIn: Sustainability Indicators: A Compass on the Road
Towards Sustainability. Wuppertal Paper no. 81, Wuppertal. .
Srinivasa-Raju, K., Duckstein, L., Arondel, C., 2000. Multicriterion analysis of
sustainable water resources planning: a case study in Spain. Water Resour.
Manag. 14 (6), 435–456.
Thornton, G., Franz, M., Edwards, D., Pahlend, G., Nathanaile, P., 2007. The
challenge of sustainability: incentives for brownfield regeneration in Europe.
Environ. Sci. Policy 10 (2), 116–134.
UNCSD, 2001. Indicator of Sustainable Development. Guidelines and
Methodologies. United Nations Commission for Sustainable Development,
New York.
van der Steen, P., 2011. Application of Sustainability Indicators within the
Framework of Strategic Planning for Integrated Urban Water Management. A
Training Manual for Process Facilitators of Urban Strategic Planning
Processes. SWITCH, UNESCO-IHE Institute for Water Education.
van Leeuwen, K., Chandy, P., 2013. The cities blueprint: experiences with the
implementation of 24 indicators to assess the sustainability of the urban
water cycle. Water Sci. Technol. 13 (3), 769–781.
van Leeuwen, K., Frijns, J., van Wezel, A., 2011. Indicators for the Sustainability
of the Urban Water Cycle. Watercycle Research Institute (KWR), The
Netherlands.
van Leeuwen, K., Frijns, J., van Wezel, A., van de Ven, F.H.M., 2012. Cities
blueprints: 24 indicators to assess the sustainability of the urban water
cycle. Water Resour. Manag. 26, 2177–2197.
von Winterfeldt, D., Edwards, R., 1986. Decision Analysis and Behavioral
Research. University Press, Cambridge.
Water UK, 2000. Towards Environmental Sustainability: Indicators for the UK
Water Industry. Water UK, London.
Water UK, 2010. Sustainable Indicators 2009–10. Water UK, London.
WCED, 1987. Our Common Future (Brundtland Report). World Commission on
Environment and Development. Oxford University Press, New York.