Energy & Buildings 209 (2020) 109711

Contents lists available at ScienceDirect

Energy & Buildings

journal homepage: www.elsevier.com/locate/enbuild

A comparative analysis of data-driven methods in building energy

benchmarking
Yong Ding a,b,∗, Xue Liu a,b,∗
a
Joint International Research Laboratory of Green Buildings and Built Environments (Ministry of Education), Chongqing University, Chongqing 400045, China
b
National Centre for International Research of Low-carbon and Green Buildings (Ministry of Science and Technology), Chongqing University,
Chongqing 400045, China

a r t i c l e i n f o a b s t r a c t

Article history: A reasonable building energy efficiency benchmarking program plays an important role in energy con-
Received 17 July 2019 sumption control and supervision. Previous studies have focused on the process of establishing a single
Revised 12 November 2019
benchmarking method, but few have compared the performances and outcomes of different methods. To
Accepted 17 December 2019
fill this gap, this paper selects three benchmarking methods—multiple linear regression (MLR) based on
Available online 19 December 2019
Energy Star, stochastic frontier analysis (SFA) and the descriptive statistics method (DSM) based on the
Keywords: national energy consumption standard in China—to develop benchmarking models. We demonstrate each
Energy performance method using data on the energy and building characteristics of 45 four- and five-star hotel buildings
Energy benchmarking located in Chongqing, China. To compare the consistency, robustness and explanatory ability of the three
Data-driven approaches methods, we first utilize the Spearman rank correlation analysis to test whether these methods have
Comparative analysis consistent energy efficiency ranks and then present Sankey diagrams to further reveal the interactions of
the estimated energy efficiency grades obtained from the three methods. It is found that the results of
DSM and SFA are most consistent, while MLR vs. SFA and MLR vs. DSM present significant differences in
evaluating building energy performance. In addition, DSM is more robust for evaluating the ranks of sam-
pled buildings, while SFA is more robust for evaluating energy efficiency grades. Furthermore, we discuss
the explanatory ability of each method. In addition to the building characteristics, the design and op-
erational characteristics of the HVAC system have great effects on building energy consumption. Finally,
we present suggestions for policy-makers regarding the development and implementation of the building
energy benchmarking program in Chongqing and for the management of buildings with different energy
performances to further improve the energy efficiency.
© 2019 Elsevier B.V. All rights reserved.

1. Introduction transformed by enhancing the efficiency of the supply and man-

1.1. Background
The total energy consumption for building operation in China
accounted for 20% of the total energy consumption in the coun-
try in 2015, while public and commercial building energy use ac-
counted for 30% of the total building energy consumption [1].
Facing the contradictions of increasing building energy consump-
tion alongside global warming, resource shortages and environ-
mental pollution, the Chinese government proposed the “Energy
Supply and Consumption Revolution Strategy (2016–2030)” in 2017
[2], indicating that China’s energy strategy should be gradually
∗
Corresponding authors at: Joint International Research Laboratory of Green
Buildings and Built Environments (Ministry of Education), Chongqing University,
Chongqing 40 0 045, China.
E-mail addresses: dingyongqq@163.com (Y. Ding), cheerps12@126.com (X. Liu).
aging the demand. Specifically, the implementation of double con-
trol of the total energy quantity and intensity is being promoted in
China. In addition, China has promised to control its carbon diox-
ide emissions so that the emissions would reach a peak value by
no later than 2030 [3]. As a consequence, there is an ongoing effort
to put forward a series of policies for energy conservation, includ-
ing building energy retrofitting, clean energy, energy management,
and energy benchmarking. In these policies, energy benchmarking
is the key first step in meeting the requirements of the energy con-
sumption intensity and quantity control in China.
Building energy benchmarking defines a public yardstick for en-
ergy use performance during a given period, considering weather
conditions, physical characteristics, management situations and
device utilization [4]. It is regarded as a feasible and cost-effective
means for comparing the energy usage level of a building with
those of its peers and identifying inefficient buildings, especially
when compared to on-site energy auditing and energy simulation.

https://doi.org/10.1016/j.enbuild.2019.109711
0378-7788/© 2019 Elsevier B.V. All rights reserved.
2 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711
Some popular benchmarking tools have been proposed and
adopted by developed countries, such as the Energy Star rating in
the U.S. [5], VDI 3807 in Germany [6], and the Energy Smart tool
in Singapore [7]. In China, the “Standard for Energy Consumption
of Buildings, GB/T51161-2016 (shortened to Standard) [8] was re-
cently released for evaluating the energy consumption of buildings,
representing the first standard setting an energy usage quota for
buildings at the national level [9]. The Standard notes the energy
quotas for buildings located in different climate zones; however,
differences in economic conditions, energy policies and architec-
tural forms may all lead to differences in building energy usage
levels within cities, which indicates that the energy quota values
in the Standard may not be representative in certain cities. This
characteristic increases the difficulties for local governments in
conducting reasonable energy benchmarking programs at the city
level. There is still no official operating energy usage or efficiency
evaluation program for public buildings in Chongqing. To help
policy-makers better understand how to choose the best bench-
marking method and identify inefficient buildings, this paper col-
lected a sample of hotel buildings in Chongqing, considered build-
ing designs and operational characteristics, applied three bench-
marking methods to compare the obtained energy performance as-
sessments and put forward potential suggestions for policy-makers.
1.2. Literature review
1.2.1. Current benchmarking methods
According to previous studies, current benchmarking methods
can be divided into three main categories: i) physical models, ii)
energy performance indicators, and iii) data-driven approaches.
The physical model-based benchmarking method focuses on estab-
lishing a benchmarking model with known input parameters by
using simulation tools and then comparing the actual energy per-
formance to the results of the benchmarking model. To make the
input parameters more representative of the dataset, researchers
have selected average values as input parameters for simulations
[10]. Although the simulation approach can quantify the influence
of each parameter on energy consumption, there are time and
effort costs, as well as a gap between the actual and simulated
energy performance due to the significant uncertainty of building
model inputs [11]. Another physically based model mentioned in
some studies [12,13] proposes a method based on exergy analysis
to define an exergy efficiency indicator for assessing the energy
performances of buildings and accounting for thermodynamic en-
ergy flows. However, this method was developed based on detailed
physical parameters and may not be feasible for use in evaluating
the energy performances of a large number of buildings.
The energy performance indicator, or energy usage intensity
(EUI), is simple to use for ranking sampled buildings and under-
standing energy usage levels according to descriptive statistics
values, such as the mean and quantile values of EUI [14,15].
Commonly used EUI measures include the energy consumption
per unit area, energy consumption per person, and energy con-
sumption per unit room [16–18]. However, the main drawback of
EUI is that it does not consider other important energy drivers,
such as occupancy and the HVAC system, leading to unreliable
benchmarking results. Based on such limitations, some studies
have proposed a normalized EUI using the degree-day method
[19], energy disaggregated indicators [20], or a detailed bottom-up
indicator system [21]. However, some limitations still exist when
these methods are applied. For example, normalizing an EUI using
the degree-day method relies on the assumption that building
energy consumption and heating/cooling degree-days have a linear
correlation, energy disaggregated indicators do not consider the
effects of occupancies, and a bottom-up indicator system may
not be suitable for application to benchmark building energy
performance on a city level.
To improve the reliability of a single energy benchmarking
indicator and reduce the dimensions of the building characteristics
of sampled buildings, a data-driven approach has been found to be
a solution, and regression analysis is most widely used to assess
building energy performance [16,22,23]. The regression analysis
establishes a linear regression model using an EUI and its key
explanatory variables, then an energy efficiency ratio is defined
according to the actual EUI and predicted EUI. A recent study
has improved the linear regression method by adopting quantile
regression, which does not require normal or constant variance for
a dataset [24]. Although regression analysis is easy to apply and
accepted by engineers and policy-makers, there are still significant
limitations. First, residuals from a regression model not only
represent relatively inefficient items in buildings but also contain
random items, which may include measurement errors, statistical
noise and the effects of unexplained factors. Additionally, a large
sample size and comprehensive explanatory variables are needed
for linear regression analysis since the analysis is sensitive to
outliers. Furthermore, linear regression models are not likely to
capture the non-linear nature of building energy consumption and
its drivers.
Unlike linear regression, machine learning (ML) methods, such
as artificial neural networks, random forests, and regression trees,
which enable learning non-linear relations and obtaining higher
accuracies in the research field of predicting building energy
consumption and building energy optimization [25–27], have also
been adopted in recent studies of building energy benchmarking
[28–30]. Similar to the MLR method, the energy efficiency ratio
is defined as the actual EUI divided by the model-predicted
EUI and used to quantify the relative energy performance of a
building. It should be noted that the accuracy performance of this
approach highly relies on a large sample size for training, testing
and validating ML models to ensure their reliability. Another
data-driven approach discussed in current studies is clustering
analysis, which classifies buildings based on multiple dimensions
of building features using the k-means clustering method [31].
The centroid of each cluster is considered the energy performance
benchmarking reference. However, the main shortcoming of the
clustering method is that we do not know how to classify new
buildings that are not part of the samples used for clustering.
Some recent studies have also used frontier analysis to mea-
sure building energy efficiency by fitting frontiers and actual
EUIs. Frontier analysis can be divided into data envelopment
analysis (DEA) and stochastic frontier analysis (SFA). DEA uses
a linear programming technique to measure the efficiencies of
decision-making units [32–34], which is a useful approach because
it considers the complex nature of the relations between multiple
inputs and outputs. However, the main drawback of DEA is that
it attributes residuals to inefficient items and ignores the impacts
of random items. Moreover, it cannot provide a specific equation
relating the input and output due to its nonparametric nature. To
improve these defects, the SFA approach has been introduced in
some studies to evaluate energy efficiency performance [35–37].
SFA is a parameter frontier approach that assumes a function
to give the maximum possible output as a function of certain
inputs. The advantage of SFA compared with DEA and the multiple
linear regression (MLR) approach is that this model can estimate
inefficiency and data noise according to the deviations from the
frontier and the actual energy consumption using the assump-
tions about the distribution of the measurement errors and the
inefficiency terms [38]. That is, SFA acknowledges the presence
of measurement errors and other sources of statistical noise and
enables the separation of inefficiency from data noise, while MLR
assumes that all residuals come from inefficiency. This method of
 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711
Table 1
Summary of comparisons of benchmarking methods in the references.
Reference Applied method Validated method
Chen et al. [41] Lorenz curve Practical EUIs and overall
energy consumption
Gao and Malkawi [31] K-means clustering Energy Star scores
Yang et al. [37] Decision trees and Energy Star scores and
stochastic frontier analysis actual EUI
Papadopoulos and Xgboost and k-means Energy Star scores
Kontokosta [30] clustering
eliminating statistical noise from residuals contributes to a more
accurate estimation of efficiency.
1.2.2. Comparisons of benchmarking methods
The aforementioned studies have a common goal, namely,
developing an evaluation indicator of building energy perfor-
mance while reducing the effects derived from the core energy-
influencing factors. Most of the studies draw attention to proposing
a new benchmarking method, with only a few studies mentioning
the validation of the proposed method by comparison with other
methods. Table 1 summarizes the comparisons mentioned in
current literature, including the applied benchmarking method,
validated method, sample size and differences. In each reference
except reference [37], only one method was used to validate the
reliability of the proposed method. No matter how small or sig-
nificant the differences, it is difficult to define which methods are
“accurate” since none of the results given by a certain benchmark-
ing method can be considered as ground truth. Note that these
references indicate that Energy Star seems to perform unstably re-
garding energy benchmarking results. In summary, there is a lack
of a full study considering the comparison of different benchmark-
ing methods. Meanwhile, conducting a comparison of different
benchmarking methods regarding building energy performance
evaluation is very worthwhile, since unlike the comparisons in
other research fields—e.g., those of building energy consumption
prediction, which focus on the accuracy or time consumption of
the applied methods [39,40]—we draw more attention to the con-
sistency among different methods rather than absolute accuracy.
Based on the above reasoning, we present two main contri-
butions to the performance and application of current commonly
used energy benchmarking methods. We first selected three rep-
resentative benchmarking methods—MLR based on Energy Star,
SFA and the DSM based on the Standard in China—to discuss
whether they can provide consistent benchmarking results and
which method has the most robust performance, and then we
compared the three methods according to consistency, robustness
and explanatory ability. Second, we present suggestions for policy-
makers regarding how to select and apply benchmarking meth-
ods to manage buildings with different energy performances to
further improve the energy efficiency. The reasoning behind the
selection of these three methods is, first, because Energy Star is
one of the most popular energy benchmarking tools and is always
regarded as a reference for validation. Second, SFA is able to ac-
count for random effects and technical inefficiency, which is a su-
periority compared with the ordinary least squares estimation ap-
plied in MLR. Finally, DSM is the method that was adopted and
recommended recently in a national standard in China; thus, we
were concerned about the relationships between the correspond-
ing evaluation results and those given by the other two popu-
lar benchmarking methods. We did not consider other data-driven
methods such as machine learning or clustering analysis due to
the limitation of the sample size in this study. This study provides
a first and an important step in developing and implementing a
building energy benchmarking program in Chongqing city, helping
3
Sample size Differences
212 buildings Small
1964 buildings Validation based on 4 buildings. Half are almost
the same, and half are greatly different.
10,153 The proposed method is more consistent with the
buildings actual EUI than with Energy Star scores.
7487 buildings Significant
local governments enhance their understanding of how to choose
an appropriate benchmarking method and formulate energy effi-
ciency policies.
2. Data and methodology
2.1. Data acquisition
An effective energy benchmarking program should be based on
a sufficient dataset. However, unlike in the U.S., where some pub-
licly available databases, such as the EIA CBECS (Energy Informa-
tion Administration’s Commercial Buildings Energy Consumption
Survey) [42] and the DOE BPD (Department of Energy’s Building
Performance Database) [43], have been established, there is no na-
tional or local database of building energy performance in China.
Supported by the Chongqing Housing and Urban-Rural Committee,
this study collected data from the energy audit reports of a to-
tal of 48 hotel buildings that were classified as energy retrofitting
demonstration projects in Chongqing. The selected information
contains data on building energy consumption and potential en-
ergy influencing factors. Most current studies focus on the inherent
design characteristics of buildings, such as the gross floor area and
floor type, and seldom consider equipment and operation charac-
teristics. According to one study [44], six key factors that influence
real energy use are the building envelope, building equipment (en-
ergy systems), building operation and maintenance, weather, in-
door comfort criteria, and occupant behaviors. Therefore, this study
comprehensively considered potential energy-influencing factors,
and the variables used in this study are summarized as follows:
(1) Energy consumption
The energy bills for 12–24 months were collected from energy
audit reports. The energy bill data involved all types of monthly
hotel energy consumption data, including electricity and natural
gas, which were verified by electric power and gas companies to
ensure their factuality.
(1) Potential influencing factors
(a) Building characteristics: building age; hotel ratings; loca-
tion; gross floor area, number of guest rooms; number of
stories; floor area of guest rooms, dining, conference and
entertainment zones; underground floor area; floor area
with air conditioning; glazing ratio; performance index of
external wall, roofs and windows.
(b) Occupancy: annual average occupancy rate.
(c) Equipment parameters: lighting power density of guest
rooms; air conditioning system type; chiller type of air
conditioning system; rated energy efficiency ratio (REER) of
chiller.
(d) Operation: heating and cooling period.
(e) Weather: daily average outdoor temperature used to calcu-
late heating- and cooling-degree days (the heating reference
degree is 18°C and the cooling reference degree is 26°C).
4 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711

Table 2
Statistical summary of data collected from the surveyed hotels.

Variables Max Min Average Standard deviation

Construction year 2013 1998 2008 4

Gross floor area (m2 ) 97,425 10,984 39,078 19,800
Number of building stories 61 4 19 11
Number of guest rooms 680 59 240 111
Heating-degree days (◦ C · day) 1158 822 950 124
Cooling-degree days (◦ C · day) 452 218 258 84
Occupancy rate (%) 92 20 62 16
Glazing ratio (%) 74 15 38 14
Heating period (hours) 3160 681 1732 609
Cooling period (hours) 3740 884 2457 697
Uwall (Exterior walls) (W/m2 K) 3.75 0.5 1.21 0.60
Uroof (Roofs) (W/m2 K) 1.45 0.28 0.72 0.26
Uwindow (windows) (W/m2 K) 6.60 2.20 3.50 1.15
Rated EER of chiller 8.33 0.93 4.27 1.80
Lighting power density (kW/m2) 28.13 4.45 13.20 5.93
Total energy per unit area (kWh/m2 /year) 318.7 39.8 151.9 63.7
Total energy per guest room (MWh/room/year) 79.1 4.0 24.5 12.2

all information was available. The sampled hotels can be divided

into two types according to the service quality offered: four-star
hotels (20 hotels) and five-star hotels (25 hotels). The investigation
data included the energy consumption data and information of
potential energy-influencing factors in the hotels that influence
the energy consumption.
Table 2 provides a statistical summary of part of the data
collected from the 45 hotels. It is noted that all the selected hotels
have a central air conditioning system. The REER of these central
chiller plants ranges from 0.93 to 8.33. The significant difference in
REER is due to the direct-fired absorption chiller/heaters applied in
some hotels. The percentages of each type of cooling and heating
source in the air conditioning system are provided in Table 3.
For lighting systems, only the lighting power density of the guest
rooms was investigated in this work because the guest rooms are
the main functional areas in hotels and have relatively long-term
use.
Since the energy usage of hotel buildings includes different
energy categories, the total energy consumption should be calcu-
lated according to the same energy categories. The widely used
Fig.. 1. Location map of the 45 sampled buildings.
EUI unit used for most building categories is kWh/m². Therefore,
natural gas consumption should be converted into electricity. The
conversion coefficient of natural gas is 7.131 kWh/m3 [45]. The
Notably, the energy consumption of the data center is not
statistical summary of the energy consumption of the sampled
included in the annual energy consumption and EUI values men-
hotels is shown in Table 2.
tioned in this study. The outdoor temperature data were collected
for 5 years (2012–2016) because energy bills were collected during
2.3. Benchmarking methods
these years and the corresponding weather differences should be
considered in this work. In addition, weather differences across lo-
2.3.1. Multiple linear regression analysis based on energy star
cations can be ignored because all the buildings are in Chongqing,
There were three basic steps in developing the benchmarking
where there is only one climate zone.
model based on multiple linear regression analysis: (1) determi-
nation of the EUI, (2) development of the MLR model, and (3)
2.2. The sampled building characteristics
establishment of the benchmarking table.

The 48 sampled buildings are distributed in multiple districts
in Chongqing, as shown in Fig. 1. This district is in China’s hot
summer and cold winter zone, where the average temperature
of the coldest month is from 4 to 13°C and that of the warmest
month is from 22 to 36°C, making both cooling and heating
systems important and complex in this district. The main energy
sources of the sampled hotels are electricity and natural gas. Elec-
tricity is mainly used for lighting, plug-in devices, water pumps,
fans and terminals of the HVAC system, elevators, domestic hot
water pumps, etc. Natural gas is mainly used for cooking, boilers
of HVAC systems, domestic hot water boilers, etc.
The selected sample initially comprised 48 hotels, of which
45 were finally used. Three hotels were excluded because not
(1) Determination of the energy usage intensity
There are several types of EUIs, including the annual energy
consumption per unit area and the annual energy consumption
per unit per room. To select the most appropriate type of EUI
for the sampled buildings in this study, the Pearson correlation
coefficient (r) was adopted to obtain a building energy-influencing
factor that is more significantly correlated with total energy
consumption compared to the rest of the factors. The degree of
correlation between the building energy consumption and various
influencing factors can be measured by Eq. (1).
n
(xi − x̄ )(yi − ȳ )
r=  n i=1
 n (1)
i=1 (xi − x̄ ) i=1 (yi − ȳ )
 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711
Table 3
Cooling and heating source of the samples.
Cooling and Water-cooled chiller
heating source (screw/centrifugal) & gas boiler
Percentage (%) 62
Here, r ranges between (−1, +1). When r is less than 0, it
indicates a negative correlation, and in the opposite case, there is
a positive correlation. In addition, the larger is the value of |r|, the
stronger is the correlation between the two variables; otherwise,
the weaker is the correlation between the two variables.
(1) Development of a multiple linear regression model
This study used the stepwise regression method to develop a
benchmarking model. This method uses an automatic procedure
to determine the best explanatory variables among the candi-
date variables mentioned in the Section 2.1 and minimizes the
collinearity problems between the variables for use in a regression
model. Each step introduces the most significant variable, checks
whether or not the selected variables in the model are signifi-
cant, and removes the insignificant variables from the candidate
explanatory variables based on a specific criteria (the F-test was
used in this study, and the criteria were as follows: probability of
F-value to enter ≤0.05, probability of F-value to remove ≥0.10). In
this way, the final stepwise regression model was able to include
the best explanatory variables. The developed MLR model is given
by the following equation:
Y = β0 + β1 X1 + β2 X2 + · · · + β p X p + ε
where Y is the EUI, β 0 is the intercept, β i are the regression
coefficients, Xi represents the significant influencing factors
(i = 1, 2, . . . , n), and ɛ is the residual.
(1) Benchmarking rating table
According to the benchmarking model, each predicted EUI ob-
tained from the MLR model can be calculated. The actual EUI of
each building is divided by its predicted EUI to calculate an en-
ergy efficiency ratio (EER), as shown in Eq. (3). A lower EER indi-
cates that a building uses less energy than predicted; otherwise, it
is more efficient.
EUIActual
EER =
EUIPredicted
To determine the rank of buildings based on the EER, the
cumulative probability density distribution plot of the EERs was
generated first, and then a best-fit curve through the plot was
determined. In particular, a gamma distribution was used to find
the best gamma curve by minimizing the sum of the squared dif-
ferences. Once a specific gamma curve was determined, we could
calculate the percentile of rank given a certain EER. Then, the en-
ergy efficiency score could be calculated as (1 − percentile ) × 100.
For instance, the percentile of the gamma curve is 20%, which
corresponds to an energy efficiency score of 80.
2.3.2. Stochastic frontier analysis
In this section, the cost function in the form of the frontier
model was selected since we intended to evaluate the efficiency
by exploring the relationship between the EUI and potential
influencing variables. The EUI was measured as the annual en-
ergy consumption per unit area. Then, we needed to determine
appropriate inputs from among the sampled data. This process
considered only two situations containing three and four input
variables in the SFA model and used the enumeration method to
list all possible input variable combinations. The output of the SFA
model could be calculated based on each combination. Only three
5
Air source heat Direct-fired absorption
pump chiller/heater
13 25
and four input combinations were considered in this study due to
the limited number of sample buildings and various potential in-
dependent variables, which may have resulted in overfitting of the
model. The limited combinations considered also greatly reduced
the calculation process. The commonly used Cobb-Douglas cost
function was used in this study, as shown in Eq. (4).
β β β
yi = f (xi ; β ) = eβ0 x1i1 x2i2 . . . xkik , i = 1, 2 . . . , n (4)
The proposed SFA model for EUI is specifically as follows:
β β β
yi = f (xi ; β )eεi = eβ0 x1i1 x2i2 . . . xkik eεi , i = 1, 2, . . . , n (5)
where yi represents the actual EUI of a building and n is the num-
ber of buildings; xk is a vector of energy-influencing factors af-
fecting the building energy consumption and k is the number of
factors; the β coefficients pertain to the potential factors; and ɛi
includes the inefficiency ui and random error ν i and is written as
εi = ui + νi . After taking the log transformation of Eq. (5), the func-
tion is shown in Eq. (6).

k
ln yi = β0 + βk ln xki + ui + νi , i = 1, 2, . . . , n (6)
(2) k=1
The inefficiency items ui are non-negative variables that rep-
resent the technology energy efficiency of the building energy
performance, which are assumed to be ui ∼ iidN + (0, σu2 ), and the
noise items ν i are random errors that are independent of ui , which
are assumed to be νi ∼ iidN (0, σν2 ). There are other assumptions
for ui , such as exponential, truncated normal and gamma distri-
butions, while some studies have proposed that there is little dif-
ference in the compared average inefficiency across these distribu-
tions for ui [46]. Therefore, this paper selected the most commonly
used half-normal distribution. The identification of the inefficiency
item ui is generally based on two steps. In the first step, estimates
of the SFA model parameters θˆ are obtained by maximizing the
likelihood function l(θ ), where θ = (β0 , βk , σu2 , σv2 ) [47]. The
(3)
maximum likelihood estimates for the parameters cannot be
solved directly, so an iterative procedure (quasi-Newton method)
is applied to approximate the maximum-likelihood estimates of
the parameters [48]. The initial values used for estimating the
parameters are set according to the values of an ordinary least
squares regression. In the second step, a point estimate of the in-
efficiency term ui for each building can be obtained via the mean
of the conditional distribution ui , given ɛi [49]. Thus, the mean
value of this conditional distribution can be written as Eq. (7).
f (−μ∗ /σ∗ )
E ( ui |εi ) = μ∗ + σ ∗ (7)
1 − F (−μ∗ /σ∗ )
Here, f and F denote the probability density function and cumu-
lative distribution function of the normal distribution, respectively.
The parameterization of μ∗ = −σu2 εi /σ 2 , and that of σ∗ = σu σv /σ .
The likelihood ratio (LR) test was used to check whether there
was a difference between the buildings in terms of efficiency
in the SFA model (σu2 = 0), and the criteria required that the
chi-squared distribution with one degree of freedom from the
SFA model was larger than the critical value. In addition, the
significance value of the test results was used to select the best
SFA model, according to which the final energy efficiency score of
each building could be calculated. The energy efficiency score CEi
6 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711

for the sampled buildings was estimated as follows:

yi min f (xi ; β )evi
CEi = = = exp (−ui ) (8)
yi f (xi ; β )e(vi +ui )
The energy efficiency score CEi is defined as the ratio between
the minimum energy consumption and the actual energy con-
sumption and takes a value between 0 and 1. According to this
definition, the closer is the efficiency measure to 1, the more
efficient is the energy use of a building considered.

2.3.3. Descriptive statistics method

The Standard proposes a benchmarking method based on the
DSM to evaluate building energy consumption. Specifically, the
Standard classifies two types of buildings based on the potential
for natural ventilation in buildings. Type A represents normal-sized
buildings with floor areas of less than 20,0 0 0 m2 that have high
potential to apply natural ventilation, while Type B represents
large, complex buildings that need to use mechanical ventilation
instead. The Standard uses the annual energy consumption per
unit area as the EUI to evaluate the level of building energy usage.
It defines two indicators of the EUI, including the constraint value
and leading value, which are calculated according to the mean
value and lower quartile value, respectively. Considering the limi-
tation of using a single EUI for evaluation, the Standard also takes
the annual average occupancy rate and the percentage of guest
room area relative to the total building area as correction factors
so that the benchmarking can be more reasonable. As mentioned
in the Standard, the correction method is written as follows:
 H0
  R
 and GFASER , respectively. The coefficient of determination (R2 ) of
Eac = Ea × 0.4 + 0.6 × × 0.5 + 0.5 ×
H R0
where Eac represents the corrected actual EUI, kWh/m2 ; Ea is the
actual EUI, kWh/m2 ; H is actual annual occupancy rate of the
hotel,%; H0 is the standard annual average occupancy rate and
is equal to 50%; R is the actual percentage of guest room area
relative to the total building area,%; and R0 stands for the standard
percentage of guest room area and is equal to 70%. The correction
method for the actual EUI should be applied when both the annual
average occupancy rate and the percentage of guest room area are
beyond the standard values.
3. Results
3.1. Multiple linear regression based on energy star
3.1.1. Determination of the energy usage intensity and potential
drivers
(a) Energy usage intensity
The Pearson correlation coefficients of the annual energy con-
sumption with consideration of the potential energy influencing
factors were calculated, and the results of the main factors are
shown in Table 4.
The GFA is the most significant influencing factor of the annual
energy consumption, and the correlation coefficient of the GFA is
clearly higher than those of other factors, which conforms to the
conclusions reached in some studies [19,50]. This result indicates
that the energy use per unit area should be used for evaluating
the EUI of hotel buildings in these samples. Other significant
factors, including ZONE, STAR5 (dummy variable differentiating
four- and five-star hotels), STORY, ROOM, GFAOFF and CP, all have
significant value at the 0.01 level. The factor ZONE shows that
hotels located in well-developed districts can consume more
energy. In addition, GFAOFF reflects a negative correlation with
the annual energy consumption, and this may be because the EUI
values of conference and office areas are lower than the EUI of the
main functional area, and the occupation times of the former are
also limited. Therefore, the higher is GFAOFF , the lower is the EUI.
Fig.. 2. Cumulative distribution curve of the EER. (For interpretation of the refer-
ences to color in this figure, the reader is referred to the web version of this arti-
cle.)
3.1.2. Multiple linear regression analysis
Stepwise regression was used to obtain the relationship be-
tween the EUI and independent variables and the corresponding
result is given in Table 5. Only three independent variables were
reserved in the final MLR model, all of which had a significant lin-
ear relation with the EUI in the regression model, with p-values of
less than 0.05. The parameters XCP , XHP and XSER represent CP, HP
(9) the proposed model is 0.700, and the adjusted R2 is equal to 0.677,
indicating that the model can adequately explain the correlation
between the EUI and the input variables, and the linear combina-
tion of three variables accounts for 67.7% of the variance in the EUI.
3.1.3. Developing a benchmark table
The predicted EUIs were obtained from the MLR model, and
then the EERs of each building were calculated. A gamma curve
(the red line in Fig. 2) was used to fit the cumulative probability
density distribution of the EER, as shown in Fig. 2. The best-fit
gamma function is shown in Eq. (10).
x γ (α , β x )
F (x; α , β ) = ∫ f (t; α , β )dt = (10)
0 (α )
Here, x represents the EER of the sampled buildings; F(x; α , β )
stands for the percentile corresponding to the EER; and the param-
eters α and β are 9.210 and 9.217, respectively. According to the
gamma function, a representative benchmarking rating table can
be generated, as shown in Table 6. For example, if the score is 80,
the percentile should be 20%, and the corresponding EER is 0.717.
3.2. Benchmarks based on stochastic frontier analysis
The SFA was performed using R programming language with
the ‘frontier’ package [51]. All available explanatory variables were
used to determine the best combination of input variables using
the enumerate method. According to the LR test results, we can
remove most of the obtained SFA models due to their insignificant
LR test values, and the best model with the most significant LR
test value was chosen. Hence, the independent variables of the
final SFA model included GFASER , FLOOR, HTCEWALL and HVAC1,
where HVAC1 is a dummy variable with 1 indicating that the
building adopts chiller units for the air conditioning system and
0 indicating otherwise, while the other variables are numeric
explanatory inputs. Since the Cobb-Douglas cost function was used
in this paper, both the dependent and independent variables were
log-transformed. The maximum-likelihood estimation results of
 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711 7

Table 4
Pearson correlations of potential factors for annual energy consumption.

Variable Description Pearson correlation P-value

AGE Building construction year 0.091 0.550

ZONE Hotel location district 0.405∗∗ 0.006
STAR4 4-star hotel −0.377∗ 0.011
STAR5 5-star hotel 0.475∗∗ 0.001
GFA Gross floor area 0.774∗∗ 0.000
GUA Gross underground area 0.104 0.496
STORY Number of building stories 0.469∗∗ 0.001
ROOM Number of guest rooms 0.477∗∗ 0.001
OCCUPANCY Annual occupancy rate 0.118 0.495
GFADIN The percentage of GFA for dining area −0.076 0.620
GFAOFF The percentage of GFA for convention centers and office area accounting for the GFA −0.382∗∗ 0.010
GFASER The percentage of entertainment and leisure area accounting for the GFA 0.145 0.328
GFAAC The percentage of GFA for air-conditioned area accounting for the GFA 0.206 0.185
HDD18 °C Heating degree days 0.084 0.584
CDD26 °C Cooling degree days 0.166 0.276
GLAZINGR Glazing ratio 0.326 0.068
HTCEWALL Heat transfer coefficient of external wall 0.083 0.594
HTCEWINDOW Heat transfer coefficient of external window −0.008 0.961
REER Rated energy efficiency ratio of chiller 0.335∗ 0.026
HVAC1 Water-cooled chiller 0.316∗ 0.035
HVAC2 Direct-fired absorption chiller −0.137 0.369
HVAC3 Air source heat pump −0.277 0.065
CP Cooling period 0.451∗∗ 0.002
HP Heating period 0.318∗ 0.033
LPD Lighting power density 0.109 0.481
∗
Correlation is significant at the 0.05 level (2-tailed).
∗∗
Correlation is significant at the 0.01 level (2-tailed).

Table 5
Results from multiple regression analysis of the EUI.

Nonstandardized coefficient Standardized

B Standard error coefficients beta t p-value overall R2

(constant) −18.662 20.647 – −0.904 0.371

XCP 0.031 0.014 0.325 2.142 0.038 0.700
XHP 0.048 0.016 0.450 3.065 0.004 0.677a
XSER 161.863 62.699 0.236 2.582 0.014
a
stands for adjusted R2 .

Table 6 Table 8
Benchmarking rating table. Parameter estimates for the SFA model
by the maximum-likelihood technique.
Percentile EER Score
Parameter Estimate
10% 0.607 90
20% 0.717 80 λ 3.678
30% 0.804 70 σu2 0.348
40% 0.884 60 σv2 0.025
50% 0.963 50 γ 0.931
60% 1.047 40 log[L(H0 )] −20.381
70% 1.142 30 log[L(H1 )] −19.017
80% 1.261 20 LR test 0.049∗
90% 1.438 10
Significance codes:.
∗
0.1.

Table 7
Coefficient estimates for the SFA model by the maximum-
likelihood technique.
resents the most variation in the actual EUI that comes from
Variable Coefficient value Standard error Probability inefficiency, with only 7% of the random error contained in the
Intercept 2.104 0.717 0.003∗∗∗
error item. An LR test was used to test whether there were
GFASER 0.222 0.073 0.002∗∗∗ differences between the sampled buildings in terms of energy
FLOOR 0.161 0.091 0.074∗ efficiency. According to Table 8, the LR test provided a statistic of
HTCEWALL 0.104 0.127 0.412 2.73 (=−2 × (−19.017 − (−20.381 ))), which exceeded the 90%
HVAC1 0.348 0.107 0.001∗∗∗
critical value of 2.71. Hence, this result demonstrates that there are
Significance codes:,∗ ∗ ∗ ,0.01,∗ ∗ ,0.05, ‘∗ ’0.1. significant differences in energy efficiency between the buildings.
The coefficients of the SFA model, including GFASER , FLOOR,
HTCEWALL and HVAC1, are all positive and show a positive relation-
the coefficients and parameters for the SFA model are shown in ship between the corresponding variable and the EUI. This result is
Tables 7 and 8. partly the same as the findings in [35]; that is, a growth in enter-
The parameter gamma (γ ) is close to 1 and shows that the tainment area in a building will lead to an increase in the build-
inefficiency item accounts for 93% of the error item, which rep- ing’s EUI. It is worth mentioning that the input variables are dif-
8 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711
Fig.. 3. Relation between the predicted frontier and the actual EUI. (For interpretation of the references to color in this figure, the reader is referred to the web version of
this article.)
ferent in the final SFA and MLR models. One possible reason could
be related to the different function forms used in the two models.
Another possible reason could be the use of different methods of
independent variable selection in the two models. In particular, the
MLR model used a stepwise regression, while the SFA model deter-
mined independent variables according to the results of an LR test.
As shown in Fig. 3, the purple line represents the predicted
frontier (the minimum EUI with the highest energy efficiency) and
is equivalent to the actual EUI, demonstrating that the energy per-
formance of the buildings on the line is efficient. The scatters are
closer to the purple line, meaning that the corresponding buildings
are more efficient in energy utilization. However, several buildings
were found to have an actual EUI lower than the frontier because
of random error items. Overall, there is a clear gap between the
actual EUI and the fitted frontier for sampled buildings according
to the slope of the two lines in Fig. 3.
Fig. 4 shows the distribution of the energy efficiency scores ob-
tained from the SFA model. In general, the scores estimated from
the SFA model ranged from 0.26 to 0.93, with an average score
of 0.66. The majority of the sampled buildings were evaluated
between 0.5 and 0.89, which means that most of the buildings
have relatively good energy performance.
3.3. Benchmarks based on the standard
Because all the sampled buildings have adopted central air con-
ditioning systems to improve the indoor thermal environments,
they are all considered to be Type B buildings. According to the
requirements of the correction method in the Standard, the EUIs
of five hotels need to be corrected since their annual average
occupancy rate and percentage of guest room area exceed the
standard values. The constraint value is 150 kWh/m2 , and the
leading value is 108 kWh/m2 . This result signifies that a building
is regarded as an inefficient energy user if its EUI is higher than
150 kWh/m2 , while a building can be considered a leader in
energy efficiency if it has an EUI lower than 108 kWh/m2 . When
the EUI is between these two indicators, it indicates a normal
performance in which there may still be some energy saving
potential. Fig. 5 displays the probability density curves with kernel
density estimation for the corrected EUI from the Standard. The
area between the best and worst performers is narrow, which
might exacerbate the difficulties in the benchmarking process for
existing buildings since the requirement of the constraint value is
higher than the value of the 75th percentile applied in another
study [19]. However, it is beneficial in enhancing the motivation
for the energy-efficient design of new buildings and to encourage
users to apply advanced technologies for efficiency improvements.
4. Comparative analysis of benchmarking methods
This section compares the energy performance assessments
based on the above methods, including the MLR based on Energy
Star, SFA and the DSM. Each of these data-driven benchmarking
methods provided a set of definite benchmarking results for the
sampled buildings. Evaluating the accuracy of each benchmarking
method is a challenge for a benchmarking program since it is
time-consuming and costly to acquire the practical energy effi-
ciencies of individual buildings in large-scale building sampling,
leading to an absence of ground truth to verify the obtained en-
ergy efficiency scores of each method. Hence, this section verifies
the energy performance by comparing whether the benchmarking
results of the three methods are consistent. In other words, we
focus on the consistency and robustness of the benchmarking
results instead of a comparison to the ground truth of energy
performance. In addition, benchmarking does not merely put
forward an accurate energy performance assessment or a ranking
table. A better explanatory ability is another important property of
benchmarking methods [37,52]. This requires benchmarking meth-
ods to provide suggestions for improving the energy performance
of buildings or a better understanding of benchmarking programs.
 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711 9

Fig.. 4. The frequency distribution of the energy efficiency scores based on SFA.

Fig.. 5. The corrected EUI of sampled buildings.

10 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711

Table 9
Spearman correlation test of the ranking results based on
the three benchmark methods.

Variable MLR SFA DSM

MLR 1 – –
SFA 0.439(0.003) 1 –
DSM –0.505 (0.000) –0.797 (0.000) 1

4.1. Consistency and robustness

The Spearman rank correlation coefficient was applied to test

the relationship between two ranked energy efficiency scores from
the three methods. Unlike the Pearson correlation coefficient, it is
a nonparametric statistic without critical requirements for the dis-
tribution of variables. The Spearman rank correlation coefficients
with corresponding p-values within the brackets are shown in
Table 9.
The significance levels (p-values) of the Spearman correlation
coefficients reflect that the energy efficiency rankings of the
sampled buildings from different benchmarking methods have
correlations with each other. The rankings of SFA and the DSM are
most consistent according to the values of the Spearman correla-
tion coefficients. However, differences still exist in the results of
the three methods. For instance, the MLR and SFA benchmarking
methods have the weakest correlation between their ranking re-
sults, with only a 0.439 correlation coefficient, and the correlation
coefficient of MLR and the DSM is approximately 0.5, indicating
that this correlation is also not strong. Different from MLR and
SFA, the output of the DSM is described as the EUI, which means
that is the higher EUI, the poorer is the energy performance of
the building. Therefore, the correlations between the DSM and the
other two benchmarking methods are negative.
To further detect the differences among the benchmarking
methods, we first divided the benchmarking results based on the
three methods into several grades. For instance, the results based
on MLR and SFA were divided into four grades similar to the build-
ing energy grades proposed in the law passed by New York City
[53], which included “0–19 , “20–49 , “50–89 and “90 or above”
and “0–0.19 , “0.2–0.49 , “0.5–0.89 and “0.9 or above”. The
results based on the DSM were divided into three grades, A, B and
C, according to the constraint values and leading values defined in
the Standard, i.e., the grade “A” indicates an EUI under the leading
value; “B” indicates an EUI between the constraint and leading
value; and “C” indicates an EUI beyond the constraint value.
Fig. 6 uses Sankey diagrams to present the interactions between
the pairwise comparisons of the three benchmarking methods. For
the pair of MLR and SFA, 42% of the sampled buildings received
different grades. The significant difference mainly exists for low
performances, i.e., the clusters of “0–19 and “20–49 from MLR
are split in the cluster of “0.5–0.89 from SFA. The difference in
the interaction between MLR and the DSM is the most significant,
with 47% of buildings evaluated as having different grades. More-
over, there are some extreme cases in which the buildings with Fig.. 6. Comparison of the benchmarking results based on the three methods
low MLR energy efficiency scores (“0–19 and “20–49 ) received (a) MLR (left node) and SFA (right node) MLR (left node) and DSM (right node) SFA
high DSM energy grades ("A” and “B” grade). The difference (left node) and DSM (right node).
between SFA and the DSM is smaller than that between any
other pair of benchmarking methods, with 40% of the sampled
buildings obtaining different benchmarking results. The significant correlation coefficients of the DSM and MLR and of the DSM and
differences appear in the medium-grade performance determined SFA are higher than the coefficient of MLR and SFA according to
by SFA, i.e., buildings with a score of “50–89 are split between Table 9, which indicates that the DSM may provide more robust
the “A”, “B” and “C” grades. benchmarking results compared to the other two methods. If we
In terms of the robustness of each benchmarking method, DSM focus on the grades of sampled buildings instead of their ranks,
and SFA perform robustly in different situations. For example, if we SFA performs more robustly than do MLR and the DSM according
focus on the ranking of the sampled buildings, the DSM is more to Fig. 6 because it has fewer differences in energy efficiency
robust than are the other two benchmarking methods because the scores relative to MLR and DSM than they have with each other.
 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711
4.2. Explanatory ability
The MLR method based on Energy Star provides three inde-
pendent variables, namely, the percentage of entertainment and
leisure area and cooling and heating periods, that significantly
affect the EUIs in the sampled buildings. The variables of cooling
and heating periods are related to the operational characteristics
of the HVAC system. This result suggests that hotels in this sample
should adopt measures that can reduce the operation time of
HVAC systems to improve the energy performance. The SFA model
in this paper indicates that almost all of the input variables are re-
lated to design parameters that seldom change during the building
operation period, such as the number of building floors, the type of
chiller units and the U-values of external walls. The MLR and SFA
methods reveal that the determinations of EUIs are related to the
design and operational characteristics of the HVAC systems, which
suggests that energy retrofitting for these sampled buildings could
be dependent on the implementation of effective measurements
in the HVAC systems. A similar result was obtained in an investi-
gation of 68 retrofitted demonstration projects in Chongqing [54].
Although the MLR and SFA benchmarking methods enable
providing some potential suggestions for energy conservation
measurements for the sampled buildings, they are unlikely to
provide detailed solutions for individual buildings in which there
is high energy use or energy waste. Furthermore, the obtained
energy efficiency scores can only reflect the relative energy per-
formances among the sampled buildings. Therefore, on-site energy
audits are still needed to further detect energy saving potentials
for individual buildings. The best use case for these benchmarking
methods is to determine which buildings are considered to be
inefficient in a sample and try to understand the core drivers of
energy consumption in the buildings. The explanatory ability of
the benchmarking results allows different parties, from building
users and energy service companies to governmental policy-
makers, to discover deeper insights that can be used to make
better energy efficiency decisions and policies.
Compared with the other two methods, the DSM has a lower
explanatory ability, as reflected in two aspects. On one hand, the
correction factors (annual average occupancy rate and percentage
of guest room area) are inherent building characteristics that are
related to the building function and business rather than to the
energy performance. On the other hand, the correction method
mentioned in the Standard does not provide a detailed explanation
of how the correction formulas were conducted.
5. Discussion
5.1. Policy implications
This study discussed how to apply three methods to evaluate
the energy performances of sampled buildings in Chongqing and
compared the performances of the methods based on consis-
tency, robustness and explanatory ability. The results can assist
governmental policy-makers in developing a building energy
benchmarking program.
According to the comparative analysis, there is not sufficient
evidence to determine the best or worst benchmarking method,
since there are significant differences in the estimated efficiency
scores (or grades) between the three methods, and there is a lack
of ground truth of the energy performance for sampled buildings,
as well. Hence, policy-makers are recommended to select the most
robust benchmarking method or apply multiple benchmarking
methods at the same time to avoid misleading policy conclusions
obtained by using only one unreliable benchmarking method.
In particular, Fig. 6 presents that SFA has the best robustness.
As an example, Fig. 4 shows the frequency distribution of the
11
energy efficiency scores estimated by SFA. The different colored
bars suggest the measures that the government could adopt
for different buildings. The yellow bars represent the 22% of
buildings that are inefficient and may receive a financial penalty
and be required to implement energy retrofitting. The blue bars
correspond to the most efficient 4% of buildings that could be
established as demonstration buildings to encourage relatively
inefficient buildings to further improve their energy efficiency and
provide an example of advanced energy efficiency technologies
and management strategies. The largest gray bars with 0.5–0.89
efficiency scores refer to less efficient buildings, which could be
encouraged through incentive schemes to conduct energy audits
and retrofit to further improve their energy performance.
The interpretability of benchmarking methods indicates the
building information that should be collected for benchmarking.
Current studies focus on inherent building characteristics and
have shown the importance of such information in influencing
the building energy consumption. According to the MLR and SFA
models, this study shows that the design and operational charac-
teristics of HVAC systems also have significant effects on building
energy consumption. Therefore, the information of the HVAC
system should also be considered in data collection for developing
energy benchmarking programs, local or national energy databases
and energy performance disclosure policies.
5.2. Limitations and future work
Similar to any data-driven method, the benchmarking methods
we discussed in this study would give more accurate benchmark-
ing results if a sufficient number of buildings were sampled and
the data of comprehensive energy influencing factors could be
acquired. Although this study collected various details of potential
energy drivers, including the physical, operational characteristics
and occupancy behaviors of buildings, its sample was still limited
in size and far smaller than the samples in similar previous studies
in other countries [24,35,37]. This situation indeed increased the
difficulties of developing an accurate energy benchmarking model
in China. Another limitation is that the actual energy efficiency
performances of the sampled buildings were not available to verify
the estimated energy efficiency scores relative to the ground truth
for the benchmarking methods in this study. In future work, we
will first expand the number of sampled buildings. Specifically, we
plan to use a larger available database (such as the CBECS database
in the U.S.) to compare the performances of the discussed bench-
marking methods, confirm whether the differences still occur
and analyze the robustness of each method to further determine
the most robust benchmarking method. Then, more data-driven
methods including machine learning will be considered to make a
comprehensive comparison. In addition, almost all of the sampled
hotels are retrofitted demonstration projects in Chongqing, and we
intend to obtain the energy bills from the post-retrofitting period
to quantify the energy savings and evaluate the effects of energy
retrofitting at the city level.
6. Conclusions
Data-driven methods are widely used to benchmark the energy
performances of buildings, but few studies have compared the
performances or accuracies of these methods. In this paper, we se-
lected the widely used MLR based on Energy Star, SFA and the DSM
based on the energy consumption standard in China to determine
the performance of each benchmarking method. The energy and
building characteristic information of 45 four- and five-star hotels
in Chongqing was collected for a case study. Since it is difficult
to obtain the actual energy efficiencies of individual buildings, we
focused on the consistency and robustness instead of the absolute
12 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711
accuracy of the three benchmarking methods to quantify their per-
formance via Spearman correlation analysis and Sankey diagrams.
Some interesting findings were obtained from the results of the
comparative analysis of the three benchmarking methods. The re-
sults of DSM and SFA were found to be the most consistent, while
those of MLR and SFA and of MLR and DSM showed significant
differences in terms of the evaluated energy performances accord-
ing to the low Spearman coefficients and more buildings receiving
different energy efficiency grades. Furthermore, the DSM was more
robust in evaluating the ranks of the sampled buildings, and SFA
was more robust in evaluating the energy efficiency grades, and
it could be potentially recommended for use by policy-makers to
conduct an energy benchmarking program at the city level.
The intention of this paper was to compare the performance
of data-driven benchmarking methods that are commonly used in
current studies to determine the most appropriate benchmarking
method that could be recommended to policy-makers. This paper
confirmed that using only one benchmarking method may lead
to distorted evaluations of the energy performances of sam-
pled buildings. Policy-makers are recommended to utilize multiple
benchmarking methods or select the most robust method to obtain
the assessment results that are likely to be the most approximate
to the ground truth. Moreover, policy-makers are suggested to
apply different strategies such as mandatory schemes or incentive
policies to manage buildings with different energy performances.
Finally, the explanatory power of the benchmarking methods in-
dicates the influence of the design and operational characteristics
of the HVAC system on the building energy consumption. These
characteristics are suggested to be further collected to develop
local or national databases and energy performance disclosure
policies. In future work, we intend to expand the sample size
by using a public database, such as the CBECS in the U.S. and
compare more data-driven methods to test the consistency and
robustness of energy performance assessments based on different
benchmarking methods.
Declaration of Competing Interest
None.
Acknowledgments
The authors would like to thank Dr. Travis Walter at the
Lawrence Berkeley National Laboratory for his many useful and
helpful suggestions for improving this study. The authors are
thankful to the Chongqing Housing and Urban-Rural Committee
and the energy service companies for granting access to rele-
vant energy audit reports. This work is also supported by the
project “Standard System for Post-assessment of Green Build-
ing Performance” (the National Key Projects in 13th Five-Year,
project no. 2016TFC0700105), National Natural Science Founda-
tion of China (Grant number 51978095), the project "A Big Data
Platform of Supervison, Management and Evaluation for Insti-
tional Buidlings in Chongqing" and the China Scholarship Council
(Student no. 201806050085).
Supplementary materials
Supplementary material associated with this article can be
found, in the online version, at doi:10.1016/j.enbuild.2019.109711.
References
[1] Building Energy Research Center in Tsinghua University, Annual Report on
China Building Energy Efficiency (in Chinese), 2017, China Architecture & Build-
ing Press, Beijing, 2017.
[2] X. Yang, S. Zhang, W. Xu, Impact of zero energy buildings on medium-to-long
term building energy consumption in China, Energy Policy 129 (2019) 574–586,
doi:10.1016/j.enpol.2019.02.025.
[3] UNFCCC (United Nations Framework Convention on Climate Change), The Paris
agreement, 2015, < http://unfccc.int/paris agreement/items/9485.php >(Ac-
cessed June 24, 2019).
[4] W. Chung, Review of building energy-use performance benchmarking method-
ologies, Appl. Energy. 88 (2011) 1470–1479, doi:10.1016/j.apenergy.2010.11.022.
[5] Energy Star. < http://www.energystar.gov/buildings >(Accessed June 24, 2019).
[6] The Association of German Engineers, Characteristic values of energy con-
sumption in building heating and electricity, VDI 3807 part 2-2014. 2014, Ger-
man.
[7] National University of Singapore and National Environment Agency, Technical
guide towards energy smart office, 2013. 2013, Singapore.
[8] MoHURD (Ministry of Housing and Urban-Rural Construction Development,
China), Standard For Energy Consumption of Buildings, China Architecture and
Building Press, Beijing, 2016 GB/T51161-2016.
[9] D. Yan, T. Hong, C. Li, Q. Zhang, J. An, S. Hu, A thorough assessment of China’s
standard for energy consumption of buildings, Energy Build. 143 (2017) 114–
128, doi:10.1016/j.enbuild.2017.03.019.
[10] Y. Sheng, Z. Miao, J. Zhang, X. Lin, H. Ma, Energy consumption model and en-
ergy benchmarks of five-star hotels in China, Energy Build. 165 (2018) 286–
292, doi:10.1016/j.enbuild.2018.01.031.
[11] D. Yan, W. O’Brien, T. Hong, X. Feng, H. Burak Gunay, F. Tahmasebi, A. Mah-
davi, Occupant behavior modeling for building performance simulation: cur-
rent state and future challenges, Energy Build. 107 (2015) 264–278, doi:10.
1016/j.enbuild.2015.08.032.
[12] D.B.D.E. Santo, An energy and exergy analysis of a high-efficiency engine tri-
generation system for a hospital: a case study methodology based on an-
nual energy demand profiles, Energy Build. 76 (2014) 185–198, doi:10.1016/j.
enbuild.2014.02.014.
[13] P. Gonçalves, A.R. Gaspar, M.G. Da Silva, Energy and exergy-based indicators
for the energy performance assessment of a hotel building, Energy Build. 52
(2012) 181–188, doi:10.1016/j.enbuild.2012.06.011.
[14] A. Juaidi, F. AlFaris, F.G. Montoya, F. Manzano-Agugliaro, Energy benchmarking
for shopping centers in Gulf coast region, Energy Policy 91 (2016) 247–255,
doi:10.1016/j.enpol.2016.01.012.
[15] J. Zhao, Y. Xin, D. Tong, Energy consumption quota of public buildings based on
statistical analysis, Energy Policy 43 (2012) 362–370, doi:10.1016/j.enpol.2012.
01.015.
[16] Z. Wei, W. Xu, D. Wang, L. Li, L. Niu, W. Wang, B. Wang, Y. Song, A study of
city-level building energy efficiency benchmarking system for China, Energy
Build 179 (2018) 1–14, doi:10.1016/j.enbuild.2018.08.038.
[17] Y.S. Kim, J. Srebric, Impact of occupancy rates on the building electricity con-
sumption in commercial buildings, Energy Build. 138 (2017) 591–600, doi:10.
1016/j.enbuild.2016.12.056.
[18] P.O. Oluseyi, O.M. Babatunde, O.A. Babatunde, Assessment of energy consump-
tion and carbon footprint from the hotel sector within Lagos, Nigeria, Energy
Build. 118 (2016) 106–113, doi:10.1016/j.enbuild.2016.02.046.
[19] Y. Xin, S. Lu, N. Zhu, W. Wu, Energy consumption quota of four and five star
luxury hotel buildings in Hainan province, China, Energy Build. 45 (2012) 250–
256, doi:10.1016/j.enbuild.2011.11.014.
[20] C. Yan, S. Wang, F. Xiao, A simplified energy performance assessment method
for existing buildings based on energy bill disaggregation, Energy Build. 55
(2012) 563–574, doi:10.1016/j.enbuild.2012.09.043.
[21] H. Li, X. Li, Benchmarking energy performance for cooling in large commercial
buildings, Energy Build. 176 (2018) 179–193, doi:10.1016/j.enbuild.2018.07.039.
[22] W. Xuchao, R. Priyadarsini, L. Siew Eang, Benchmarking energy use and green-
house gas emissions in Singapore’s hotel industry, Energy Policy 38 (2010)
4520–4527, doi:10.1016/j.enpol.2010.04.006.
[23] Y. Ding, Z. Zhang, Q. Zhang, W. Lv, Z. Yang, N. Zhu, Benchmark analysis of elec-
tricity consumption for complex campus buildings in China, Appl. Therm. Eng.
131 (2018) 428–436, doi:10.1016/j.applthermaleng.2017.12.024.
[24] J. Roth, R. Rajagopal, Benchmarking building energy efficiency using quantile
regression, Energy 152 (2018) 866–876, doi:10.1016/j.energy.2018.02.108.
[25] M.W. Ahmad, M. Mourshed, Y. Rezgui, Trees vs neurons: comparison between
random forest and ann for high-resolution prediction of building energy con-
sumption, Energy Build. 147 (2017) 77–89, doi:10.1016/j.enbuild.2017.04.038.
[26] F. Smarra, A. Jain, T. de Rubeis, D. Ambrosini, A. D’Innocenzo, R. Mangharam,
Data-driven model predictive control using random forests for building energy
optimization and climate control, Appl. Energy 226 (2018) 1252–1272, doi:10.
1016/j.apenergy.2018.02.126.
[27] A. Jain, F. Smarra, M. Behl, R. Mangharam, Data-driven model predictive con-
trol with regression trees-An application to building energy management, ACM
Trans. Cyber Phys. Syst. 2 (1) (2018) 1–12, doi:10.1145/3127023.
[28] F. Khayatian, L. Sarto, G. Dall’O’, Application of neural networks for evaluat-
ing energy performance certificates of residential buildings, Energy Build. 125
(2016) 45–54, doi:10.1016/j.enbuild.2016.04.067.
[29] S. Papadopoulos, E. Azar, W.L. Woon, C.E. Kontokosta, Evaluation of tree-based
ensemble learning algorithms for building energy performance estimation, J.
Build. Perform. Simul. 11 (2017) 322–332, doi:10.1080/19401493.2017.1354919.
[30] S. Papadopoulos, C.E. Kontokosta, Grading buildings on energy performance
using city benchmarking data, Appl. Energy 233-234 (2019) 244–253, doi:10.
1016/j.apenergy.2018.10.053.
[31] X. Gao, A. Malkawi, A new methodology for building energy performance
 Y. Ding and X. Liu / Energy & Buildings 209 (2020) 109711
benchmarking: an approach based on intelligent clustering algorithm, Energy
Build. 84 (2014) 607–616, doi:10.1016/j.enbuild.2014.08.030.
[32] S. Önüt, S. Soner, Energy efficiency assessment for the antalya region hotels in
Turkey, Energy Build. 38 (2006) 964–971, doi:10.1016/j.enbuild.2005.11.006.
[33] W.S. Lee, Benchmarking the energy efficiency of government buildings with
data envelopment analysis, Energy Build. 40 (2008) 891–895, doi:10.1016/j.
enbuild.2007.07.001.
[34] J. Keirstead, Benchmarking urban energy efficiency in the UK, Energy Policy 63
(2013) 575–587, doi:10.1016/j.enpol.2013.08.063.
[35] J. Buck, D. Young, The potential for energy efficiency gains in the Cana-
dian commercial building sector: a stochastic frontier study, Energy 32 (2007)
1769–1780, doi:10.1016/j.energy.20 06.11.0 08.
[36] M. Filippini, L.C. Hunt, US residential energy demand and energy efficiency:
a stochastic demand frontier approach, Energy Econ. 34 (2012) 1484–1491,
doi:10.1016/j.eneco.2012.06.013.
[37] Z. Yang, J. Roth, R.K. Jain, DUE-B : data-driven urban energy benchmarking of
buildings using recursive partitioning and stochastic frontier analysis, Energy
Build. 163 (2018) 58–69, doi:10.1016/j.enbuild.2017.12.040.
[38] D. Aigner, K. Lovell, P. Schmidt, Upland development and prospects for the ru-
ral poor: experience in northern Thailand, J. Econom. 6 (1977) 21–37, doi:10.
1016/0304- 4076(77)90052- 5.
[39] H.F. Deng, D. Fannon, M.J. Eckelman, Predictive modeling for us commer-
cial building energy use: a comparison of existing statistical and machine
learning algorithms using CBECS microdata, Energy Build. 163 (2018) 34–43,
doi:10.1016/j.enbuild.2017.12.031.
[40] T. Ostergard, R.L. Jensen, S.E. Maagaard, A comparison of six metamodeling
techniques applied to building performance simulations, Appl. Energy 211
(2018) 89–103, doi:10.1016/j.apenergy.2017.10.102.
[41] Y.B. Chen, H.W. Tan, U. Berardi, A data-driven approach for building en-
ergy benchmarking using the Lorenz curve, Energy Build. 169 (2018) 319–331,
doi:10.1016/j.enbuild.2018.03.066.
[42] EIA 2019. Commercial Buildings Energy Consumption Survey (CBECS). < http:
//www.eia.gov/consumption/commercial/about.cfm> (Accessed 24 June 2019).
13
[43] United States Department of Energy. Building performance database.
<https://www.energy.gov/eere/buildings/building- performance- database- bpd
>(Accessed 24 June 2019 ).
[44] H. Yoshino, T. Hong, N. Nord, IEA ebc annex 53: total energy use in buildings—
Analysis and evaluation methods, Energy Build. 152 (2017) 124–136, doi:10.
1016/j.enbuild.2017.07.038.
[45] Y. Jiang, X. Yang, The electricity equivalent calculation used in the energy anal-
ysis, Energy China. 32 (2010) 5–11 (in Chinese).
[46] W.H. Greene, A gamma-distributed stochastic frontier model, J. Econom. 46
(1990) 141–163, doi:10.1016/0304- 4076(90)90052- U.
[47] G.E. Battese, G.S. Corra, Estimation of a production frontier model: with appli-
cation to the pastoral zone of eastern australia, Aust. J. Agric. Econ. 21 (1977)
169–179, doi:10.1111/j.1467-8489.1977.tb00204.x.
[48] T. Coelli, A guide to frontier version 4.1: a computer program for stochastic
frontier production and cost function estimation, Comput. Progr. Stoch. Front.
7 (1996) 1–33, doi:10.10 07/BF0 0158774.
[49] J. Jondrow, K. Lovell, I.S. Materov, P. Schmidt, On the estimation of technical
inefficiency in the stochastic frontier production function model, J. Econom. 19
(1982) 233–238, doi:10.1016/0304-4076(82)90 0 04-5.
[50] J.C. Wang, A study on the energy performance of hotel buildings in Taiwan,
Energy Build. 49 (2012) 268–275, doi:10.1016/j.enbuild.2012.02.016.
[51] Coelli, T., Henningsen, A., 2017. Frontier: stochastic frontier analysis. R Package
Version 1.1–2. < https://cran.r-project.org/web/packages/frontier >(Accessed 24
June 2019).
[52] C. Koulamas, A.P. Kalogeras, R. Pacheco-torres, J. Casillas, L. Ferrarini, Suitability
analysis of modeling and assessment approaches in energy efficiency in build-
ings, Energy Build. 158 (2018) 1662–1682, doi:10.1016/j.enbuild.2017.12.002.
[53] Institute for Market Transformation. Coming to NYC: Building Energy Grades;
2018. < https://www.imt.org/coming- to- nyc- building- energy- grades/ >(Ac-
cessed 24 June 2019).
[54] D. He, Y. Huang, Y... Ding, Systematically analysis and study on technical mea-
sures of energy savings for typical public buildings in Chongqing, J. Chongqing
Univ. Sci. Technol. (Nat. Sci. Ed.) 16 (2014) 112–116.