Journal o f Hydrology, 34 (1977) 129--149 129

,.:) Elsevier Scientific Publishing Company, Amsterdam -- Printed in The Netherlands

A PROCEDURE FOR THE SELECTION OF OBJECTIVE FUNCTIONS

FOR HYDROLOGIC SIMULATION MODELS

M.H. DISKIN* and E. SIMON

Faculty o f Civil Engineering, Technion -- Israel Institute o f Technology, Haifa (Israel)
Tahal -- Water Planning for Israel Ltd., Tel Aviv (Israel)
(Received April 30, 1975; revised and accepted March 30, 1976)

ABSTRACT

Diskin, M.H. and Simon, E., 1977. A procedure for the selection of objective functions for
hydrologic simulation models. J. Hydrol., 34: 129--149.

Hydrologic simulation models are calibrated by comparing observed data with data
generated by ;he models. The comparison is made in an optimization procedure using an
objective function adopted for that purpose and a set of data which is a subset of all data
available or observable. The choise of the set of data and of the objective function to be
used for any given model is a subjective decision which influences the values of the model
parameters and the performance of the model. The set of data used should be comparable
to the engineering application for which the model is intended. The objective function can
be chosen by a procedure outlined in the article in which a number of possible functions
are considered and compared with reference to one or more engineering applications.

INTRODUCTION

R e c e n t y e a r s h a v e w i t n e s s e d a large i n c r e a s e in t h e n u m b e r of c o n c e p t u a l
h y d r o l o g i c m o d e l s p r o p o s e d or a d o p t e d f o r t h e s i m u l a t i o n o f t h e b e h a v i o r o f
w a t e r s h e d s . T h e s i m u l a t i o n m o d e l s c o n s i d e r e d h e r e i n are t h o s e b a s e d on w a t e r
balance equations applied at discrete time intervals to the elements of the
m o d e l s as well as t o t h e e n t i r e m o d e l . M o d e l s d e s c r i b e d in t h e t e c h n i c a l litera-
t u r e r a n g e f r o m t h e v e r y c o m p l i c a t e d c o m p r e h e n s i v e S t a n f o r d m o d e l (Craw-
f o r d a n d L i n s l e y , 1 9 6 6 ) to t h e v e r y s i m p l e s p e c i a l p u r p o s e l i n e a r r e g r e s s i o n
m o d e l ( D i s k i n , 1 9 7 0 b , D i s k i n e t al., 1 9 7 3 ) . T h e p o t e n t i a l user o f h y d r o l o g i c
m o d e l s m a y be c o n f u s e d b y t h e w i d e c h o i c e o f m o d e l s a v a i l a b l e a n d t h e
n e c e s s i t y t o c h o o s e b e t w e e n v a r i o u s m o d e l s all o f w h i c h a p p a r e n t l y p r o d u c e
e q u a l l y g o o d results. S o m e g u i d e l i n e s are t h u s n e e d e d f o r t h e b e n e f i t o f t h e
users o f h y d r o l o g i c m o d e l s . T h e s e will be u s e f u l also t o t h e h y d r o l o g i s t w h o
m a y wish t o p r o p o s e or d e v e l o p y e t a n o t h e r m o d e l .

*Visiting Professor, Department of Hydrology and Water Resources, University of Arizona,

Tucson, Arizona, U.S.A.
130

The basic rule for the selection of a hydrologic model is, according to
Dawdy (1969), to adopt the simplest model compatible with the uses for
which the model is intended. Except for some obvious simple cases (Diskin
et al., 1973) the development of a model to meet a given set of conditions
and demands is not a simple task. The above rule is, however, useful for
practical applications only if some yardstick or measure is available for
assessing the degree to which the model meets the requirements of its users.
To avoid bias, it is advisable to develop the measure for the utility of the
model with reference to a number of objective functions and not only the
one used in the optimization procedure.
Following Box and Jenkins (1970), Diskin (1970a), Dooge (1972), James
(1972), and Diskin et al. (1973), the following procedure may be formulated
for the selection of a hydrologic model.
(a) Define the problems to be solved by the model or the range of applica-
tions for the model.
(b) Define the processes which take place in the watershed, their possible
effect on the applications intended for the model, and the constraints on
these processes.
(c) Formulate the various procedures proposed or available for the solution
of the problems to be solved.
(d) Select the type of model likely to meet the required applications, and
define the extent of the phenomena to be represented by the model.
(e) Define the structure of the model and the equations representing its
various components.
(f) Select the time increments used for representing the input and o u t p u t
data and for carrying out the computations.
(g) Select an objective function to be used in the optimization scheme.
(h) Calibrate the model with the aid of past records using a suitable opti-
mization procedure and the objective function adopted.
(i) Examine the performance of the model in the light of its intended uses.
(j) Refine the model if necessary. This may include introducing changes in
the structure of the model and in the time increments used.
(k) Apply the model to the problem or problems for which it was intended
and evaluate the results produced.
Steps (e)--(j) may be repeated a number of times in the light of results ob-
tained in the final stages of the analysis.
It should be pointed out at this stage that the selection of an objective
function for the optimization procedure is in itself a subjective decision
which influences the optimal values of the model parameters. Thus the opti-
mal set of parameters is optimal only in the context of the objective function
selected.
In this article the problem of the selection of an objective function is
examined and some recommendations are outlined for the method of choos-
ing the function to be used. The objective function is normally defined as a
function of the difference between computed and observed data during the
 131

calibration period. In the present work the comparison is based n o t on all the
measured data b u t on a specified subset of them. The subset used in any given
case is such th at the objective f unc t i on becomes oriented towards a certain
engineering application. Moreover, in some cases the comparison may be
made n o t between the c o m p u t e d and observed values but between param-
eters derived from these.

OBJECTIVE FUNCTIONS AND MEASURES OF MODEL PERFORMANCE

The most c o m m o n l y used objective function for hydrologic simulation

models is the sum of squared deviations, defined by:
U = ).2(y i -- x i ) 2 (1)
in which Yi are the observed values of r u n o f f for a specified time interval and
x i are the values of r u n o f f generated by the model for the same period of
time. The n u m b e r of items c om par e d in the above summation is N and the
value of the f u n c t i on to be minimized is U. The value of U depends on indi-
vidual values of x i which in turn depend on the structure of the model con-
sidered and the values of its parameters.
Other objective functions used in the present study are listed in Appendix
I. In all objective functions Y i and x i appear as the variables and the form of
the expression defining Uj is such that it gives more or less emphasis to vari-
ous parts o f the data. It should also be not e d that some objective functions
may b eco me zero also when n o t all errors are zero but also when their mean
becomes zero.
Each o f the objective functions can also serve as a measure of performance
o f any given model. In this role the function is n o t used in the optimization
process, b u t its value is c o m p u t e d for any given set of parameter values and
the p er f o r man ce of the model judged by the closeness of this value to the
ideal value. The ideal value is zero for most objective functions but, some
o t her finite value may be the limit to which the objective function tends.
In the p r o ced u r e described herein a n u m b e r of functions are considered
simultaneously in the study of a given model. One of these serves as an objec-
tive function, while the others are used as measures of model performance.

ENGINEERING APPLICATIONS OF HYDROLOGIC MODELS

The best simulation model according to some investigators is that which

simulates at small time intervals all the processes which take place in the real
watershed. Hydrologists who adopt this view argue that such a model is
superior because the information needed for all conceivable uses can be ob-
tained directly from the data generated by the model. A little objective
thinking leads, however, to the conclusion t hat the above argument is not
sound. Thus, if the annual yield of a watershed is all the information needed,
the value obtained by summing up 35,040 values of r u n o f f values for 0.25-h
132

intervals is not necessarily better than that obtained by adding twelve m o n t h l y

r u n o f f values or even a value of yearly r u n o f f generated directly by a suitable
model. Similarly, if only the annual peak flow is needed there is very little
justification in computing hourly values of infiltration, evaporation, ground-
water storage, surface storage, etc., for the whole year for the purpose of
selecting the one value needed for the application in mind.
Another important problem involved in the choice of the most suitable
model is the choice of the set of observed data which will be used in the
calibration and verification of the model. These should be selected in such a
way that the set of generated data, needed for the engineering application
intended, would represent as closely as possible the variables that would be
selected if an unlimited array of various real measurements were available.
Thus, if for example, the data needed for a certain design problem are the
low flow values during winter months it is reasonable to base the comparisons
used for model calibration on these data or at least give these data extra
weight in the optimization scheme. Similarly, if a model is used in computing
the m o n t h l y contribution of a watershed to groundwater it is hard to justify
the use of a model calibrated on the basis of measured and simulated daily
runoff values using an objective function which tends to give extra weight to
}arge r u n o f f values.
The influence of the intended engineering application on the selection of
the objective function used with a given model was first recognized by Dawdy
and Thompson (1967). Further indications of the importance of this factor
are given by Lichty et al. (1968), Dawdy (1969) and Dawdy et al. (1972). An
example of an objective function based on two applications, proposed by
Dawdy et al. (1972), is a function expressed by the following equation:
UD = E(ln q i -- lnPi) 2 + 0.5 )_;(In u i in wi) 2 (2)
where q i a n d P i are observed and generated peak storm flows, v i and w i a r e
observed and generated volumes of storm flows, respectively.
In the present work the term "engineering application" was interpreted in
terms of the set of data selected for the application intended. It is obvious
that of all the information observed or observable in a watershed only a small
fraction is actually used in any given case. The reasons for this are that some
data are not relevant for the application or in some cases that the added data
increase the cost of the design without actually improving the results. If a
simulation model is used as an aid for the design it is reasonable to specify
that the calibration of the model should be based on the set of data which is
relevant for the application for which the model is intended.
A large number of engineering applications may be formulated. Some com-
mon examples are all values of daily or monthly runoff, maximum yearly
values of daily runoff, all daily runoff values that are not zero or all such
values larger than some specified threshold, etc. A list of engineering applica-
tions used in the present study is given in Appendix II.
 133

OUTLINE OF STUDY

The p r o ced u r e for the selection of objective functions presented in this

article was developed as part of a basic study of hydrological conceptual
models (Simon, 1974). The study included a critical review of various objec-
tive functions and measures of model effectiveness, a study of the conse-
quences o f adopting various sets of data to represent typical engineering
applications, an examination of the effect of using various time increments
to discretize the input and o u t p u t of the model, and a sensitivity analysis of
the model parameters.
The study was limited, because of practical considerations of time and
funds available, to a small n u m b e r of models and to data typical for the semi-
arid climate of Israel. Altogether, some ten models of progressively increasing
c o m p l e x i t y have been studied. These range from simple one- and two-param-
eter models with one storage element to a fairly complicated model (identi-
fied as model 12) which has eight parameters and two storage elements. How-
ever, in the main part of the study only two models were extensively used.
One was model 12 m e nt i one d above and the other was model 15 described
below. The data used were derived from three ephemeral streams located in
the central part o f Israel. The results obtained were of course influenced by
the choice o f models and data but t hey may be indicative of possible results
obtainable under ot her conditions. The m e thod of analysis used is, however,
considered to be o f sufficiently general interest for practicing hydrologists.

WATERSHED AND BASIC DATA

The three watersheds used are shown in Fig.1 and some o f their properties
are listed in Table I. These watersheds were chosen mainly because of the
availability of rainfall and r u n o f f data of acceptable quality but they repre-
sent also different conditions within the broad climatic classification of semi-
arid watersheds.
The Ekron watershed near Bet-Elazari is a 62-km 2 watershed located in the
low hills area o f the coastal plane 25 km south of Tel Aviv. Most of the water-
shed has alluvial soils which are cultivated. The Upper Sorek watershed near
Ein Karem reservoir is in the central mountain range. It has an area of 79 km ~
and it is located a bout 10 km NW of Jerusalem. A small part of the watershed
is actually within the builtup area of the city. Most of the rest of the area is
uncultivated and includes some w o o d e d hills. The soil in the watershed is
terra-rosa and brown soils of small thickness overlying karstic limestone for-
mations. Some farming is carried out in the valley and in narrow terraces on
the steep slopes of the watershed. The Kishon watershed upstream of the
Kefar Baruch reservoir is a fairly large watershed (468 km 2) located about 45
km SW o f Haifa. A large portion o f the watershed is within the flat area of
the valley of Yezre'el characterized by heavy clay soils. The rest of the area is
on the steep slopes o f the Efrayim range. The low parts of the watershed are
134

0 10 20 30 40 50 km

LAKE
TIBERIA:
25O HAIFA

200

~o TEL

so.
EKRON
JERUSALEM/

lo.__~o

f 100

I
Fig.1. L o c a t i o n m a p .
150

I
200

extensively cultivated and fairly well drained b y a s y s t e m o f drainage chan-

nels. T h e m o u n t a i n o u s parts are m o s t l y f o r e s t e d b u t t h e r e are also a few
villages in these parts. T h e small cities o f Afula and Jenin are also within the
b o u n d a r i e s o f the watershed.
T h e climate in t h e t h r e e watersheds, as in the rest o f Israel, is c h a r a c t e r i z e d
b y a rainy season and a d r y season. T h e rainy season is f r o m O c t o b e r to April.
Only very small quantities o f rainfall are r e c o r d e d for the first and last m o n t h s
o f the above p e r i o d so t h a t the effective rainfall season is f r o m N o v e m b e r t o
March. Mean annual rainfall a m o u n t s for the t h r e e w a t e r s h e d s are listed in
'Fable I. T h e n u m b e r o f rainy days is b e t w e e n 40 and 60 per year. T h e m e a n
p o t e n t i a l e v a p o t r a n s p i r a t i o n for the rainy season are also given in Table I f o r
each o f the watersheds studied.
The m e t e o r o l o g i c a l d a t a used as i n p u t to the models c o n s i d e r e d in the s t u d y
included measured values o f daily rainfall, which were averaged over the area
 135

TABLE I

Some properties and data for the watersheds studied

Watershed

Ekron Sorek Kishon

Area of watershed (km 2) 62 78 468

Length of main stream (kin) 12 17 49
Slope of main stream (%) 0.89 1.85 1.50
Maximum elevation (m) 200 885 450
Minimum elevation (m) 35 550 55
Mean annual rainfall (1931--1960) (mm) 500 600 480
Mean rainy seasons potential evapotranspiration
(Oct.--Apr.) (mm) 625 623 610
Mean annual measured runoff (mm) 55.2 12.6 13.1
Flow regime floods floods floods + base flow

of each watershed by the Thiessen method, and mean monthly values of daily
evapotranspiration for each watershed, which were in the range of 1.8--4.0
mm/d. These average values were derived from actual observations of daily
evaporation from Class-A pans multiplied by a correction factor of 0.78.
Because of the long rainless season the various storage elements in the models
were assumed to be empty at the beginning of each rainy season.
R u n o f f data used were of two kinds. One was actual daily runoff, measured
by the Hydrological Service of Israel and published in its Hydrological Year-
books. The other was synthetic daily runoff data produced by the Tahal
simulation model which is a local adaptation of the Stanford mark IV model
developed by the Water Planning for Israel Company (Tahal). The parameters
used in the Tahal model for generation of these synthetic data were derived
by an optimization procedure using actually measured daily r u n o f f data. The
reason for using the synthetic data instead of the measured values was the
desire to eliminate noise in the measured data. It was thought that the use of
noiseless data could give a better basis for the various comparisons. Compari-
sons and conclusions based on measured data were found, however, to produce
the same results as those based on synthetic data.
One of the problems considered in the study was the effect of the length
of the data series. For this purpose two series lengths were considered, a short
series of 4 years and a long series of 16 years duration. While most years selected
for each of these series in each watershed were consecutive, this was not so in all
cases. It was considered more important to include years with high, medium
and low r u n o f f rather than have consecutive years. This procedure was justi-
fied because the annual long rainless periods made each year's r u n o f f indepen-
dent of previous years. Results reported in the example given below are those
related to the short series of data. Similar findings were, however, obtained
with the long series of data.
136

PROCEDURE FOR OBJECTIVE FUNCTION SELECTION

The first step in the proposed procedure is to define a number of objective

functions and to use each one in turn to derive the optimal parameters of the
model adopted, using the set of data defined by the engineering application
considered. The set of parameter values Pj thus obtained for a given model and
engineering application are the optimal for the objective function Uj considered.
Using these parameters Pj, the values of other objective functions are computed
and used as measures of model performance. The values obtained, denoted by
Wjk, are of course higher than the value of the objective function obtained by
the optimal parameters for that function. The relationship between the two
variables is given by:
wjk > vj, j¢ k (3)
wik = uj, j= (4)
The closeness of the value Wjk to the minimum value of another objective
function Uk is an indication of how well can one objective function (Uk) be
represented by another objective function (Uj).
Arranging the results obtained by the above procedure in a tabular form
produces a matrix with elements Wjk. The elements thus give the value of
each function denoted by the index k and used now as a measure of model
effectiveness for all sets of model parameters derived by use of other objective
functions identified by the index j.
The matrix which contains the actual values of objective functions Wjk is
transformed in the next stage of the procedure to another matrix in which
ranks are assigned to the various objective functions instead of the actual
values. The ranks Rjk are assigned according to the relative magnitude of the
computed values of Wjk in each group of values identified by a given index k,
starting with a value of 1 for the lowest value. The ranks assigned to the vari-
ous objective functions obtained by use of a set of parameters Pj are thus
integers indicating how well this set of parameters could give rise to low
values of other objective functions. Naturally the value of the kth objective
function was lowest for the parameters Pk so that Rkk = 1 for all cases.
A set of optimal parameters Pj which could produce low values of the ranks
Rjk for a large number of other objective functions may be considered to be
superior to other sets of optimal parameters. The objective function U] used
in the derivation of this set of parameters may, therefore, be adopted as the
recommendable objective function. The " b e s t " objective function of those
considered might be defined and identified by adding up the ranks assigned to
all functions according to each value o f j (i.e., Y,jRjk), and choosing the func-
tion for which this sum has the lowest value. Theoretically the lowest possible
value for this sum is 2N R -- 1 where NR is the number of values added.
The results obtained depend of course also on the engineering application
for which the model is intended, as expressed by the set of data selected for
 137

computing the various objective functions. If a number of engineering applica-

tions are contemplated for a given model the above procedure for the selection
of the most suitable objective function may be extended by performing the
above analysis for each of the engineering application separately. As a result
it is possible to rank the sets of parameters or objective functions obtained
for each engineering application according to how well they could also repre-
sent the other engineering applications. The new ranks thus obtained may be
used to point out the objective functions which receive a good evaluation (a
low rank) for a number of engineering applications. These may be adopted as
good general purpose objective functions when a model is used for more than
one application.

SIMULATION MODELS USED

As mentioned above two hydrologic simulation models have been used in

the main part of the study. These were model 12 which was applied to the
Kishon watershed and model 15 used for the Ekron and Sorek watersheds.
The reason for employing model 12 for the Kishon watershed was the occur-
rence of longer periods of low r u n o f f due to base flow which could not be
successfully simulated with the simpler model 15. Both models are moisture
accounting models using daily values of precipitation, evapotranspiration and
runoff. Model 12 is shown schematically in Fig.2 but a description of its struc-
ture is not included because the example presented herein is based on model
15. A detailed description of model 12 is available elsewhere (Simon, 1974;
Simon and Diskin, 1975}.
A schematic drawing of model 15 is shown in Fig.3. It consists of a single
storage element of m a x i m u m capacity SM which represents both soil storage
and depression and interception storage in the watershed. The moisture con-
tents S of this storage element at the start of each day is augmented by the

EI r+
+ I ' ~I +FC
VIMI+-+~ ~L/4IAG'OX~ +AL
El P
+ I+v Ip~LL~ ~ '
I \
[ + R2
i

SM
XK __ : R 1
XC t__ R
t J

1+
Fig.2. Structure o f m o d e l 12. Fig.3. Structure o f m o d e l 15.
138

dally precipitation P and depleted by the daily evaporation E, to yield a tem-

porary new value o f moisture cont ent s S':
S' = S + P - E (5)
The evaporation E is taken equal to the daily potential evapotranspiration Ep
only if the storage element is full and daily precipitation less than 1.0 mm. If
the storage element is n o t full the evaporation is reduced in p r o p o r t i o n to the
ratio (S + P)/SM. On days when the precipitation is more than 1.0 mm
(P > 1.0) the above values of E are reduced by 50%.
The occurrence of r u n o f f in the model depends on the relationship between
the moisture contents S' and the threshold storage SA. As long as S' ~< SA no
flow is p r o d u ced by the model either no groundwater (i.e., G -- 0) or to sur-
face r u n o f f (R = 0). The value of S for the end of the day is taken in this case
to be equal to S'. If S' is larger than SA but less than SM some surface r u n o f f
R is p r o d u c e d as well as some flow to groundwater G. The equation for the
surface r u n o f f is based on a spatially variable infiltration c o n c e p t which leads
to the following expression:
R = RI = XN(XN + 1 ) - ' ( S ' - SA)[(S' - S A ) / ( S M - SA)] l/xN (6)
where XN is a model parameter which describes the variability of the infiltra-
tion capacity with the area in the watershed.
If the value o f S' is larger than the m a x i m u m capacity SM o f the storage
element an additional c o n t r i b u t i o n of surface r u n o f f R2 is produced the
magnitude of which is c o m p u t e d by:
R: = S' - SM (7)
The value o f R 1 in this case is c o m p u t e d by the following expression obtained
by substituting SM for S' in eq.6:
R , = XN(XN + 1)-l(SM - SA) (8)
The total r u n o f f is equal to the sum of the t w o c o m p o n e n t s :
R = R2 + R~ = (S' - SM) + XN(XN + 1 ) - I ( S M - SA) (9)
The flow to groundwater G, which does n o t c o n t r i b u t e to the yield o f the
watershed and may be considered as a loss, is c o m p u t e d for the t w o cases
discussed above by:
a = XK(S' - SA - R) (10)
where XK is an additional parameter of the model representing the percola-
tion process. The value of moisture S remaining in storage at the end of the
day is given by:
S=S'-(R +a) (11)

This value is also the initial value used as a starting poi nt for next day's com-
putation.
 139

The four parameters of model 15 (SM, SA, XK and XN) have by definition
only positive (or rather non-negative) values, with the additional requirements
that SA < SM and XK ~< 1.0. The maximum value of XN is unlimited. In prac-
tical cases the value of XN is expected to be in the range of 0.2 ~< XN ~< 3.0.

APPLICATION OF SELECTION PROCEDURE

The procedure for the selection of the most suitable objective function is
illustrated below with reference to model 15 using a data set identified in the
study as engineering application 1 for the Ekron watershed. To facilitate com-
parison between the observed and generated values this set of data included
runoff values for all days on which either the measured or the generated
values were not zero.
As stated above, the first step in the analysis was the calibration of the
model selected for each watershed, using daily non-zero r u n o f f data corre-
sponding to engineering application 1. The results consisted of the optimal
parameters P] for each of the objective function U1 used. The set of optimal
parameters for each objective function was derived by the pattern search
method of optimization described by Green (1970). The starting point of the
search, i.e., the initial values of the parameters, were the same for all objective
functions considered. At the conclusion of this stage of the analysis values of
Wjk were c o m p u t e d for all objective functions using the sets of optimal
parameters PJ obtained by the pattern search procedure. In some of the cases
it was discovered that only local minima were reached since some of the Wlk
values were lower than the minimum computed for the optimal set of param-
eters. In these cases the pattern search procedure was again employed to find
a new set of optimal parameters. The starting point in these cases were the
set of parameters --Pi which gave the lowest value of Wlk. New values of Wlk
were computed with the new set of optimal parameters and the whole process
was repeated if necessary.
An example of the results obtained at this stage is given in Tables II and
III. Table II lists the values of the parameters for each of the objective func-
tions listed in Appendix I. Also given are values of the mean yearly flow as
derived from the generated data using the corresponding sets of optimal param-
eters and observed rainfall data for the 4 years period used in the calibration stage.
The values listed should be compared to the observed mean value of
2.983.106 m 3. Table III gives the values of each of the various objective func-
tions c o m p u t e d with the aid of data generated by model 15 using each of the
sets of parameters listed in Table II.
The next stage in the analysis was to replace the actual values of Wlk by
ranks Rjk according to the magnitude of the items included in each group of
Wjk values identified by a given value of the index k. The table of ranks derived
from Table III is shown in Table IV for the Ekron data. In cases where two or
more entries in Table III have the same rounded value, the corresponding ranks
140

'FABLE II

Optimal parameters and mean annual flow for model 15 according to various objective
functions using short-period data for the Ekron watershed

Objective Optimal model parameters Mean annual

function - flow
SM SA XK XN (10" m ~)

j= 1 161.8 93.5 0.813 0.845 2.866

2 136.8 80.7 0.943 0.809 3.090
3 161.7 93.6 0.832 0.851 2.835
4 152.0 89.8 0.889 0.862 3.016
5 165.0 93.3 0.771 0.868 2.984
6 136.2 80.3 0.943 0.808 3.103
7 165.1 92.6 0.771 0.876 2.999
8 111.4 52.9 0.818 0.631 2.900
9 165.1 93.4 0.771 0.868 2.982
10 175.4 108.5 0.799 0.831 2.719
11 153.9 97.5 0.988 0.775 2.507
12 152.3 84.3 0.844 0.817 2.800

in Table IV were d e t e r m i n e d a c c o r d i n g to the value o f the n e x t significant digit

n o t s h o w n in Table III.
I n s p e c t i o n o f the results in Table IV and in similar tables p r e p a r e d for
o t h e r w a t e r s h e d s led to the r a t h e r obvious c o n c l u s i o n t h a t s o m e o f the objec-
tive f u n c t i o n s c o u l d be o m i t t e d . This was based on the finding t h a t essentially
identicai ranks were o b t a i n e d for a n u m b e r o f objective f u n c t i o n s so t h a t o n l y
one o f these f u n c t i o n s c o u l d r e p r e s e n t the others. F o l l o w i n g this line o f rea-
soning the n u m b e r of objective f u n c t i o n s used in f u r t h e r c o m p a r i s o n s was
r e d u c e d to nine. This was achieved b y a d o p t i n g UI to represent U,, U3 and UT,
and the f u n c t i o n U2 t o represent Uz and U6.
The sums o f the ranks in vertical lines in Table IV give some i n d i c a t i o n o f
the relative merits of the various objective f u n c t i o n s as described above. Thus,
for a p p l i c a t i o n 1 r e p r e s e n t e d b y the set o f daily n o n - z e r o d a t a used as basis
for Table IV, it appears t h a t the l o w e s t sum is o b t a i n e d for the c o l u m n j = 1,
indicating t h a t the set o f p a r a m e t e r s P1 derived with the aid o f objective func-
t i o n Ut also gave low values o f m o s t o t h e r objective f u n c t i o n s . This result
d e p e n d s o f c o u r s e on the w a t e r s h e d and o n the set o f d a t a used in the analysis.
R e p e a t i n g the a b o v e p r o c e d u r e for the three w a t e r s h e d s studied, using in each
case daily n o n - z e r o r u n o f f d a t a for the s h o r t periods with the a p p r o p r i a t e
m o d e l led to similar o r d e r i n g o f the nine objective f u n c t i o n s a c c o r d i n g t o the
sums w h i c h were used to indicate the relative merits o f the f u n c t i o n s . Each o f
these nine objective f u n c t i o n s was thus assigned a m e r i t n u m b e r ranging f r o m
1 to 9 a c c o r d i n g to the m a g n i t u d e o f the sum o f ranks o b t a i n e d for t h a t ob-
jective f u n c t i o n f o r the three w a t e r s h e d s considered.
Values o f the merit n u m b e r s assigned to t h e various objective f u n c t i o n s b y
T A B L E III

Values of B~¢ c o m p u t e d by various o b j e c t i v e f u n c t i o n s identified b y i n d e x k using sets of p a r a m e t e r s Pj

j = 1 2 3 4 5 6 7 8 9 10 11 12

k = 1 0.176 0.371 0.178 0.217 0.182 0.385 0.185 0.514 0.182 0.226 0.291 0.225
2 119.3 83.2 106.8 97.7 146.5 83.3 146.8 89.9 146.4 135.8 102.6 100.7
3 0.333 0.462 0.330 0.360 0.351 0.471 0.352 0.554 0.351 0.380 0.409 0.368
4 0.205 0.247 0.203 0.193 0.215 0.250 0.215 0.297 0.215 0.219 0.271 0.219
5 0.039 0.036 0.050 0.011 0.0001 0.040 0.005 0.028 0.0005 0.089 0.159 0.061
6 1.326 1.015 1.227 1.149 1.526 1.014 1.530 1.057 1.526 1.492 1.252 1.170
7 0.026 0.055 0.026 0.032 0.026 0.057 0.026 0.076 0.026 0.033 0.043 0.033
8 1.633 1.344 1.580 1.536 1.839 1.347 1.854 0.942 1.839 2.123 1.702 1.307
9 0.067 0.239 0.100 0.119 0.002 0.252 0.016 0.146 0.0007 0.134 0.274 0.165
10 0.173 0.404 0.212 0.243 0.177 0.424 0.215 0.288 0.176 0.144 0.403 0.315
11 2.157 2.680 2.211 2.294 2.805 2.751 3.051 2.559 2.808 1.109 0..403 2.567
12 0.059 0.060 0.058 0.058 0.064 0.061 0.064 0.058 0.064 0.085 0.073 0.053
142

TABLE IV

Ranks derived from Wik v~ues given in Table III

j = 1 2 3 4 5 6 7 8 9 10 11 12

it = 1 1 10 2 6 4 11 5 12 3 8 9 7
2 8 1 7 4 11 2 12 3 10 9 6 5
3 2 10 1 6 4 11 5 12 3 8 9 7
4 3 9 2 1 4 10 6 12 5 8 11 7
5 7 6 9 4 1 8 3 5 2 11 12 10
6 8 2 6 4 11 1 12 3 10 9 7 5
7 2 10 3 6 5 11 1 12 4 7 9 8
8 7 3 6 5 10 4 11 1 9 12 8 2
9 4 10 5 6 2 11 3 8 1 7 12 9
10 2 11 5 7 4 12 6 8 3 1 10 9
11 3 8 4 5 10 9 12 6 11 2 1 7
12 5 6 3 4 9 7 10 2 8 12 11 1
Tot~ 52 86 53 58 75 97 86 84 69 94 105 77

the a b o v e p r o c e d u r e are listed in Table V in the c o l u m n h e a d e d " A p p l i c a t i o n

No. 1". Also listed are values o f merit n u m b e r s assigned t o the same objective
f u n c t i o n s b y the o t h e r sets of d a t a representing d i f f e r e n t engineering applica-
tions as d e f i n e d in A p p e n d i x II. The values of the merit n u m b e r s in Table V
are based on tables o f W]k and Rik similar to Tables III and IV discussed
above. Thus, for each engineering application o p t i m a l sets o f p a r a m e t e r s P],
derived b y the various objective f u n c t i o n s using all n o n - z e r o r u n o f f data, were
used to generate the o u t p u t s a p p r o p r i a t e for the a p p l i c a t i o n considered. F r o m
these o u t p u t s values o f Wit~ were c o m p u t e d and ranks Rjk were assigned as
before. S u m m i n g up the ranks for the three w a t e r s h e d s led to t h e merit n u m -
bers given in Table V for each o f the engineering applications c o n s i d e r e d . In
the derivation o f merit n u m b e r s for applications 4 and 5 o n l y s o m e o f the
objective f u n c t i o n s c o u l d be used as n o t e d in A p p e n d i x II.
I n s p e c t i o n o f Table V shows that, as e x p e c t e d , the merit n u m b e r assigned
t o a n y particular objective f u n c t i o n is n o t t h e same for all engineering applica-
tions for w h i c h the m o d e l is intended. F o r the six applications c o n s i d e r e d
herein, s o m e o f the objective f u n c t i o n s appear to give c o n s i s t e n t l y l o w e r {i.e.,
better) values o f merit n u m b e r s while others give c o n s i s t e n t l y higher values.
The s u m o f the m e r i t n u m b e r s for each objective f u n c t i o n t a k e n over all
engineering applications can thus be used as s o m e i n d i c a t i o n o f t h e over-all
p e r f o r m a n c e o f the objective f u n c t i o n .
It appears f r o m Table V t h a t for the watersheds, m o d e l s a n d applications
c o n s i d e r e d in this s t u d y , objective f u n c t i o n U4, based o n the s u m o f a b s o l u t e
deviations gives the best p e r f o r m a n c e . N e x t are U1 and U9 in the s e c o n d and
third place, respectively. T h e w o r s t f u n c t i o n s are Us, Us and U11.
The l o c a t i o n s o f the merit n u m b e r s in Table V p o i n t to a possible link
 143

TABLE V

Merit numbers assigned to various objective functions using the short series of data

Objective Application No. Total

function
1 2 3 4 5 6

1 1 4 1 1 6 6 19
2 5 9 7 5 4 1 31
4 2 5 2 3 1 2 15
5 9 6 9 8 5 7 44
8 6 8 8 9 8 5 44
9 4 3 5 4 2 3 21
10 7 1 4 2 7 8 29
11 8 2 3 5 9 9 36
12 3 6 6 7 3 4 29

b e t w e e n t h e m a t h e m a t i c a l f o r m u l a t i o n of t h e o b j e c t i v e f u n c t i o n s and t h e
engineering a p p l i c a t i o n . T h e s o u r c e o f this link can be t r a c e d to t h e t y p e o f
d a t a selected for each o f the engineering a p p l i c a t i o n s used.

CONCLUSIONS

T h e d e v e l o p m e n t of h y d r o l o g i c s i m u l a t i o n m o d e l s requires a f e w m a j o r
decisions on t h e p a r t o f the h y d r o l o g i s t w h o is engaged in this task. T h e s e
include t h e d e f i n i t i o n o f t h e s t r u c t u r e o f the m o d e l and t h e s e q u e n c e o f c o m -
p u t a t i o n s used t o c o n v e r t the rainfall i n p u t into r u n o f f , the a d o p t i o n o f a
basic t i m e i n c r e m e n t f o r c a r r y i n g o u t the c o m p u t a t i o n , and the selection and
d e f i n i t i o n o f t h e o b j e c t i v e f u n c t i o n for t h e o p t i m i z a t i o n p r o c e d u r e a d o p t e d
f o r t h e c a l i b r a t i o n o f the m o d e l . Each o f these decisions, has an i m p o r t a n t
e f f e c t on t h e results o b t a i n e d with t h e m o d e l . T h e decisions are, h o w e v e r ,
m a d e in m a n y cases o n the basis o f i n t u i t i o n and e x p e r i e n c e o f t h e h y d r o l o g i s t
and are t h u s n o t free f r o m subjective c o n s i d e r a t i o n s .
T h e a i m of t h e p r e s e n t s t u d y was to investigate the e f f e c t s o f t h e o b j e c t i v e
f u n c t i o n s e l e c t i o n and t o arrive if possible at s o m e r e c o m m e n d a t i o n s or guide-
lines f o r this selection w h i c h will r e d u c e t h e a p p a r e n t s u b j e c t i v i t y involved.
T h e results o b t a i n e d are p e r f o r c e l i m i t e d b y the t y p e o f d a t a and m o d e l s in-
c l u d e d in the s t u d y b u t it is e x p e c t e d t h a t similar studies using o t h e r t y p e s
o f d a t a or o t h e r m o d e l s will yield similar conclusions.
T h e m a i n c o n c l u s i o n o f the s t u d y is a d e m o n s t r a t i o n o f t h e i n a d e q u a c y o f
a single o b j e c t i v e f u n c t i o n to serve as a universal t o o l f o r the o p t i m i z a t i o n o f
h y d r o l o g i c models. It has b e e n d e m o n s t r a t e d t h a t t h e r e is a definite link
b e t w e e n t h e m a t h e m a t i c a l f o r m u l a t i o n o f an o b j e c t i v e f u n c t i o n a n d the t y p e
o f e n g i n e e r i n g a p p l i c a t i o n for w h i c h the m o d e l is used. It a p p e a r s t h a t b e t t e r
results are o b t a i n e d if the o b j e c t i v e f u n c t i o n is selected a c c o r d i n g to t h e
144

engineering application for which the results will be used. For example, ob-
jective functions U1, U3 and UT, all based on the sum of squared deviations,
produce the best results for each of engineering applications 1, 3 and 4. These
three applications refer respectively to all daily non-zero r u n o f f values, to the
m a x i m u m m o n t h l y values of daily runoff, and to all daily r u n o f f values
higher than 0.5.106 m 3. The same objective functions give fairly poor results
if used for other engineering applications such as application 5 referring to
all daily r u n o f f values lower than 0.5.106 m 3 or to application 6 which is
based on the flow duration curve. Other combinations of objective functions
and engineering applications which produce good results are identified by the
merit numbers listed in Table V where a low number signifies a desireable
combination. The user of a simulation model should thus calibrate his model
with such an objective function that is best capable of generating the type of
data that interests him.
Another important outcome of the study is the desireability of considering
more than one objective function in the optimization procedure for a given
model and a given engineering application. There are two reasons for this
recommendation. One is that the best function for the given application is
not known in advance. The second reason is that an optimization procedure
utilizing a number of objective functions (in prallel) is less likely to lead to a
local minimum for the one objective function adopted.
Considering the optimal sets of parameters obtained by the various objec-
tive functions (Table II), it appears that the ranges of values obtained for any
parameter are relatively small. The same observation was repeated in similar
tables, not reproduced in this article, prepared by using other watersheds and
other models. These results point to the great sensitivity of the models con-
sidered to the values of the parameters since relatively small changes in the
values of the parameters lead to large change in the values of the objective
functions. Another conclusion drawn from Table II is the ineffectiveness of
using the mean annual flow as a criterion for the selection of model param-
eters. It appears that results very near to the observed value may be obtained
with almost any set of parameters derived from the various objective functions
considered.
Each of the objective functions considered in this article may also be used
as a measure of model effectiveness. In this role the proximity of the com-
puted value of this function to its ideal value is used as an additional criterion
for the selection of the best objective function for a given application.
The results presented herein do not lead to a recommendation that any one
objective function is the best for all cases. The procedures developed contribute,
however, to the reduction of the subjectivity involved in choosing an objective
function for a given case. It is thus possible to choose an objective function
that will be significant to the engineering application considered and to arrive
at an optimal set of parameter values that is nearer to the global optimum
than the set obtained with other procedures.
 145

ACKNOWLEDGEMENTS

The paper is based on material prepared for and included in a D.Sc. Thesis
written by E. Simon (1974), with M.H. Diskin acting as research supervisor.
Partial financial support for the research project was obtained from the
Research Fund of the Faculty of Civil Engineering, Technion -- Israel Institute
of Technology, from the Israel Hydrological Service and from a research grant
of the I.B.M. Company. Assistance in developing and running the c o m p u t e r
programs was received from the C o m p u t e r Center of the Technion and from
personnel o f the D e p a r t m e n t of C om put e r Sciences in the Technion.

APPENDIX I

Definition of objective functions used

In the preliminary stages of the study m e n t i o n e d in this article, a large num-

ber of objective functions have been defined and studied. Some of these have
been collected from various sources in the technical literature and others have
been defined as part of the study. As the work progressed some of the func-
tions were discarded so that only twelve functions were retained for the main
part o f the study.
The equations defining the twelve functions are given below. Each function
is identified in the article by the value of the subscript j. There is no signifi-
cance to the order in which t hey are presented. The n o t a t i o n used is uniform
for all equations. The value of the objective function is d e n o t e d by Uj, the
observed values are d e n o t e d by Yi and the generated values by x i. Summations
are carried o u t over the n u m b e r of items N. The n o t a t i o n in eqs. AI-9--AI-11
is that ~ denotes the mean of the series Yi, Vy the variance, Sy the standard
deviation, Csy the coefficient of skew and Cky the coefficient of kurtosis.
Corresponding symbols apply to statistical parameters of the x i series. The
definitions of these parameters are as given by Yevjevich (1972).
To conserve space the origin and properties of the various functions are
n o t discussed here. Some notes on each of the functions listed are given by
Simon and Diskin (1975) and a detailed discussion of these and ot her objec-
tive functions are given by Simon (1974).
U, = W(yi xi) 2 (AI-1)
U2 = V [ ( 2 ( y i - xi) }/(yi + xi)] 2 (AI-2)
U3 = [ {N~Z(yi - x i ) 2 }'/~]/Wy i (AI-3)
Ua = (~ l Y i - - x i l ) / Z yi (AI-4)
Us = I S , ( y i - - xi)l/~Zyi (AI-5)
U6 = ( l / N ) [ { 2 ( y i -- x i ) } / ( y i + xi)] 2 (AI-6)
146

U7 = [ E ( y i - xi) 2]/[E(yi - 2¢)2] (AI-7)

Us = [ "lIE(yi 1/3 - xil/3) z } 3/2 ]/NV~Ey i (AI-8)

U9 = 2 L ( y - x ) / ( S y + Sx)L + 2[(Vy - V x ) / ( V y + Vx)l +

+ 21 (2?-- 2)/(:~ + 2)1 tAI-9)
Ulo = U9 + ICsy - Csxl (AI-10)
UII = Ulo + [Cky -- CkxL (M-11)
Uxz = [E(yi '/' - xiV')2]/EYi (AI-12)

APPENDIX II

Definitions o f various engineering applications

A d o p t i n g the t e r m engineering application to r e p r e s e n t a specific set o f

data, the f o l l o w i n g six engineering applications have b e e n d e f i n e d for the
p r e s e n t study. This is by n o means a c o m p l e t e list o f possible applications
and m a n y o t h e r s m a y be f o r m u l a t e d . T h e applications listed below are, how-
ever, fairly c o m m o n , and t h e y serve to illustrate the a p p r o a c h d e v e l o p e d
herein. In the article the applications are identified b y their n u m b e r s as assign-
ed in the following list.
Application 1 includes all daily n o n - z e r o r u n o f f values. Daily r u n o f f values
which are zero for b o t h m e a s u r e d and simulated d a t a occur, in the semi-arid
watersheds c o n s i d e r e d herein, on m o r e t h a n half the n u m b e r o f days o f ttae
rainy season and have been e x c l u d e d . The series thus c o n t a i n s Nz entries for
which e i t h e r t h e observed r u n o f f or the simulated r u n o f f or b o t h are n o t zero.
Application 2 includes the m a x i m u m y e a r l y values o f daily r u n o f f in the
observed and in the generated data. T h e n u m b e r o f items included is equal to
the n u m b e r o f years o f d a t a Ny. It should be n o t e d t h a t observed and generated
d a t a included are n o t necessarily for the same dates. Also, some o f the items
may have zero values if the flow is zero for the entire h y d r o l o g i c year, as m a y
h a p p e n in some arid and semi-arid watersheds.
Application 3 includes the m o n t h l y m a x i m u m values o f daily r u n o f f in the
observed and in the g e n e r a t e d data for the rainy season. T h e n u m b e r of items
in each series is equal to the n u m b e r o f m o n t h s o f data. F o r a 7 - m o n t h rainy
season N m = 7Ny. T h e items o f observed and g e n e r a t e d data are n o t always
for the same dates. S o m e o f the items in the series m a y have zero values.
Application 4 includes all values o f observed and g e n e r a t e d daily r u n o f f
larger t h a n an arbitrary threshold. T h e value o f t h r e s h o l d selected in the
p r e s e n t s t u d y was 0.5- 106 m 3 which ensured t h a t the n u m b e r o f values ob-
tained was a b o u t twice the n u m b e r o f years o f record. T h e n u m b e r o f items
included in the observed series is thus usually n o t equal to the n u m b e r o f
items in the g e n e r a t e d series. T h e o n l y objective f u n c t i o n s which c o u l d there-
fore be used for calibration o f models for this application are Ug, U~0 and U~.
 147

A p p l i c a t i o n 5 includes all values of observed and generated non-zero daily

r u n o f f which are smaller than the threshold value of 0.5.106 m 3. The above
remarks concerning the number of items in the observed and generated sets
and the objective functions which could be used are valid also to this applica-
tion.
A p p l i c a t i o n 6 is based on the flow--duration curve for the observed and
for the generated data. For this purpose the range of daily r u n o f f values was
divided into a number of segments Ns defined by:
Ns = 4 log Qmax + 2 {AWl)
where Q m a x is the m a x i m u m value of the observed or generated daily r u n o f f
(in 10 a m 3 units) whichever is larger. The length of the segments was equal on
a logarithmic scale and the end-points of segments were chosen so that each
log cycle was divided into four segments. The number of segments (Ns) thus
obtained was usually between 14 and 20. The same number of segments was
used in any given case for both observed and generated data. The values of Yi
and xi according to this application were not runoff values but frequencies,
i.e., the numbers of daily r u n o f f values falling in each segment when they were
arranged according to magnitude in the flow--duration curves. Some of the
values Yi or x i could according to the above definition be zero values.

APPENDIX III

Notation -- Symbols used in this paper

Ckx coefficient of kurtosis of the series xi

Cky coefficient of kurtosis of the series Yi
Csx coefficient of skew of the series x i
Csy coefficient of skew of the series Yi
E daily evaporation
Ep daily potential evapotranspiration
G generated daily ground water flow
i index of items included in the data series
] serial number identifying the objective function
k index of objective functions
N length of the series or the total number of items included in it, length of observed
record
Nm number of months of data
Ns number of segments in flow duration curve
Ny number of years of data
Nz number of non-zero items in the series of residuals (yi x i)
NR number of values added up in Table IV
P daily precipitation
P~,Pi a set of optimized parameters
Pi generated peak storm flows
Qmax maximum observed or generated daily runoff
qi observed peak storm flows
R generated daily surface runoff in model 15
R, generated daily surface runoff in model 15 when S' is less than SM
148

APPENDIX III (continued)

R, additional contribution of surface runoff in model 15 when S' is greater than SM

Rick rank of objective function on which the calibration was based
Rjk ranks replacing values of objective functions in matrix jk
S moisture content of the single storage element in model 15
S' temporary value of S
SA a parameter in model 15
SM a parameter in model 15
U value of an objective function
UD objective function based on peak flow and volume of runoff
Uk, Uj an objective function identified by index k or j (Appendix I)
Vx variance of the series x i
V>. variance of the series Yi
vi observed volume of storm flows
wi generated volume of storm flows
Wkk value of objective function obtained by the optimal parameters for that function
values of other objective functions computed according to Pj
X mean of the series x i
xi generated values of runoff, number of items in segments of generated flow
duration curve
XK a parameter in model 15
XN a parameter in model 15
mean of the series Yi
Yi observed values of runoff, number of items in segments of observed flow dura-
tion curve

REFERENCES

Box, G.E. and Jenkins, G.M., 1970. Time Series Analysis, Forecasting and Control. Holden
Day, San Francisco, Calif., 553 pp. (see especially Ch. 1, pp. 1--19).
Crawford, N.H. and Linsley, R.K., 1966. Digital simulation in hydrology; Stanford water-
shed model IV. Dep. Cir. Eng., Standord Univ., Stanford, Calif., Tech. Rep. No. 39.
Dawdy, D.R., 1969. Considerations involved in evaluating mathematical modeling of urban
hydrologic systems. U.S. Geol. Surv., Water Supply Pap. No. 1591-D.
Dawdy, D.R. and Thompson, T.H., 1967. Digital computer simulation in hydrology. J. Am.
Water Works Assoc., 59(6): 685--688.
Dawdy, D.R., Lichty, R.W. and Bergmann, J.M., 1972. A rainfall--runoff simulation model
for estimation of flood peaks for small drainage basins. U.S. Geol. Surv. Prof. Pap.,
506-B.
Diskin, M..H., 1970a. Objectives and techniques of watershed modeling. Proc. ARS--SCS
Workshop on Watershed Modeling, Tucson, Ariz., pp. 9.1--9.35.
Diskin, M.H., 1970b. Definition and uses of the linear regression model. Water Resour.
Res., 6(6): 1668--1673.
Diskin, M.H., Buras, N. and Zamir, S., 1973. Application of a simple hydrologic model for
rainfall--runoff relations of the Dalton watershed. Water Resour. Res., 9(4): 927--936.
Dooge, J.C.I., 1972. Mathematical models of hydrologic systems. Proc. Int. Symp. on
Mathematical Modeling Techniques in Water Resources Systems, pp. 171--189.
Green, R.F., 1970. Optimization by the pattern search method. TVA, Division of Water
Control Planning, Knoxville, Tenn., Res. Pap., No.7.
James, W., 1972. Developing simulation models. Water Resour. Res., 8(6): 1590--1592.
 149

Lichty, R.W., Dawdy, D.R. and Bergmann, J.M., 1968. Rainfall--runoff model for small
basin flood hydrograph simulation. Proc. Symp. on The Use of Analog and Digital
Computers in Hydrology, Tucson, Ariz. IASH Publ. No. 81, pp. 356--367.
Simon, E., 1974. A basic study of conceptual models for the hydrologic behavior of
watersheds and their engineering applications. D.Sc. Thesis, Technion -- Israel Institute
of Technology, Haifa. (In Hebrew, with English summary.)
Simon, E. and Diskin, M.H., 1975. Objective function formulation and their effect on
hydrologic simulation models. Faculty of Civil Engineering, Technion -- Israel Institute
of Technology, Haifa, Publ. No. 216.
Yevjevich, V., 1972. Probability and statistics in hydrology. Water Resour. Publ., Fort
Collins, Colo., 302 pp.