See

discussions, stats, and author profiles for this publication at:

https://www.researchgate.net/publication/225426727

EASY-FIT: A software system

for data fitting in dynamic
systems
Article in Structural and Multidisciplinary Optimization February 2002
Impact Factor: 1.97 DOI: 10.1007/s00158-002-0174-6

CITATIONS

READS

42

85

1 author:
Klaus Schittkowski
University of Bayreuth
124 PUBLICATIONS 3,989
CITATIONS
SEE PROFILE

Available from: Klaus Schittkowski

Retrieved on: 05 June 2016

EASY-FIT: A Software System for Data Fitting in

Dynamic Systems
Klaus Schittkowski
Universitt Bayreuth
Abstract
EASY-FIT is an interactive software system to identify parameters in explicit model functions,
dynamic systems of equations, Laplace transformations, systems of ordinary differential equations, differential algebraic equations, or systems of one-dimensional time-dependent partial differential equations with or without algebraic equations. Proceeding from given experimental
data, i.e. observation times and measurements, the minimum least squares distance of measured
data from a fitting criterion is computed, that depends on the solution of the dynamic system.
The software system is implemented in form of a relational database under MS-Access 97
running under MS-Windows. The underlying numerical algorithms are coded in Fortran and are
executable independently from the interface. Model functions are either interpreted and evaluated
symbolically by a program called PCOMP, that allows in addition automatic differentiation of
nonlinear model functions, or by user-provided Fortran subroutines. In the latter case interfaces
for the Fortran compilers Watcom F77/386, Salford FTN77, Lahey F77L-EM/32 and Microsoft
Fortran PowerStation are provided.

Introduction

Data fitting plays an important role in many natural science, engineering and
other disciplines. The key idea is to estimate unknown parameters in a mathematical model that describes a real life situation, by minimizing the distance
of some known experimental data from the theoretical model function values. Thus also model parameters that cannot be measured directly, can be
73

identified by a least squares fit and analysed subsequently in a quantitative

way. To sum up, parameter estimation or data fitting, respectively, is extremely important in all practical situations, where a mathematical model and
corresponding experimental data are available to describe the behaviour of a
dynamic system.
The purpose of the documentation is to introduce a software system called
EASY-FIT that can be used to compute parameters by a least squares fit interactively. The mathematical model that must be provided by the system
analyst, has to belong to one of the following categories:

explicit model functions

dynamic systems of equations (steady-state systems)
Laplace equations
ordinary differential equations
differential algebraic equations
one-dimensional, time-dependent partial differential equations
one-dimensional partial differential algebraic equations

The general mathematical model to be analysed and the corresponding

software contains additional features with the goal to apply it to a large variety
of different applications:
1. More than one fitting criterion can be defined, i.e. more than one experimental data set can be fitted within a model formulation.
2. The fitting criteria are arbitrary functions depending on the parameters to
be estimated, the solution of the underlying dynamic system, and the time
variable.
3. The model may possess arbitrary equality or inequality constraints and
upper and lower bounds for parameters to be estimated. Constraints are
allowed to depend also on the solution of the dynamic equation at given
time and concentration or spatial values, respectively.
4. Dynamic models may possess so-called switching points where the integration is restarted, e.g. for non-continuous input functions or model
changes.
5. An additional independent model parameter called concentration can be
introduced.
6. Differential algebraic equations can be solved up to index 3. Consistent
initial values for index-1-formulations are computed by the system.
7. In case of partial differential equations, also coupled ordinary differential
equations and non-continuous transitions between different areas can be
taken into account.
8. Partial differential equations may possess algebraic equations e.g. to solve
higher order systems. Consistent initial values are computed internally.
9. The model functions may be defined by their Laplace transforms. The
74

backtransformation is done internally.

10. Gradients can be evaluated by automatic differentiation, i.e. without additional round-off, truncation or approximation errors, and without compilation of Fortran code.
11. Partial differential equations may be defined by their flux functions to
facilitate the input. Also hyperbolic equations can be solved successfully
by special variants of the upwind formula.
12. Alternatively data can be fitted w.r.t. the L 1 (sum of absolute values) or
the L (maximum of absolute values) norm.
Only for illustration purposes we denote the first independent model variable the time variable of the system, the second one the concentration variable
and the dependent data as measurement values of an experiment. By this the
probably most frequent application is described. On the other hand, these
terms may have any other meaning within a model depending on the underlying application.
Nonlinear model functions can be evaluated symbolically. Thus any compilation and link of Fortran subroutines is not required whenever model functions are defined or altered in this way. A particular advantage of this approach is the automatic differentiation of model functions to avoid numerical
truncation errors. The corresponding program is called PCOMP, see Dobmann, Liepelt and Schittkowski [13], and is part of the executable codes.
The data fitting model, alternative phrases are parameter estimation or system identification, is outlined in Section 2. Is is shown, how the dymamic
systems have to be adapted to fit into the least squares formulation required
for starting an optimization algorithm.
A brief review of available numerical algorithms is presented in Section 3.
Only some basic features of the underlying ideas are presented. More details
are found in the references. The codes allow the numerical identification of
parameters in any of the seven situations under investigation. The executable
files are called MODFIT.EXE and PDEFIT.EXE.
EASY-FIT is in permanent use to solve real life problem, i.e. problems
with some realistic practical background. Application areas are for example
pharmacy, biochemistry, chemical engineering, physics, and mechanical engineering. A few applications are listed in Section 4. Moreover we present
a case study by which we investigate the acidification of groundwater, where
the diffusion is modeled by convection and dispersion. The example shows
how a typical application problem must be implemeted for EASY-FIT.
The interactive system EASY-FIT proceeds from a relational database for
storing model information, experimental data and results. A context sensitive
help is included containing additional technical and organizational information e.g. about the input of data and optimization tolerances. The organization
of the software system is outlined in Section 5.
75

Fig. 1: EASY-FIT: Creation of a New Problem

Fig. 2: EASY-FIT: Input of Experimental Data

76

Parameter Estimation Models

EASY-FIT is an interactive software system to identify parameters in explicit

model functions, dynamic systems of equations, Laplace transformations,
systems of ordinary differential equations, differential algebraic equations or
systems of one-dimensional, time-dependent partial differential equations optionally with additional algebraic equations. Proceeding from given experimental data, i.e. observation times and measurements, the minimum least
squares distance of measured data from a fitting criterion is computed, that
depends on the solution of the dynamic system. The model function may
depend also on an additional independent variable that could represent for
example a concentration value of an experiment.
The basic mathematical model is the least squares problem, i.e. the problem of minimization of a sum of squares of nonlinear functions of the form

l
min i=1 fi (p)2
gj (p) = 0 ,
p IRn : gj (p) 0 ,
pl p pu

j = 1, ..., me ,
j = me + 1, ..., m ,
.

(1)

Here we assume that the parameter vector p is n-dimensional and that all
nonlinear functions are continuously differentiable with respect to p. Upper
and lower bounds are treated independently from the remaining constraints.
All least squares parameter estimation algorithms proceed from the above
formulation, although in the one or other case different approaches are available to define the objective functions. The assumption, that all problem functions must be smooth, is essential. The efficient numerical algorithms under
consideration are based more or less on the Gauss-Newton method, that requires first derivatives.
Alternatively the L 2 -norm may be changed to the L 1 -norm, i.e.,

l
min i=1 |fi (p)|
gj (p) = 0 ,
p IRn : gj (p) 0 ,
pl p pu

j = 1, ..., me ,
j = me + 1, ..., m ,

(2)

or the L -norm leading to

min maxi=1,...,l |fi (p)|
gj (p) = 0 ,
p IRn : gj (p) 0 ,
pl p pu

j = 1, ..., me ,
j = me + 1, ..., m ,
.

(3)

77

Fig. 3: EASY-FIT: Discretization and Integration Tolerances

Fig. 4: EASY-FIT: Model Functions

78

2.1 Explicit Model Functions

In this case we investigate parameter estimation problems, where one vectorvalued model function is available in explicit form, the so-called fitting criterion function, with one additional variable called time, and optionally with another one called concentration. We proceed from r measurement sets, given
in the form
k
) , i = 1, . . . , nt , j = 1, . . . , nc , k = 1, . . . , r,
(ti , cj , yij

(4)

where nt time values, nc concentration values and n t nc r corresponding experimental measurement values are defined. Together with a vector-valued
model function
h(p, t, c) = (h1 (p, t, c), . . . , hr (p, t, c))T ,
we get the above least squares formulation by
k
k
(hk (p, ti , cj ) yij
),
fs (p) = wij

(5)

where s runs from 1 to l = n t nc r in any order. Moreover we assume that

k
there are suitable weighting factors w ij
> 0 given by the user, that are to
reflect the individual influence of a measurement on the whole experiment.
The underlying idea is to minimize the distance between the model function at certain time and concentration points and the corresponding measurement values. This distance is denoted the residual of the problem. In the ideal
case the residuals are zero indicating a perfect fit of the model function by the
measurements.
Since we assume that h(p, t, c) does not depend on an additional dynamic
system function, we call it an explicit model function. Otherwise h(p, t, c)
may depend in addition on the solution vector y(p, t, c) of an auxiliary problem, e.g. an ordinary differential equation, that is implicitly defined. Models
of this kind are considered in the subsequent sections.

2.2 Dynamic Systems of Equations

Our special goal is to estimate parameters in systems of nonlinear equations,
which may depend also on the time variable. Typical applications are for
example steady-state system as obtained by neglecting the transient terms
of an ordinary differential equation. To simplify the notation we omit the
concentration variable c. In this case the model function h(p, t) depends in
addition on the solution vector of a system of nonlinear equations, i.e.
h(p, t) = h(p, z(p, t), t) ,

(6)
79

where z(p, t) IRm is implicitly defined by the solution z of the system

s1 (p, z, t)

= 0 ,

sm (p, z, t) = 0 .

(7)

The system functions are assumed to be continuously differentiable with

respect to variables p and z. Moreover we require the regularity of the system,
i.e. that the system is solvable and that the derivative matrix


si (p, z, t)
zj
has full rank for all p with p l p pu and for all z, for which a solution
z(p, t) exists. Consequently the function z(p, t) is differentiable with respect
to all p in the feasible domain.
Now let t be fixed and let z(p, t) be a solution of the system of equations.
If we denote S(p, z) := (s 1 (p, z, t), . . . , sm (p, z, t))T for all p and z, we
get from the identity S(p, z(p, t)) = 0, which is to be satisfied for all p, the
derivative
p S(p, z(p, t)) + z S(p, z(p, t))z(p, t) = 0 .
Here p S(p, z) and z S(p, z) denote the partial derivative matrices of
S(p, z) with respect to the parameters p and z, respectively. In other words,
the desired Jacobian matrix z(p, t) is obtained from the linear system
p S(p, z(p, t)) + z S(p, z(p, t))V = 0 ,

(8)

where V is a m n-matrix. Note that we describe here the implicit function

theorem. Since z S(p, z(p)) is nonsingular, the above system is uniquely
solvable.

2.3 Laplace Models

In many practical applications, the model is available in form of a Laplace
formulation to simplify the underlying analysis. The numerical algorithms
described in this paper, are able to proceed directly from the Laplace transform and to compute its inverse internally by a quadrature formula.
The advantage of a Laplace formulation is that the numerical complexity of nonlinear systems can be reduced to a lower level. Linear differential
equations, e.g., can be transformed into algebraic equations, and linear partial
differential equations with constant coefficients can be reduced to ordinary
differential equations. The simplified systems are often solvable by analytical considerations.
80

If the model function is given in form of a Laplace transform, say

H(p, s, c), the inversion is performed numerically by a method proposed
by Stehfest [42]. Proceeding from coefficients which can be evaluated before
starting the parameter estimation algorithm, we compute the expression
2q

h(p, t, c) =

i ln 2
ln 2 
, c) .
vi H(p,
t i=1
t

(9)

h(p, t, c) is the numerical inverse Laplace transform of H(p, s, c) subject to

an accuracy given by the number q, and defines our fitting criterion in this
situation. The function may depend in addition on a concentration variable
c. It is recommended to use q = 5 or q = 6. Any smaller value decreases
the required accuracy, any larger value introduces additional round-off errors.
For the practical models we have in mind, the numerical instabilities induced
e.g. by oscillating function values, do not appear.
Note that another advantage of the above formula is, that we get easily
the gradient of the fitting function w.r.t. the parameters to be estimated, if
derivatives of the Laplace transform p H(p, s, c) are available.

2.4 Systems of Ordinary Differential Equations

We proceed now from r measurement sets, given in the form (4) and suppose
that the fitting criteria depend in addition on the solution y(p, t, c) of a system
of ordinary differential equations, i.e.
h(p, t, c) = h(p, y(p, t, c), t, c) ,

(10)

where y(p, t, c) is implicitly defined by a system of s ordinary differential

equations
y 1 =
y s =

F1 (p, y, t, c) , y1 (0) = y10 (p, c) ,

Fs (p, y, t, c) , ys (0) = ys0 (p, c) .

(11)

Without loss of generality we assume that the initial time is zero. The
initial values of the differential equation system y 10 (p, c), . . ., ys0 (p, c) may
depend on one or more of the system parameters to be estimated, and on the
concentration parameter c.
There are many practical situations where a model changes during integration over the time variable, and where initial values are to be adopted.
A typical example is a pharmacokinetic application with an initial infusion
and subsequent applications of drug doses by injection. In a formal way we
describe the model changes by
81

y 10 =
y s0 =

F10 (p, y 0 , t, c) , y10 (0) = y 01 (p, c) ,

Fs0 (p, y 0 , t, c) , ys0 (0) = y 0s (p, c) ,

(12)

for 0 t 1 and
y 1i = F1i (p, y i , t, c) , y1i (i ) = y i1 (p, c, y1i1 (p, i , c)) ,

y si = Fsi (p, y i , t, c) , ysi (i ) = y is (p, c, ysi1 (p, i , c)) ,

(13)

for i t i+1 , i = 1, . . . , nb . nb is the number of break or switching

points i with 0 < 1 < . . . < nb < T , where nb +1 := T is the last experimental time. The initial values of each subsystem are given by functions
y ij (p, c, y) depending on the parameters to be estimated, the actual concentration value, and the solution of the previous interval at the break point i .
Internally the integration of the differential equation is restarted at a break
point. It is possible that break points become variables to be adapted during
the optimization process, if they are not known in advance.
Constraints of the form
gj (p)
gj (p)

= 0 , j = 1, ..., me ,
0 , j = me + 1, ..., m

(14)

are allowed, where the restriction functions may depend on the solution of the
dynamic system at predetermined time and concentration values, i.e.
gj (p) = gj (p, y(p, tj , cj ), tj , cj )

(15)

for j = 1, ..., m. The m predetermined time and, if available, concentration

values must coincide with some of the given measurement values. If not, the
given data are rounded to the nearest experimental value. If constraints are to
be defined independently from given measurement data, it is recommended to
insert dummy experimental values with zero weights at the desired time and
concentration points t j and cj , respectively.
In some cases it is possible that multiple shooting reduces the number of
iterations of the least squares algorithm drastically, in particular if measurements are available for all state variables. So-called shooting points are introduced which may coincide with the experimental time values. Then the integration is performed successively from one shooting point to the subsequent
one, cf. Ascher and Petzold [2]. To guarantee that a continuous numerical solution of the ODE is obtained, we have to add initial values for each shooting
interval to the parameters to be optimized, and nonlinear equality constraints
obtained by the requirement that a solution variable at the end of a shooting
82

interval must coincide with the initial value of the subsequent one. A special
advantage is that starting values needed for the optimization algorithm are
easily obtained from the measurement values, see Bock [7].

2.5 Systems of Differential Algebraic Equations

Parameter estimation problems based on differential algebraic equations, are
very similar to those based on ordinary differential equations. The only difference is that we allow additional algebraic equations in the system, i.e. we
get additional system variables
y 1 =

F1 (p, y, z, t, c)

y s1 = Fs1 (p, y, z, t, c)
0=
G1 (p, y, z, t, c)

0=
Gs2 (p, y, z, t, c)

, y1 (0) = y10 (p, c) ,

, ys1 (0) = ys01 (p, c) ,
, z1 (0) = z10 (p, c) ,

(16)

, zs2 (0) = zs02 (p, c) .

Now y(p, t, c) and z(p, t, c) are solution vectors of a joint system of s 1 +s2
differential and algebraic equations (DAE). The system is called an index-1problem or an index-1-DAE, if the algebraic equations can be solved w.r.t. z,
i.e. if the matrix
z G(p, y, z, t, c)

(17)

possesses full rank. In all other cases we get DAEs with a higher index, see
e.g. Hairer and Wanner [18] for a suitable definition and more details.
Similar to systems of ordinary differential equations, it is possible to define
switching or break points, and to modify initial values at these points, that
may depend on the parameters to be estimated, the concentration variable
and the solution w.r.t. the previous interval.
For simplicity we consider now only problems of index one, although the
numerical algorithms are capable to treat also higher index models. Moreover
problems with higher index can be transformed to problems of index one by
successive differentiation of the algebraic equations.
We have to be very careful when defining the inital values of the model,
since they must satisfy the consistency equation
G1 (p, y 0 (p, c), z 0 (p, c), t, c) = 0 ,
...
Gs2 (p, y 0 (p, c), z 0 (p, c), t, c) = 0 .

(18)

Here the intial values y 10 (p, c), . . . , ys01 (p, c) and z10 (p, c), . . . , zs02 (p, c) are
functions depending on the parameters to be estimated, and the concentration
variable as declared by the user.
83

Parameter estimation problems based on DAE models, can be solved by

the program MODFIT. The code checks whether the consistency condition
is satisfied or not when starting an integration cycle. In the latter case consistent initial values are computed by solving the above nonlinear system of
equations w.r.t. z, if the intial values for the differential equations are inserted. Also if break points exist, consistent initial values are evaluated before
restarting the integration procedure.
Again we have to assume that the observation times are strictly increasing,
and get a fitting criterion of the form
h(p, t, c) = h(p, y(p, t, c), z(p, t, c), t, c) .
As before, constraints of the form
gj (p)
gj (p)

= 0, j = 1, ..., me ,
0, j = me + 1, ..., m

(19)

are allowed, where the restriction functions may depend on the solution of the
dynamic system at predetermined time and concentration values, i.e.
gj (p) = g j (p, y(p, tj , cj ), z(p, tj , cj ), tj , cj )

(20)

for j = 1, ..., m.
As in case of ODEs, also the shooting technique and internal numerical
evaluation of gradients can be applied.

2.6 Systems of One-Dimensional Time-Dependent Partial Differential Equations

Partial differential equations are extensions of ordinary differential equations,
where an additional space or spatial variable is introduced together with its
first and optionally also some higher partial derivatives. Again we assume
without loss of generality, that the initial time is zero. This assumption facilitates the description of the mathematical model and is easily satisfied in
practice by a suitable linear transformation of the time variable.
The model we want to investigate now, is defined by a system of n p onedimensional partial differential equations in one or more spatial intervals, see
also Schittkowski [36]. These intervals that could describe e.g. certain areas
with different diffusion coefficients, are given by the outer boundary values
xL and xR that define the total integration interval w.r.t. the space variable x,
and optionally some additional internal transition points x a1 , . . . , xana 1 . Thus
we get a sequence of n a + 1 boundary and transition points
xa0 := xL < xa1 < . . . < xana 1 < xana := xR .
84

(21)

For each integration interval, we define a one-dimensional, time-dependent

partial differential equation of the form
uit = F i (x, t, f i (x, t, v, ui , uix ), fxi (x, t, v, ui , uix ), v, ui , uix , uixx , p) , (22)
where x IR is the spatial variable with x ai1 x xai for i = 1, . . . , na ,
t IR the time variable with 0 < t T , v IR no the solution vector of the
coupled system of ordinary differential equations, u i IRnp the system variable we want to compute, and p IR n the parameter vector to be identified
by the data fitting algorithm.
Optionally the right-hand side may depend also on a so-called flux function
f i (x, t, v, ui , uix ) which is introduced either to facilitate the declaration of the
model functions or for being able to apply special upwind formulae in case
of hyperbolic equations, when usual approximation schemes break down e.g.
when shocks propagate over the integration interval. In this case only scalar
equations can be treated and the underlying equation is of the form
uit = (f i (x, t, ui , p))x , i = 1, . . . na .

(23)

A solution of the coupled system depends on the spatial variable x, the

time variable t, the parameter vector p, and is therefore written in the form
v(t, p) and ui (x, t, p) for i = 1, . . . , na .
For both end points x L and xR we allow boundary conditions of the form
u1 (xL , t, p)

uL (t, p) ,

una (xR , t, p)

uR (t, p) ,

u1x (xL , t, p)

u
L (t, p) ,

unx a (xR , t, p)

u
R (t, p)

(24)

for 0 < t ts . It is very essential to understand, that we do not require the

evaluation of all boundary functions. Instead a user may omit some of them
depending on the structure of the PDE model. Note that boundary information is also contained in coupled ordinary differential equations introduced
subsequently.
Transition conditions between the different areas may be defined in addition. They are allowed at most at a transition point x ai and have the form
ui (xai , t, p)

i+1 a
cR
(xi , t, p), p) ,
i (t, u

ui+1 (xai , t, p)

i a
cL
i (t, u (xi , t, p), p) ,

uix (xai , t, p)

i+1 a
a
cR
(xi , t, p), ui+1
x (xi , t, p), p) ,
i (t, u

a
ui+1
x (xi , t, p)

i a
i
a
cL
i (t, u (xi , t, p), ux (xi , t, p), p)

(25)

85

with 0 < t ts , i = 1, . . . , na 1. The transition conditions are required

to obtain a unique solution, and depend on the underlying model structure
similar to boundary conditions.
Since the starting time is assumed to be zero, initial conditions must have
the form
(26)
ui (x, 0, p) = ui0 (x, p) , i = 1, . . . , na

 a
and are defined for all x xi1 , xai , i = 1, . . . , na .
If the partial differential equations are to be coupled to ordinary differential
equations, we proceed from an additional ODE-system of the form
j
(xj , t, p), p)
v j = Gj (t, v, uij (xj , t, p), uixj (xj , t, p), uixx

(27)

for j = 1, . . . , no with initial values

v(0, p) = v0 (p) ,

(28)

that may depend again on the parameters to be estimated. The system has
no components, i.e. v = (v 1 , . . . , vno )T . Coupling of ordinary differential
equations is allowed at arbitrary points within the integration interval and the
corresponding area is denoted by the index i j . The spatial variable value
xj belongs to the j-th area, i.e. x j [xaij 1 , xaij ) or xj [xana 1 , xana ],
respectively, j = 1, . . . , no , and is called coupling point.
Coupling points are rounded to their nearest line when discretizing the
system. The right-hand side of the coupling equation may depend on the corresponding solution of the partial differential equation and its first and second
derivative w.r.t. the space variable at the coupling point under consideration.
Similar to systems of ordinary differential equations, it is possible to define
break points and modified initial values at these points, that may depend on
the parameters to be estimated, the spatial variable and the solution w.r.t. the
previous interval.
To indicate that the fitting criteria h k (p, t) depend also on the solution of
the differential equation at the corresponding fitting point, where k denotes
the index of a measurement set, we use the notation
k
(xk , t, p))
hk (p, t) = hk (p, t, v(t, p), uik (xk , t, p), uixk (xk , t, p), uixx

(29)

and insert hk instead of hk into the data fitting function. Again the fitting criteria may depend on solution values at a given spatial variable value w.r.t. to
an integration interval defined by the index
variable


 i k . The spatial
 xk belongs
to the ik -th integration area, i.e. x k xaik 1 , xaik or xk xana 1 , xana ,
respectively, k = 1, . . . , r, where r denotes the total number of measurement
sets. The fitting criterion may depend on the solution of the partial differential equation and its first and second derivative w.r.t. the space variable at the
86

fitting point. Fitting points are rounded to their nearest line when discretizing
the system.
As for ordinary differential equations, dynamic constraints of the form
gj (p) = 0 , j = 1, ..., me ,
gj (p) 0 , j = me + 1, ..., m

(30)

are allowed, where the restriction functions may depend on the solution of the
partial differential equation and its first and second spatial derivatives at predetermined time and spatial values, and the solution of the coupled ordinary
differential equation at predetermined time values, i.e.
k
(xj , tj , p))
gj (p) = g j (xj , tj , p, v(tj , p), ujk (xj , tj , p), ujxk (xj , tj , p), ujxx
(31)
for j = 1, ..., m. Here u jk denotes the corresponding integration area that
contains the spatial parameter x j . The m predetermined time values must
coincide with some of the given measurement values. If not, the given data are
rounded to the nearest experimental data. The corresponding spatial values
must coincide with a line obtained by the equidistant discretization of the
underlying integration interval.
In order to achieve smooth fitting criteria and constraints, we assume that
all model functions depend continuously differentiable on the parameter vector p. Moreover we assume that the discretized system of differential equations is uniquely solvable for all p with p l p pu . A collection of 20
examples of partial differential equations that can be solved by the presented
approach, and comparative numerical results are found in Schittkowski [36].

2.7 Systems of One-Dimensional Partial Differential Algebraic

Equations
PDAEs are based on the same model structure as one-dimensional, timedependent partial differential equations. The only difference is that additional
algebraic equations are permitted as in case of DAEs. Typical examples are
higher order partial differential equations, e.g.,
ut = f (x, t, u, uxx, uxxxx, p) ,
transformed into a second order equation by introducing additional variables,
e.g.,
ut = f (x, t, u, w, wxx , p) ,
0 = w uxx ,
or distributed systems of the form
u1t
u2x

=
=

f1 (x, t, u1 , u2 , p) ,
f2 (x, t, u1 , u2 , p) ,
87

with initial values u1 (x, 0, p) = u1 (x, p) , u2 (0, t, p) = u2 (t, p), transformed

into the PDAE
u1t = f1 (x, t, u1 , u2 , p) ,
0
= u2x f2 (x, t, u1 , u2 , p)
or
u2x = f2 (x, t, u1 , u2 , p) ,
0
= u1t f1 (x, t, u1 , u2 , p) ,
respectively.
Thus we proceed again from a sequence of n a + 1 boundary and transition
points
(32)
xa0 := xL < xa1 < . . . < xana 1 < xana := xR
and consider the PDAE-system
uit
0

= F1i (x, t, f i (x, t, v, ui , uix ), fxi (x, t, v, ui , uix ), v, ui , uix , uixx , p) ,

= F2i (x, t, f i (x, t, v, ui , uix ), fxi (x, t, v, ui , uix ), v, ui , uix , uixx , p) ,
(33)
i = 1, . . . na , where x IR is the spatial variable with x ai1 x xai for
i = 1, . . . , na , t IR the time variable with 0 < t T , v IR no the solution
vector of the coupled system of ordinary differential equations, u i IRnp the
system variable we want to compute, and p IR n the parameter vector to be
identified by the data fitting algorithm.
But now the state variables u i are divided into so-called differential variables and algebraic variables, i.e. u i = (ui1 , ui2 )T , where the number of
algebraic variables is identical to the number of algebraic equations summarized by the vector F 2 . Also we allow flux functions, switching points,
constraints for parameters and state functions, and coupled ordinary differential equations. All these extensions are treated in the same way as for partial
differential equations without algebraic equations.
However we must handle initial and boundary conditions with more care.
We have to guarantee, that at least the boundary and transition conditions
satisfy the algebraic equations
0

= F2i (xai , t, f i (xai , t, v, ui , uix ), fxi (xai , t, v, ui , uix ), v, ui , uix , uixx, p)

(34)

for i = 1, . . . na . If initial conditions for discretized algebraic equations

are violated, then the corresponding system of nonlinear equations is solved
internally proceeding from initial values given. In other words, consistent initial values are computed automatically, where the given data serve as starting
parameters for the nonlinear programming algorithm applied.
Consequently we allow only index-1-systems unless it is guaranteed, that
consistent initial values for the discretized DAE are available. Jumps or discontinuities at initial values of algebraic equations do not make sense.
88

Numerical Algorithms

EASY-FIT serves as a user interface for the parameter estimation programs

MODFIT and PDEFIT, that are also executable independently from the user
interface of EASY-FIT. One of its features is the automatic generation of input
files in ASCII format for the codes mentioned above. Model functions are
either defined symbolically to be executed by the automatic differentiation
tool PCOMP, or must be given in form of Fortran codes.
The corresponding data and code organization is outlined in Section 5. In
this section we describe very briefly the underlying numerical algorithms that
are implemented.

3.1 Data Fitting Algorithms

The parameter estimation programs contain interfaces for four different least
squares algorithms, where only one code is also capable to solve L 1 - and L problems, i.e. problems, where the sum of absolute residual or the maximum
of absolute residual values is to be minimized.
DFNLP: By transforming the original problem into a general nonlinear programming problem in a special way, typical features of a Gauss-Newton and
quasi-Newton least squares method are retained, see Schittkowski [34] for
details. In case of minimizing a sum of absolute function values or the maximum of absolute function values, the problem is transformed into a smooth
nonlinear programming problem by introducing additional variables and constraints. The resulting optimization problem is solved by a standard sequential quadratic programming code called NLPQL, cf. Schittkowski [33].
DN2GB: The subroutine is a frequently used unconstrained least squares algorithm and was developed by Dennis, Gay and Welsch [12]. The mathematical method is also based on a combined Gauss-Newton and quasi-Newton
approach.
DSLMDF: First successive line searches are performed along the unit vectors
by comparing function values only. The one-dimensional minimization is
based on a quadratic interpolation. After the search cycle the Gauss-Newtontype method DFNLP is executed with a given number of iterations. If a solution is not obtained with sufficient accuracy, the search cycle is repeated, see
Nickel [27] for details.
NELDER: This is a very simple search method developed by Nelder and
Mead [26]. Successively edges of a simplex are exchanged by comparing
function values only. The edge with highest function value is replaced by another one generated by a move through the center of the actual simplex to the
other side.
89

Fig. 5: EASY-FIT: Optimization Parameters and Tolerances

Fig. 6: EASY-FIT: Simulation Plot

90

Note that only DFNLP is able to take linear or nonlinear constraints into
account. However all other algorithms satisfy upper and lower bounds of the
parameters to be estimated. The algorithms are also capable to solve problems
with large residuals.
The choice of algorithm NELDER is only useful when the other algorithms
fail because of non-differentiable model functions, a very bad starting point
or large round-off errors in the function evaluation.

3.2 Solving Systems of Dynamic Equations

The program MODFIT is executed to solve parameter estimation problems
based on dynamic systems of equations or steady-state systems, respectively.
To solve the corresponding systems of nonlinear equations, they are treated
as general nonlinear programming problems and solved by the Fortran code
NLPQL, see Schittkowski [33]. Objective function is the sum of squares of
the system parameters, and the constraints are identical to the nonlinear system of equations given. The algorithm proceeds from a successive quadratic
approximation of the Lagrangian function and linearization of constraints. To
get a search direction, a quadratic programming problem must be solved in
each iteration step, a subsequent line search stabilizes the algorithm.
Also the initial values required for the start of an optimization cycle, must
be predetermined by the user in a suitable way. They may depend on the parameters of the outer optimization problem. The system of nonlinear equations
must be solved for each experimental time value. Moreover the gradients of
the model function h(p, z(p, t), t) are calculated analytically by the implicit
function theorem. In this case a system of linear equations must be solved for
each time value by numerically stable Householder transformations.
As mentioned above, the parameter estimation algorithm DFNLP does also
require the execution of subroutine NLPQL for solving the modified nonlinear programming problem. To avoid execution conflicts, DFNLP uses reverse
communication for evaluating function and gradient values.

3.3 Numerical Laplace-Back-Transformation

The executable program MODFIT is also executed to solve parameter estimation problems, where the model functions are defined in the Laplace space.
In this case, constraints are not allowed.
Since we suppose that analytical back-transformation is not available, we
have to apply a numerical quadrature formula, see Bellman et al. [3] for details. In our case we use the quadrature formula of Stehfest [42]. Proceeding
from a given Laplace transform H(p, s, c) of an unknown model function
h(p, t, c) subject to the parameter vector p to be optimized, we compute the
91

coefficients
min(i,q)

vi := (1)q+i

k=(i+1)/2

k q+1 (2k)!
,
(q k)!k!(k 1)!(i k)!(2k i)!

(35)

which are independent of H and which can be evaluated before starting the
main procedure, where function values are to be computed. Then
2q

h(p, t, c) =

ln 2 
i ln 2
, c)
vi H(p,
t i=1
t

(36)

is a numerical approximation formula for h(p, t, c).

The parameter for controlling the accuracy is the number q. When working
with double precision arithmetic, it is recommended to use q = 5 or q = 6.
Any smaller value decreases the required accuracy, any larger value introduces additional round-off errors. For the practical models we have in mind,
the numerical instabilities induced e.g. by oscillating function values, do not
appear.
A particular advantage of the formula is, that the derivative of h(p, t, c)
w.r.t. the parameters to be estimated, i.e. p, are easily obtained from the
derivatives of H(p, s, c).

3.4 Numerical Algorithms for Ordinary Differential Equations

Within MODFIT it is possible to select among seven different solvers:
DOPRI5

DOP853

ODEX
RADAU5
SDIRK4
SEULEX
IND-DIR

92

Explicit Runge-Kutta method of order 4/5 based on Dormand

and Prince [30] formula, cf. Hairer, Nrsett and Wanner [19].
Steplength is adapted internally.
Explicit Runge-Kutta method of order 8 based on Dormand
and Prince [30] formula, cf. Hairer, Nrsett and Wanner [19].
Local error estimation and step size control is based on embedded formulae of order 5 and 3.
Extrapolation method based on GBS algorithm with variable
order and stepsize, see Hairer, Nrsett and Wanner [19].
Implicit Radau-type Runge-Kutta method of order 5 for stiff
equations, cf. Hairer and Wanner [18].
Diagonally implicit Runge-Kutta method with 5 stages, cf.
Hairer and Wanner [18].
Extrapolation algorithm based on linearly implicit Euler
method, cf. Hairer and Wanner [18].
Runge-Kutta-Fehlberg method of order 4 to 5, cf. Shampine
and Watts [39] with additional sensitivity analysis implemented by Benecke [4].

Note that the first six codes use dense output, i.e. the integration is performed over the whole interval given by first and last time value, and intermediate solution values are interpolated. In these cases gradients w.r.t. parameters to be estimated, are obtained by external numerical differentiation.
The codes RADAU5, SDIRK4 and SEULEX are able to solve also stiff equations.
The last algorithm IND-DIR is capable to evaluate derivatives of the solution of the ODE internally w.r.t. the parameters to be estimated, i.e. by
analytical differentiation of the Runge-Kutta scheme.
It is possible that the right-hand side of an ODE is non-continuous w.r.t.
integration time, e.g. if non-continuous input functions exist. Especially in
case of short peaks, the integration routine might not realize the peak at all
because of a big time step. Moreover the numerical approximation of gradients could become unstable in case of discontinuities. Thus MODFIT allows
to supply an optional number of time values, so-called break points, where
the integration of the ODE is restarted with initial tolerances, e.g. with the
initially given stepsize. The integration in the preceeding interval is stopped
at the time value given minus a relative error in the order of the machine
precision.
Restarts of the integration routine are needed only for the codes DOPRI5,..., SEULEX because of their dense output facilities. In case of
IND-DIR, the integration is restarted at each experimental time value. By
introducing artificial time values with zero weight, additional restarts at discontinuities can be simulated easily. Note also that break points can be treated
as optimization variables, i.e. may vary from one iteration step to the other.

3.5 Numerical Algorithms for Differential Algebraic Equations

Algebraic differential equations are solved by any of the following implicit
methods:
RADAU5
SDIRK4
SEULEX

Implicit Radau-type Runge-Kutta method of order 5 for stiff

equations, cf. Hairer and Wanner [18].
Diagonally implicit Runge-Kutta method with 5 stages, cf.
Hairer and Wanner [18].
Extrapolation algorithm based on linearly implicit Euler
method, cf. Hairer and Wanner [18].

DAEs with an index higher than one can be solved only by RADAU5. If
consistent initial values cannot be provided by the user, the corresponding
nonlinear system of equations is treated as general nonlinear programming
problem with equality constraints. A minimum norm solution is computed
93

by the sequential quadratic programming method NLPQL of Schittkowski

[33]. The initial values given for the algebraic variables are used as starting
values for NLPQL.
For reasons outlined in the previous section, it is possible that the righthand side of a DAE becomes non-continuous w.r.t. integration time. Thus it
is possible to supply an optional number of time values, where the integration
of the DAE is restarted.

3.6 Numerical Algorithms for Partial Differential Equations

The underlying idea is to transform the partial differential into a system of
ordinary differential equations by discretizing the model functions w.r.t. the
spatial variable x. This approach is known as the method of lines, see e.g.
Schiesser [32].
For the i-th integration interval of the spatial variable, we denote the number of discretization points by n i , i = 1, . . . , na . We proceed from uniform
grid points within each interval and get a discretization of the whole space
interval from xL to xR . To approximate the first and second partial derivatives of u(x, t, p) w.r.t. the spatial variable at a given point x, several different
alternatives have been implemented in PDEFIT, see also Schittkowski [38]:
a) Polynomial approximation: We compute an interpolating polynomial subject to some neighbouring values. The number of interpolation points depends on the polynomial degree selected. Polynomials of order 3, 5, 7 or
9 are computed from Newtons formula. By differentiating the interpolation
formulae, first and second order approximations are obtained. In case of Neumann boundary conditions, Hermite interpolation is applied for being able to
exploit known derivative values.
b) Difference Formulae: First and second derivatives can be approximated
by difference formulae, see e.g. Schiesser [32]. Difference formulae with 3
and 5 points for first derivatives are available, that can be applied recursively
to get also the second derivatives. Alternatively a 5-point difference formula
for second derivatives is implemented as well. The difference formulae are
adapted at the boundary to accept given function and gradient values.
c) Upwind Formulae for Hyperbolic Equations: In case of one hyperbolic
equation, i.e.
uit = (f i (x, t, ui , p))x

(37)

with a so-called flux function f , approximation by polynomials or difference

formulae might become unstable especially if non-continuous boundary conditions are supplied to describe e.g. the propagation of shocks, see again
Schiesser [32] for some numerical examples. The following update formulae
are available for solving hyperbolic equations:
94

-simple upwind formula

-TVD with upwind factor -1/3
-Fromms TVD-scheme
-third order upwind-biased TVD-scheme
-second order TVD-scheme
-central difference TVD-scheme
-first order high resolution TVD-upwind
-second order central-difference high resolution
For more information, see the original literature, e.g. Yee [50], Chakravarthy
and Osher [9], [10], [11], Sweby [44], Wang and Richards [47], and Yang and
Przekwas [49]. TVD stands for total variation diminishing and the correponding one parameter family of upwind formulae was proposed by Chakravarthy
and Osher [9]. In this case a certain stability criterion requires that the internal time stepsizes of the ODE-solver do not become too small compared to
the spatial discretization accuracy. Because of the black box approach used,
the stepsizes however cannot be modified and we have to suppose that the
criterion remains satisfied.
Whenever a boundary condition in Dirichlet-form is given, cf. (26) and
(27), then we know the value of the boundary function and use it to interpolate
or approximate the function u(x, t, p) as decribed above. In other words the
corresponding function value in the right-hand side of the discretized system
is replaced by the value given. Alternatively a boundary condition may appear
in Neumann-form, see (26) or (27). In this case the derivative values at the
boundary are replaced by the given ones before evaluating the second order
spatial derivative approximations.
Ordinary differential equations are added to the discretized system without
any further modification. Since arbitrary coupling points are allowed, they
are rounded to the nearest line of the discretized system. In the same way
fitting criteria can be defined at arbitrary values of the spatial variable.
When defining the transition function, it is important to have the underlying flux direction in mind. If, for example, the flux is in the direction
of the spatial variable and we want to define a continuous transition at x ai ,
then we have to formulate the corresponding transition function in the form
a
i
a
ui+1
k (xi , t, p) = uk (xi , t, p) in order to guarantee, that the boundary values
w.r.t. xL are spread over the interval.
For the same reasons outlined in the previous sections, it is possible that
the right-hand side of a PDE becomes non-continuous w.r.t. integration time.
Thus it is possible to supply an optional number of time values, where the integration of the ODE is restarted with initial tolerances, e.g. with the initially
given stepsize. The integration in the preceeding interval is stopped at the
time value given minus a relative error in the order of the machine precision.
95

Break or switching points are either constant or optimization variables to be

adapted by the optimization code.
In many application models we need to compute an integral w.r.t. the spatial variable x e.g. to evaluate a mass balance, i.e.


xa
j

xa
j1

ui (x, t, p)dx ,

where the integral is taken over the j-th area where the PDE is defined, j =
1, . . . , na , and where i = 1, . . . , n p . The integral is evaluated by Simpsons
rule and and can be retrieved from a common block or, alternatively, through
a special construct of the PCOMP language.

3.7 Numerical Algorithms for Partial Differential Algebraic

Equations
The basic idea is now to transform the partial differential into a system of
differential algebraic equations by discretizing the model functions w.r.t. the
spatial variable x. It is assumed that always the last n ae equations of the n p
given ones are algebraic.
Again we denote the number of discretization points by n i , i = 1, . . . , na ,
for the i-th integration interval of the spatial variable. We proceed from uniform grid points within each interval and get a discretization of the whole
space interval from x L to xR . To approximate the first and second partial
derivatives of u(x, t, p) w.r.t. the spatial variable at a given point x, we may
apply polynomial approximation or difference formulae as outlined in the
previous section. Thus we get a system of differential algebraic equations,
that can be solved then by any of the available integration routines.
Boundary conditions have to satisfy the algebraic equations. Consistent
initial values are computed within the code PDEFIT, where the given data
serve as starting parameters for the nonlinear programming algorithm applied. Consequently we allow only index-1-systems unless it is guaranteed,
that consistent initial values for the discretized DAE are available. Also any
jumps or discontinuities at initial values of algebraic equations do not make
any sense.

3.8 Statistical Error Analysis

Proceeding from the assumption that the model is linearized in a neighbourhood of an optimal solution and that all experimental data are Gaussian and
independent, some statistical data can be evaluated:
96

Variance/covariance matrix of the problem data

Correlation matrix of the problem data
Estimated variance of residuals
Confidence intervals for the individual parameters subject to the significance levels 1%, 5% or 10%, respectively.

Case Study

4.1 Application Problems

The authors database of EASY-FIT contains more than 500 real life and academic data fitting problems based on about 400 different mathematical models. In some cases it is also possible to transform an optimal control problem
to a parameter estimation problem. A few practical applications solved by
EASY-FIT and the underlying numerical routines are:
1. Substrate diffusion in a metabolically active cutaneous tissue, see Schittkowski [37], Boderke, Schittkowski, Wolf, Merkle [8], Steingsser [43],
or Blatt, Schittkowski [6] for an optimal control model
2. Acetylene reactor, see Wansbrough [48] or Birk, Liepelt, Schittkowski,
Vogel [5] for an optimal control approach
3. Drying of maltodextrin in a convection oven, see Frias, Oliveira, Schittkowski [15]
4. Receptor-ligand binding study, see Rominger, Albert [31] or Schittkowski [35]
5. Multibody system of a truck, see Simeon, Grupp, Fhrer, Rentrop [41],
Simeon, Rentrop [40] or Fhrer, Leimkuhler [16]
6. Manutec r3 robot, see Otter, Trk [29]
7. Mass transfer in sorbing porous media, see Hoch [22], Van Genuchten,
Wierenga [45], or Andersson, Olsson [1]
8. Ammonium and nitrate fertilization in forest soils, see Fischer [14]
9. Distillation column, see Jourdan [23], or Kuhn, Schmidt [24]
10. Cooling during rolling-mill operations, see Hedrich [21] or Groch [17]
11. Signal evaluation in periodic mechanical systems, see Zschieschang and
Rockhausen [51]
12. Design of horn radiators for satellite communication by least squares optimization, see Hartwanger and Schittkowski [20]
13. Isomerization in protein unfolding and refolding, see Odefey, Mayr,
Schmid [28], or Mayr, Odefey, Schutkowski, Schmid [25]
14. Thermodynamic coupling for GroEL-mediated unfolding, see Walter,
Lorimer, Schmid [46]
97

4.2 Groundwater Flow

The mathematical model to be introduced in form of a case study, describes a
tracer experiment that was conducted at the lake Grdsjn in Sweden, see Andersson and Olsson [1], to investigate acidification of groundwater pollution.
To perform an experiment, a catchment of 1.000 m 2 was covered by a roof. A
tracer impulse consisting of Lithium-Bromide was applied with steady-state
flow conditions. Tensiometer measurements of the tracer concentration are
documented in a distance of 40 m from the center of the covered area.
The diffusion equations proposed by Van Genuchten and Wierenga [45]
were chosen by Hoch [22] to analyze the diffusion process and to get a simulation model. A two-domain approach was selected in form of two equations,
to model the mobile and the immobile part of the system. The first one describes the diffusion of the flow through the soil by convection and dispersion,
the second one the diffusion orthogonal to the flow direction.
m

cm
cim
2 cm
cm
(x, t) + im
(x, t) = m Dm
(x, t)
(x, t) m Vm
2
t
t
x
x
cim
(x, t) = (cm (x, t) cim (x, t))
im
t

for 0 < t 2.55 and 0 < x < L = 80 with boundary conditions

Dm cm
5800 , if t < t0 ,
(0, t) =
cm (0, t)
0,
otherwise,
Vm x
cm (L, t) +

Dm cm
(L, t) = 0
Vm x

for 0 t 2.55. Initial conditions are c m (x, 0) = 0 and cim (x, 0) = 0 for
0 < x < L.
Here we are interested in fitting the function
h(p, t) = cm ( 12 L, t)

Dm cm 1
( L, t)
Vm x 2

defined for 0 < t 2.55. We let t 0 = 0.01042, im = 1 m , and

Vm = 100.
Our intention is to give an impression how a typical application problem
can be formulated by the PCOMP interpreter language of EASY-FIT. The
order of variables and model functions is mandatory, whereas their names
can be chosen arbitrarily. In this particular case, the order of the parameters
and state variables is:
98

1. parameters to be estimated (p)

2. state variables identifying solution of partial differential equations (u)
3. variables identifying first spatial derivatives of solution of partial differential equations (u x )
4. variables identifying second spatial derivatives of solution of partial differential equations (u xx)
5. spatial variable (x)
6. time variable (t)
In a similar way the order of model function is prescribed:
1. functions defining right-hand side of system of partial differential equations (F (x, t, u, ux , uxx, p), see (22))
2. initial values of system of partial differential equations at t = 0 (u 0 (x, p),
see (26))
3. boundary derivatives w.r.t. x (
c L (t, p), cR (t, p), see (24))
4. fitting criterion (h(p, t, u, u x , uxx ), see (29))
Boundary functions are defined first at left boundaries of all areas, then at
right boundaries of all integration areas. In the present example we do not
have to take flux functions, transition functions between different integration
areas, Dirichlet boundary conditions, coupled ordinary differential equations,
or constraints into account. The model is implemented then in the PCOMP
language as follows:
*

REAL CONSTANT
t0 = 0.0104167
C0 = 5800
Vm = 100

C
C-----------------------------------------------------C
*
VARIABLE
Tm, Dm, alpha
Cim, Cm, Cim_x, Cm_x, Cim_xx, Cm_xx
x, t
C
C-----------------------------------------------------C
*
FUNCTION Cim_t
Tim = 1 - Tm
Cim_t = alpha/Tim*(Cm - Cim)
C
*
FUNCTION Cm_t
Cm_t = Dm*Cm_xx - Vm*Cm_x - alpha/Tm*(Cm - Cim)
C
C-----------------------------------------------------C
*
FUNCTION Cim0
Cim0 = 0.0
C
*
FUNCTION Cm0
Cm0 = 0.0
C
C------------------------------------------------------

99

C
*

C
*

FUNCTION Cm_x_left
IF (t.LT.t0) THEN
Cm_x_left = (Cm - C0)*Vm/Dm
ELSE
Cm_x_left = Cm*Vm/Dm
ENDIF
FUNCTION Cm_x_right
Cm_x_right = -Cm*Vm/Dm

C
C-----------------------------------------------------C
*
FUNCTION Cm_fit
Cm_fit = Cm - Dm/Vm*Cm_x
C
C-----------------------------------------------------C
*
END

Parameters to be estimated are m , , and Dm . Initial and final optimization parameters are shown in the subsequent table together with their significance intervals subject to a significance level of 5 %.
parameter

initial value

final value

m
Dm

0.5
100.0
1.0

0.347704
348.981
3.86361

confidence interval
lower
upper
0.337491 0.357818
326.485
371.476
3.39786
4.32936

25 lines are used to discretize the spatial interval. The resulting system
of 50 ordinary differential equations is solved by RADAU5 subject to an error tolerance of RELERR=ABSERR=1.0D-6. After 24 iterations the least
squares code DN2GB computes the scaled residual value 0.0067, see Figure
4.2. The corresponding surface plots for mobile and immobile part are shown
in Figure 4.2. Statistical significance intervals are quite tight at least subject
to the first two parameters.
A couple of typical screen shots of EASY-FIT for generating a new model,
starting an optimization run, and interpreting results are shown in figures 1 to
6 on pages 76, 78 and 90.

100

50

data 2
222222
2
2
fit
criterion
2
2
22 2 22222222
2
2
2222
2 22
2 222
222
22
22
2
22 2222
2 2 2222222
2
2 2 222
22222
22
2222
2222222
2

45
40

h(p, t)

35
30
25
20
15
10
5
0
0

0.5

1.5
t

2.5

Fig. 7: Final Trajectories for Groundwater Flow

Software Organization

EASY-FIT consists of a database containing models, data and results, and of

underlying numerical algorithms for solving the parameter estimation problem depending on the mathematical structure, i.e.
MODFIT

PDEFIT

parameter estimation in explicit functions, steady-state systems, Laplace transforms, ordinary differential and differential algebraic equations
parameter estimation in one-dimensional, time-dependent
partial differential algebraic equations

EASY-FIT requires a lot of system resources to run in a smooth and efficient way. Especially the run time system of the MS-Access 97 1 database
needs as much CPU memory as possible. Recommended hardware is a Pentium processor with 32 MB memory on board and at least 200 MHz. A full
installation requires about 40 MB on hard disk. The system runs under Windows 95 and Windows NT.
EASY-FIT comes with the run-time and royalty-free version of MSAccess. Plots are generated either by internal plot facilities or optionally by
the external graphic systems MS-Graph5 and GNUPLOT 2 . Numerical data
1 MS,

Windows, Microsoft Access are registered trademarks of Microsoft Corp.

by T. Williams, C. Kollin

2
1986-93
c

101

model function
cm (x, t)
100
50
0
0 10
20

30 40
50
x

60 70
80

0.5

1.5

2.5

model function
cim (x, t)
100
50
0
0 10
20

30 40
50
x

60 70
80

0.5

Fig. 8: Surface Plots for Groundwater Flow

102

1.5
t

2.5

describing problems and models are kept in a separate database with file
name EASY_DAT.MDB to allow easier maintenance and updates.
Parameter estimation problems are solvable without a Fortran compiler, if
the model functions are defined in form of the PCOMP modeling language.
In this case nonlinear model functions are interpreted and evaluated during
run time together with their derivatives. The full version of EASY-FIT allows
also the most flexible input of the underlying model in form of Fortran code,
and has an interface for the Watcom F77/386 3, the Salford FTN77 4 , the Lahey
F77L-EM/32, and the Microsoft Fortran PowerStation 5 compiler, where the
compiler and linker options can be altered and adapted to special needs.
To install EASY-FIT, one has to insert the CD-ROM and to execute the
setup program SETUP.EXE. Installation assumes that the run-time version of
MS-Access 97 is to be loaded together with EASY-FIT (typical and custom
setup).
EASY-FIT is delivered with Fortran source codes for the main numerical
codes MODFIT.FOR and PDEFIT.FOR, and for interfaces to the PCOMP interpreter in the files MODFITEX.FOR and PDEFITEX.FOR, respectively. It
is then possible to link numerical algorithms with user-provided dimensioning parameters or to solve problems where model functions are defined in
form of a Fortran code. These files together with additional object codes of
internal numerical algorithms e.g. for solving the optimization problem or
for integrating differential equations, are copied to a separate directory by the
setup program, if required by the user. If the submitted default compiler interface is to be changed e.g. from the Watcom to Salford or Lahey compiler,
one has to set the corresponding compiler name, some path names, and the
compiler and linker execution commands.
For more information about EASY-FIT and its capabilities, contact the
author under
klaus.schittkowski@uni-bayreuth.de
A demo version is available through the URL
http://www.uni-bayreuth.de/departments/math/kschittkowski/home.htm

3 WATCOM

is a registered trademark of WATCOM Systems Inc.

is a trademark of Salford Software Ltd.
5 PowerStation is a trademark of Microsoft Corporation
4 FTN77

103

References
[1] Andersson F., Olsson B. eds. (1985): Lake Gdsjn. An acid forest lake and its catchment,
Ecological Bulletins, Vol. 37, Stockholm
[2] Ascher U.M., Petzold L.R. (1998): Computer Methods for Ordinary Differential Equations and
Differential-Algebraic Equations, SIAM, Philadelphia
[3] Bellman R.E., Kalaba R.E., Lockett J. (1966): Numerical Inversion of the Laplace Transform,
American Elsevier, New York
[4] Benecke C. (1993): Interne numerische Differentiation von gewhnlichen Differentialgleichungen, Diploma Thesis, Department of Mathematics, University of Bayreuth
[5] Birk J., Liepelt M., Schittkowski K., Vogel F. (1998): Computation of optimal feed rates and
operation intervals for turbular reactors, to appear: Journal of Process Control
[6] Blatt M., Schittkowski K. (1998): Optimal Control of One-Dimensional Partial Differential
Equations Applied to Transdermal Diffusion of Substrates, in: Optimization Techniques and
Applications, L. Caccetta, K.L. Teo, P.F. Siew, Y.H. Leung, L.S. Jennings, V. Rehbock eds.,
School of Mathematics and Statistics, Curtin University of Technology, Perth, Australia,
Volume 1, 81 - 93
[7] Bock H.G. (1983): Recent advantages in parameter identification techniques for ODE, Proceedings of the International Workshop on Numerical Treatment of Inverse Problems in
Differential and Integral Equations, Birkhuser, 95-121
[8] Boderke P., Schittkowski K., Wolf M., Merkle H.P. (1998): A mathematical model for diffusion
and concurrent metabolism in metabolically active tissue, submitted for publication
[9] Chakravarthy S.R., Osher S.(1984): High resolution schemes and the entropy condition, SIAM
Journal on Numerical Analysis, Vol. 21, No. 5, 955-984
[10] Chakravarthy S.R., Osher S. (1984): Very high order accurate TVD schemes, ICASE Report
No. 84-44
[11] Chakravarthy S.R., Osher S. (1985): Computing with high resolution upwind schemes for hyperbolic equations, Lectures in Applied Mathematics, Vol. 22, 57-86
[12] Dennis J.E.jr., Gay D.M., Welsch R.E. (1981): Algorithm 573: NL2SOL-An adaptive nonlinear
least-squares algorithm, ACM Transactions on Mathematical Software, Vol. 7, No. 3, 369383
[13] Dobmann M., Liepelt M., Schittkowski K. (1995): Algorithm 746: PCOMP: A FORTRAN
code for automatic differentiation, ACM Transactions on Mathematical Software, Vol. 21,
No. 3, 233-266
[14] Fischer P. (1996): Modellierung und Simulation der Ammonium- und Nitrat-Dynamik in strukturierten Waldbden unter besonderer Bercksichtigung eines dynamischen, hierarchischen
Wurzelsystems, Diploma Thesis, Department of Mathematics, University of Bayreuth
[15] Frias J.M., Oliveira J.C, Schittkowski K. (1998): Modelling of maltodextrin DE12 drying
process in a convection oven, submitted for publication
[16] Fhrer C., Leimkuhler B. (1991): Numerical solution of differential-algebraic equations for
constrained mechanical motion, Numerische Mathematik, Vol. 59, 55-69
[17] Groch A.G. (1990): Autmatic control of laminar flow cooling in continuous and reversing hot
strip mills, Iron and Steel Engineer, 16-20
[18] Hairer E., Wanner G. (1991): Solving Ordinary Differential Equations II. Stiff and DifferentialAlgebraic Problems, Springer Series Computational Mathematics, Vol. 14, Springer
[19] Hairer E., Nrsett S.P., Wanner G. (1993): Solving Ordinary Differential Equations I: Nonstiff
Problems, Springer Series Computational Mathematics, Vol. 8, Springer
[20] Hartwanger C., Schittkowski K. (1998): Computer Aided Design of Horn Radiators for Satellite Communication by Least Squares Optimization, submitted for publication
[21] Hedrich C. (1996): Modellierung, Simulation und Parameterschtzung von Khlprozessen in
Walzstraen, Diploma Thesis, Department of Mathematics, University of Bayreuth

104

[22] Hoch R. (1995): Modellierung von Fliewegen und Verweilzeiten in einem Einzugsgebiet unter
stationren Fliebedingungen, Diplomarbeit, Fakultt fr Biologie, Chemie und Geowissenschaften, Universitt Bayreuth
[23] Jourdan, M. (1997): Simulation und Parameteridentifikation von Destillationskolonnen,
Diploma Thesis, Department of Mathematics, University of Bayreuth
[24] Kuhn U., Schmidt G. (1987): Fresh look into the design and computation of optimal output
feedback controls for linear multivariable systems, International Journal on Control, Vol.
46, No. 1, 75-95
[25] Mayr L.M., Odefey C., Schutkowski M., Schmid F.X. (1996): Kinetic analysis of the unfolding
and refolding of ribonuclease T1 by a stopped-flow double-mixing technique, Biochemistry,
Vol. 35, No. 17, 5550-5561
[26] Nelder J.A., Mead R. (1965): A simplex method for function minimization, The Computer
Journal, Vol. 7, 308
[27] Nickel B. (1995): Parameterschtzung basierend auf der Levenberg-Marquardt-Methode in
Kombination mit direkter Suche, Diploma Thesis, Department of Mathematics, University
of Bayreuth
[28] Odefey C., Mayr L.M., Schmid F.X. (1995): Non-prolyl cis-trans peptide bond isomerization
as a rate-determining step in protein unfolding and refolding, Journal of Molecular Biology,
Vol. 245, 69-78
[29] Otter M., Trk S. (1988): The DFVLR models 1 and 2 of the Manutec r3 robot, DFVLR-Mitt.
88-3, DFVLR, Oberpfaffenhofen
[30] Prince P.J., Dormand J.R. (1981): High order embedded Runge-Kutta formulae, Journal on
Computational Applied Mathematics, Vol. 7, 67-75
[31] Rominger K.L., Albert H.J. (1985): Radioimmunological determination of Fenoterol. Part I:
Theoretical fundamentals, Arzneimittel-Forschung/Drug Research, Vol.35, No.1a, 415-420
[32] Schiesser W.E. (1991): The Numerical Method of Lines, Academic Press, San Diego
[33] Schittkowski K. (1985/86): NLPQL: A FORTRAN subroutine solving constrained nonlinear
programming problems, Annals of Operations Research, Vol. 5, 485500
[34] Schittkowski K. (1988): Solving nonlinear least squares problems by a general purpose SQPmethod, in: Trends in Mathematical Optimization, K.-H. Hoffmann, J.-B. Hiriart-Urruty,
C. Lemarechal, J. Zowe eds., International Series of Numerical Mathematics, Vol. 84,
Birkhuser, 295-309
[35] Schittkowski K. (1994): Parameter estimation in systems of nonlinear equations, Numerische
Mathematik, Vol. 68, 129-142
[36] Schittkowski K. (1997): Parameter estimation in partial differential equations, Optimization
Methods and Software, Vol. 7, No. 3-4, 165-210
[37] Schittkowski K. (1998): Parameter Estimation in a Mathematical Model for Substrate Diffusion in a Metabolically Active Cutaneous Tissue, to appear: Proceedings of the Optimization
Day, Perth, Australia, June 29-30
[38] Schittkowski K. (1998): PDEFIT: A FORTRAN code for parameter estimation in partial differential equations, to appear: Optimization Methods and Software
[39] Shampine L.F., Watts H.A. (1979): The art of writing a Runge-Kutta code, Applied Mathematics and Computations, Vol. 5, 93-121
[40] Simeon B., Rentrop P. (1993): An extended descriptor form for the simulation of constrained
mechanical systems, in: Advanced Multibody System Dynamics, W. Schiehlen ed., Kluwer
Academic Publishers, p. 469-474
[41] Simeon B., Grupp F., Fhrer C., Rentrop P. (1994): A nonlinear truck model and its treatment
as a multibody system, Journal of Computational and Applied Mathematics, Vol. 50, 523532
[42] Stehfest H. (1970): Algorithm 368: Numerical inversion of Laplace transforms, Communications of the ACM, Vol. 13, 47-49
[43] Steingsser I. (1994): The organized HaCaT cell culture sheet: A model approach to study
epidermal peptide drug metabolism, Dissertation, Pharmaceutical Institute, ETH Zrich

105

[44] Sweby P.K. (1984): High resolution schemes using flux limiters for hyperbolic conservation
laws, SIAM Journal on Numerical Analysis, Vol. 21, No. 5, 995-1011
[45] Van Genuchten M.T., Wierenga P.J. (1976): Mass transfer studies in sorbing porous media. 1.
Analytical solutions, Soil Sci. Soc. Am. Journal, Vol. 44, 892-898
[46] Walter S., Lorimer G.H., Schmid F.X. (1996): A thermodynamic coupling mechanism for
GroEl-mediated unfolding, Biochemistry, Vol. 93, 9425-9430
[47] Wang Z., Richards B.E. (1991): High resolution schemes for steady flow computation, Journal
of Computational Physics, Vol. 97, 53-72
[48] Wansbrough R.W. (1985):Modeling chemical reactors, Chemical Engineering, Vol. 5
[49] Yang H.Q., Przekwas A.J. (1992): A comparative study of advanced shock-capturing schemes
applied to Burgers equation, Journal of Computational Physics, Vol. 102, 139-159
[50] Yee H.C. (1985): Construction of a class of symmetric TVD schemes, Lectures in Applied
Mathematics, Vol. 22, 381-395
[51] Zschieschang T., Rockhausen L. (1996): Zur Signalauswertung und Modellierung bei periodischen Vorgngen mit transienten Anteilen, Report, Institute of Mechanics, Technical
University of Chemnitz-Zwickau, Chemnitz, Germany

106