Century
John Gu
kenheimer
1
Complexity vs. aggregation
We then dis
uss issues of numeri
al analysis that arise in simulation.
2
not an argument that we should avoid a fo
us on simulation, but rather a plea
that we should examine the enterprise. As with large teles
opes in astron
omy, many areas of s
ien
e stand to reap enormous benets from thoughtful
planning and investment in te
hnology that will help simulation eorts.
Determinism
Is the weather predi
table? There is abundant eviden
e that global fore
ast models display sensitive dependen
e to initial
onditions. Perturbations
within the measurement error of
urrent observations lead to dierent fore
asts. It is likely that this sensitivity is not an artifa
t, but an inherent prop
erty of atmospheri
ir
ulation. Let us assume here that the atmospheri
ow
is turbulent and that disturban
es grow to ee
t global
ir
ulation. What
are the impli
ations for simulation? Dynami
al systems analysis of
haoti
systems gives insight into the answer to this question. It
laries how unpre
di
tability
an arise in a deterministi
,
lo
kwork universe. Sensitive depen
den
e to initial
onditions is a property of many dynami
al systems, from
simple nonlinear me
hani
al linkages to population models. The
haoti
na
ture of su
h systems has been studied extensively, to the point that there
are solid mathemati
al foundations for quantifying their unpredi
tabiity us
ing su
h quantities as invariant measures, entropy, Lyapunov exponents and
fra
tal dimensions. Still, the appli
ation of these
on
epts to the simulation
of
omplex systems remains problemati
.
Fa
ed with systems that display sensitive dependen
e to initial
ondi
tions, long term predi
tion of the full state of the system as a fun
tion of
time is simply impossible. Weather fore
asts ve weeks in advan
e, let alone
ve years,
an spe
ify at best average properties of the weather. Operational
weather fore
asts have begun to employ ensemble fore
asting based on simu
lations of several initial states. The usefulness of dynami
al systems methods
is problemati
for simulations with attra
tors whose dimensions are large.
Probabilisti
methods about system behavior seem more suited to these sit
uations, but assumptions about underlying statisti
al distributions are hard
to verify. For example, the statisti
al properties of turbulent
uid
ows re
lated to
oherent geometri
stru
tures remain a
ontroversial subje
t. Monte
Carlo methods embody systemati
approa
hes to the modeling of sto
hasti
systems. Produ
ing high delity simulations in
omplex systems with many
parameters using Monte Carlo methods is an even larger task than it is for
deterministi
models with stable asymptoti
states. When fa
ed with noisy,
3
unpredi
table systems like the brain, we have only a primitive understanding
of whi
h aspe
ts of dynami
al behavior of
omplex systems we
an hope to
simulate.
Modern mole
ular biology has invested great eort to determine the rea
tion
pathways of living organisms. This resear
h produ
es graphs that show how
important biologi
al mole
ules are produ
ed, modied and used to regulate
physiologi
al pro
esses. Frequently missing from this work is an understand
ing of the kineti
s of the pathways. We see information about the stru
ture
of the pathway, but we do not understand how it works. In parti
ular, we
have diÆ
ulty predi
ting the fun
tional
onsequen
es of drugs, mutations or
other modulatory ee
ts on the network. For example, the deleterious side
ee
ts of
al
ium blo
kers used as
ardia
drugs illustrate the importan
e
of understanding biologi
al kineti
s. Mole
ular biologists have assumed that
fun
tion will be evident from stru
ture. They have made the elu
idation
of stru
ture the prin
ipal goal of the subje
t. However, the
ase of neu
ral networks demonstrates that a single morphologi
al network may support
a diverse set of behaviors. Mole
ular biologists in
reasingly re
ognize that
systems modeling will fa
ilitate their understanding of
omplex biologi
al
pro
esses.
Apart from
onsiderations of symmetry, we have had little su
ess in
relating system stru
ture to fun
tion. Dynami
al systems theory provides
a
ontext for guiding our intuition of dynami
al phenomena that we expe
t
to observe in generi
systems. Using results from dierential topology and
singularity theory, we have a
oherent view of phenomena that we regard
as typi
al and phenomena that are ex
eptional. When there is symmetry
in a system, we know how to modify the theory. We believe that there
are ar
hite
tural prin
iples whi
h are important in building robust
omplex
systems, but there is little theory to support our intuition. In the
ontext of
spe
i
appli
ations su
h as ele
troni
ir
uit design, we build hierar
hi
al
systems of astounding
omplexity with millions of elements. Con
epts su
h
as hierar
hy and feedba
k
ontrol have not been in
orporated into a general
theory of nonlinear dynami
al systems. Our la
k of insight into how system
ar
hite
ture
onstrains dynami
al behavior limits the power of simulation as
a tool for studying
omplex systems.
4
Continuous, Dis
rete and Hybrid Models
De
omposition of natural systems into dierent "s
ales" is one of the
en
tral tasks in produ
ing high delity simulations. We seek to understand
how ma
ros
opi
behavior results from physi
al laws that operate on smaller
s
ales. For example, we would like to understand fra
ture in terms of atom
isti
properties of materials. Redu
ing all
omplex phenomena to atomi
intera
tions is
learly a hopeless task. It is preposterous to model the ef
fe
ts of global
limate
hange on natural populations and agri
ulture on an
atomi
s
ale. Re
ognizing when we
an separate physi
al s
ales and en
ap
sulate smaller s
ale information in models that operate at larger s
ales has
5
been a
entral issue within
ondensed matter physi
s. In studying popula
tion biology, e
onomies, or the brain the issue of aggregating small s
ales is
fuzzier and more
hallenging.
Only re
ently have we a
quired the
omputing resour
es required to sim
ulate detailed multis
ale models. One of the areas that is growing rapidly is
the simulation of models with large numbers of
omponents. In appli
ations
as varied as mole
ular dynami
s, battleeld simulations and traÆ
ow, we
build sto
hasti
models from whi
h we seek to observe emergent behavior at
the system level. Su
h simulations form the basis for a whole set of
om
puter games. The
hallenge with these eorts is to obtain results that t the
real world. We seldom know whi
h details of
omponent behavior are most
signi
ant for determining system properties, so it would be prudent to have
systemati
ways of evaluating the ee
ts of un
ertainty in model
omponents
upon system behavior. Su
h methods hardly exist at this time.
Numeri
al Analysis
Numeri
al implementation of dynami
al systems models depends upon ap
proximations that themselves are subtle. The simplest, most dire
t numeri
al integration algorithm (the Euler method) is subje
t to substantial errors.
These errors
an a
umulate to give qualitatively in
orre
t predi
itions about
long time dynami
s, as happens with the harmoni
os
illator. Histori
ally,
numeri
al solution methods for dierential equations addressed stringent lim
itations on the speed and
ost of performing arithmeti
. The dramati
im
provements of digital
omputers during the past fty years have
ompletely
transformed these parts of mathemati
s. The speed of
omputation in sim
ulation of physi
al systems as a limiting fa
tor has largely been repla
ed by
issues su
h as memory hierar
hies, roundo errors inherent in
oatingpoint
arithmeti
, and extra
ting useful information from very large data sets.
Consider the problem of simulating ele
tri
power systems. Reliable ele

tri
power produ
ed with minimal environmental impa
t is vital to the world
today. Adequate
apa
ity to handle anti
ipated loads and realtime mon
itoring of operations are essential for these systems. Simulating network
models at rst sight seems like a straightforward task in numeri
al integra
tion. However, when we look a bit
loser we nd te
hni
al issues that are
bothersome. One issue is that the equations for a network are naturally ex
pressed as dierentialalgebrai
equations rather than as ordinary dierential
equations. The mathemati
al theory of DAE's is more
omplex than that of
6
ODE's. For DAE's, not all initial
onditions in the phase spa
e are
onsis
tent with the equations. Moreover, there are points in phase spa
e where the
algebrai
onstraints inherent in the equations are satised, but there still
are no solutions (or multiple solutions) with these initial
onditions. The
mathemati
al underpinnings of the theory of DAE's remain in
omplete. Re
stri
tive assumptions on models are required to guarantee that simulation
algorithms will work. Still DAE's are
ommon in engineering appli
ations
and
annot be ignored. In addition to power systems, DAE's arise naturally
as models of
onstrained me
hani
al systems.
Beyond Simulation
Simulation viewed as the evolution of spe
i
initial
onditions for a dynami
al
omputer model is unlikely to dire
tly answer many of the questions that
we ask. For example,
onsider the problem of tting parameters to experi
mental data. In HodgkinHuxley models for neurons, a typi
al ve
tor eld
may have a ten dimensional phase spa
e and forty parameters. The model
of a network of ten neurons built from single
ompartment HodgkinHuxley
neurons will have a phase spa
e of dimension approximately one hundred
with several hundred parameters. If there are many model parameters that
annot be measured dire
tly, then we are left with a
omplex \inverse" prob
lem of using simulation data to optimize the parameters. In the
ase of the
neural network, the problem is further
ompli
ated by the distortion of the
primary voltage measurements that o
urs due to unmodeled spatial ee
ts
in the system. This means that the most useful data for model
omparisons
is likely to be related to qualitative properties like the period of os
illations or
the stability boundaries for dierent dynami
al states as physi
al parameters
are varied. Obtaining this information by sampling traje
tories
an only be
done for a small number of parameters be
ause the number of required tra
je
tories grows exponentially with the number of parameters. Thus, solution
of these parameter identi
ation problems seems to require algorithms that
go beyond simulation. The problems of tting model parameters inhibit the
reation of high delity models. As des
ribed previously, in
reasing model
resolution to in
lude smaller s
ales in a problem may in
rease the number of
parameters that must be determined faster than the delity of the models
improve.
Bifur
ation theory for dynami
al systems provides a framework for dire
t
determination of information about how system behavior
hanges qualita
7
tively with parameter variations. Implementation of algorithms based on
this theory is a step towards
omputing parameter ranges that produ
e de
sired behavior. For bifur
ations of equilibria, stability boundaries
an be
determined without numeri
al integration by formulating dening equations
for bifur
ations from the derivatives of the ve
tor eld. These methods have
been implemented in a
ontinuation setting to
ompute
urves of
odimension
one Hopf and saddlenode bifur
ations in twoparameter families of ve
tor
elds and
urves of
odimension two bifur
ations in threeparameter families
of ve
tor elds. We need better algorithms to
ompute multidimensional
ontinuation of submanifolds of bifur
ations and to treat bifur
ations of pe
riodi
orbits reliably.
There are additional geometri
questions about dynami
al systems that
are important for varied appli
ations. Computation of mixing properties of
uid
ows has been greatly fa
ilitated by regarding the instanteous velo
ity
elds as generating a dynami
al system of streamlines and
omputing in
variant manifolds of these dynami
al systems. The stagnation points of the
uid
ow are saddle points of the dynami
al system and their stable and
unstable manifolds give separation boundaries for the
uid
ow. Computing
the interse
tions of these manifolds and the evolution of their turnstile stru

tures gives approximations to the mixing properites of these
uids. These
te
hniques have been used to investigate the design of industrial rea
tors,
hemi
al rea
tion rates and
uid transport of o
ean eddies. Be
ause invari
ant manifolds be
ome highly
onvoluted with sharp bends, substantial
are
is need to
ompute them a
urately.
We desire the ability to robustly and routinely
ompute far more about
the qualitative properties of dynami
al systems than we
an today. There are
phenomena that play a prominent role in the qualitative theory that o
ur
on very ne s
ales in many examples. Developing
onsistent,
onverged
al
ulations requires that these s
ales be resolved. Sin
e the phenomena often
involve singularities and bifur
ations,
lassi
al algorithms need to be mod
ied and extended to work with these problems. Mathemati
al theory has
guided this work, leading to the
reation of algorithms that solve
hallenging
problems. Unlike prevailing trends in
omputational s
ien
e, the problems
have often been small and the
omputing highly intera
tive. The interplay
between
lassi
al and modern mathemati
s, geometry and numeri
al analysis
and
omputational s
ien
e will
ontinue to be important to progress in the
use of simulation as a powerful s
ienti
tool.