1 9780898719543 ch2 PDF

Downloaded 06/14/18 to 165.230.225.19. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.
php
Chapter 2
Manifolds and Numerical

Continuation
2.1 Manifolds
2.1.1 Definitions
For the purposes of this book it is sufficient to consider only differentiable manifolds
that are naturally embedded in finite-dimensional Euclidean spaces and can be defined
locally as the regular null sets of continuously differentiable functions with values in other
Euclidean spaces. For a more general (and more abstract) definition see [201, §4.1].
Definition 2.1.1. Let k,N E N,N > k >_ 1. A subset S of RN is called a k-
dimensional C l - manifold if for every t E S there exist a neighborhood U of t in RN and
a C'-function G : U -* R N-k such that G x (±) has rank N - k and {x E U : G(x) =
0} = S n U. We say that G is a local defining function for S; if we want to stress that
G takes values in a multidimensional space, then we call G a local defining system. The
number N - k is called the codimension of the manifold.
Note. It is a fundamental property that if Gl is another R' _ !c- valued C l - function
defined in a neighborhood of that locally vanishes on the manifold defined by G and
has full linear rank N - k, then Gl defines (locally) the same manifold. We will not prove
this here; it is an easy corollary to Lemma 6.2.3. It will not be used in this chapter but
will be used extensively in Chapter 5.
Definition 2.1.2. If N> k >_ 1 and G : RN -+ R N-k is a Cl- function, then G is
called a global defining function for the manifold {x E R N : G x (x) has rank N - k}. The
points of this manifold are called ordinary points of the zero set of G.
Definitions 2.1.1 and 2.1.2 generalize in the obvious way to C 1 -manifolds (1> 1) and
to C°°-manifolds. C-manifolds will also be called smooth manifolds.
Let us assume without loss of generality that in Definition 2.1.1 the submatrix of
G x (±) that is related to the first N - k variables is nonsingular. Then we call the last k
variables the basic variables and collect them in XB E Rk; the others are called nonbasic
29
30 Chapter 2. Manifolds and Numerical Continuation
Downloaded 06/14/18 to 165.230.225.19. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
variables and are collected in XN E R I-k . So formally x = (xg ). The implicit function
theorem tells us that there exists a continuously differentiable mapping Gi defined in
a neighborhood of xB E Rk and with values in R N- k such that Gz (xB) = xN and
G(G 1 (XB), XB) - 0. The mapping Cl is locally unique and inherits the differentiability
properties of G. If, in particular, G is a smooth function, then so is G i . The map
1[ßk --f S : XB ' ('B)) is a homeomorphism from a neighborhood of xB E R k onto a
neighborhood of E S. It is also a diffeomorphism; i.e., its Jacobian has full rank k.
2.1.2 The Tangent Space

Consider a C l -map G : RN - ► R N-k (N > k _> 1). Let e(t) be a C l -curve in the zero set
S of G, i.e.,
G2(e(t))=0 for 1<i<N-k. (2.1)
By taking derivatives we obtain
0G 1 . 0G 1 . 0G1 .
^1+a xe ^2+...+ eN-a (2.2)
äx1 a-N
Hence the tangent vector to each curve in S is in the kernel of the Jacobian in the
point. We prove that the converse is true in a precise sense if the Jacobian has full rank.
Proposition 2.1.3. Let G : R N _ RN-(N > k >_ 1) be a C l -map and suppose
that x E Rn , G(^) = 0, and G x () has full rank N - k. If p = (pi,... , pN)T is in
the kernel of J = G x (x), then there exists a curve x = e(t) in RN which lies locally in
S = {x E R N : G(x) = 0} and for which C(0) = t, (0) = p.
Proof. Without loss of generality we assume that the first N - k columns of J are
linearly independent. By the implicit function theorem there exist functions
Fi(XN-k+1, ... ‚ XN) (1 < i < N - k)

such that
Xi = F1(ZN-k+1, ... , XN) (1 < i < N - k)
and
Gi(F1(xN-k+1, ... , xN), ... , FN-k(xN-k+1, ... , xN), XN-k+1, ... , xN) = 0 (2.3)
in a neighborhood of xN-k+1, ... , XN) T E Rk

(
.
Now define
î(t) = F'i(xN-k+1 + tpN-k+1, ... , xN + tpN) (1 < i < N — k), (2.4)
î (t) = .ti + tpi (N — k + 1 < i < N). (2.5)
These two equations determine a curve (t) in RN with (0) = x and locally (t) E S.
We now prove that (0) = p.
From (2.5) it follows that
X1(0)=pi if N-k+l<I<N. (2.6)

2.1. Manifolds 31
By taking derivatives of (2.3) we find

8Gi aF; 3Gi
axjaxl+axl 0 (1 <i<N-k,N-k+l<l<N). (2.7)
j<N-k
After multiplication with (t) and summation over 1 we obtain
aG i aF; • aGi
E äx axl ^l +
j<N-k i 1»k1>N-k
8x1^t = 0, .i: (2.8)
where the identity is with respect to t and l = pl (N = k + 1 <_ l < N).

From (2.4) we infer
•
!2- <j<N-k). (2.9)
a^l (1
1>N-k
Combining (2.8) and (2.9) we obtain

acti. aci•
(2.10)
äx ^j + äx ` = 0.
j<N-k 1>N-k
On the other hand,

_ aG (2.11)
ax^ p3 + ax1 pl = 0.
j<N-k 3 1>N-k 1
Since [ay: ]1<i,j<N-k has full rank, it follows from (2.10) and (2.11) that ßj(0) = pj also
if1<j<N-k. 0
Definition 2.1.4. Let k, N E ICY, N > k >_ 1, S a k-dimensional C l - manifold in R N ,
x E S and G: U -+ R N-k a local defining function of S at Jr. Then the null space of
Gx (2) is called the tangent space of S at x.
Proposition 2.1.3 proves that the above definition is independent of the choice of the
local defining function G at 2 E S.
2.1.3 Examples
Example 2.1.5. Let N = 3, k = 2, R E R, R> 0. Define G : R 3 - R, ( x1, x2, x3) T ->
x1 + x2 + x3 - R2 . The zero set of G is the sphere with center in the origin and radius
R; G x has full rank 1 at every point of the sphere and so constitutes a global defining
function.
Example 2.1.6. Let N = 3, k = 2. A torus in JR 3 has an obvious parametrization
with two real parameters. Locally each part of the torus is diffeomorphic to an open set
in R 2 . By local elimination of the parameters one obtains local defining functions. The
existence of a global defining function is less obvious, but we can take
(x1, x2, x3) T - (

x1 + x2 -4) 2 +x -1.
Example 2.1.7. Let N = 2, k = 1 and define C: 1R 2 -> ][l , (x1, x2) T -► xi -x. The
zero set of G consists of the two straight lines x1 = x2 and x1 = -x2. This set does not
constitute a manifold since at (0, 0) T the set of tangent vectors does not form a linear
space. However, if we define
S = {(xi, x2) T : x1 - x2 = 0, (x1, x2) T (0, o)T},
then S is a manifold with dimension 1 and G is a defining function at each point of

S. We remark that S is a disconnected set that consists of four connected parts (open
half-lines).
Example 2.1.8. Let N = 4, k = 3 and identify R 4 with the space of 2 x 2 matrices.
Define det : R 4 -- R, (all, a12, a21, a22) T -^ a11a22 - a12a21. The zero set of det is the
set of singular 2 x 2 matrices. The vectors (1, 0, 0, 0) T and (0, 0, 0, 1) T are both tangent
vectors to curves in the zero set of det at the origin, but (1, 0, 0, 1) T is not. Hence the
zero set of det does not constitute a manifold. If we delete the origin, then
S {xER 4 :det(x)=0,x#0}
is the set of rank-1 matrices and constitues a three-dimensional differentiable manifold

since det has a nonvanishing Jacobian at each point of S.
Example 2.1.9. Let N = 6, k = 4 and identify R 6 with the space of 3 x 2 matrices.
Define
det1 : R 6 -> R, ( a11, a12, a21, a22, a31, a32) T a11a22 - a12a21,
det2 : R 6 -> R, (a11 , a12, a21, a22, a31, a32) T -4 a11a32 - a12a31,
det3 : JR 6 --f R, (all , a12, a21, a22, a31, a32) T - a21a32 - a22a31 •
Now
S={xE R6:detl(x)=det2(x)=det3(x)=0,x#0}
is the set of rank-1 3 x 2 matrices. If x E S, then at least one entry of x is nonzero. For
example, suppose that all # 0. Then (detl = 0, det2 = 0) define S locally near x and
the Jacobian of this system is nonsingular at x. So S is a four-dimensional manifold and
local defining systems can be found by taking appropriate pairs from (detl = 0, det2 =
0, det3 = 0). We remark that each pair is actually a defining system for S in an open
everywhere dense subset of R 6 .
2.2 Branches and Limit Points

The case of a one-dimensional manifold (k = 1) deserves special attention. It arises most
naturally if one of the variables is singled out as a distinguished bifurcation parameter.
Let us then subdivide the independent variables as (x, a), where x E R N , a E R. Let
G : ]RN+ RN be a C l - function and let Z be the zero set of G, S the set of all
ordinary points in Z. Then S is a differentiable manifold with dimension 1. The maximal
connected components of S are called branches; they are necessarily diffeomorphic with
real intervals.

2.2. Branches and Limit Points 33
AC
Figure 2.1: The zero set of a nonlinear function.
A possible picture of Z is given in Figure 2.1. Obviously D, E are not in S. S has

eight maximal connected components (branches). The point D is called a transcritical
bifurcation point; E is a pitchfork bifurcation. The distinction between transcritical bi
-furcationpsdhfrkbiucatonsexlyuforchieas
distinguished bifurcation parameter.
A, B, C are in S but are special with respect to the parameter a because the tangents
in these points are orthogonal to the a-axis. The parameter a cannot be used to param-
eterize S in a neighborhood of A or B. It can be used in a neighborhood of C, but the
parameterization will then not be diffeomorphic as we would probably prefer it to be.
Definition 2.2.1. A point (x, a) E S is called a limit point or fold if Gx is singular.
All other points of S are called regular points.
Proposition 2.2.2. In a limit point G x has rank deficiency 1. There exists a vector
î # 0, unique up to a scalar multiple, such that V T G X = 0. For this vector V T G a # 0.
Proof. Since [G x G a ] has rank N, G x has rank at least (N — 1). 0
Proposition 2.2.3. Let s be a diffeomorphic parameterization of a branch through
the ordinary point (x, a). Then (x, a) is a limit point if and only if a, = ds = 0.
Proof. From G(x(s), a(s)) - 0 it follows that G x x, + G a a, - 0. If (x, a) is a limit
point and Vi is the corresponding left singular vector of G, then
(SG T G^)xs+( T Ga)as=0.
Hence a, = 0. Conversely, if a, = 0 then G x x, = 0. Since (x', a,) T # 0 we infer x, # 0

and hence G x is singular. 0
In the setting with a distinguished bifurcation parameter, limit points are often en-
countered as quadratic turning points. We saw this already in Chapter 1. To introduce
it now formally, we start from the identity
G(x(s), a(s)) - 0

1.5
0.5
-0.5
6.25 os 0.75 1 125 1S 1.75
Figure 2.2: A quadratic turning point.
from which
Gxxs + Gaas = 0,
Gxxxaxs + Gxaasxs + Gxxss + Gxaxsas + Gaaasas + Gaass = 0.
In a limit point a 8 = 0; after multiplication with cb T we obtain
W T Gxxxsxs + ( l T Ga)as
s = 0.
Hence a ss # 0 if and only if jiT G xx x,x s # 0. We remark that x, is a nonzero right

singular vector of G.
Definition 2.2.4. A limit point is a simple or quadratic turning point if a„ # 0.
It follows that a(s) locally behaves as cs 2 where s = 0 corresponds to the limit point
and c is a nonzero constant. In particular, a 8 changes sign. See Figure 2.2.
Remark 2.2.5. Of course one can define limit points of higher order. For example,
a hysteresis point is defined by the conditions a s = 0, a„ = 0, a 83 0, or, equivalently,
by
G,, singular,
') T Gxxxsxs = 0,
+ 3 T Gxxxsxss 0
(check this!). In transcritical and pitchfork bifurcation points one has IpTG a = 0. This
case also deserves further study. See Chapter 7.
2.3 Numerical Continuation

2.3.1 Natural Parameterization
We consider the problem of computing a solution branch of the equation
G(x,a) = 0 (x E R N , a E R, G(x, a) E R N ). (2.12)

2.3. Numerical Continuation 35
Figure 2.3: Continuation by natural parameterization.
Numerical continuation is a technique to find consecutive points of a solution branch to

(2.12). We adapt our notation to this splitting of the variables, keeping in mind that the
splitting is irrelevant in some cases.
Suppose that we already found a solution point (x', ao) T . To find a new one, call it
(xi, al ) T , we need a starting point (2T , ä) T and a strategy to determine (xi, ai) T
One possibility is to choose & and to fix al = ä. This is the historical method [196].
(See Figure 2.3.) Then xl is determined by solving
G(x,a,) = 0. (2.13)
Solving this system by Newton's method we construct a sequence x 0 , x 1 , x 2 , ... with

xo = 2 = xo and
Gx(xk,a,)Axk = — G(x k ,a,), (2.14)
x k+1 = x k + / x k (2.15)
for k = 0, 1, 2, .... Of course any other method to solve nonlinear systems can be used
instead of (2.14), (2.15), in particular quasi-Newton methods.
Geometrically this amounts to approximating the curve first by a straight line (pre-
dictor step) and then correcting in a hyperplane a = al (corrector step). See Figure
2.3.
This suggests another predictor, namely, along the tangent line. If we parameterize
by a, then from the identity
G(x(a), a) = 0 (2.16)
we infer that
Gyxä = —G a . ( 2.17)
Hence x. is found by solving a system similar to (2.14), and the predictor is i =
xo + (ä — ao)xä(0). See Figure 2.4.
Figure 2.4: Prediction along the tangent.
2.3.2 Pseudoarclength Continuation

The methods proposed in §2.3.1 have difficulties if a branch of (2.12) contains limit points;
a is not a good parameterization of the curve in the neighborhood of such points.
Suppose that a point (xi, a1) T and a previous point (x', ao) T with tangent vector
to are known. To find a tangent vector tl at (xi , al) T we remark (as in §2.1.2) that
[Gx Ga] ti = 0. (2.18)
This equation determines tl up to a scalar factor. To preserve the orientation of the

branch we require
t o tl = 1. (2.19)
If we decompose tl in a natural way as tl = () and do similarly for to, then we

can write (2.18) and (2.19) as
cx G a l tix) 10 1
_ (2.20)
tpx)T tÖ-) L 1
where G x is an n x n matrix, G a and toxl are n-vectors, and to' l is a scalar. A matrix
with form
[C A
T
B
D
with A E ][fin x n, B, C E 1Rn x m , D E Rm x m will be called a bordered matrix with
(2.21)
borderwidth m.
Suppose that a steplength As is chosen (this choice will be discussed further in §2.3.3).
Then our predictor is
x = x 1 + 1 tl
x) , & = a, + I s ti`
A Yl . ( 2.22)

Figure 2.5: Continuation along a branch with limit points.
This is called a pseudoarclength predictor because As measures arclength along the

tangent line and therefore approximates the arclength along the branch.
There are several possibilities for the corrector step; we discuss four of them.
1. A solution to (2.12) is sought in a coordinate hyperplane through (x, &); i.e., an
equation of the form x i = ii ( i E {1,... , n}) or a = & is added. For a picture of the
former case, see Figure 2.5. The latter case is the one discussed in §2.3.1, which fails if
Gx is singular. In the former case the system is
G(x, a) = 0, (2.23)
eT(x — i) = 0, (2.24)
where (e i ) , = b ij (1 < j < n). The Jacobian of (2.23)-(2.24) is hence
M=
[ Gx
0^.(2.25)
a
One has to choose i so that (2.25) is nonsingular. From a comparison of (2.25) and
(2.18) it follows that M is singular if and only if the ith component of tl is 0.
The strategy proposed in [201] and implemented in the code PITCON (Pittsburgh
Continuation Code; see §2.4) is, therefore, to choose that value of i for which I ti I i s
maximal (with a small correction:
J
if !!1 <IM and itl^ l > (toj I whereby (tij j ? µ1ä1
IIti II Poll Ilti ll Atoll
for a threshold value p(0 < µ < 1), then one chooses j instead of i).
2. A solution to (2.12) is sought in the hyperplane orthogonal to ti. The system for
the computation of the next point on the branch is, therefore,
Figure 2.6: Correction in the hyperplane orthogonal to the tangent.
G(x, a) = 0, (2.26)
a)T
ti T (x — x) + t1 (a — ä) = 0 (2.27)
with Jacobian
Gx G^
t1x) T tla) (2.28)
This is called Keller's method [151]. See Figure 2.6. We remark that orthogonality is
not scale invariant. If x and a have different physical dimensions (as is often the case),
then the meaning of orthogonality in (x T , a) T -space is rather arbitrary. Therefore, in
AUTO [80], [84] the possibility is provided to replace (2.27) with
9x t1x1T (x — x) + 9a t1^ 1T (a — a) = 0 (2.29)

with weights B x > 0 and 8 a > 0.
3. In a variant of the previous method the tangent vector tl is replaced after every
corrector step by the normalized singular vector of the Jacobian at the point from where
the correction step was started. This is the strategy in CONTENT [165].
4. A solution to
G(x + Ax, & + La) = 0 (2.30)
is sought so that II (öä) jj is minimal.
Since G(x, a) is nonlinear, this cannot be done exactly in one step. However, we can
perform an iterative algorithm similar to Newton's method. More precisely, we choose
(x°, a°) = (i, &) and xk+ 1 = xk + Ax , a k+l = a c + Aa, where Ax, Aa is the
minimum-norm solution to
G.(x k , a I`)Lx + G a (X k , a i`)Aa = —G(x k , a^`). (2.31)
The iterative scheme, which uses (2.31) as a step, is called the Gauss—Newton iteration
[74], [75]. See Figure 2.7. It is analyzed, and quadratic convergence is proved in [74].

Figure 2.7: Gauss—Newton method.
We note that computing the minimum-norm solution to an underdetermined system

of linear equations is a standard problem in numerical linear algebra; it is part of the
more general problem of computing minimum norm least squares solutions. This in
turn is the numerical procedure to apply the pseudoinverse or Moore—Penrose inverse
of a given possibly nonsquare and possibly rank-deficient matrix. The recommended
stable algorithm uses a QR decomposition with pivoting of the matrix [106], [230], [70].
However, in the context of continuation and bifurcation problems it is also natural to use
a bordered matrix approach. We note that the solution to (2.31) is unique except for a
multiple of the nullvector of [G., G a ]. Hence if one solves the systems
) t(.)T a) 1
Gx Ga t( t(- = 0 (2.32)
4) 1 1
Gx G. tlx —G
T = , (2.33)
t (.) t(a) Ala 0
then the general solution to (2.31) is given by
Ox = Aix + 77t (x) ,

(2.34)
Aa = Ala + r)t (a) ,
where i E R is arbitrary. The vector [ ö^ ] has minimal norm if
((Ox) T Da) ( t(«) ) =0;

i.e.,
(Lix)T tixi + (Lla)t(a) (2.35)
11th1 2
Extensions of this method to more general minimum norm and least squares problems
can be found in [120].
Remark. In each of the four methods the solution of systems with G. is replaced
by the solution of systems with form
Gx b
M = cT d
i.e., bordered matrices with borderwidth 1. The underlying idea is that M can be non-
singular even if G x is singular. We will discuss bordered matrices in detail in Chapter 3.
2.3.3 Steplength Control

Steplength control is an important part of a continuation method. If the steplength is
too small, then a lot of unnecessary work is done. If it is too large, then the corrector
algorithm may converge to a point on a different branch of (2.12) or not converge at all.
It is possible to make an a priori estimate of a good steplength, although such an
estimate is never completely reliable. Suppose that we have computed a point (uk, ak)
with normalized tangent vector tk and the preceding point (xk_l, ak-1) with normalized
tangent vector tk-1. Let tk = (tN T , t(a) ) T and tk-1 = ( t
splittings of tk and tk-1, respectively.
ky i ,
t() l ) T be the natural
,
For a pseudoarclength As the predictor is
X = xk + ( s)t, (2.36)
ä = ak + ( Ls)t. (2.37)
Since the prediction is along the tangent line the error has order (Ls) 2 for small (Ls)
(Taylor expansion!). An estimate of the error may be obtained by comparing the predic-
tion with one of order (As) 3 , namely,
q(As) _ a k)+( Ls)tk + 2 (Os) 2 wk, (2.38)
whereby
Wk =
Indeed, from
Ask (tk — tk-1) AS k
=I \ a k / — \ a k -1 / I
9(0)= ( ak 1 ' d ( A )) (U) = tk, I d( A s )
)
) (—Ask) = tk-1,
it follows that (2.38) is an estimate with order max {IAskIhAsI 2 , As1 3 } ( check this!).
Suppose that a tolerance e is chosen for the absolute error in (i, &). The error in
q(As) is about 2(As) 2 Ilwkli; therefore, the choice
As - II 2 ^ I (2.39)
w
is a reasonable one.
This choice is used (with some additional constraints) for steplength control in PITCON
(see §2.4). The following corrector step can of course lead to a rejection of this stepsize
and replacement by a smaller one.
It is also possible to base the steplength control completely on the convergence of
Newton iterates in the corrector step. A simple way to do this is implemented in CON-
TENT. The convergence is declared successful if both ii Gil and the length of the last
correction vector are below certain user-chosen threshold values after no more than three
correction steps; the steplength is then multiplied by 1.3. It is declared unsuccessful if
the thresholds are not reached after a user-chosen number 'max > 3 of correction steps;
the steplength is then halved and corrections are started again. In the intermediate case
convergence is declared successful, but the steplength is left unchanged.
A more sophisticated method is used in AUTO [80], [84]. We discuss this further.
Suppose the corrector step for the computation of (xi', ak ) T starts from (x (0)T , a (0) )T =
(xk T , a, ) T and computes consecutively the values (xkl)T, a kl) ) T , (xk2)T, a^2) ) T , ...whereby
x kl+1) _ xkl) + Ax(t+l), a ka+ 1 ) = a
( ' ) + Aa ' + 1) . For every correction one computes the
(
norm
E(l+1) = II (Ax (l+1)T Aa(d+1))T
Ile
and the iteration is stopped if e ( '+ 1) < T whereby T is a threshold value, e.g., T =
10 -5 ll(4 1 , ak_1)T iI if the machine precision is about 10 -1 s
One decides that the iteration is unsuccessful if the condition
e(t+1) < T
(2.40)
is not satisfied after 10 iterations or if e(l+l) > e ( l ) In the case of unsuccessful iter-
ations the steplength is halved; below a certain minimal threshold the continuation is
interrupted.
Now let Osk_1 be the steplength of the previous step; i.e.,
(xk T ak) T = (X _
' i + ASk-ltk-1
, ak-1 + Ask-itk_i
)T
is the point that is reached after successful correction step has yielded (xk, ak).
Let No be the number of iterations that was needed to satisfy (2.40), and let us denote
the norms of the corrections by ek" ) 1 (1 < n < No).
We now compute a new steplength Ask to find a suitable starting point (xk+l, äk+l) T
(xk,ak)T +ASk (tky)T,tka))T.
Let us denote by c (-) the (yet unknown) norms of the corrections that will be needed
when starting from this point. From the hypothesis of quadratic convergence (theory of
Newton iterates) we find
E ( ") = C(ei"-1))2 = C1+2(e="-2))4 (2.41)

= C1+2+4( e ^n-3))8 = ,. 1_ 1)(6i l ) )(2 ' 1)
= C(2 (2.42)
fori=k-1 andfori=k.
We suppose further that
= a(Asti) (i = k — 1, k).
(
l) (2.43)
A theoretical analysis, based on quadratic convergence, leads to an approximation of the

form ei l) = c (ts^) 2 , but this is true only in the limit for small (As s ), and we have no
reason to choose As smaller than necessary. The estimate (2.43) is in practice better if
the steps are not very small.
We want to choose (Ask) so that (2.40) is satisfied after K steps whereby K is
chosen heuristically. We always perform at least three Newton iterations, and ideally
this number should be sufficient. So No > 3 and we set
K— 3ifN0=3,
No-1ifNo>3.
From (2.42) and (2.43) it follows that

(2K-3)
e(K) Ds
(K)
k-1
— (ASk-1)
If we require (as a safety measure) that e (K) ^ 0.1T instead of e(K) < T, then we get
(21)
0.1?
Ask = ASk_1. K) (
2.3.4 Convergence of Newton Iterates

Since convergence of Newton iterates is essential for most algorithms discussed in this
book (not only in continuation), we take the opportunity to recall some basic facts.
Suppose we are solving
F(x) = 0 (x E R', F(x) E R"), (2.44)
using a starting point x ° and the iterations
FF (x k )Lx k = —F(x"), (2.45)
x k+1 = xk + AX k . (2.46)
An important property of this process is its affine invariance. Namely, if A is an arbitrary

nonsingular matrix, then (2.44) is equivalent to the solution of
G(x) := AF(x) = 0. (2.47)
The sequence Ix ° , x 1 , x 2 , ... } of Newton iterates is independent of A; from (2.45) it

follows indeed that
G'(xk)Axk = -G(x k ).
A convergence test of the form

Oxk +1II < E
or monotonicity test
I 'Lxk +1 II < OII Ax k II (2.48)
for a fixed 9 (0 < 9 < 1) is also affine invariant. On the other hand, the apparently
natural test
IIF(xk +l)II sOIIF(x k )II (2.49)
(0 < 9 < 1) is not affine invariant. If F(x k ) and F(x k +l) are linearly independent vectors
(as they usually are), then for any given 9 > 0 it is in fact possible to find a nonsingular
matrix A such that
AF(xk +l)II > OIIAF(x k )
On the other hand, an affine invariant convergence and monotonicity test can be
based on IIFx (x k ) -1 F(x k + 1 )II. To be precise, the monotonicity test would be
-1 F(x k
IIFx(x k ) +l)II s 9 IIFx(x k ) - 1 F(x k )II (2.50)
(0 < 9 < 1); affine invariance follows from
(AFF(xk))-1(AF(xk +1)) = FF ( x k)- 1F( x k +1)
We remark that in (2.50) the computation of the right-hand side requires little additional
work since Fx (x k ) -1 F(x k ) = -Lx k is already known. The left-hand side requires a new
solution with the system F (xk). If these systems are solved by a factorization method
(e.g., LU or QR), then the factorization of Fx (xk) is done already in the computation of
/xk, and the additional work of solving a system is small. In [75] the reduced Newton
correction Axk+l is defined by
F(xk)Oxk+l = - F(x k+l ) ( 2.51)
and the test (2.50) hence becomes
IIL(x k + l ) II <- 9II x k ll. (2.52)
In [75] the value 0 = 0.5 is recommended in (2.52).

2.3.5 Some Practical Considerations

To start a continuation algorithm one needs a starting point (ui ao) T , a tangent vector
,
in that point, an initial steplength, and an orientation along the tangent.

A starting point is usually found by ad hoc methods. In many cases the problem has
a trivial solution for certain parameter values. If only a guess is available, then we can
try to improve it using a Newton iteration or any other solution method for nonlinear
systems. A tangent vector is found by solving the system
L eT d j L t Û) I = I, I , (2.53)
where c E R", d E R are chosen fairly arbitrarily; it is only required that the matrix in
(2.53) be nonsingular. An initial steplength is usually found by trial and error. For the
orientation only two choices are possible and both may be worthwhile.
2.4 Notes and Further Reading

1. For a more complete treatment of numerical continuation methods we refer to
[201], [74], [22] (code COLCON for boundary value problems), and [6].
Several good continuation codes are freely available. One of them is PITCON. For
related literature see [201], [198], [73], [199], [200]. It can be obtained from the NETLIB li-
brary at http://www.netlib.org as contin/pcon6l.f (single precision) or contin/dpcon6l.f
(double precision). For the computation of implicitly defined manifolds contin/manpak
is also provided.
ALCON is another continuation code, developed at the Konrad Zuse Institut in Berlin.
It can be obtained from http://elib.zib.de/pub/Packages/alcon-S. One of its main fea-
tures is the use of Gauss—Newton iterations in the corrector stage.
An extensive list of continuation and related software is given in [9]. Unfortunately,
such lists tend to become obsolete rather quickly.
2. Most codes work fine in simple low-dimensional problems as those we consider in
the exercises for this chapter. On the other hand, some problems require a good contin-
uation code and a careful handling of tolerance and threshold parameters. Examples in
this book include Exercise 15, §5.6 and the computations in §7.7.
3. To detect bifurcations it is often necessary to monitor certain test functions while
computing branches. We will see many examples in Chapters 4 and 5 and beyond.
It has been suggested that these test functions can also be used in the choice of the
steplength. Roughly the idea is to decrease the steplength if the behavior of the test
functions suggests that a bifurcation point might be nearby. Alhough reasonable, this
idea has not yet found its way into software.
2.5 Exercises
Note. CONTIN is a generic name for a continuation code, which the reader may write
or, preferably, download from one of the sites mentioned in §2.4. The exercises that refer

2.5. Exercises 45
to CONTIN were tested with PITCON.
1. Prove that the set of rank-2 matrices in R 3 " 3 is a differentiable manifold with
dimension 8 and give a global defining function.
2. Prove that the set of rank-1 matrices in R3X3 is a differentiable manifold with
dimension 5 and describe local defining systems for every matrix in the manifold.
3. Prove that the set of 2 x 2 matrices with exactly one zero eigenvalue is a differ-
entiable manifold in R4 . Give a local defining function for every matrix in the
manifold. Is it also a global defining function?
4. The set of real normal 2 x 2 matrices (AA T = AT A) does not constitute a manifold.
It is a finite disjoint union of manifolds of dimensions 1, 2, and 3. Prove this. (Such
structures are called stratified sets; we omit the precise definition.)
5. Use CONTIN to compute 25 points of the curve y = x 2 starting with x = 1, y = 1

with initially increasing values of x.
6. Use CONTIN to find the smallest solution to the equation

x 2 -3x+2=0,x>0 (startx=0) (Hint: G(Y,X)=Y—X 2 +3X-2=0).
7. Compute ' with CONTIN.
8. Use CONTIN to compute a number of solution points to x 2 — y 3 = 0, starting with

(x, y) = (-1, 1) with initially decreasing values of y. Can you continue the branch
to points with x > 0? Then try again with x 2 — y 3 + e = 0, where e is a positive or
negative number. Explain what happens.
9. Find with CONTIN all real roots of the cubic equation x 3 — 6x 2 + 11x — 6 = 0.
10. Find with CONTIN all real roots of the equation x 7 — 6x + 1 = 0.
11. Continue with CONTIN the solution branch to the system
G(X(1), X(2)ß A) _ h2 r 12 12 J L eX(2) J

X (2) ] + A [
with h = 3, starting point (0, 0, 0) in the direction of initially decreasing A- values

(cf. §1.4). Questions:
(a) Does one stay in the symmetric space S = {(X (1), X(2), A) : X(i) = X(2)}?
(b) Is there a turning point with respect to A? (1, 1, —0.8277287)
12. Find an asymmetric solution branch to the problem in the previous exercise by
starting from a point in the neighborhood of the symmetry-breaking bifurcation
point (3, 3, —0.3360627).

13. Consider the map G : R 3 —> R 2 , where
G(x) _ xl — x2 + 5x2 — 2x2 + 34x3 — 47

x1 + x2 + x2 - 14x2 + 10x3 - 39 )
[201, p. 146].
(a) Which one of xl, x2 i x3 can be used easily to parameterize the solution set of
the system? Why?
(b) Give an explicit parameterization of the solution set by this variable.
(c) Give a starting point with x2 = 0.
(d) Find with CONTIN a point with x2 > 0 and xl = 0. Is there guaranteed to be
one? Can you also find it with analytical methods?
(e) Is there a point with x2 > 0 and x3 = 0?
(f) Find with CONTIN a maximum of xl in the region x2 > 0. Can you also find
it with analytical methods?
14. Consider the following eutrophication model:
il = xl(0.2(A1 — xl — x2) — 0.445x3 — 4),

X= —0.0455x2x3 + 4x1,
X3 = A2(10 — x3) — 2.67x3(0.445x1 + 0.0455x2).
Here x3 is the concentration of oxygen in the hypolimnion of a flat lake; xl and x2

are the nutrient densities stored in biomass and detritus, respectively. The param-
eter Al is the sum of all nutrients, including those in dissolved form and assumed
to be constant; A2 determines the transportation of oxygen from epilimnion to
hypolimnion (cf. Werner [237]).
(a) Compute for A2 = 0.7 a curve of stationary solutions in (xl,X2 i x3, A1) -space
starting from (xi = 0, x2 = 0, x3 = 10, Al = 42.25) in the direction of initially
increasing xl.
(b) Find a turning point of this curve with respect to Al. Conclusion: There
exist parameter pairs (Al, A2) for which two equilibrium states (xl, x2 i X3) are
possible.
(Remark: You may have starting problems. See §7.6.2 for a study of the local
behavior near the starting point.)
15. Consider the function
G(x, y) = —y 2 + x + x 2 + 0.1x 4 + 0.01x 6 — 0.001x 8
(x,yEIR).

2.5. Exercises 47
(a) Compute with CONTIN all turning points (with respect to either x or y) of the
branch of G(x, y) = 0 through (0, 0).
(b) Another solution branch goes through (-3.570165,4.442488). Compute also
all turning points on this branch.
(c) Make a sketch of the computed solution branches to G(x, y) = 0 in R 2 .
(d) Prove that the solution set to G(x, y) = 0 is bounded.
16. Consider the function
G(x, y) = 7x 6 + x 3 y + xy + y 2 — 8y 6 .
(a) Compute all solutions of the equation G(x, y) = 0 that lie on the circle {(x, y)
x 2 + y 2 = 9 }. (Hint: There is precisely one in each quadrant.)
(b) Do the computed points exhibit a form of symmetry? Was this to be expected?
(c) Compute for every i (1 < i < 4) a solution branch of G(x, y) = 0 through Pi
and follow it into the interior of the disc {(x, y) : x 2 + y 2 < 9} until it leaves
the disc again.
(d) Do all these branches pass through (0, 0)?
(e) Compute with CONTIN all limit points in the disc { (x, y) : x 2 + y 2 <_ 9} with
respect to either x or y.
(f) What can you guess from the computed points concerning the tangents at
(0, 0) to all branches?
(g) Compute the intersection points of all branches with the coordinate axes.
Make a sketch of all solutions to the equation G(x, y) = 0 inside the disc
{(x, y) : x 2 + y 2 < 9 }. Indicate all the computed special points.
17. The 3-box Brusselator describes the Belusov—Zhabotinski reaction in a system of
three cyclically coupled cells (see §4.2 for some background). In the ith cell a
component with concentration Xi and one with concentration Yi react in a process
that involves the parameters A, B. Adjacent cells influence each other by diffusion,
which is governed by diffusion coefficients D 1 , D2 and a coupling coefficient A. The
global equation of the so formed dynamical system is
X1 =D1(X2 - 2X1+X3)+A(A - (B+1)X1+X1Y1),
= D2(Y2 - 2Y1 + Y3) + \(BX1 - X1Y1),

X2 = Dl (X3 — 2X2 + X,) + A(A — (B + 1)X 2 + X2Y2),
i'2=D2(Y3 - 2Y2+Y1)+,\(BX2 - X2Y2),
X3 = D l (X, — 2X3 + X2 ) + A(A — (B + 1)X 3 + X3Y3),
Y3 =D 2 (Yl -2Y3 +Y2 )+A(BX 3 — X3Y3 ).
We assume D l =1, D 2 =10, A=2.1, B=3.5.
(a) Write a driving program for CONTIN to compute a branch of equilibrium so-
lutions with )► as the free parameter. Compute a number of points on a
branch through the point with coordinates Xl = X2 = 1.087734, Yl = Y2 =
1.8488063, X3 = 4.124552, Y3 = 1.0389936, \ = 5.
(b) Are the symmetry relations Xl = X2, Yl = Y2 preserved along the branch?
(c) This question is similar to (a) for the branch through Xl = X3 = 2.5975837,
Yi = Y3 = 1.4653965, X2 = 1.1048325, Y2 = 1.8634635, A = 5.
(d) This question is similar to (b) for the branch computed in (c).
(e) Does there exist a fully symmetric solution for A = 5, i.e., a solution with
Xl = X2 = X3, Yi = Y2 = Y3? If so, then find for this solution the precise
values X1, Y1. Describe the solution branch with free parameter A through
this point.
(f) Find (without computing) four not-yet-mentioned equilibrium solutions for
A =5.
(g) Compute with CONTIN a branch of equilibrium solutions through the point
with coordinates Xl = 1.1001799, Yl = 2.079928, X2 = 1.9698811, Y2 =
1.7320476, X3 = 3.229939, Y3 = 1.2280244, A = 9.
(h) Is the so-computed branch a closed one? Compute all limit points of this
branch. How many do you find? How many points does the branch contain
with A = 5? How many with A = 9?
(i) Can you find seven equilibrium points with A = 9 that are not on the branch
computed in (g)?

1 9780898719543 ch2 PDF

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

1 9780898719543 ch2 PDF

Hochgeladen von

Copyright:

Verfügbare Formate

Downloaded 06/14/18 to 165.230.225.19. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.

Manifolds and Numerical

2.1.2 The Tangent Space

Fi(XN-k+1, ... ‚ XN) (1 < i < N - k)

in a neighborhood of xN-k+1, ... , XN) T E Rk

^i(t) = F'i(xN-k+1 + tpN-k+1, ... , xN + tpN) (1 < i < N — k), (2.4)

^i (t) = .ti + tpi (N — k + 1 < i < N). (2.5)

By taking derivatives of (2.3) we find

After multiplication with (t) and summation over 1 we obtain

where the identity is with respect to t and l = pl (N = k + 1 <_ l < N).

Combining (2.8) and (2.9) we obtain

On the other hand,

(x1, x2, x3) T - (

S = {(xi, x2) T : x1 - x2 = 0, (x1, x2) T (0, o)T},

then S is a manifold with dimension 1 and G is a defining function at each point of

is the set of rank-1 matrices and constitues a three-dimensional differentiable manifold

2.2 Branches and Limit Points

Figure 2.1: The zero set of a nonlinear function.

A possible picture of Z is given in Figure 2.1. Obviously D, E are not in S. S has

(SG T G^)xs+( T Ga)as=0.

Hence a, = 0. Conversely, if a, = 0 then G x x, = 0. Since (x', a,) T # 0 we infer x, # 0

6.25 os 0.75 1 125 1S 1.75

Figure 2.2: A quadratic turning point.

Hence a ss # 0 if and only if jiT G xx x,x s # 0. We remark that x, is a nonzero right

2.3 Numerical Continuation

Figure 2.3: Continuation by natural parameterization.

Numerical continuation is a technique to find consecutive points of a solution branch to

Solving this system by Newton's method we construct a sequence x 0 , x 1 , x 2 , ... with

Figure 2.4: Prediction along the tangent.

2.3.2 Pseudoarclength Continuation

[Gx Ga] ti = 0. (2.18)

This equation determines tl up to a scalar factor. To preserve the orientation of the

If we decompose tl in a natural way as tl = () and do similarly for to, then we

Figure 2.5: Continuation along a branch with limit points.

This is called a pseudoarclength predictor because As measures arclength along the

Figure 2.6: Correction in the hyperplane orthogonal to the tangent.

9x t1x1T (x — x) + 9a t1^ 1T (a — a) = 0 (2.29)

Figure 2.7: Gauss—Newton method.

We note that computing the minimum-norm solution to an underdetermined system

then the general solution to (2.31) is given by

Ox = Aix + 77t (x) ,

where i E R is arbitrary. The vector [ ö^ ] has minimal norm if

((Ox) T Da) ( t(«) ) =0;

2.3.3 Steplength Control

For a pseudoarclength As the predictor is

q(As) _ a k)+( Ls)tk + 2 (Os) 2 wk, (2.38)

Newton iterates) we find

E ( ") = C(ei"-1))2 = C1+2(e="-2))4 (2.41)

A theoretical analysis, based on quadratic convergence, leads to an approximation of the

From (2.42) and (2.43) it follows that

2.3.4 Convergence of Newton Iterates

F(x) = 0 (x E R', F(x) E R"), (2.44)

using a starting point x ° and the iterations

FF (x k )Lx k = —F(x"), (2.45)

An important property of this process is its affine invariance. Namely, if A is an arbitrary

G(x) := AF(x) = 0. (2.47)

The sequence Ix ° , x 1 , x 2 , ... } of Newton iterates is independent of A; from (2.45) it

A convergence test of the form

(0 < 9 < 1); affine invariance follows from

(AFF(xk))-1(AF(xk +1)) = FF ( x k)- 1F( x k +1)

F(xk)Oxk+l = - F(x k+l ) ( 2.51)

and the test (2.50) hence becomes

IIL(x k + l ) II <- 9II x k ll. (2.52)