Sie sind auf Seite 1von 2

Mathematical Tripos Part II Lent 2005 Professor A. Iserles

Numerical Analysis – Lecture 1 1

1 Iterative methods for linear algebraic systems

Problem 1.1 (Positive deﬁnite systems). We consider linear systems of equations of the form Ax = b, where A is an n × n real positive deﬁnite symmetric matrix and b R n is known. Such systems occur, for example, in numerical methods for solving elliptic partial differential equations.

Example 1.2 (Poisson’s equation on a square). We seek the function u (x, y ), 0 x, y 1, that satisﬁes 2 u = f on the square, where f is a given function and where u is given on the boundary of the square. The best known ﬁnite difference method (the ﬁve-point formula) results in the approximation

(∆x) 2 2 u (x, y ) u (x x, y ) + u (x + ∆x, y ) + u (x, y x) + u (x, y + ∆x) 4u (x, y ), (1.1)

where x = 1/(m + 1) for some positive integer m . Thus, we estimate the m 2 unknown function values u (p x, q x), 1 p, q m , by letting the approximation to (∆x) 2 2 u (p x, q x) equal (∆x) 2 f (p x, q x) at these values of p and q . This yields an n × n system of linear equations, where

n = m 2 . The matrix of this system, A say, has the following properties.

Lemma 1.3 The matrix A of Example 1.2 is symmetric and positive deﬁnite.

= j then the ith and j th points of the grid (p x, q x),

0 implies A i,j = A j,i = 1, which proves the

Proof Equation (1.1) implies that if A i,j

1 p, q m , are nearest neighbours. Hence A i,j symmetry of A.

Therefore A has real eigenvalues and eigenvectors. We consider the eigenvalue equation Ax = λ x , and let

=

0 for i

=

 k be an integer in {1, , n} such that |x k | = max {|x i | : 1 ≤ i ≤ n}, where x i is the ith component of x. Then we address the identity n

(A k,k λ )x k =

j =1 j = k

A k,j x j .

Assume ﬁrst that λ < 0. Since A k,k = 4, A k,j ∈ {0, 1} for k

are nonzero for each k , it follows by the triangle inequality that

= j and at most four off-diagonal elements

|(A k,k λ )x k | =

n

j =1 j = k

A k,j x j

n

j =1 j = k

|A k,j ||x j | ≤ 4 max |x j | = 4|x k | < (4+ |λ |)|x k | = |(A k,k λ )x k |,

j =1 , ,n

a contradiction. Therefore λ 0. If λ = 0 then

(A k,k λ )|x k | = 4|x k | =

n

j =1 j = k

|A k,j ||x j |

implies that |x j | = |x k | whenever A k,j = 1. Continuing by induction, this implies that |x j | ≡ |x k | for all j , but this leads to contradiction in the equations that have fewer than four off-diagonal terms: such

1 Please email all corrections and suggestions to these notes to A.Iserles@damtp.cam.ac.uk. All handouts are available on the WWW at the URL http://www.damtp.cam.ac.uk/user/na/PartII/Handouts.html.

1

equations occur at the boundary of the grid. It follows that all the eigenvalues are positive and A is positive

deﬁnite.

Method 1.4 (Simple iteration). We write Ax = b in the form

(A B )x = B x + b,

where the matrix B is chosen so that it is easy to solve the system (A B )x = y for any given y . Then simple iteration commences with an estimate x (0) of the required solution, and generates the sequence

x ( k +1) , k = 0, 1, 2,

, by solving

(A B )x ( k +1) = B x ( k ) + b,

k = 0, 1, 2,

If the sequence converges to a limit, lim k x ( k ) = xˆ , say, then the limit has the property (A B )xˆ = B xˆ + b . Therefore xˆ is a solution of Ax = b as required.

Revision 1.5 (Conditions for convergence). Method 1.4 requires both A and A B to be nonsingular. Moreover, deﬁning xˆ by Axˆ = b, we recall from the IB Numerical Analysis course that it is instructive to write the expression (1.4) in the form (A B )(x ( k +1) xˆ ) = B (x ( k ) xˆ ). Indeed we deduce the relation

x ( k +1) xˆ = H (x ( k ) xˆ ) = H k +1 (x (0) xˆ ),

where H = (A B ) 1 B . We found in Part IB that the required convergence of the sequence x ( k ) ,

k

, is achieved for all choices of x (0) if and only if H has the property ρ(H ) < 1. Here

ρ(H ) is the spectral radius of H , which means the largest modulus of an eigenvalue of H . (Some of the eigenvalues may have nonzero imaginary parts.)

Methods 1.6 (Jacobi and Gauss–Seidel). Both of these methods are versions of simple iteration for the case when A is symmetric and positive deﬁnite. 2 In the Jacobi method the matrix B has a zero diagonal but the off-diagonal elements of B are those of A. In other words, B is deﬁned by the condition that A B is the

= 0, 1, 2,

 diagonal matrix whose diagonal elements are the nonzero numbers A i,i , i = 1, 2, , n. In the Gauss–Seidel method one sets B i,j = 0 for j ≤ i and B i,j = A i,j for j > i, so A − B is a

lower-triangular matrix with nonzero diagonal elements. The components of x ( k +1) satisfy

i

j

=1

A i,j x

(

j

k +1)

=

n

j

= i+1

A i,j x

(

j

k )

+ b i ,

i = 1, 2,

, n,

and it is straightforward to calculate them in sequence by forward substitution.

Let us return to Example 1.2. Denoting by u p,q our approximation to u (p x, q x), we have the equations

u p 1 ,q + u p+1 ,q + u p,q 1 + u p,q +1 4u p,q = (∆x) 2 f (p x, q x).

Arranging grid points by columns (so called natural ordering), we obtain

 Jacobi: u ( Gauss–Seidel: u (

k

)

k

k )

p1 ,q + u p+1 ,q + u p,q 1 + u p,q +1 4u p,q

(

p,q +1 4u p,q

p1 ,q + u p+1 ,q + u p,q 1

(

(

k )

k )

(

(

k )

k )

(

k +1)

(

k +1)

+1)

(

k +1)

+ u

= (∆x) 2 f (p x, q x);

= (∆x) 2 f (p x, q x).

We will ﬁnd that both methods converge for Example 1.2 by applying the following theorem. Its proof will be given in the next handout.

Theorem 1.7 (The Householder–John theorem). If the real symmetric matrices A and A B B are both positive deﬁnite, then the spectral radius of H = (A B ) 1 B is strictly less than one.

2 Actually, it is sufﬁcient for all diagonal elements of A to be nonzero.

2