Introduction to Simulation - Lecture 8

1-D Nonlinear Solution Methods

Jacob White

Thanks to Deepak Ramaswamy Jaime Peraire, Michal

Rewienski, and Karen Veroy

Outline
Nonlinear Problems
Struts and Circuit Example

Richardson and Linear Convergence

Simple Linear Example

Newtons Method

Derivation of Newton
Quadratic Convergence
Examples
Global Convergence
Convergence Checks

Nonlinear
problems
( x0 , y0 )

( x2 , y2 )

Strut Example

( x1 , y1 )

Given: x0, y0, x1, y1, W

Find: x2, y2

Load force
W

Need to Solve f x

W

Struts Example

Nonlinear Problems

Reminder: Strut Forces

L0  L
EAc
L0

f
fx
f

(0,0)
L

x1 , y1

fy

fx
X

fy
L

SMA-HPC 2003 MIT

H  L0  L 

x1
f
L
y1
f
L
2
1

2
1

x y

Nonlinear
problems

Strut Example

( x1 , y1 )

( x0 , y0 )

 x2  x0    y2  y0 

L1

 x2  x1    y2  y1 

L2

f1
( x2 , y2 )

x2  x0
H ( Lo  L1 )
L1
x2  x1
H ( Lo  L2 )
L2

f1x

f2

f2 x

Load force
W

1x

 f2 x

1y

 f2 y  W

Nonlinear
problems

Strut Example

Why Nonlinear?

y2  y1
H ( Lo  L2 ) 
L2
y2  y0
H ( Lo  L1 )  W
L1

Pull Hard on the

Struts

The strut forces change

in both magnitude and
direction

Nonlinear
problems

v1
10v

Circuit Example

v2

10

1
I r  Vr
10

+
- Vd

Vd

I d  I s (e
Need to Solve

Id  Ir
I vsrc  I r

0
0

Vt

 1)

Nonlinear
problems

Solve Iteratively

Hard to find analytical solution for

f ( x)

Solve iteratively
0
guess at a solution x
x0
repeat for k = 0, 1, 2, .

k 1

W x

k 1
f
x
|0
until

Ask
Does the iteration converge to correct solution ?
How fast does the iteration converge?

Richardson
Iteration

Definition

Richardson Iteration Definition

k 1

x  f (x )

An iteration stationary point is a solution

k 1

f ( xk )

xk

x* ( Solution)

Richardson
Iteration

Example 1

f ( x)
Start with

0.7 x  10
0

x1

x 0  f ( x 0 ) 10

x2

x1  f ( x1 ) 13

x 6 14.27

x3

x 2  f ( x 2 ) 13.9

x7

x4

x3  f ( x3 ) 14.17

x8 14.28

14.25
14.28
Converged

Richardson
Iteration

Example 1

f ( x)

x x

0.7 x  10

Richardson
Iteration

Example 2

f ( x)
Start with

2 x  10

x0

x1

x0  f ( x0 ) 10

x2

x1  f ( x1 )

x3

x2  f ( x2 ) 130

x4

x3  f ( x3 )

40
400

No convergence !

Richardson
Iteration

Convergence

Setup

Iteration Equation
Exact Solution

k 1

x  f (x )
*

x N
f (x )
0

Computing Differences

k 1

x

x  x  f (x )  f (x )
Need to Estimate

Richardson
Iteration

f (v )  f  y 

Convergence

Mean Value Theorem

wf  v 
v  y
wx

v > v, y @

v

Richardson
Iteration

Convergence

Use MVT

Iteration Equation
Exact Solution

k 1

x  f (x )
*

x N
f (x )
0

Computing Differences

k 1

x

x  x  f (x )  f (x )
wf  x  k
*
1 
x x
wx

Richardson
Iteration

If

1

And
Then
Or

Convergence

Richardson Theorem

wf  x 
wx

d J  1 for all x s.t. x  x  G

x x G

k 1

x dJ x x

lim k of x

k 1

x

lim k of J x  x

Linear Convergence

Richardson
Iteration

Example 1

f ( x)

x x

0.7 x  10

Richardson
Iteration

Problems

Convergence is only linear

x, f(x) not in the same units:
x is a voltage, f(x) a current in circuits
x is a displacement, f(x) a force in struts
Adding 2 different physical quantities
But a Simple Algorithm
Just calculate f(x) and update

Newtons method

Another approach

From the Taylor series about solution

df k *
f ( x )  f ( x )  ( x ) ( x  xk )
dx
*

Define iteration
Do k = 0 to .
1
df k
k 1
k
x
x  ( x ) f ( xk )
dx

df k
if ( x )
dx

until convergence

1

exists

Newtons Method

Graphically

Newtons Method

Example

Newtons Method

x x

Example

Newtons Method
0

f ( x* )

Convergence

2
df
d
f
k
k
k
*
f ( x )  ( x )( x  x )  2 ( x )( x*  x k ) 2
dx
dx
k
*

some x [ x , x ]

Mean Value theorem

truncates Taylor series

But

df k k 1 k
f ( x )  ( x )( x  x )
dx
k

by Newton
definition

Convergence

Newtons Method

Contd.
2

df k k 1 *
( x )( x  x )
Subtracting
dx
Dividing through

d f
k
* 2

x
x
x
(
)(

)
2
d x

2
df
d
f
k 1
k 1
k
*
* 2

( x  x ) [ ( x )]
(
)(

)
x
x
x
2
dx
d x

1

2
df
d
f

Suppose ( x)
( x) d L for all x
2
dx d x

then x

k 1

x dL x x

* 2

Convergence is quadratic if L is bounded

Convergence

Newtons Method

Example 1

f ( x) x 2  1 0,
df k
( x ) 2 xk
dx
k

2x (x

k 1

k 1

2x (x
or ( x

k 1

find x ( x* 1)

 x

x )

1
k

 x )  2x (x  x )
*

x )

1
k
* 2
(x  x )
k
2x

 x

 x  
2

Convergence is quadratic

Convergence

Newtons Method

Example 2
2

f ( x) x 0, x 0
df
df k
Note :
not bounded
k
dx
(x ) 2 x
dx
away from zero
2
k
k 1
k
2 x ( x  0) ( x  0)
1 k
*
k 1
k
for x z x 0
x 0
x 0
2
1
*
*
( xk  x )
or ( xk 1  x )
2
1

Convergence is linear

Newtons Method

Convergence

Examples 1 , 2

Newtons Method

Convergence

1

2
df
d f
Suppose ( x)
( x) d L for all x
2
dx d x

if L x0  x* d J  1
then xk converges to x*

Proof

x1  x * d L ( x0  x * ) x0  x *
x1  x * d J x0  x *
x2  x * d LJ x0  x * x1  x *
or x2  x * d J 2 x1  x * d J 3 x0  x *
x3  x * d J 4 x 2  x * d J 7 x 0  x *

Newtons Method

Convergence

Theorem
df
d2 f
If L is bounded (
bounded away from zero ;
bounded)
2
dx
dx
then Newton's method is guaranteed to converge given a "close
enough" guess

Always converges ?

Newtons Method

Convergence

Example
Convergence Depends on a Good Initial Guess
f(x)

x1

x1
x

SMA-HPC 2003 MIT

Convergence

Newtons Method

Convergence Checks

Need a "delta-x" check to avoid false convergence

f(x)

k 1

k 1

 x ! H xa  H xr x

f x

SMA-HPC 2003 MIT

k 1

 H fa

k 1

Convergence

Newtons Method

Convergence Checks

Also need an "f  x  " check to avoid false convergence

f(x)

f x
x*

x
SMA-HPC 2003 MIT

 x  H xa  H xr x

! H fa
X

x k 1 x k
k 1

k 1

k 1

Summary
Nonlinear Problems
Struts and Circuit Example

Richardson and Linear Convergence

Simple Linear Example

1-D Newtons Method

Derivation of Newton
Quadratic Convergence
Examples
Global Convergence
Convergence Checks