Emacs A 072006 Mark

Elements
of
Matrix Modeling and Computing
with
MATLAB
Robert E. White
Department of Mathematics
North Carolina State University
white@math.ncsu.edu
Updated on 07-20-06.
Copyright: CRC/Chapman Press a Division of
Taylor and Francis Group
Contents
List of Figures
vii
List of Tables
xi
Preface
xiii
Introduction
xv
1 Vectors in the Plane

1.1 Floating Point and Complex Numbers
1.2 Complex Valued Functions . . . . . .
1.3 Vectors in R2 . . . . . . . . . . . . . .
1.4 Dot Product and Work . . . . . . . . .
1.5 Lines and Curves in R2 and C . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
10
19
27
38
2 Vectors in Space
2.1 Vectors and Dot Product
2.2 Cross and Box Products .
2.3 Lines and Curves in R3 .
2.4 Planes in R3 . . . . . . . .
2.5 Extensions to Rq . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
47
47
56
67
76
86
= d: Unique Solution
Matrix Models . . . . . . . . . . . . . .
Matrix Products . . . . . . . . . . . . .
Special Cases of Ax = d . . . . . . . . .
Row Operations and Gauss Elimination
Inverse Matrices . . . . . . . . . . . . .
OX Factorization . . . . . . . . . . . . .
Determinants and Cramers Rule . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
95
95
105
117
127
138
149
159
3 Ax
3.1
3.2
3.3
3.4
3.5
3.6
3.7
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4 Ax = d: Least Squares Solution

171
4.1 Curve Fitting to Data . . . . . . . . . . . . . . . . . . . . . . . . 171
4.2 Normal Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 182
v
CONTENTS
vi
4.3
4.4
Multilinear Data Fitting . . . . . . . . . . . . . . . . . . . . . . . 191

Parameter Identication . . . . . . . . . . . . . . . . . . . . . . . 199
5 Ax
5.1
5.2
5.3
= d: Multiple Solutions
209
Subspaces and Solutions in R3 . . . . . . . . . . . . . . . . . . . 209
Row Echelon Form . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Nullspaces and Equilibrium Equations . . . . . . . . . . . . . . . 230
6 Linear Initial Value Problems

6.1 First Order Linear . . . . . . . . . . . .
6.2 Second Order Linear . . . . . . . . . . .
6.3 Homogeneous and Complex Solution . .
6.4 Nonhomogeneous Dierential Equations
6.5 System Form of Linear Second Order . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
243
243
250
257
263
272
7 Eigenvalues and Dierential Equations

281
7.1 Solution of x0 = Dx by Elimination . . . . . . . . . . . . . . . . . 281
7.2 Real Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . 289
7.3 Solution of x0 = Dx + f (w) . . . . . . . . . . . . . . . . . . . . . . 296
8 Image Processing in Space Domain
311
8.1 Matrices and Images . . . . . . . . . . . . . . . . . . . . . . . . . 311
8.2 Contrast and Histograms . . . . . . . . . . . . . . . . . . . . . . 321
8.3 Blurring and Sharpening . . . . . . . . . . . . . . . . . . . . . . . 331
9 Image Processing in Frequency Domain
9.1 Laplace and Fourier Transforms . . . . .
9.2 Properties of DFT . . . . . . . . . . . .
9.3 DFT in Rq Rq . . . . . . . . . . . . .
9.4 Frequency Filters in Rq Rq . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
343
343
351
361
370
A Solutions to Odd Exercises
381
Bibliography
397
Index
399
List of Figures
1.1.1
1.1.2
1.2.1
1.2.2
1.3.1
1.3.2
1.3.3
1.3.4
1.4.1
1.4.2
1.4.3
1.4.4
1.4.5
1.4.6
1.5.1
1.5.2
1.5.3
1.5.4
1.5.5
Complex Numbers as Arrows . . . . . . .

Norm(} 2 ) and Angle(} 2 ) . . . . . . . . .
A!ne, Square and Square Root of z . . .
Solutions of } 12 = 1 . . . . . . . . . . . .
A Vector in the Plane . . . . . . . . . . .
f2 = d2 + e2 . . . . . . . . . . . . . . . .
f2 = e2 + d2 2de cos() . . . . . . . . .
$ $

$
$

$

d + e >
d e and v e . . . . . . . . . .
Trigonometric Identity and Dot Product
Area and Dot Product . . . . . . . . . .
Linearly Independent Vectors . . . . . . .
Work and a Ramp . . . . . . . . . . . . .
Torque on a Wheel . . . . . . . . . . . .
Work with Independent Paths . . . . . .
Line Given a Point and Direction . . . .
Minimum Distance of Point to a Line . .
Cycloid and Wheel . . . . . . . . . . . .
Cycloid . . . . . . . . . . . . . . . . . . .
Two-tone Signal . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4
7
13
17
20
21
22
23
31
33
34
35
35
36
39
41
43
44
45
2.1.1
2.1.2
2.1.3
2.2.1
2.2.2
2.2.3
2.2.4
2.3.1
2.3.2
2.3.3
2.3.4
2.4.1
2.4.2
Point in Space . . . . . . . . .
Vector in Space . . . . . . . .
Vector Addition in Space . . .
Unit Vector Cross Products . .
Projected Area . . . . . . . .
Box Product and Volume . . .
Determinant and Volume . . .
Vector Equation and Minimum
Distance between Two Lines .
Helix . . . . . . . . . . . . . .
Projectile in Space . . . . . . .
Normal and Point . . . . . . .
Three Points . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
48
49
50
58
58
64
65
68
71
73
75
77
78
vii
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
Distance
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
LIST OF FIGURES
viii
2.4.3
2.4.4
2.5.1
2.5.2
2.5.3
Linear Combination of Vectors

Minimum Distance to a Plane
Mesh of Image Matrix . . . . .
Imwrite of Image Matrix . . .
Negative Image Matrix . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
79
81
92
92
92
3.1.1
3.1.2
3.1.3
3.1.4
3.2.1
3.2.2
3.3.1
3.4.1
3.5.1
3.6.1
3.6.2
3.7.1
Box with Fixed Volume . . . . . . .

Cost of a Box . . . . . . . . . . . .
Two-bar Truss . . . . . . . . . . . .
Two-loop Circuit . . . . . . . . . . .
Heat Conduction in a Wire . . . . .
Steady State Heat Diusion . . . .
Temperature in Wire with Current .
Six-bar Truss . . . . . . . . . . . . .
Five-bar Truss . . . . . . . . . . . .
Three-loop Circuit . . . . . . . . . .
Potential in a Single-loop Circuit . .
Three-tank Mixing . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
101
102
102
103
112
114
125
134
149
155
156
167
4.1.1
4.1.2
4.1.3
4.2.1
4.4.1
4.4.2
Sales Data . . . . . . . . . . . . . . .
Least Squares Function for Sales Data
Radioactive Decay . . . . . . . . . . .
World Population Prediction . . . . .
US Population and Logistic Model . .
Temperature Data and Curve Fit . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
174
176
179
188
204
207
5.3.1 Bar e with Four Forces . . . . . . . . . . . . . . . . . . . . . . . 237

5.3.2 Fluid Flow in Four Cells . . . . . . . . . . . . . . . . . . . . . . 239
6.2.1
6.3.1
6.4.1
6.5.1
6.5.2
Mass-Spring System . . . . . . . . . .
Variable Damped Mass-Spring . . . .
Forced Mass-Spring . . . . . . . . . .
Series LRC Circuit . . . . . . . . . . .
Tuned Circuit with Modulated Signal
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
251
263
271
273
279
7.3.1 Heat Diusion in Thin Wire . . . . . . . . . . . . . . . . . . . . 308

8.1.1
8.1.2
8.1.3
8.1.4
8.1.5
8.1.6
8.1.7
8.1.8
8.1.9
Pollen Image . . . . . . . .
Enhanced Pollen Image . . .
Aerial Photo . . . . . . . . .
Enhanced Aerial Photo . . .
Mars Rover Photo . . . . . .
Enhanced Mars Rover Photo
Moon . . . . . . . . . . . . .
Sharper Moon Image . . . .
Plot of the Matrix C . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
312
312
313
313
314
314
315
315
317
LIST OF FIGURES
ix
8.1.10 Image of Letter C . . . . . . . . . .

8.1.11 Negative Image . . . . . . . . . . . .
8.1.12 Matrix NCSU . . . . . . . . . . . .
8.1.13 Image of NCSU . . . . . . . . . . .
8.1.14 Negative Image of NCSU . . . . . .
8.1.15 Center Grain in Pollen . . . . . . . .
8.2.1 Histogram of Pollen Image . . . . . .
8.2.2 Histogram of Lighter Pollen Image . .
8.2.3 Lighter Pollen Image . . . . . . . . .
8.2.4 Piecewise Linear Function . . . . . .
8.2.5 Histogram for Enhanced Pollen Image
8.2.6 Higher Contrast Pollen Image . . . .
8.2.7 Mars Rover Image Using Power 1/2 .
8.2.8 Mars Rover Image Using Power 2 . .
8.3.1 Deblurred 1D Image . . . . . . . . . .
8.3.2 Original NCSU . . . . . . . . . . . . .
8.3.3 Blurred NCSU . . . . . . . . . . . . .
8.3.4 Deblurred NCSU . . . . . . . . . . . .
8.3.5 Increased Contrast Pollen . . . . . . .
8.3.6 Brighter and Sharper Pollen . . . . .
8.3.7 Original Moon Image . . . . . . . . .
8.3.8 Brightened and Sharpened . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
317
318
318
319
319
320
322
324
325
326
328
328
330
330
334
336
336
337
338
339
340
341
9.2.1 DFT of Sine and Cosine . . . . . .

9.2.2 Noisy Sine Function . . . . . . . .
9.2.3 Filtered Sine Image . . . . . . . .
9.3.1 2D DFT of Sine and Cosine . . .
9.3.2 Noisy 2D Sine Wave . . . . . . . .
9.3.3 Mesh Plot of Noisy Sine Wave . .
9.3.4 DFT of Noisy Sine Wave . . . . .
9.3.5 Low-pass Filter . . . . . . . . . .
9.3.6 Filtered DFT of Sine Wave . . . .
9.3.7 Filtered Sine Wave . . . . . . . . .
9.4.1 Noisy NCSU Image . . . . . . . .
9.4.2 Low-pass Filtering of NCSU . . .
9.4.3 Ideal Low-pass NCSU . . . . . . .
9.4.4 Band-reject Filtering of NCSU . .
9.4.5 Band-reject Filtered NCSU . . . .
9.4.6 Light and Noisy Aerial Image . .
9.4.7 Filtering Aerial Image . . . . . . .
9.4.8 Filtered Aerial Image . . . . . . .
9.4.9 Micro Chip Image . . . . . . . . .
9.4.10 Sharpening of Micro Chip Image
9.4.11 Sharpened Micro Chip Image . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
354
359
360
364
367
368
368
369
369
370
372
373
373
374
374
376
376
377
378
378
379
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
List of Tables
4.1.1
4.1.2
4.1.3
4.3.1
4.3.2
4.3.3
4.3.4
4.4.1
4.4.2
Computer Sales Data . . . .

World Population Data . . .
Radioactive Decay Data . . .
Multlinear Data . . . . . . .
Price Data for Three Markets
Home Appraisal Data . . . .
Three-tank Mixing Data . . .
US Population Data . . . . .
Temperature Data . . . . . .
.
.
.
.
.
.
.
.
.
xi
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
173
174
178
191
193
195
197
202
205
Preface
An important objective of this book is to provide "math-on-time" for second
year students of science and engineering. The student should have had one
semester of calculus. The student most likely would take this matrix course
concurrently with the second semester of calculus or would use this text for independent study of these important topics. This text lls in often missed topics
in the rst year of calculus including complex numbers and functions, matrices, algebraic systems, curve tting, elements of linear dierential equations,
transform methods and some computation tools.
Chapters one and two have introductory material on complex numbers, 2D
and 3D vectors and their products, which are often covered in the beginning of
multivariable calculus. Here a connection is established between the geometric
and algebraic approaches to these topics. This is continued into chapters three,
four and ve where higher order algebraic systems are solved via row operations,
inverse matrices and LU factorizations. Linearly independent vectors and subspaces are used to solve over and under determined systems. Chapters six and
seven describe rst and second order linear dierential equations and introduce
eigenvalues and eigenvectors for the solution of linear systems of initial value
problems. The last two chapters use transform methods to lter distorted images or signals. The discrete Fourier transform is introduced via the continuous
versions of the Laplace and Fourier transforms. The discrete Fourier transform
properties are derived from the Fourier matrix representation and are used to
do image ltering in the frequency domain.
The rst ve chapters can be used as a two-credit course (28 50-minute
classes). Among the nine chapters there is more than enough material for a
three-credit course. This three-credit matrix course when coupled with a nineor ten-credit calculus sequence can serve as a more "diverse" alternative to the
traditional twelve-credit calculus sequence. The twelve-credit calculus sequence
can be adapted to this alternative by reducing the precalculus, moving some of
2D and 3D vectors and dierential equations into the matrix course, and using
computing tools to do the complicated computations and graphing.
Most sections have some applications, which should indicate the utility of the
mathematics being studied. Seven basic applications are developed in various
sections of the text and include circuits, trusses, mixing tanks, heat conduction, data modeling, motion of a mass and image lters. The applications are
xiii
PREFACE
xiv
developed from very simple models to more complex models. The reader can
locate sections pretaining to a particular application by using the index.
R
MATLAB is used to do some of the more complicated computations. Although the primary focus is to develop by-hand calculation skills, most sections at the end have some MATLAB calculations. The MATLAB m-les used
in the text are listed in the index and are included in the books Web site:
http://www4.ncsu.edu/~white. The approach to using computing tools includes: rst, learn the math and by-hand calculations; second, use a computing
tool to conrm the by-hand calculations; third, use the computing tool to do
more complicated calculations and applications.
I hope this book will precipitate discussions concerning the core mathematical course work that scientists and engineers are required to study. Discrete
models and computing have become more common, and this has increased the
need for additional study of matrix computation, and numerical and linear algebra. The precise topics, skills, theory and appropriate times to teach these
are certainly open for discussion. The matrix algebra topics in this book are
a small subset of most upper level linear algebra courses, which should be enhanced and taken by a number of students. This book attempts to make a
bridge from two- and three-variable problems to more realistic problems with
more variables, but it emphasizes skills more than theory.
I thank my colleagues who have contributed to many discussions about the
content of this text. And, many thanks go to my personal friends and Liz White
who have listened to me emote during the last year.
Bob White
MATLAB is a registered trademark of The MathWorks, Inc. For product
information, please contact:
The MathWorks, Inc.
3 Apple Hill Drive
Natick, MA 01760-2098 USA
Tel: 508-647-7000
Fax: 508-647-7001
E-mail: info@mathworks.com
Web: www.mathworks.com ?http://www.mathworks.com/A.
Introduction
One can view an pq matrix as a table of objects with p rows and q columns.
The objects are usually real or complex numbers, but they could be characters
or records of information. A simple example is data for the last 12 months of
car sales where there are p = 12 rows and q = 2 columns. The rst column
will have the months number and the second column will have the number of
cars sold in the corresponding month. By examining the data one would like
to make a prediction about futures sales. This is where the modeling enters.
If the graph of the sales versus months "looks" like a straight line, then the
data may be modeled by a linear function of time | = pw
b + f. The slope p
b
and intercept f must be chosen so that the computed sales are "close" to the
car sales data. This is done by appropriate manipulations of the two column
vectors and computing a solution of the resulting system of algebraic equations.
Once p
b and f have been found, the predicted sales for w larger than 12 can
easily be calculated by evaluating the linear function. The modeling process is
complicated by incorrect sales data, changing prices and other models such as
a parabolic function of time.
This text examines a variety of applications, which have matrix models and
often have algebraic systems that must be solved either by-hand calculations
or using a computing tool. Applications to projectiles, circuits, mixing tanks,
trusses, heat conduction, motion of a mass, curve tting and image enhancement
will be initially modeled in very simple ways and then revisited so as to make
the model more accurate. This is typical of the modeling process where there is
an application, a model, mathematical method, computations and assessment
of the results. Then this cycle is repeated so as to enhance the applications
model.
The rst two chapters deal with problems in two- and three-dimensional
space where the matrices have no more than three rows or columns. Here
geometric insight can be used to understand the models. In Section 2.5 the
extension to higher dimensions is indicated for vectors and matrices, solution
to larger algebraic systems, more complicated curve tting, time dependent
problems with systems of dierential equations and image modeling. Chapters
three, four and ve have the basic matrix methods that are required to solve
systems in higher dimensions. Chapters six and seven contain time dependent
models and introduce linear systems of dierential equations. The last two
xv
INTRODUCTION
xvi
chapters are an introduction to image and signal processing.

Most sections have some by-hand matrix calculations in the numbered examples, some applications and some MATLAB computations, see [4] and [6].
The focus is on the by-hand calculations, and one should carefully study the
numbered examples. Each numbered example usually has two exercises associated with it. There are also additional exercises, which may ll in some parts of
the text, be related to applications or use MATLAB. This text is not intended to
be a tutorial on MATLAB, but there are a number of short codes that may help
you understand the topics being discussed. The by-hand calculations should be
done, and MATLAB should be used to conrm these calculations. This will give
you condence in both your understanding of the by-hand matrix computation
and the use of MATLAB. Larger dimensional problems can easily be done using
MATLAB or other computer software.
The following matrices are used in Chapters 3, 4, 5 and 9, and they can
be generalized to larger matrices enabling one to cross the bridge from models
with few variables to many variables.
5
6
5
6
0 0 0
1 0 0
]=7 0 0 0 8
L=7 0 1 0 8
0 0 0
0 0 1
5
6
5
6
1 0 0
1 7 10
4 8
H32 (3) = 7 0 1 0 8
X =7 0 2
0 3 1
0 0
3
5
6
5
2 1 0 200
1 1
0
b = 7 0 3@2 1
[D g] = 7 1 2 1 0 8
[X g]
0 1 2
70
0 0 4@3
6
5
5
2 1 0
0
8 6 4
9 1 2 1 0 :
9 6 12 8
1
:
D = (1@10) 9
D=9
7 0 1 2 1 8
7 4 8 12
0
0 1 2
2 4 6
6
5
5
6
1 1
1 2 3 4 5
9 2 1 :
:
UHI = 7 0 0 1 2 1 8
OV = 9
7 3 1 8
0 0 0 0 0
4 1
5
6
1 1 1 1
9 1 } }2 }3 :
:
F4 = 9
7 1 }2 1 }2 8
1 }3 }2 }
6
200
100 8
410@3
6
2
4 :
:
6 8
8
Chapter 1
Vectors in the Plane

This chapter contains geometric and algebraic descriptions of objects in two
dimensional space, R2 > and in the complex plane, C. The objects include vectors, lines, complex valued functions and some curves. Fundamental operations
include vector addition and dot product. The basic properties of complex numbers and complex valued functions are introduced. Applications to navigation,
work, torque, areas and signal representation via phasors are given.
1.1
Floating Point and Complex Numbers
In this section we rst discuss the integers and rational numbers. The oating
point numbers, which are used in computers, are a nite subset of the rational
numbers. The real and complex numbers are natural extensions of these. The
complex numbers also can be represented by directed line segments or vectors in
the plane. Although initially complex numbers may appear to be of questionable
value, they will be used extensively in the chapters on dierential equations and
image processing.
1.1.1
Rational Numbers
The integers are the set of whole numbers and include both positive, negative
and zero
Z { 2> 1> 0> 1> 2> }=
The addition and product of two integers are also integers. Any integer can
be uniquely factored into a product of prime numbers (an integer that is only
divisible by itself and one). For example, 90 = 51 32 21 =
The rational numbers are fractions of integers p@q where q is not zero and
p and q are integers
Q {p@q : p> q 5 Z> q 6= 0}=
1
CHAPTER 1. VECTORS IN THE PLANE
The set of rational numbers has a countable but innite number of elements.
Also, the addition and product of two rational numbers are rational numbers.
1.1.2
Real Numbers
Any real number is approximated by a sequence of rational numbers. Traditionally, one uses a base ten decimal expansion with {l > h 5 Z and 0 {l ? 10
{ = (={1 {g )10h
({1 @10 + + {g @10g + )10h =
This expansion is either a bounded increasing or decreasing sequence of rational
numbers and, therefore, by the completeness axiom for the real numbers it must
converge.
R {({1 @10 + + {g @10g + )10h : {l > h 5 Z> 0 {l ? 10}=
Real numbers contain the rational numbers, but not all real numbers are
rational. For example, consider { = 31@2 where 3 is a prime number. If { were
a rational number, then { = p@q giving 3 = p2 @q2 and 31 q2 = p2 . The
left side has an odd number of prime factors 3, and the right side has an even
number of prime factors 3. This contradicts the unique factorization property
and, hence, { cannot be a rational number.
1.1.3
Floating Point Numbers
Computers use a nite subset of the rational numbers to approximate any real
number. This set of numbers may depend on the computer being used. However, they do have the same general form and are called oating point numbers. Any real number { can be represented by an innite decimal expansion
{ = (={1 {g )10h , and by truncating this we can dene the chopped
oating point numbers.
Let { be any real number and denote a oating point number by
i o({) = ={1 {g 10h
({1 @10 + + {g @10g )10h =
This is a oating point number with base equal to 10 where {1 is not equal
to zero, {l are integers between 0 and 9, the exponent h is an integer between
given integers i and j and g is a positive integer called the precision of the
oating point system
F {({1 @10 + + {g @10g )10h : {1 6= 0> 0 {l ? 10> 0 ? g}=
Associated with each real number, {, and its oating point approximate
number, i o({), is the oating point error, i o({) {. This error decreases as
the precision, g, increases. Each computer calculation has some oating point
1.1. FLOATING POINT AND COMPLEX NUMBERS
error. Moreover, as additional oating point calculations are done, there may
be an accumulation of these oating point errors.
Example 1.1.1. Let { = 1=5378 and i o({) = 0=154 101 where g = 3. The
oating point error is
i o({) { = =0022=
The error will accumulate with any further operations containing i o({), for
example, i o({)2 = =237 101 and
i o({)2 {2 = 2=37 2=36482884 = =00517116=
Repeated calculations using oating point numbers can accumulate to signicant errors.
1.1.4
Complex Numbers
Any real number times itself must be positive unless it is zero. The square root
of any negative number cannot be positive and, therefore, not a real number!
The set of complex numbers is dened to be real numbers plus another real
number times l (1)1@2
C {d + el : d> e 5 R and l = (1)1@2 }=
If } 5 C> then real(}) = d is called the real part and imag(}) = e is called the
imaginary part. Two complex numbers are equal if and only if both the real
and imaginary parts are equal.
Complex numbers are often represented graphically where the real part is
located on the horizontal axis and the imaginary part is located on the vertical
axis. The complex number is then either viewed as point in the complex plane
or as directed line segment or arrow from the origin 0 + l0 to the complex point
} = d + el= Figure 1.1.1 illustrates this for } = 2 + 1l= One point of confusion
concerns the starting point of the arrow, which is the origin, but often the arrow
is moved parallel to its original position, as is illustrated by the top dashed arrow
in Figure 1.1.1 that is parallel to }= This is done to depict addition of two arrows
by placing the beginning of the arrow at the end of another arrow.
There are several important operations: conjugate of }, modulus of }, addition, subtraction, product and division. The conjugate of } is another complex
number formed by the reection of } about the real axis. The modulus of } is
the length of the vector. The algebraic operations are dened in the natural
way by using l2 = 1=
Denitions 1.1.1. Attributes of complex numbers } = d + el and z = f + gl=
}
}
|}|
arg(})

z if and only if d = f and e = g>

d el or frqm(}) d el,
(d2 + e2 )1@2 or norm or modulus of } and
arctan(e@d) or argument or angle of }=
Figure 1.1.1: Complex Numbers as Arrows
Denition 1.1.2. The polar representation of a complex number with =

arg(}) is
} = |}| (cos() + l sin()=
s
Example 1.1.2.sLet } = 3 +s
l. Then
1@2
} = 3 l, 3} = 3 3 + 3l
= 2.
s and |}| = 4
Also, duj(}) = arctan(1@ 3) = @6 and
} = 2(cos(@6) + l sin(@6))=
Solving equations with unknown complex numbers } = { + l| requires one
to nd two real numbers { and |= So, there must be two real equations, which
can be found by setting equal the real and imaginary parts of a given complex
equation. For example, consider the linear complex equation
} + 7 + 2l = 3 + 4l=
This is equivalent to equating the real and imaginary parts
real(} + 7 + 2l) = 3 and
imag (} + 7 + 2l) = 4=
Addition of complex numbers is dened by adding the real and imaginary parts,
} + 7 + 2l = ({ + l|) + 7 + 2l = ({ + 7) + (| + 2)l and, then, the above two real
equations are
{ + 7 = 3 and
| + 2 = 4=
Thus, the complex solution of } + 7 + 2l = 3 + l4 is } = 4 + 2l=
Denitions 1.1.3. Operations with complex numbers } = d+el and z = f+gl=
}z
}z
}@z
(d f) + (e g)l
(d + el)(f + gl) = (df eg) + (ef + dg)l and
d + el
d + el f gl
df + eg ef dg

+ 2
l=
=
= 2
f + gl
f + gl f gl
f + g2
f + g2
Example 1.1.3. Let } = 2 + l and z = 1 + 3l= Then

}z = 1 + 7l, }@z = 5@101@2 5@101@2 l and
2} 5z = (4 + 2l) (5 + 15l) = 1 13l=
Example 1.1.4. Let } = frv(2@3) + l sin(2@3)= Note the modulus is one so
that the angle between thes} vector and the real axis is 2@3= Compute } 3
} 3 = (1@2 + s
3@2 l)3
s
= (1@2 + s3@2 l)2 (1@2 +s 3@2 l)
= (1@2 3@2 l)(1@2 + 3@2 l)
= 1=
Example 1.1.5. The modulus can be computed as the square root of the
product of the complex number and the conjugate of the complex number
|}|2 = }} = (d + el)(d el) = d2 + e2 =
This is useful in expressing ratios of complex number in terms of real and
imaginary parts
1+l
3 + 2l
=
=
=
1 + l 3 2l
3 + 2l 3 2l
(3 + 2) + (3 2)l
32 + 22
5
1
+ l=
14 14
The zero complex number has a zero real and imaginary parts, and we
simply write 0 = 0 + l0= The unit complex number has real part equal to one,
imaginary part equal to zero, and we write 1 = 1+l0= The multiplicative inverse
of a non-zero complex number } = d + el is another complex number
} 1
=
=
1 d el
d + el d el
d
e
+ 2
l=
2
2
d +e
d + e2
The following eleven algebraic properties are easily established. The rst ve
properties are the additive group properties, the next ve are the multiplicative
group properties for non-zero complex numbers and the last is the distributive
property.
Theorem 1.1.1 (Field Properties of Complex Numbers) Let }> z> z1 > z2 5 C=
Then
} + z 5 C, } + 0 = }, } + (}) = 0, } + z = z + },
} + (z1 + z2 ) = (} + z1 ) + z2 ;
}z 5 C, }1 = }, }(} 1 ) = 1 when } 6= 0, }z = z},
}(z1 z2 ) = (}z1 )z2 ;
}(z1 + z2 ) = }z1 + }z2 =
Example 1.1.6. This example illustrates how one can solve complex algebraic
equations where we must solve for both the real and imaginary parts of an
unknown complex number } = { + l|= Here we use the above eld properties
1 + 4l
= 2+l
}+1
1 + 4l = (2 + l)(} + 1)
= (2 + l)} + (2 + l)=
Now solve for }
}
=
=
=
(1 + 4l) (2 + l)
(2 + l)
1 + 3l 2 l
2+l 2l
1
7
+l =
5
5
So, { = 1@5, | = 7@5, which is easy to verify

1 + 4l
(1@5 + (7@5)l) + 1
5 + 20l
6 + 7l
5 + 20l 6 7l
=
6 + 7l 6 7l
170 + 85l
=
85
= 2 + l=
The complex numbers are extensions of the real numbers, and many of
the algebraic and absolute value properties extend to the complex numbers.
For example, if d and f are real numbers and |d| is the absolute value, then
|df| = |d| |f|. When d and f are extended to complex numbers and absolute
value is extended to the modulus of a complex number, then one can show
Figure 1.1.2: Norm(} 2 ) and Angle(} 2 )
|}z| = |}| |z| = In order to see this is true, let } = d + el and z = f + gl and
show |}z|2 = |}|2 |z|2 :
}z
2
|}z|
= (d + el)(f + gl) = (df eg) + (ef + dg)l

= (df eg)2 + (ef + dg)2
= (df)2 2(df)(eg) + (eg)2 +
(ef)2 + 2(ef)(dg) + (dg)2
= d2 (f2 + g2 ) + e2 (f2 + g2 )
= d2 + e2 (f2 + g2 )
2
= |}| |z| =

2
If } = z, then we could write this as } 2 = |}|
Another property of the product }z is arg(}z) = arg(}) + arg(z), and
this with } = z is illustrated in Figure 1.1.2= Figure 1.1.2 was generated by
the MATLAB code complex_prod.m and command quiver.m to generate the
vectors. The proof follows from the trigonometric identities
cos( + *) = cos() cos(*) sin() sin(*) and
sin( + *) = sin() cos(*) + cos() sin(*)=
Let } = |}| (cos() + l sin()) and z = |z| (cos(*) + l sin(*)) so that

}z
= |}| (cos() + l sin()) |z| (cos(*) + l sin(*))

= |}| |z| (cos() cos(*) sin() sin(*)) +
l(sin() cos(*) + cos() sin(*))
= |}z| (cos( + *) + l sin( + *))=
Additional properties are summarized in the following theorem.

Theorem 1.1.2 (Norm Properties of Complex Numbers) Let }> z 5 C= Then
|}z| = |}| |z| > arg(}z) = arg(}) + arg(z)>
|} + z| |}| + |z| and
|}| = 0 if and only if } = 0 + l0=
1.1.5
Complex Operations Using MATLAB
MATLAB also uses i to represent the square root of minus one. The following
are some complex numbers and operations:
AA z = 1 + i*2
z=
1.0000 + 2.0000i
AA w = 2*z +1
w=
3.0000 + 4.0000i
AA z^2
ans =
-3.0000 + 4.0000i
AA w*z
ans =
-5.0000 +10.0000i
AA z/w
ans =
0.4400 + 0.0800i.
The MATLAB command quiver(x,y,dx,dy,s) produces a plot of a vector that
begins at the point (x,y) and ends at the point (x+dx,y+dy), see Figure 1.1.2.
The following graphs a vector representation of a complex number:
z=
1.0000 + 2.0000i
AA plot(z,*)
AA hold on
AA quiver(0,0,real(z),imag(z),1)
This produces a graph with the complex number marked by a *, and then
inserts an arrow from the origin to the complex number. The MATLAB code
complex_prod.m also illustrates this.
1.1.6
Exercises
1.
Consider the real number 1@3.

(a).
Find i o(1@3) with base 10 and three digits, g = 3.
2
2
(b).
Compute (i o(1@3)) and the error = (i o(1@3)) (1@3)2 =
1@2
2.
Let s be prime number. Show s
is not a rational number.
3.
Let } = 7 l and z = 2 + 3l=
(a).
Graph }, z, z and 2z + 3}=
(b).
Compute }z and }@z=
4.
Let } = 2 + 3l and z = 2 3l=
(a).
Graph }, z, z and 2z + 3}=
(b).
Compute }z and }@z=
5.
For } = cos(2@3) + l sin(2@3) compute 1 + } + } 2 =
6.
For } = cos(2@4) + l sin(2@4) compute 1 + } + } 2 + } 3 =
7.
Use the properties in Theorem 1.1.1 to solve for the complex number
} = { + |l
} + 7 + 2l = 3 + 4l=
8.
9.
Prove the remaining properties in Theorem 1.1.1.

Find the real and imaginary parts of
10.
2+l
=
3 7l
Find the real and imaginary parts of
11.
2+l
+ 5=
10 + 7l
Solve for the complex number } = { + |l
12.
2 + 4l
= 7 + l=
}+l
Solve for the complex number } = { + |l
2 + 4l
= 1 + l=
2} l
13.
Prove the remaining properties in Theorem 1.1.2.
14.
Does the quadratic formula hold when the coe!cients d> e and f are
complex numbers? You may wish to consider some special cases such as d =
1> e = 0 and f = l= One needs to be able to dene the square root of a complex
number!
15.
Use MATLAB and enter } = 2 + 3l and z = 2 3l=
(a).
Graph } and z=
(b).
Compute }z and }@z=
(c).
Use quiver() to plot the vectors for } and z=
16.
Consider the calculations
in Figure 1.1.2 given by the MATLAB code
s
complex_prod.m. Let } = 3 + 1l=
(a).
Use by-hand calculations to nd } 2 , } 3 , their norms and arguments.
(b).
Modify complex_prod.m to conrm your by-hand calculations.
10
1.2
Complex Valued Functions
Mappings from the complex numbers into the complex numbers can be viewed
as functions from the plane into the plane. Several examples such as d} + e or
} 2 were introduced in the previous section and here additional functions will
be described. An important objective is to develop the Euler formula for the
exponential function of an imaginary number
hl| = cos(|) + l sin(|)=
This formula is extremely useful and has applications to trigonometric identities, the solution of algebraic and dierential equations, circuits and signal
processing.
1.2.1
Linear and Power Functions
The linear complex function is

i (}) = d} + e=
The constants d and e are given complex numbers, and } = {+l| is any complex
number where { and | are real numbers. This means
i (}) = x({> |) + ly({> |) where
the functions x({> |) and y({> |) have real values. The solution of i (}) = g =
g1 + lg2 is equivalent to equating the real and imaginary parts and reduces to
solving two real equations for { and |
x({> |) = g1 and y({> |) = g2 =
(1.2.1)
Example 1.2.1. Consider solving i (}) = (1 + l)} + 3 l = 7l= Let } = { + l|

so that
i (}) = (1 + l)({ + l|) + 3 l
= ({ | + 3) + l({ + | 1)=
Thus, x({> |) = { | + 3 and y({> |) = { + | 1= In order to solve i (}) = 7l =
0 + 7l> equate the real and imaginary parts as in equation (1.2.1)
{ | + 3 = 0 and { + | 1 = 7=
This algebraic system can be solved by adding the two equations 2{ + 2 = 7 so
that { = 5@2 and | = 11@2= Thus, the solution of i (}) = (1 + l)} + 3 l = 7l
is } = 5@2 + (11@2)l= The solution can also be found by using the properties in
Theorem 1.1.1
}=
ge
7l (3 l)
3 + 8l
3 + 8l 1 l
5 + 11l
=
=
=
=
=
d
1+l
1+l
1+l 1l
2
1.2. COMPLEX VALUED FUNCTIONS
11
The linear rational complex function is

i (}) =
d} + e
f} + g
where constants d> e> f and g are complex numbers.

Example 1.2.2. Let d = 1> e = l> f = 1 and g = l giving
i (}) =
}+l
=
}l
This function can be represented by two real valued functions x({> |) and y({> |)
i (}) =
=
=
=
=
}+l
}l
({ + l|) + l
({ + l|) l
{ + l(| + 1) { l(| 1)
{ + l(| 1) { l(| 1)
{2 + | 2 1
2{
+l 2
2
2
{ + (| 1)
{ + (| 1)2
x({> |) + ly({> |)=
The solution of i (}) = 2 + l can be found by equating the real and imaginary
parts x({> |) = 2, y({> |) = 1 and solving for { and |= Or, one can use the
properties in Theorem 1.1.1 to nd } as follows:
}+l
}l
}+l
}(1 (2 + l))
}(1 l)
}
= 2+l
= (2 + l)(} l) = (2 + l)} (2 + l)l
= (2 + l)l l
= 3l + 1
1 + 3l 1 l
2 + 4l
=
=
= 1 + 2l=
1+l 1l
2
A quadratic complex function is

i (}) = d} 2 + e} + f=
The coe!cients d, e and f are given complex numbers, and one would like to
solve i (}) = d} 2 + e} + f = 0= In order to generalize the quadratic formula
to the complex case, the square root of a complex number needs to be dened.
Since the square root function is the inverse of the square function, let us focus
on
i (}) = } 2 =
12
In Section 1.1 we used the polar representation } = |}| (cos() + l sin()) and
Theorem 1.1.2 with } = z to obtain
} 2 = |}|2 (cos(2) + l sin(2))=
This suggests that the square root of complex number should be
} 1@2 |}|1@2 (cos(@2) + l sin(@2))=
A complex square root function i (}) =
values so that its square is }
s
( })2
{ + l|
(1.2.2)
s
} = x + ly must have complex
= (x + ly)2
= (x2 y 2 ) + l2xy=
Thus, one must choose x and y such that { = x2 y 2 and | = 2xy= The use of
trigonometric identities conrms
1@2
x = |}|
1@2
cos(@2) and y = |}|
sin(@2)
are the solution of these two equations and, hence, equation (1.2.2) is correct
for the denition of square root of }=
s
Example 1.2.3. Use equation (1.2.2) to nd the square root of } = 1 + 3l.
The modulus of } is 2 and the angle is @3 or 60 degrees so that } = 2(cos(@3)+
l sin(@3))= Equation (1.2.2) gives
(1 +
s 1@2
3l)
= 21@2 (cos(@6) + l sin(@6))

s s
=
2( 3@2 + l(1@2))
p
s
3@2 + l(1@ 2)=
=
Note the solution of the complex quadratic equation } 2 = 1 +

minus this square root.
s
3l is plus or
Figure 1.2.1 illustrates the linear, square and square root functions of a
complex variable; this was generated by the MATLAB code complex_power.m.
The complex numbers in the domain correspond to the domain semicircle with
radius equal to three. The linear function has values in the complex plane where
the domain semicircle has been doubled in radius and shifted to the right. The
square function has values in the complex plane where the domain semicircle
has been expanded in both the radial direction and in an angular direction to
one complete circle of radius equal to three squared. The smallest curve depicts
the square root function. Now, the domain semicircle has been contracted in
both the radial and angular components to one half of the semicircle and with
radius equal to the square root of three.
13
Figure 1.2.1: A!ne, Square and Square Root of z
1.2.2
Exponential Function e}
The exponential function of a complex variable } = { + l| at rst look is

h} = h{+l| = h{ hl| . But, what is the meaning of hl| ? Find functions x(|) and
y(|) such that
hl| = x(|) + ly(|)
has properties similar to the exponential function of a real variable. Two properties that one would like to hold are
hl0
g l|
h
g|
= 1 and
(1.2.3)
= lhl| =
(1.2.4)
By equating the real and imaginary parts in equations (1.2.3) and (1.2.4) we
have
x(0) = 1 and y(0) = 0 and
gx
gy
= y and
= x=
g|
g|
(1.2.5)
(1.2.6)
Note the x(|) = cos(|) and y(|) = sin(|) satisfy these equations.
In order to justify using the exponential function to represent this complex
valued function, use the power series expansions, which are studied in the second
14
semester of calculus,
h{
{2
{3 {4 {5
+
+
+
+ >
2!
3!
4!
5!
4
{
+
and
4!
{5
+
=
5!
= 1+{+
{2
2!
{3
sin({) = {
3!
cos({) = 1
Extend the exponential functions domain from the real numbers to the complex
numbers by replacing { by l|
hl|
(l|)2 (l|)3 (l|)4 (l|)5

+
+
+
+
2!
3!
4!
5!
|3 |4
|5
|2
l +
+ l +
= 1 + l|
2!
3!
4!
5!
|4
|3 |5
|2
+
] + l[|
+
]
= [1
2!
4!
3!
5!
= cos(|) + l sin(|)=
= 1 + l| +
Denition 1.2.1. Eulers Formula is

hl| = cos(|) + l sin(|)=
The general complex exponential function is

i (}) = h} = h{+l| = h{ hl| = h{ (cos(|) + l sin(|))=
Also, any complex variable } = { + l| can be represented by using hl =
cos()+l sin() where = arctan(|@{)= This gives the exponential representation
of a complex variable
}
1.2.3
= { + l|
= |}| (cos() + l sin ())
= |}| hl =
Application to Trigonometric Identities
Four consequences of Eulers formula are presented. First, since the cosine is
an even function and the sine function is an odd function,
hl{
= cos({) + l sin({)
= cos({) l sin({)
= hl{ =
15
Second, this leads to the representation of the cosine and sine functions
cos({) = (hl{ + hl{ )@2 and
sin({) = (hl{ hl{ )@(2l).
(1.2.7)
(1.2.8)
Third, consider the product of hl{ and hl| and use the trigonometric identities
for the sum of angles
hl{ hl|
= (cos({) + l sin({))(cos(|) + l sin(|))

= [cos({) cos(|) sin({) sin(|)]
+l[sin({) cos(|) + cos({) sin(|)]
= cos({ + |) + l sin({ + |)
= hl({+|) =
Fourth, another identity, which follows from the repeated use of hl{ hl| = hl({+|)
with | = {> is de Moivres formula
(cos({) + l sin({))q
= (hl{ )q
= hl{q
= cos(q{) + l sin(q{)=
(1.2.9)
This formula for dierent choice of q leads to a variety of trigonometric identities. Equation (1.2.9) can be used to reduce computations with large exponents.
Example 1.2.4. Use de Moivres formula to evaluate (3 + 4l)10 . First, write
3 + 4l in exponential form 3 + 4l = 5hl where = tan1 (4@3)= Second, use
equation (1.2.9) with q = 10 on the second factor
(3 + 4l)10
1.2.4
=
=
=
=
=
(5hl )10
510 (hl )10
510 hl10
510 (cos(10) + l sin(10))
510 (0=9885 + 0=1512l)=
Application to Solution of } q = g
Consider the case q = 2 where the solution of } 2 = g is plus or minus the square
root of g= Let g = |g| hl and
1@2 l@2
(|g| hl )1@2 = |g|
The second solution is |g|1@2 hl@2 = |g|1@2 hl@2 hl2@2 =

There should be q solutions of } q = 1= In order to discover these, represent
one by 1 = hl2 and, hence,
11@q = (hl2 )1@q = hl2@q =
16
This leads to all q solutions } = (hl2@q )m = hl(2@q)m where m = 0> 1> ===> q 1=
They are solutions because
}q
=
=
=
=
(hl(2@q)m )q
hl(2)m
cos((2)m) + l sin((2)m)
1=
Denition 1.2.2. Let m = 0> 1> > q1= Then hl(2@q)m are solutions to } q = 1
and are called the qwk roots of unity.
Example 1.2.5. Let q = 3 and nd the three solutions of } 3 = 1= Here
2@q = 2@3 or 120 degrees. The three solutions are
hl(2@3)0
hl(2@3)1
hl(2@3)2
= cos((2@3)0) + l sin((2@3)0) = 1
= cos((2@3)1) + l sin((2@3)1) = 0=5000 + 0=8660l
= cos((2@3)2) + l sin((2@3)2) = 0=5000 0=8660l=
Another special case with q = 12 is illustrated in Figure 1.2.2 where the 12

vectors with radius equal to one are the solutions. The other 12 vectors have
increasing moduli and have staggered angles
(1=05h(l2(1+=05)@q) )m =
The gure was generated by the MATLAB code euler_form.m.
The solution of the more general problem } q = g is straightforward once
the complex number g is written in polar form
g = |g| hl = |g| hl hl2
}
1@q
= g1@q = |g|
(hl hl2 )1@q
= |g|1@q hl@q hl2@q =

l2@q
(1.2.10)
m q
Dene z h
and use the fact (z ) = 1 to obtain all q solutions } =
g1@q zm where m = 0> 1> ===> q1= A summary of the above is given in the following
theorem.
Theorem 1.2.1 (Solutions of } q = g)= Let z hl2@q and g = |g| hl = Then
z satises z = z1 > zq = 1 and 1+z+ +zq1 = 0= Moreover, the solutions
1@q l@q m
h
z = g1@q zm where m = 0> 1> > q 1=
of } q = g are given by } = |g|
Example 1.2.6. Let q = 3 and nd the three solutions of } 3 = 1 + l= Here
2@q = 2@3 and g = 1 + l = 21@2 hl@4 = The three solutions are
21@6 hl@12 hl(2@3)0
21@6 hl@12 hl(2@3)1
21@6 hl@12 hl(2@3)2
=
=
=
=
=
=
21@6 (cos(@12 + (2@3)0) + l sin(@12 + (2@3)0))

1=0842 + 0=2905l
21@6 (cos(@12 + (2@3)1) + l sin(@12 + (2@3)1))
0=7937 + 0=7937l
21@6 (cos(@12 + (2@3)2) + l sin(@12 + (2@3)2))
0=2905 1=0842l=
17
Figure 1.2.2: Solutions of } 12 = 1
1.2.5
Complex Functions Using MATLAB
Complex functions are evaluated using MATLAB as are real functions. The
computations in Example 1.2.6 are easy to do. The second solution is
AA a = exp(i*2*pi/3)
a=
-0.5000 + 0.8660i
AA b = 2^(1/6)*exp(i*pi/12)
b=
1.0842 + 0.2905i
AA a*b
ans =
-0.7937 + 0.7937i
AA z = a*b
z=
-0.7937 + 0.7937i
AA z^3
ans =
1.0000 + 1.0000i.
The MATLAB code complex_power.m, see Figure 1.2.1, uses an implied loop
to generate a row vector of values for the parameter w= Then a row vector of
complex values is computed to form the semicircle, which is to be the domain
of three functions. Array operations given by }=s are used to compute the
18
square and square root functions. An array operation on a row vector produces
another row vector whose components are given by the operation. For example,
if d = [2 3]> then d=2 = [22 32 ]=
The MATLAB code euler_form.m, see Figure 1.2.2, uses a for-loop to generate a sequence of vectors associated with the complex exponential function
raised to a variety of powers. The vectors are created by the MATLAB command quiver(x,y,dx,dy,1), which graphs a vector beginning at the point ({> |)
and ending at ({ + g{> | + g|).
1.2.6
Exercises
Let i (}) = (2 + l)} + 7 + l where } = { + l|

(a).
Find x({> |) and y({> |) such that i (}) = x + ly=
(b).
Find the solution of i (}) = 10 2l=
2.
Let i (}) = (3 + l)} + 1 + 2l where } = { + l|
(a).
(b).
3.
Let i (}) = (} + 2l)@(} l) where } = { + l|
(a).
(b).
4.
Let i (}) = (2} + l)@(} + l) where } = { + l|
(a).
(b).
Find the solution of i (}) = 1 + 2l=
5.
Use equation (1.2.2) to nd the square root of 1 + l= s
6.
Use equation (1.2.2) to nd the square root of 1 + 3l=
7.
Use the complex quadratic formula to solve d} 2 + e} + f = 0=
(a).
Solve the quadratic equation } 2 + 2} l = 0=
(b).
Check your answer.
8.
Consider the completing square method of developing the quadratic formula.
(a).
Prove: if } and z are complex numbers and }z = 0> then either
} = 0 or z = 0=
(b).
Use this fact to derive the quadratic formula for complex numbers.
9.
Use de Moivres formula in equation (1.2.9) to compute (1 + l)10 =
10.
Use de Moivres formula in equation (1.2.9) to compute (3 + 2l)7 =
11.
Use Theorem 1.2.1 to nd all three solutions to } 3 = 3 + 4l=
s
12.
Use Theorem 1.2.1 to nd all four solutions to } 4 = 1 + 3l=
13.
Let z hl2@q = Show z satises z = z1 > zq = 1 and 1 + z + +
q1
z
= 0=
14.
Use MATLAB to verify the third equation in Example 1.2.6.
15.
Use MATLAB to experiment with dierent size semicircles in the code
complex_power.m.
16.
Use MATLAB to experiment with powers q = 3> 6 and 12 in the code
euler_form.m.
1.
1.3. VECTORS IN R2
1.3
19
Vectors in R2
Complex numbers are represented either by ordered pairs of real numbers or as

directed line segments. The rst number in the pair is the real part and the
second number is the imaginary part. The directed line segment or arrow starts
at the origin and ends at the complex point. There are a number of important
similar examples such as displacement in the plane of a mass, force acting on
a mass as well as the velocity and acceleration of a mass. As in the complex
numbers, these quantities can be added, subtracted and scaled, and have length
and angles.
1.3.1
Vector Denition and Notations
$

$
There are a number of notations of vectors. One is the l > m unit vectors often
used in physics textbooks. Other notations list the components in either a row
or a column. There is also some confusion about the dierence between a point
in a plane and a vector in a plane; often the notation is the same! Part of the
problem is the need to have vectors with more than two or three components,
which will be considered after the rst two chapters.
$
Denition 1.3.1. A vector in R2 is an ordered list of two real numbers
d =
[d1 d2 ]. One can visualize this by forming the directed line segment from the
origin point (0> 0) to the point (d1 > d2 )=
Notation. Points in the plane R2 will be denoted by (d1 > d2 )> and vectors will
be considered as either row or column vectors:

$
d = [d1 d2 ] denotes a row vector >

d1
a =
denotes a column vector =
d2
$
d=
aW = [d1 d2 ] is called the transpose of the column vector a so that aW =
Example 1.3.1. The standard unit basis vectors are

$

1
and
l = [1 0] or e1 =
0

$

0
m = [0 1] or e2 =
=
1
Any vector can be represented as a linear combination of unit vectors. For
$
example, if
d = [2 3], then either by row vectors
$

$

$
d = 2 l + 3 m or by column vectors
a = 2e1 + 3e2 =
These vectors are illustrated in Figure 1.3.1.
20
Figure 1.3.1: A Vector in the Plane

Movement of a mass in a plane involves displacement, velocity and force
vectors. If one moves a mass from the origin to a point (g1 > g2 )> then there is
displacement in both the horizontal and the vertical directions. This requires
two numbers, and we agree to list the horizontal term rst so that

$
g = [g1 g2 ] = displacement vector.
In order to move an object, a force must be exerted on the mass, and this
will also have horizontal and vertical components. By adjusting the size of the
components, the direction and magnitude of the force can be determined to
obtain the desired motion. Let i1 be the force in the horizontal direction, i2
be the vertical force and denote the force vector as

$
i = [i1 i2 ] = force vector.
The resulting motion will have speeds in the horizontal and vertical directions.
The velocity vector lists these speeds so that the velocity vector also has a
direction and magnitude. The speed in the horizontal direction is y1 and when
this is coupled with the vertical speed y2 we get a velocity vector

$
y = [y1 y2 ] = velocity vector.
Newtons law of motion can be applied in both the horizontal and vertical
directions
gy1
gy2
i1 = p
and i2 = p
=
gw
gw
1.3. VECTORS IN R2
21
Figure 1.3.2: f2 = d2 + e2
The vector notation for Newtons law of motion is
$

i = [i1 i2 ]
gy1
gy2
= [p
p
]
gw
gw
$

= p d where
gy1 gy2
$

d = [
] = acceleration vector.
gw gw
As in complex vectors there are operations with vectors such as addition,
length and angles. Two very useful theorems are related to the length of a
side of a triangle, which is opposite one of the angles. When the angle is 90
degrees, the Pythagorean theorem gives a very useful relationship between the
three sides. There are number of proofs for the Pythagorean theorem, but the
following is a very nifty proof that is based on geometry. Consider Figure 1.3.2
where the right triangle has sides of length d> e and f with f being opposite the
90 degree angle. The gure is drawn with d larger than e. The area of the big
square is f squared, and it is the union of four identical triangles and a small
square
f2
= 4(de@2) + (d e)2
= 2de + d2 2de + e2
= d2 + e2 =
Theorem 1.3.1 (Pythagorean) Consider a right triangle with sides a, b and

c. If c is opposite the right angle, then
f2 = d2 + e2 =
22
Figure 1.3.3: f2 = e2 + d2 2de cos()

An important consequence of the above, where sin() = e@f and cos() =
d@f> is the trigonometric identity
1 = sin2 () + cos2 ()=
Another result pertains to the case when the angle is not 90 degrees as is
illustrated in the triangle in Figure 1.3.3 where the triangle with sides d> e and
f is a union of two right triangles. The left right triangle has base equal to
e cos() and height equal to e sin()= The other right triangle has the same
height, but the base is d e cos() (assuming d A e)= Therefore, applying the
Pythagorean theorem to the right triangle on the right side gives
f2
=
=
=
=
(e sin())2 + (d e cos())2
e2 sin2 () + d2 2de cos() + e2 cos2 ()
e2 (sin2 () + cos2 ()) + d2 2de cos()
e2 + d2 2de cos()=
Theorem 1.3.2 (Cosine Law) Consider a triangle with sides a, b and c. If c

is opposite the angle , then
f2 = e2 + d2 2de cos()=
1.3.2
Vector Operations
Vectors can be scaled, added and subtracted by either viewing the vectors
geometrically or as symbolic objects. Figure 1.3.4 illustrates vector scaling by
$

$

$
either elongating or shrinking the vector e . Two vectors
d and e can be
$

added by moving vector e parallel to its original position so that its beginning
$
point is the end point of vector
d . The addition corresponds to the diagonal
$

$

$
$
of the parallelogram formed by vectors
d and e . If
d is added to e , then
$

$
the subtraction is the other diagonal from the end of
d to the end of e =
1.3. VECTORS IN R2
23
$ $

$
$

Figure 1.3.4: $
d + e >
d e and v e
$

$

$
Denitions 1.3.2. Let
d = [d1 d2 ], e = [e1 e2 ] and 0 = [0 0] be the zero
$

$
vector.
d equals e means both d1 = e1 and d2 = e2 = The addition of two
vectors is another vector whose components are the sum of the components of
the two vectors
$

$

d + e [ d1 + e1 d2 + e2 ]=
Let v be a real number. The scalar product of a vector is another vector whose
components are the scalar times the components of the vector
$
v
d [ vd1
vd2 ]=
These operations are similar to those for complex numbers. However, it is

important to note that vector products, which could be similar to complex products, are not dened. There are special vector products called "dot", "cross",
"box" and "array" products, which will be introduced in the subsequent sections. The following theorem lists some basic algebraic properties of the above
operations.
$

$
f be vectors and
Theorem 1.3.3 (Vector Space Properties) Let
d > e and $
let v and w be real numbers. Then the following rules hold
$

$
d + e
$

$

d + 0
$
(v + w)
d
$

$

v( d + e )

$
$

$ $

$
$
$
e +$
d > (
d + e )+
f =
d +( e +
f )>
$

$

$

$

= d and d + ( d ) = 0 ;
$
$
$
$
$
$
= v
d + w
d , v(w
d ) = (vw)
d > 1
d =
d and
$

$

= vd +ve =
=
24
$

$
$
$
The proofs are straightforward. The proof of
d + e = e +
d uses the
commutative property for addition of real numbers
$

$
d + e = [ d1 + e1
= [ e1 + d1
$ $

= e +
d=
d2 + e2 ]
e2 + d2 ]

$
$

$

Example 1.3.2. Let
d = [2 4], e = [1 2] and v = 3= Then $
d + e = [1
$

$

$
6]>
d e = [3 2] and v e = [3 6]=
Example 1.3.3. One can use the above basic properties to solve vector equa$

$
$
d and e are given
tions for an unknown vector
{ = [{1 {2 ]= Suppose vectors
$

$
$
$
and
{ must satisfy the vector equation
d +
{ = e . Solve this by adding the
$

vector d to both sides and use the algebraic rules
$ $

$
$
$
(
d +
{)
d = e
d

$
$

$

$

$
({ + d) d = e d
$ $

$

$
$
{ + (
d
d) = e
d
$

$

$
{ = e d=

Denitions 1.3.3. Let $
d = [d1 d2 ]. The length or Euclidean norm of the
$

vector d is given by the Pythagorean theorem
q
$
k
d k d21 + d22 =
The angle of the vector is
= arctan(d2 @d1 )=

The unit vector in the direction of $
d is

$
d
1
$
d=
= $
$

kdk
kdk
The cosine law may be rewritten in terms of the norms of the vectors
$

$

$

$
$
$
d > e >$
f =
d e and the angle between the vectors
d and e
$

2
$2
$
2
$
$
d e = k
d k + e 2 k
d k e cos()=
$
This allows one to compute the angle between two given vectors. In fact, this
can be simplied to easily compute cos()> sin() and the area of the parallelogram formed by two vectors. In Section 1.4 this will be applied to the
computation of work and torque.
1.3. VECTORS IN R2
25
$

$

$

Example 1.3.4. If
d = [2 4] and e = [1 2]> then $
d e = [3 2]>
$2

$2

2
$
d e = 13> k
d k = 20> e = 5 so that
$
s s
13 = 20 + 5 2 20 5 cos()
= 20 + 5 20 cos()
cos() = 12@20 = 3@5
= cos1 (3@5) 53=13 degrees.
$
The unit vector associated with
d is

$
s
s
[2 4]
d
= s = [1@ 5 2@ 5]=
$

kdk
2 5
1.3.3
Application to Navigation of an Aircraft
A pilot wishes to y northeast with a speed of 400 miles per hour. The means
the desired velocity vector has length equal to 400 and the speed in the east
$
and north directions are equal. Let this desired velocity be the vector
y = [y1
$

y2 ] where k y k = 400 and y1 = y2 A 0
y12 + y12 = 4002 =
s
Thus, y1 = y2 = 400@ 2= An added di!culty is the wind velocity, which is
coming from the northwest and has a speed equal to 100 miles per hour. Let
$

$
z = [z1 z2 ] where k
z k = 100 and z1 = z2 A 0
z12 + z12 = 1002 =
s
$

Thus, z1 = z2 = 100@ 2= What should the heading velocity k = [k1 k2 ]
be so that the desired velocity is attained? The aircraft must be pointed north
$
of northeast and have a speed larger than 400. The desired velocity
y is the
$

$

resultant of the wind velocity z and the heading velocity k > which means
$

$

$
y =
z + k=
By the basic rules for vector operations
$

$
$
k =
y
z
s
s
s
s
[k1 k2 ] = [400@ 2 400@ 2] [100@ 2 100@ 2]
s
s
= [300@ 2 500@ 2]=
The speed of the aircraft should be

q
s
s
s
$
k = (300@ 2)2 + (500@ 2)2 = 100 18 424=26>
and the angle from the east direction should be
s
s
arctan((500@ 2)@(300@ 2)) = arctan(5@3) 59=04 degrees=
26
1.3.4
Vectors Using MATLAB
The vector notation in MATLAB is very similar to what has been used in this
text and uses square brackets to enclose both row and column vectors. The
overhead arrow and bold notations are not used to distinguish row and column
vectors. A row vector such as d = [2 5] is converted to a column vector by
using a prime, [2 5].
AA a = [2 5]
a=
25
AA e1 =[1 0]
e1 =
1
0
AA e2 = [0 1]
e2 =
0
1
AA aa = a(1)*e1 + a(2)*e2
aa =
2
5
AA b = [-1 5]
b=
-1 5
AA c = 2*a + 3*b
c=
1 25
The MATLAB command quiver(x,y,dx,dy,s) can be used to graph vectors
starting at the point ({> |) and ending at the point ({ + g{> | + g|) and using
$
v = 1= The following generates a graph of the two vectors
d = [2 5] and
$

e = [1 3].
AA quiver(0,0,2,5,1)
AA hold on
AA quiver(0,0,-1,3,1)
1.3.5
Exercises
$

$
$
1.
Consider the vector
d = [2 5]= Write it in l m and column forms.
$

$

$
2.
Consider the vector
d = [1 3]= Write it in l m and column forms.
$

$

$ $
$
$

$
$
$
3.
Let
d = [2 5] and e = [1 3]. Graph
d > e > e >
d + e and
d e=
$

$

$

$

$
$
$
4.
Let
d = [2 3] and e = [1 5]. Graph 2
d > e > e >
d + e and
$

$

d 2e =
1.4. DOT PRODUCT AND WORK
27
$

$

$
$
$
5.
Let
d = [2 5] and e = [1 3]. Solve
d +
{ = e=
$

$ $
$
$
6.
Let
d = [1 7] and e = [1 3]. Solve 2
d = e +
{=
$

$ $

$
$
$
7.
Consider Theorem 1.3.3. Prove (
d + e )+
f =
d +( e +
f )=
$

$

$

$

8.
Consider Theorem 1.3.3. Prove v( d + e ) = v d + v e =
$
$
$
d = [1 7]. Compute k
d k, the angle and unit vector of
d=
9.
Let
$

$

$
10.
Let d = [1 3]. Compute k d k, the angle and unit vector of
d=
$

$

11.
Let d = [2 5] and e = [1 3]. Graph these vectors and use the cosine
law to nd the angle between them.
$

$
12.
Let
d = [1 2] and e = [1 3]. Graph these vectors and use the
cosine law to nd the angle between them.
13.
Consider the aircraft heading problem. If the desired velocity is to the
northeast with speed 500 and the wind velocity has speed 120 and from the
north, nd the heading velocity.
14.
Consider the aircraft heading problem. If the desired velocity is to the
northwest with speed 500 and the wind velocity has speed 120 and from the
east, nd the heading velocity.
$

$ $
$
$

$
$
$
d > e > e >
d + e and
d 2 e where
d = [2
15.
Use MATLAB to compute 2

$
3] and e = [1 5]=
$

$

$
$
d and e where
d = [2 3] and e = [1 5]=
16.
Use MATLAB to graph
1.4
Dot Product and Work
In this section we continue the study of the parallelogram that is formed by

$

$
two vectors
d and e in the plane. The vector version of the cosine law allows
$

$
one to compute cos() where is the angle between the vectors
d and e = This
calculation is important because it leads to a number of applications such as
an easy derivation of trigonometric identities, area of the parallelogram and
calculation of work and torque in 2D and 3D space.
1.4.1
Cosine Identity and Dot Product
$

$
Let
d = [d1 d2 ] and e = [e1 e2 ] have the angle between the vectors. The
vector version of the cosine law is
$

2
$2
$
2
$
$
d e = k
d k + e 2 k
d k e cos()=
(1.4.1)
$
By denition of vector subtraction and the norm
$

2
d e
$
= (d1 e1 )2 + (d2 e2 )2
= d21 2d1 e1 + e21 + d22 2d2 e2 + e22
$2
2
$
= k
d k + e 2(d1 e1 + d2 e2 )=
28
Place this into the left side of equation (1.4.1) and cancel the norms
$2
$2
$
2
2
$
$
$
k
d k + e 2(d1 e1 + d2 e2 ) = k
d k + e 2 k
d k e cos()
$
$
2(d1 e1 + d2 e2 ) = 2 k
d k e cos()
$
$
d1 e1 + d2 e2 = k
d k e cos()=
(1.4.2)
Equation (1.4.2) is of fundamental importance. The left side is an easy calculation, and the right side makes a connection with the geometry. At the end of
this section an application will be given to work being done in a constant force
eld, which is independent of three possible paths.
$

$
Theorem 1.4.1 (Cosine Identity) Let
d = [d1 d2 ] and e = [e1 e2 ] be nonzero vectors and let be the angle between the two vectors. Then the cos() can
be computed by
d1 e1 + d2 e2
cos() =
$ =
$
k
d k e
$

$
Denition 1.4.1. The dot product of two vectors
d = [d1 d2 ] and e = [e1 e2 ]
is given by the real number on either side of equation (1.4.2). The following are
common notations for the dot product of row or column vectors, respectively,
$

$
$
$

d k e cos() or
d e d1 e1 + d2 e2 = k
aW b d1 e1 + d2 e2 = kak kbk cos()=
The cosine identity can be restated as either
cos() =
$

$
d e
$ or
$
k
d k e
cos() =
aW b
.
kak kbk
Also, the norm of a vector can be written as either

2
$
k
dk
kak2

$
= $
d
d or
= aW a=
Since 1 cos() 1> the inequality, which is called the Cauchy-Schwarz

inequality, is
$

$
$
d e k
d k e or
$
W
a b kak kbk =
29
$

$
d and e be non-zero vectors. Two vectors are parallel
if and only if they are multiples of each other, that is, there is a real number
$

$
v such that
d = v e = Two vectors are perpendicular (also called orthogonal) if
$

$
and only if the cos() is zero, that is,
d e = aW b = 0=
$

2
$
$
Example 1.4.1. Let
d = [2 3] and e = [1 2]. Then k
d k = 22 + 32 = 13>
$

$2
$
d e = 2(1) + 3(2) = 4=
e = (1)2 + 22 = 5> and
$

$
d e
4
4
s s
s
cos() =
$ = 13 5 = 65 and
$

k d k e
4
1
s
60=26 degrees.
= cos
65
Example 1.4.2. Consider the following three vectors written as column vectors

2
22
6
a=
>b =
and c =
=
3
33
4
Vectors a and b are parallel because a =(1@11)b= Vector a is perpendicular to
c because the dot product aW c = 2(6) + 3(4) = 0= Also, b and c must be
perpendicular. Consider a vector d whose rst component is 2, but the second
component is unknown. Choose the second component so that the vector d
is perpendicular to vector a= Let the vector d be represented by its transpose
dW = [2 {]= The vector d being perpendicular to a requires their dot product
to be zero

2
2 3
aW d =
{
= 2(2) + 3{ = 0=
Thus, { = 4@3 and dW = [2 4@3].
$

Denition 1.4.3. The projection
of
e onto $
d is a vector with direction the
$
$

same as d and length equal to e cos()

$
$

$
$
d
d
$
e cos() = $
$

2 d e=

kdk
kdk
The dot product has a number of interesting algebraic properties. One is
the order of the vectors does not change the value of the dot product.
$

$

d e = d1 e1 + d2 e2
= e1 d1 + e2 d2
$ $

= e
d=
Additional dot product properties listed in the following theorem are also easy
to prove.
30
$ $

$
Theorem 1.4.2 (Properties of Dot Product) Let
d > e >
f be given vectors
and let v be a real number. The dot product properties are
$

$ $

$

$

$
$
$
d e = e
d > v(
d e ) = (v
d) e >
$ $

$ $

$

d ( e +
f) = $
d e +
d $
f and
$

$

$
$

d e k d k e (Cauchy-Schwarz inequality).
Since the norm of a vector can be written in terms of the dot product
2
$
$
$
d
d>
k
dk =
several important properties of a norm can be derived from the above dot
product properties. The following norm properties are similar to the absolute
value of a real number.
$

$
Theorem 1.4.3 (Properties of Norm) Let
d and e be given vectors and let
v be a real number. The norm properties are
$

$
$
$
k
d k 0; k
d k 0 if and only if
d = 0>
$
$
kv
d k = |v| k
d k and
$

$
$

$

d + e k d k + e (triangle inequality).
The triangle inequality follows from the dot product properties
$
$

$

2

$
$
d + e ) (
d + e)
d + e = (
$
$

$ $

$
$
$
=
d (
d + e ) + e (
d + e)
$

$ $
$
$
$
$
$
=
d
d +
d e + e
d + e e
$

$
2

2
$
$
= k
d k + 2
d e +e
$
$
2
$

2
$
k
d k + 2
d e + e =
Next apply the Cauchy-Schwarz inequality to get

$2
$

2
$

2
$

d k + 2 k$
d k e + e
d + e k
$
$
$
= (k
d k + e )2 =
1.4.2
Applications to Angles and Areas
The dot product version of the cosine identity can be used to derive the formula
for the cosine of the sum of two angles
cos( + !) = cos() cos(!) sin() sin(!)=
31
Figure 1.4.1: Trigonometric Identity and Dot Product
Choose
length.
angles.
and let
that
two vectors as illustrated in Figure 1.4.1 where both vectors have unit
Then the rst and second components are the cosine and sine of their
$

$
Let the vectors
d and e have angles equal to and !> respectively,
$
$
the vector
f be the reection vector
d about the horizontal axis so

$
d = [cos() sin()]
$

e = [cos(!) sin(!)] and
$

f = [cos(!) sin(!)]=
$
$
The angle between vectors
d and
f is (!) = + ! and, therefore, by
$
$
application of the cosine identity to the vectors
d and
f
cos( + !) =
=
$

$
d
f

$
$
k d k k
fk
cos() cos(!) + sin()( sin(!))
=
1(1)
Another application of the dot product is to the computation of the area in

a parallelogram formed by two vectors, which is depicted in Figure 1.4.2. Here
$
$
the base of the parallelogram is the length of the vector
d > k
d k, and the height
32
$
of the parallelogram is e sin() and, thus,
(duhd)2
$
$
= (k
d k e sin())2
$
$
= (k
d k e )2 (1 cos2 ())=
Use the cosine identity for cos() to obtain

(duhd)
$

$
d e 2
$ 2

$
= (k d k e ) (1 (

$
) )
$
k
d k e
$2
$
2
$
$
d e )2
= k
d k e (
= (d21 + d22 )(e21 + e22 ) (d1 e1 + d2 e2 )2
= (d1 e2 e1 d2 )2 =
(1.4.3)
This formula may also be expressed using the determinant of a 2 2 matrix

formed by the column vectors that dene the parallelogram

d1 e1
det([a b]) = det(
) d1 e2 e1 d2 =
d2 e2
Then equation (1.4.3) is
(duhd)2 = (det([a b]))2 =
Theorem 1.4.4 (Area Equivalence) Let two non-zero vectors be represented

as column vectors a and b= The area of the parallelogram formed by these vectors
being non-zero is equivalent to the vectors not being parallel. In symbolic terms,
this means det([a b]) 6= 0 is equivalent to a 6= fb for all non-zero real numbers
f. Moreover, area is equal to the absolute value of det([a b]).
Another point of view generalizes to higher dimensional vectors. The matrix
product may be written as a linear combination of the column vectors a and b

d1 e1
u
0
=
v
0
d2 e2

d1 u + e1 v
0
=
d2 u + e2 v
0

d1
e1
0
u
+v
=
0
d2
e2
ua + vb = 0=
Thus, the above are also equivalent to ua + vb = 0 implies both u and v must
be zero. Such vectors are called linearly independent, which will be studied in
33
Figure 1.4.2: Area and Dot Product

more detail in Section 5.1. In Figure 1.4.3 the area of the larger parallelogram,
formed by the scaled vectors ua and vb> is uv times the area of the smaller
parallelogram. This follows from determinants

d1 u
det([ua vb]) = det(
d2 u
e1 v
e2 v
= d1 u(e2 v) e1 v(d2 u)
= uv det([a b])=
1.4.3
Applications to Work and Torque
This application is concerned with moving a mass from one point in a plane
to an other point in the plane. One can do this by using a ramp or using a
wheel. In the case of ramp the displacement vector is along the top of the ramp
and the force vector may not be parallel to the ramp, see Figure 1.4.4. For the
wheel one must be able to apply a non-zero force tangent to the perimeter of
the wheel, see Figure 1.4.5.
The movement of an object up a ramp requires a force with both non-zero
horizontal and vertical components. In order to reduce the computation of work
to the scalar formula, work equals distance times the force, we must nd the
component of the force
displacement
vector. In

vector in the direction of the
$
$
$

Figure 1.4.4 this is i cos() so that the work is g i cos()= If the given
$

$

vectors are g and i , then the work is computed by the cosine identity and
34
Figure 1.4.3: Linearly Independent Vectors

the dot product
$

$
zrun = g i =
For example, if the bottom edge of the ramp is 10 and the vertical part is 5,
$

then g = [10 5]= Suppose the force vector is 100 in the horizontal direction and
$

$

$
10 in the vertical direction so that i = [100 10]= Then the work is g i =
10(100) + 5(10) = 1050=
For the rotation of a wheel the force vector may not be tangent to the
wheel and so one must compute the component of the force acting in this
direction. This computation, in contrast to the above work problem, uses the
sine of the angle and not the cosine. Figure 1.4.5 reveals why this is the case.
The scalar value
for torque is the radius times the tangent component of the
$
$

force k u k i sin()= This value is related to plus or minus the area of the
parallelogram formed by the radius vector and the force vector.
1.4.4
Application to Work and Constant Force Vector
Consider an object that is to be moved from one point in a plane to another

point. Three possible paths are the direct path (path one in Figure 1.4.6), the
horizontal and then vertical (path two), and vertical and then horizontal (path
three). If the force vector is a constant vector, then we claim the work done
will be independent of the three paths. Let the direct path be the displacement
$

$

$
$
vector
d = g = [g1 g2 ] and the constant force vector be e = i = [i1 i2 ]=
Apply the cosine identity in equation (1.4.2)

$
$
g1 i1 + g2 i2 = g i cos()=
Figure 1.4.4: Work and a Ramp
Figure 1.4.5: Torque on a Wheel
35
36
Figure 1.4.6: Work with Independent Paths

$
$
The work along the direct path is g i cos()= The work along the second
path is g1 i1 + g2 i2 , and the work along the third path is g2 i2 + g1 i1 > which is
the same as path two because the addition of real numbers commutes. Thus,
the work done is independent of the three paths taken for constant displacement
and force vectors.
1.4.5
Dot Product Using MATLAB
The dot product of two vectors can be computed in two ways. The MATLAB
command dot(a,b) will do this if the vectors are both row or column vectors.
If the vectors are both column vectors, then one can use the transpose version
aW b> and in MATLAB this would appear as d0 e provided d and e were column
vectors.
AA a
a=
2 1
AA b
b=
1 -1
AA dot(a,b)
ans =
1
AA a*b
ans =
2 -2
1 -1
In the above d and e were initially row vectors so that d0 e will be a column
vector times a row vector, which results in a 2 2 matrix. By transposing them
37
to column vectors one can compute the dot product as a row vector times a
column vector.
AA cola = a
cola =
2
1
AA colb = b
colb =
1
-1
AA colb*cola
ans =
1
1.4.6
1.
Exercises
Verify the cosine identity in equation (1.4.2) for

s
s

$

$
d = [ 3 1] and e = [1 3]=
2.
Verify the cosine identity in equation (1.4.2) for the vectors in column
form

s
1
3
and b =
=
a=
1
1
3.
Use the cosine identity in equation (1.4.2) to nd the angle between the
vectors in column form

1
3
a=
and b =
=
2
1
4.
$

$
vectors
d = [2 1] and e = [1 1]=
$

$
5.
Find { such that
d = [3 1] and e = [{ 1] are perpendicular.
6.
Find { such that the following column vectors are perpendicular

1
3
a=
and b =
=
2
{
7.
Consider Theorem 1.4.2.

$

$ $

$

$
$
(a).
Prove v(
d e ) = (v
d) e =
d (v e )=
$

$

$
$
$
$
$
$
(b).
Verify
d ( e +
f)=
d e +
d
f for the vectors
d = [2 1],
$

$

e = [3 4] and f = [1 1]=
(c).
What is the dierence between "prove" and "verify"?
8.
Consider Theorem 1.4.3. Prove the rst three norm properties.
38
9.
Use the formula in equation (1.4.3) to compute the area of the parallelogram given by the column vectors

1
3
a=
and b =
=
2
1
10.
Use the formula in equation (1.4.3) to compute the area of the parallel$

$
ogram given by the vectors
d = [2 1] and e = [1 1]=
$

$
11.
Let
d = [2 1] and e = [1 1]= Verify the area of the parallelogram
$

$
given by the vectors 6
d and 3 e is 18 times the area of the parallelogram given
$

$
by the vectors
d and e =
12.
Consider the work in moving an object up a ramp whose horizontal edge
$

is 25 and vertical edge is 5. If the force vector is i = [10 1]> nd the work.
13.
Consider the work in moving an object up a ramp that requires a force
vector with force of 3 in the horizontal direction and 1 in the vertical direction.
$

If the displacement vector is g = [30 6]> nd the work.
$

$
14.
Use MATLAB to compute the dot product of
d = [2 1] and e = [3 4]=
$
d =
15.
Use MATLAB to compute the area of the parallelogram between
$

[1 2] and e = [3 1]=
1.5
Lines and Curves in R2 and C
In this section lines in the plane will be represented by three variations on

algebraic equations and three variations on vector equations. We will use these
descriptions to nd the point on a line that is closest to a given point not on
the line. One method uses calculus to solve the problem, and the other method
uses geometry to generate the same solution. Both these approaches can be
used to solve more general applied problems. Finally, curves in the plane will
be described by parametric equations. An important class of curves related to
signal processing can be generated by Eulers formula and complex variables.
1.5.1
Lines in the Plane
The three most common algebraic representations of a line are the slope-intercept,
the single equation and the parametric equations, respectively:
| |0
= p>
(1.5.1)
{ {0
d{ + e| + b
f = 0 or d({ {0 ) + e(| |0 ) = 0>
(1.5.2)
{ {0
| |0
=
= w= (1.5.3)
{ = g1 w + {0 , | = g2 w + |0 or
g1
g2
|
= p{ + f or
Example 1.5.1. Suppose a line contains the point (1> 2) and satises the
equation 3{ + 4| 11 = 0= Since { = 1 and | = 2 satises this equation, the
1.5. LINES AND CURVES IN R2 AND C
39
Figure 1.5.1: Line Given a Point and Direction

point is in fact on the line. Solve the given equation for | to get
|=
3
11
|2
3
{+
or
=
= p=
4
4
{1
4
In order to nd the parametric equation (1.5.3), use the second form for the
slope
3
|2
{1
|2
=
or
=
= w=
{1
4
3
4
This means | = 3w + 2 and { = 4w + 1= Note there are other possible choices
for the parameter such as w = 2b
w=
$

Three vector representations are one point and a direction vector g , two
$

distinct points, and one point and a normal vector q = The rst representation
is given by the vector version of the parametric equations
[{ |] = w[g1 g2 ] + [{0 |0 ]
$ $

$

u = wg +
u 0=
(1.5.4)
This is illustrated in Figure 1.5.1 where the direction vector and the point are
$
given; the given point is ({0 > |0 ) and the vector
u 0 starts at the origin and
$

ends this given point. The vector w g in the gure depicts an extension of the
given direction vector. Then any point on the line, represented by the vector
$

u beginning at the origin and ending at the point on the line, is a vector sum
$

$
of w g and
u 0=
If two points are given on the line by ({0 > |0 ) and ({1 > |1 ), then consider the
$
two vectors starting at the origin and ending at these points,
u 0 = [{0 |0 ] and
$

$

$

u 1 = [{1 |1 ]. Then u 1 u 0 is a vector that is parallel to the line and can
40
be used as the direction vector
$ $

$
u = wg +
u0

$
$
$
= w( u 1
u 0) +
u 0=
(1.5.5)
If a point and a normal vector are given, then the normal vector must be
$
$
perpendicular to any vector parallel to the line. If
u and
u 0 are two vectors
$

$

$
that are associated with two points on the line, then u u 0 and
q must be
perpendicular so that
$

$
$
(1.5.6)
q (
u
u 0 ) = 0=
This vector equation is related to the second version of the single equation
(1.5.2) representation
d({ {0 ) + e(| |0 ) = 0
[d e] [{ {0 | |0 ] = 0
$

$
$
q (
u
u 0 ) = 0=
Example 1.5.2. Find the vector equation (1.5.4) for the line with point (2> 1)
and parallel to the vector [1 4]=
$ $

$
u = wg +
u0
[{ |] = w[1 4] + [2 1] or
{ = 1w + 2 and | = 4w + 1=

One choice for a normal vector is $
q = [4 1] because $
q [1 4] = 0=
$ $

$
q (u
u 0)
[4 1] [{ 2 | 1]
4({ 2) + (1)(| 1)
4{ | 7
=
=
=
=
0
0
0
0=
Although the equations look dierent, it is easy to verify that the parametric
equations do satisfy this last single equation representation.
1.5.2
Minimum Distance from a Point to a Line
$

Let a line be represented by a direction vector g and point ({0 > |0 )= Let a point
$
$
(s1 > s2 ) not be on this line. Let
u 0 = [{0 |0 ] and
s = [s1 s2 ] be the vectors
associated with these two points. Any point on the line will then be a function
of the parameter w
$ $

$

u (w) = w g +
u 0=
The distance squared from the point not on the line and any point on the line
is a function of w
$
$
$
$
i (w) (
u (w)
s ) (
u (w)
s )=
(1.5.7)
41
Figure 1.5.2: Minimum Distance of Point to a Line

The objective is to nd the value of w = w0 so that
i (w0 ) = min i (w)=
w
This is illustrated in Figure 1.5.2.

The rst method for solving this minimization problem uses calculus. We
will nd w = w0 so that
gi
g2 i
(w0 ) = 0 and 2 (w0 ) A 0=
gw
gw
In order to compute the derivatives, use the basic properties of the dot product
$
$
$
$
i (w) = (
u (w)
s ) (
u (w)
s)
$

$

$
$

$
$
= (w g + u 0 s ) (w g +
u0
s)
$

$

$

$

$
$
$
$
$
s ) (
u 0
s)
= (w g + u 0 s ) w g + (w g + u 0
$

$

$

$

$

$

$

$
$

s)
= (w g ) (w g ) + 2(w g ) ( u 0 s ) + ( u 0 s ) ( u 0
$
$
$

$
$

$

$

$

$

2
= w g g + 2w g ( u 0 s ) + ( u 0 s ) ( u 0 s )=
This is a quadratic function of w whose derivatives are easily computed
$

$
$

$
$ $

gi
g2 i
$
s ) and 2 = 2 g g A 0=
= 2w g g + 2 g (
u0
gw
gw
Thus, the w0 where the rst derivative is zero is
$ $

$
s)
g (
u 0
w0 =
=
$

$
g g
42
Since the second derivative is positive, the minimum distance is the square root
of i (w) evaluated at w0 =
Example 1.5.3. Consider the line given by { + 2| = 4= The line contains the
$

points (0> 2) and (4> 0) and is parallel to g = [4 2]= If the point not on this
$
line is (2> 5), then the vector associated with the point is
s = [2 5]=
w0
=
=
=
$ $

$
s)
g (
u0
$

$
g g
[4 2] ([0 2] [2 5])
[4 2] [4 2]
2
1
=
=
20
10

1
The desired point on the line is $
u ( 10
)=
18
4
[0 2] = [ 10 10 ]=
$
1
10 g
$
+
u0 =
1
10 [4
2] + [0 2] =
For the geometric approach to this problem observe in Figure 1.5.2 that
$
$
vector from
s to
u (w0 ) must be perpendicular to the given line. This means
$

$
$
s = Now
the direction vector of the line g must be perpendicular to
u (w0 )
use the dot product to obtain

$
$
$
g (
u (w0 )
s) = 0
$

$

$
$

g (w0 g + u 0 s ) = 0
$

$
$ $
$
u 0
s) = 0
w0 g g + g (
and, hence, we get the same w0 =
Theorem 1.5.1 (Minimum Distance to a Line) Let a given point, given by
$ $

$
$
position vector
s , not be on a line
u (w) = w g +
u 0 = The distance squared i (w)
is dened by equation (1.5.7).
i (w0 ) = min i (w) has solution w0 =
w
$ $

$
g (
u 0
s)
=
$

$
g g
Variations on this important result will be considered in Section 2.3 for

$
lines in space, and in Section 4.2 for data tting using "least squares." If
u 0 is
the zero vector and the other vectors are written as column vectors, then the
minimization problem is solved by the solution of
dW d w0 = dW p=
Here d is a column vector with two rows, and w0 is called a least squares solution
to d w = p, which has two equations and one unknown.
43
Figure 1.5.3: Cycloid and Wheel
1.5.3
Curves in the Plane and Phasors
The parametric representation of a line has components that are linear polynomials of the parameter, dw + e. If one or both of the component functions are
not linear polynomials, then the set of points will not be a line. For example,
if { = w and | = dw2 > then we have a parabola. In this case the position vectors
$
for points on the parabola are
u = [w dw2 ]= Another example is a circle with
radius d where one representation is { = d cos(w) and | = d sin(w)= The position
$
vector is
u = [d cos(w) d sin(w)]= Complex variables and Eulers formula can
also describe this circle by } = dhlw = d cos(w) + ld sin(w)=
A more complicated curve is the cycloid, which is the curve given by a point
on a wheel of radius d as the wheel rolls in a straight line. As is illustrated in
Figure 1.5.3, the position vector is given by
$

$
f +
$
u = $
d + e
= [wd 0] + [0 d] + [d sin(w) d cos(w)]
= [d(w sin(w)) d(1 cos(w)]=
The angle w is the parameter, and it is given in radians so that the arclength
opposite the angle is dw= Figure 1.5.4 depicts this curve with d = 1 and was
generated by the MATLAB code cycloid.m.
Eulers formula can be used to generate sine and cosine curves with dierent
amplitudes, d, frequencies, i> and phases, . By adding the associated complex
numbers one can easily generate very complicated signals such as a persons
voice. Consider just one tone and one sine function of time w
d sin(2i w + )=
44
Figure 1.5.4: Cycloid

The sine function is the imaginary part of
dhl(2i w+) = dhl2i w hl =
At time w = 0> this is dhl , which is called a phasor where is the phase angle.
In order to have more than one tone, additional frequencies must be used.
Consider two possible tones
d1 sin(2i1 w + 1 ) and d2 sin(2i2 w + 2 )=
The signal associated with these two is simply the addition of the two sine
functions. In terms of complex variables this is the imaginary part of
} = d1 hl(2i1 w+1 ) + d2 hl(2i2 w+2 ) =
By letting time vary the complex variable will move in the complex plane, and
one can plot its imaginary part as a function of time. This is illustrated in
Figure 1.5.5, which was generated by the MATLAB code phasor.m.
1.5.4
Curves Using MATLAB
We will concentrate on graphing a curve by using parametric equations. If we

have values for the points on the curve and they are stored in two row vectors
of equal length, then the MATLAB command plot() can be used to generate a
graph. For example, in the following two row vectors each with 4 components
are created.
45
Figure 1.5.5: Two-tone Signal

AA x = [1 2 3 4];
AA y = [1 4 9 16];
AA plot(x,y)
The command plot(x,y) will generate a graph connecting the four points (1> 1),
(2> 4), (3> 9) and (4> 16)=
If one needs to have more data points, then this can be done by for-loops
or by implied loops called vector operations. The for-loops is more exible,
but it is slower. The following for-loop creates three row vectors that have 10
components.
AA for j = 1:10
t(j) = j*.5;
x(j) = t(j)+1;
y(j) = t(j)^2;
end
AA plot(x,y)
The following uses vector operations to generate the same graph. The MATLAB
command y = t.^2 is an array operation, which produces another row vector
with the same number of components as the row vector t and whose components
are the squares of those in t. For example, if t = [1 2 3], then t.^2 = [1 4 9].
AA t = .5:.5:5;
AA x = t + 1;
AA y = t.^2;
46
AA plot(x,y)
The MATLAB code cycloid.m, see Figure 1.5.4, uses vector operations, and the
code phasor.m, see Figure 1.5.5, uses for-loops with a MATLAB command pause
so that one can step through the loop.
1.5.5
Exercises
Let { 3| = 9=
(a).
Find the slope-intercept equation.
(b).
Find the parametric equations.
2.
Let { = 2w + 1 and | = 3w 2=
(a).
Find the slope-intercept equation.
(b).
Find the single algebraic equation.
3.
Let the line contain the point (2> 7) and be parallel to [1 1]=
(a).
Find the vector equation using a direction vector.
(b).
Find the parametric equations.
4.
Let the line contain the point (2> 7) and be perpendicular to [1 1]=
(a).
Find the vector equation using a normal vector.
(b).
Find the single algebraic equation.
$
5.
Find the point on the line
u = w[1 2] + [0 1] that is closest to the
point (5> 5)=
6.
Find the point on the line given by { = 2w + 2 and | = w + 1 that is
closest to the point (5> 5)=
7.
Use MATLAB to create a graph of the line given by { = w + 2 and | = 2w
with 0 w 5=
8.
Use MATLAB to create a graph of the curve given by { = w + 2 and
| = 2w2 with 0 w 3=
9.
Use the MATLAB code cycloid.m to experiment with dierent wheels,
d = 0=5> 1 and 2=
10.
Use the MATLAB code phasor.m to experiment with a variety of the
inputs:
(a).
Vary the frequencies.
(b).
Vary the amplitudes.
(c).
Vary the phase angles.
1.
Chapter 2
Vectors in Space
Vectors in space are introduced, and the dot, cross and box products are studied. Lines and planes are carefully described as well as extensions to higher
dimensional space. Applications to work, torque, inventories and visualizations
are included.
2.1
Vectors and Dot Product
A point in space can be located in a number of ways, but here the Cartesian
coordinate system will be used. You may wish to visualize this from the interior
of a room looking down into the corner. The corner is the origin; the x-axis is
the intersection of the left wall and oor; the y-axis is the intersection of the
right wall and the oor; the intersection of the left and right walls is the z-axis.
This is illustrated in Figure 2.1.1. The point (d> e> f) is located by moving d
units in the x-axis, then moving e units parallel to the y-axis, and moving f
units parallel to the z-axis. The distance from the origin to the point is given
by two applications of the Pythagorean theorem to the right triangles in Figure
2.1.1. Associated with the point (d> e> f) is the position vector from the origin
to this point.
$
Denition 2.1.1. A vector in R3 is an ordered list of three real numbers
d =
[d1 d2 d3 ]. One can visualize this by forming the directed line segment from
the origin point (0> 0> 0) to the point (d1 > d2 > d3 )=
Notation. Points in R3 will be denoted by (d1 > d2 > d3 ) and vectors will be
represented by either row or column vectors:
$

d = [d1 d2 d3 ] indicates a row vector>
5
6
d1
a = 7 d2 8 indicates a column vector =
d3
$
d=
aW = [d d d ] is called the transpose of the column vector a so that aW =
1
47
48
CHAPTER 2. VECTORS IN SPACE
Figure 2.1.1: Point in Space

Example 2.1.1. The standard unit basis vectors are
5 6
1
$

l = [1 0 0] or e1 = 7 0 8 ,
0
5 6
0
$

m = [0 1 0] or e2 = 7 1 8 and
0
5 6
0
$

n = [0 0 1] or e3 = 7 0 8 =
1
Any vector can be represented as a linear combination of unit basis vectors
$

$

$

$

d = [d1 d2 d3 ] = d1 l + d2 m + d3 n or
5
6
d1
a = 7 d2 8 = d1 e1 + d2 e2 + d3 e3 =
d3

For example, if $
d = [2 3 4], then either by row vectors
$

$

$

$
d = 2 l + 3 m + 4 n or by column vectors
a = 2e1 + 3e2 + 4e3 =
These vectors are depicted in Figure 2.1.2.
The movement of a mass in space is described by vectors with three components such as displacement, force, velocity and acceleration vectors. Newtons
2.1. VECTORS AND DOT PRODUCT
49
Figure 2.1.2: Vector in Space

law of motion in space can be derived by applying it to each of the three directions
gy1
gy2
gy3
i1 = p
> i2 = p
and i3 = p
gw
gw
gw
$

$
where p is the mass, i = [i1 i2 i3 ] is the force vector and
y = [y1 y2 y3 ]
is the velocity vector. In vector notation this means

$
i = [i1 i2 i3 ]
gy1
gy2
gy3
= [p
p
p
]
gw
gw
gw
$
= p
d where
gy
1 gy2 gy3
$

d = [
] is the acceleration vector.
gw gw gw
2.1.1
Vector Operations
Vectors can be scaled, added and subtracted by either viewing the vectors
geometrically or as symbolic objects. Two non-parallel vectors in space form
a plane in space so that the geometric approach to these operations is similar
to vectors with two components. Figure 2.1.3 illustrates vector addition and
$

$
scaling by either elongating or shrinking the vector. Two vectors
d and e
$

can be added by moving vector e parallel to its original position so that its
50
Figure 2.1.3: Vector Addition in Space
$

$
$
beginning point is the end point of vector
d . The addition
d + e corresponds
$

$

$
to the diagonal of the parallelogram formed by vectors
d and e . If e is
$

$
$
added to
d , then the subtraction
d e is the other diagonal from the end of
$

$

e to the end of d =
$

$

$
d = [d1 d2 d3 ], e = [e1 e2 e3 ] and 0 = [0 0 0]
$

$
be the zero vector.
d equals e means d1 = e1 , d2 = e2 and d3 = e3 = The
addition of two vectors is another vector whose components are the sum of the
components of the two vectors
$

$
d + e [ d1 + e1
d2 + e2
d3 + e3 ]=
Let v be a real number. The scalar product of a vector is another vector whose
components are the scalar times the components of the vector
$
v
d [ vd1
vd2
vd3 ]=
The following theorem lists some basic algebraic properties of the above operations for vectors in space R3 > which are analogous to those in the plane R2 .
51
$

$
$
Theorem 2.1.1 (Vector Space Properties) Let
d > e and
f be vectors in
3
R and let v and w be real numbers. Then the following rules hold
$

$ $

$

$ $

$

$
$
d + e = e +
d > ($
d + e )+
f =
d +( e +
f )>
$

$

$

$

$

$

d + 0 = d and d + ( d ) = 0 ;
$
$
$
$
$
$
$
d =
d and
(v + w)
d = v
d + w
d , v(w
d ) = (vw)
d > 1
$

$

$

$

v( d + e ) = v d + v e =
$

$

$
$
The proofs are straightforward. The proof of v(
d + e ) = v
d + v e uses
the distributive property of addition

$
$
v(
d + e ) = [ v(d1 + e1 ) v(d2 + e2 ) v(d3 + e3 ) ]
= [ vd1 + ve1 vd2 + ve2 vd3 + ve3 ]
= [ vd1 vd2 vd3 ] + [ ve1 ve2 ve3 ]
$

$
= v
d +ve =
$

Example 2.1.2. Let $
d = [2 4 1], e = [1 2 4] and v = 3= Then
$

$

d + e = [1 6 5]>
$

$

d e = [3 2 3] and
$

v e = [3 6 12]=
$
Denition 2.1.3. Let
d = [d1 d2 d3 ]. The length or Euclidean norm of the
$

vector d is given by the Pythagorean theorem
q
$
k
d k d21 + d22 + d23 =
As illustrated in Figure 2.1.3 two vectors in space form a plane and so the
$ $

$
cosine law may be rewritten in terms of the norms of the vectors
d > e >
f =
$

$

$

$

d e and the angle between the vectors d and e
$

2
$2
$
2

$
$
d k + e 2 k
d k e cos()=
(2.1.1)
d e = k
$
This allows one to compute the angle between two given vectors.
$
Example 2.1.3. Consider the vectors in Figure 2.1.3
d = [3 2 1] and
$

e = [1 4 2]= The vector starting
at
the
point
(1
4
2)
and
ending at (3 2 1) is
$

$

$

$
represented by d e > and d e is the distance between the end points of
$

$

$
$
2
2
$
$
$
the vectors
d and e . Then
d e = [2 2 1]>
d e = 9> k
d k = 14>
$2
e = 21, and equation (2.1.1) becomes
s s
9 = 14 + 21 2 14 21 cos()
s
= 14 + 21 14 6 cos()=
52
Thus,
s
cos() = 13@(7 6) and
s
= cos1 (13@(7 6)) 40=70 degrees.
2.1.2
Cosine Identity and Dot Product
$

$
Let
d = [d1 d2 d3 ] and e = [e1 e2 e3 ] form a plane in space and have the
angle between the vectors. The vector version of the cosine law is equation
(2.1.1). By denition of vector subtraction and the norm
$

2
d e = (d1 e1 )2 + (d2 e2 )2 + (d3 e3 )2
$
= d21 2d1 e1 + e21 + d22 2d2 e2 + e22
+d23 2d3 e3 + e23
2
$2
$
= k
d k + e 2(d1 e1 + d2 e2 + d3 e3 )=
Place this into the left side of equation (2.1.1) and cancel the norms to get
$
$
d k e cos()=
(2.1.2)
d1 e1 + d2 e2 + d3 e3 = k
As in the case for vectors with two components, equation (2.1.2) is of fundamental importance because the left side is an easy calculation, and the right
side makes a connection with the geometry.
$

$
Theorem 2.1.2 (Cosine Identity) Let
d = [d1 d2 d3 ] and e = [e1 e2 e3 ]
be non-zero vectors and let be the angle between the two vectors. Then cos()
can be computed by
d1 e1 + d2 e2 + d3 e3
cos() =
=
$
$
k
d k e
$

$
Denition 2.1.4. The dot product of two vectors
d = [d1 d2 d3 ] and e = [e1
e2 e3 ] is given by the real number on either side of equation (2.1.2). The
following are common notations for the dot product of row or column vectors,
respectively,
$

$
$

$
d e d1 e1 + d2 e2 + d3 e3 = k
d k e cos() or
aW b d1 e1 + d2 e2 + d3 e3 = kak kbk cos()=
The cosine identity can be restated as either
$

$

d e
cos() =
$

or
$
k
d k e
cos() =
aW b
.
kak kbk
53
Also, the norm of a vector can be written as either

2
$
k
dk
kak2

$
= $
d
d or
= aW a=
$

$
d and e be non-zero vectors. Two vectors are parallel
if and only if they are multiples of each other, that is, there is a real number v
$

$
such that
d = v e = Two vectors are perpendicular or orthogonal if and only if
$

$
the cos() is zero, that is,
d e = aW b = 0=
$

$
Example 2.1.4. Consider the vectors in Figure 2.1.3
d = [3 2 1] and e = [1
$

2
$2
$
$
4 2]= Then k
d k = 14> e = 21 and
d e = 3(1) + 2(4) + 1(2) = 13 so
that
$

$

13
d e
13

= s s = s and
cos() =
$
$
14 21
7 6
k
d k e
13
s
40=70 degrees.
= cos1
7 6
The dot product has a number of interesting algebraic properties. One is it
is associative with respect to a scalar multiple
$

$
v(
d e ) = v(d1 e1 + d2 e2 + d3 e3 )
= (vd1 )e1 + (vd2 )e2 + (vd3 )e3
$

$
= (v
d) e =
Additional properties are listed in the following theorems, which are analogous
to Theorems 1.4.2 and 1.4.3, and are also easy to prove. The Cauchy-Schwarz
inequality in (2.1.3) follows directly from the cosine identity in (2.1.2). Real
valued functions that satisfy equations (2.1.4)-(2.1.6) are called norms, and they
are analogous to the absolute value of a single real number. An interpretation of
the triangle inequality is the minimum distance between two points is a straight
line. These concepts generalize to higher dimensional vectors, see Sections 2.5
and 3.1.
$ $

$
Theorem 2.1.3 (Properties of Dot Product) Let
d > e >
f be given vectors in
space R3 and let v be a real number. Then the following hold
$

$ $

$

$

$
$
$
d e = e
d > v(
d e ) = (v
d) e>
$ $

$ $

$

d ( e +
f) = $
d e +
d $
f and
$

$

$
$

d e k d k e (Cauchy-Schwarz inequality);
(2.1.3)
54
$

$
Theorem 2.1.4 (Properties of Norm) Let
d > e be given vectors in space R3
and let v be a real number. Then the following hold
$

$
$
$
k
d k 0;
d = 0 if and only if k
d k = 0>
$

$

kv d k = |v| k d k and
$

$
$

$
d k + e (triangle inequality).
d + e k
2.1.3
(2.1.4)
(2.1.5)
(2.1.6)
Applications to Work
Consider moving a mass from one point in space to an other point where the
$

component changes are recorded in a displacement vector g = [g1 g2 g3 ].
A force in each direction is required, which can be recorded in a force vector
$

i = [i1 i2 i3 ]= For constant displacement and force vectors, the work is the
product of displacement distance
vector
projected

and the length of the force
$
$
$
on the displacement vector, i cos()= Then the work is g i cos()= The

work is given by the cosine identity and the dot product
$

$
zrun = g i = g1 i1 + g2 i2 + g3 i3 =
For example, if the mass is moved from the point (1> 1> 1) to the point (2> 3> 4),
$

then g = [1 2 3]= Suppose the force vectors components are 100 in the x$

direction, 10 in the y-direction and 5 in the z-direction so that i = [100 10
$

$
5]= Then the work is g i = 1(100) + 2(10) + 3(5) = 135=
2.1.4
Dot Product Using MATLAB
The MATLAB command dot(a,b) computes the dot product if the vectors are
both row or column vectors. If the vectors are both column vectors, then one
can use the transpose version aW b= In MATLAB this would appear as d0 e
provided d and e were column vectors.
AA a = [1 -1 3]
a=
1 -1 3
AA b = [4 1 7]
b=
417
AA dot(a,b)
ans =
24
AA a_col = a
AA b_col = b
AA a_col*b_col
ans = 24
55
$
The MATLAB command norm(a) computes k
d k > which is the same as

$
$

1@2
( d d ) = This is called the Euclidean norm and is one of many important
norms (see the MATLAB command help norm).
AA norm(a)
ans =
3.3166
AA (dot(a,a))^.5
ans =
3.3166
The next calculations illustrate the MATLAB command quiver3(x, y, z, dx,
dy, dz, 1), which will generate a vector in space starting at the point ({> |> })
and ending at the point ({ + g{> | + g|> } + g}).
AA quiver3(0,0,0,2,3,-1,1)
AA hold on
AA quiver3(0,0,0,3,1,2,1)
The input parameters may be arrays with the same size. Use arrays as
inputs (see MATLAB command help quiver3 for an interesting example).
AA
AA
AA
AA
2.1.5
x = [0 0];y = [0 0]; z = [0 0];

dx = [2 3];dy = [3 1];dz = [-1 2];
hold o
quiver3(x,y,z,dx,dy,dz,1)
Exercises
1.
Plot the points (3> 1> 2) and (1> 3> 1)= Indicate the corresponding vectors
starting at the origin.
2.
Plot the points (3> 1> 1) and (1> 1> 3)= Indicate the corresponding vectors
starting at the origin.
3.
Find the unit basis vector and column vector representations of

$

$
d = [3 1 2] and e = [1 3 1]=
4.
5.
Find the unit basis vector and row vector representations of

5
6
5 6
3
1
a = 7 1 8 and b = 7 1 8 =
1
3
Let
(a).
(b).
6.
Let
(a).
$

$
d = [3 1 2] and e = [1 3 1]=
$

$

$
Compute 2 e and 2
d + e=
$

$
$
$
Find the vector
{ = [{1 {2 {3 ] so that 2
d +
{ = e=
$
$

$

$

$

$

$

$

d = 2 l + 4 m n and e = l + 2 m + 3 n
$

$

$
Compute 2 e and 2
d +3e =
56
$

$

$

$

$
$
d +
{ = e=
Find the vector
{ = {1 l + {2 m + {3 n so that 3$
$

$

$

$

$

$

$
$

d = l +4m 2n
and e = l + m + 3 n
$

$
Compute k
d k and e =
$

$
(b).
Find
d e=
$

$
8.
Let
d = [0 1 2] and e = [1 4 1]=
$

$
(a).
Compute k
d k and e =
$

$
(b).
Find
d e=
9.
vectors in column form
5
6
5
6
1
1
a = 7 2 8 and b = 7 3 8 =
1
1
(b).
7.
Let
(a).
10.
$

$
vectors
d = [3 1 1] and e = [1 1 1]=
11.
Consider Theorem 2.1.3.
$

$ $

$

$
$
(a).
Prove v(
d e ) = (v
d) e =
d (v e )=
$

$

$
$
$
$
$
$
d
f for the vectors
d = [2 1
(b).
Verify
d (e +
f)=
d e +
$

$

3], e = [3 4 1] and f = [1 1 4]=
(c).
What is the dierence between "prove" and "verify"?
12.
Consider Theorem 2.1.4. Prove the results of lines (2.1.4) and (2.1.5).
13.
Consider the work in moving an object along an incline in space whose
x-direction is 25, y-direction is 20 and z-direction is 5. If the force vector is
$

i = [10 1 2]> nd the work.
14.
Consider the work in moving an object along an incline in space whose
$

$

$

$

$

displacement vector is g = 20 l + 15 m + 3 n . If the force vector is i =
$

$

$

10 l + 2 m + 1 n > nd the work.
$
d = [2 1 1] and
15.
Use MATLAB to compute the dot product of
$

e = [3 4 2] and their norms.
16.
Use MATLAB command quiver3(x,y,z,dx,dy,dz,1) to plot the vectors
$

$

d = [2 1 1] and e = [3 4 2]=
2.2
Cross and Box Products
In Section 1.4 the cosine identity was used to nd the area of a parallelogram
formed by two vectors in a plane. Then this was used to compute the torque
for the rotation about a point in the plane. The cross product will allow us
to generalize these to vectors in space and not just the xy-plane. The box
product is a combination of the dot and cross product that will enable us to
easily compute the volume of a parallelepiped formed by three vectors in space.
These concepts will be related to determinants of 3 3 matrices.
2.2. CROSS AND BOX PRODUCTS
2.2.1
57
Area and Torque in the xy-Plane
$
Any vector in the plane
d = [d1 d2 ] can be viewed as space vector whose
$

third component is zero, d = [d1 d2 0]= We use some of the results in Section
1.4 for vectors in space that are restricted to one of the coordinate planes. Let
$

$

d = [d1 d2 0] and e = [e1 e2 0] not be parallel. Then the duhd is the area
$

$
of the parallelogram formed by
d and e is
$
$
d k e sin())2
(duhd)2 = (k
= (d1 e2 e1 d2 )2

d1 e1
))2 =
= (det(
d2 e2
An application to torque, which involves rotation about the z-axis, is when
$

$

$
$

d
=

u gives the point to be rotated and e = i is the force vector. Here
$

i sin() is the component of the force acting tangent to the rotation, and
it can be directed in either a clockwise or counterclockwise motion. The convention is to use the clockwise motion as given by a right hand threaded screw
where a clockwise rotation of a screw driver sinks the screw into a mass. If one
views the screw as being below the xy-plane, and rotates the x-axis towards the
y-axis, then the screw should move in a positive z-direction. Dene the cross
product of the unit vectors according to this convention:
$

$
$
l m n
$
$

$
m n l and
$

$
$

n l m=
$

$
The cross product of
d = [d1 d2 0] and e = [e1 e2 0] is dened to be a
vector in the z-direction with magnitude equal to d1 e2 e1 d2
[d1 d2 0] [e1 e2 0] [0 0 d1 e2 e1 d2 ]=
$

$

$

$

$
$
Note, if
d = [1 0 0] = l and e = [0 1 0] = m > then l m = [0 0
$

1(1) 0(0)] = n = This is illustrated in Figure 2.2.1 where the three possible
$
$
planes are viewed from the back (negative) sides. In the case
d =
u and
$

$

$

$

e = i the third component of u i is the torque for rotation about the
z-axis.
2.2.2
Cross Product and Torque in Space
In order to understand the cross product denition for any vectors in space,
consider the area of the parallelogram formed by two vectors in space as illustrated in Figure 2.2.2. The area will be non-zero provided the two vectors are
not parallel. The area that is projected into the xy-plane is the absolute value of
58
Figure 2.2.1: Unit Vector Cross Products
Figure 2.2.2: Projected Area
the third component in the cross product of the two vectors projected into the
xy-plane. One can also project into the yz-plane and the xz-plane with similar
area formulae. The above analysis can be applied to the other two coordinate
planes to yield
[d1 0 d3 ] [e1 0 e3 ] [0 (d1 e3 e1 d3 ) 0] and
[0 d2 d3 ] [0 e2 e3 ] [d2 e3 e2 d3 0 0]=
The cosine identity eventually leads to a formula for the DUHD of the
$

$
parallelogram formed by two vectors in space
d = [d1 d2 d3 ] and e = [e1 e2
59
e3 ]
(DUHD)2
$
$
= (k
d k e sin())2
$
$
= (k
d k e )2 (1 cos2 ())
$

$

d e 2
$ 2
$

= (k d k e ) (1 (
$ ) )
$
k
d k e
$2
$
2
$
$
d e )2 =
= k
d k e (
Next substitute the vector components into the above and endure a messy
algebraic computation to obtain
(DUHD)2
= (d21 + d22 + d23 )(e21 + e22 + e23 ) (d1 e1 + d2 e2 + d3 e3 )2

..
.
= (d2 e3 e2 d2 )2 + (e1 d3 d1 e3 )2 + (d1 e2 e1 d2 )2 = (2.2.1)
The formula in equation (2.2.1) means the area in space squared is the sum of
the three "projected areas squared."
$

$
Denition 2.2.1. The cross product of
d = [d1 d2 d3 ] and e = [e1 e2 e3 ]
$

$
is another vector
d e whose components are given by the projected areas
$

$
(2.2.2)
d e [d2 e3 e2 d3 d1 e3 + e1 d3 d1 e2 e1 d2 ]
$

$

$

(d2 e3 e2 d2 ) l (d1 e3 e1 d3 ) m + (d1 e2 e1 d2 ) n =
$

$

$
$
As we show below
d e is perpendicular to both
d and e and, thus, it
$

$
is perpendicular to the plane formed by
d and e = Equation (2.2.1) implies the
norm of the cross product is equal to the area of the parallelogram formed by
the two vectors. Determinants are often used as an easy way to remember the
denition of a cross product of two vectors.
f >
$
Denition 2.2.2. Let a 3 3 matrix be given by three row vectors $
d and

$
e
5
6
f1 f2 f3
D = 7 d1 d2 d3 8 =
e1 e2 e3
The determinant of this matrix is a real number dened by expanding across
60
the top row of the matrix
d2 d3
det(D) f1 det(
)
e2 e3

d1 d3
f2 det(
)
e1 e3

d1 d2
+f3 det(
)
e1 e2
= f1 (d2 e3 e2 d3 )
f2 (d1 e3 e1 d3 )
+f3 (d1 e2 e1 d2 )=
(2.2.3)
The components of the row vectors are usually real or complex numbers.
An abuse of notation is to let the top row be replaced by a row of unit basis
vectors so that
5
$ 6
$
$
l
m
n
$

$

d e det(7 d1 d2 d3 8)=
e1 e2 e3

$

Example 2.2.1. Let $
d = [1 3 2] and e = [4 1 5]=
5 $
$ 6

$
l
m
n
$

$

d e = det(7 1 3 2 8)
4 1
5
$

$

$

= (3(5) 1(2)) l (1(5) 4(2)) m + (1(1) 4(3)) n
$

$

$

= 17 l 13 m 11 n
= [17 13 11]=
$

$
The cross product is perpendicular to both the vectors
d and e . In order
to prove this, it su!ces to show the dot product of one of the vectors with the
cross product is zero
$ $

$
(
d e)
d =
=
=
=
[d2 e3 e2 d3 d1 e3 + e1 d3 d1 e2 e1 d2 ] [d1 d2 d3 ]
(d2 e3 e2 d3 )d1 + (d1 e3 + e1 d3 )d2 + (d1 e2 e1 d2 ) d3
d2 e3 d1 e2 d3 d1 d1 e3 d2 + e1 d3 d2 + d1 e2 d3 e1 d2 d3
0=
61
Theorem 2.2.1 (Fundamental Cross Product Properties) The cross product

$

$

d e satises the following
$ $

$
(
d e)
d
$

$
$

(d e) e
$

2
d e
$
$

2
d e
$
= 0>
= 0,
$
$
= (k
d k e sin())2 and
$2
$
2
$
$
= k
d k e (
d e )2
(Lagrange identity)=
$

$
Example 2.2.2. Let
d = [1 3 2] and e = [4 1 5] be as in the rst
$

$
example so that
d e = [17 13 11]= The area of the parallelogram formed
by these vectors is
s
$

p
d e = 172 + (13)2 + (11)2 = 579 24=06=
$
The cross product has a number of interesting properties. The distributive
property is valid
$ $

$
d ( e +
f ) = [d2 (e3 + f3 ) (e2 + f2 )d3
d1 (e3 + f3 ) + (e1 + f1 )d3
d1 (e2 + f2 ) (e1 + f1 )d2 ]
= [d2 e3 e2 d3 d1 e3 + e1 d3 d1 e2 e1 d2 ] +
[d2 f3 f2 d3 d1 f3 + f1 d3 d1 f2 f1 d2 ]
$ $

$

= d e +
d $
f=
However, the cross product is not commutative and is not associative!
Theorem 2.2.2 (Algebraic Properties of Cross Product) Consider the cross
product of vectors in space. The following hold
$

$
d e
$ $

$

d ( e +
f)
$

$

$
d ( e
f)
$

$

$

d (e f)
$ $

$

$

$
$
= e
d > (v
d ) e = v(
d e )>
$ $

$
=
d e +
d $
f>
$

$$

$
$
$
d
f ) e (
d e )
f and
= (
$

$

$
= (d e ) f=
Example 2.2.3. Consider the torque vector

$

$
wrutxh =
u i=
$

$
Let
u = [1 3 1] and i = [4 1 2]= You may think of the origin as being the
location of screw or bolt. Let a wrench be attached to the bolt and have its
62
$
handle end at the end of the position vector
u = Apply a force at the handle end
$

given by the vector i . Find the components of torque governing the rotation
about x,y and z axes.
5 $
$ 6
$

l
m
n

$
$

u i = det(7 1 3 1 8)
4 1 2
$

$

$

= (3(2) 1(1)) l (1(2) 4(1)) m + (1(1) 4(3)) n
$

$

$

= 5 l + 2 m 11 n =
2.2.3
Box Product and Volume
Let three vectors in space not be in the same plane. They form a parallelepiped
as is illustrated in Figure 2.2.3. The volume is equal to the area in the base
$

$
formed by the two vectors
d and e times the height. The direction of the
height is perpendicular to the base, that is, it is parallel to the cross product
$

$

$
d e = The height is then equal to the projection of the third vector
f onto
$

$

$

$

$

d e = Let ! be the angle between the vectors f and d e so that the
$
height is k
f k cos(!)= Thus, the volume is plus or minus
$

$ $

$
$
d e (k
f k cos(!)) = (
d e )
f=
$

$

f is the real number
Denition 2.2.3. The box product of $
d > e and $
$ $

$
(
d e )
f=
Example 2.2.4. Compute the volume of the parallelepiped formed by the
$

$
$
vectors
d = [1 2 1]> e = [2 0 1] and
f = [0 1 4]=
5 $
$ 6
$

l
m
n

$
$

d e = det(7 1 2 1 8)
2 0 1
$

$

$

= (2(1) 1(0)) l (1(1) 1(2)) m + (1(0) 2(2)) n
$

$

$

= 2 l + 1 m 4 n = [2 1 4]=
The box product is
$ $

$
f = [2 1 4] [0 1 4]
(
d e)
= 2(0) + 1(1) + (4)4) = 15=
The volume of the parallelepiped is 15=
63
The box product can be computed by using determinants. Let D be the

$

$
$
3 3 matrix with rst, second and third rows
f >
d and e , respectively. Then
$ $

$

$
f (
$
(
d e )
f = $
d e)
= f1 (d2 e3 e2 d3 )
f2 (d1 e3 e1 d3 )
+f3 (d1 e2 e1 d2 )
= det(D).
Other representations of the determinant follow from rearranging the computations of det(D) in equation (2.2.3) and noting
5
f1
det(D) = det(7 d1
e1
5
d1
= det(7 e1
f1
5
d1
= det(7 d2
d3
f2
d2
e2
d2
e2
f2
e1
e2
e3
6
f3
d3 8)
e3
6
d3
e3 8)
f3
6
f1
f2 8)=
f3
(2.2.4)
(2.2.5)
$ $

$
An interpretation of equation (2.2.4) is that
d ( e
f ) gives plus or minus the
$

volume when the base of the parallelepiped is generated by the vectors e and
$

f . Equation (2.2.5) is formed by interchanging the rows and columns of the
previous matrix; the new matrix (2.2.5) is called the transpose of the matrix in
(2.2.4). If the column vectors in (2.2.5) are represented by a> b and c> then the
matrix in (2.2.5) may be written as [a b c] and det([a b c]) is plus or minus
the volume.
Theorem 2.2.3 (Volume Equivalence) Let three non-zero vectors be represented as column vectors a> b and c= The volume of the parallelepiped formed
by these vectors being non-zero is equivalent to no vector of the three being in
the plane formed by the other two vectors. In symbolic terms, this means det([a
b c]) 6= 0 is equivalent to a 6= gb+hc or b 6= gc+ha or c 6= ga+hb for all
non-zero real numbers g and h. Moreover, volume is equal to the absolute value
of det([a b c]).
Another point of view generalizes to higher dimensional vectors. The matrix
product may be written as a linear combination of the column vectors a b and
64
Figure 2.2.3: Box Product and Volume
c= In order to prove this, note

5
65 6
5 6
u
0
f1
f2 8 7 v 8 = 7 0 8
w
0
f3
6
5 6
d1 u + e1 v + f1 w
0
7 d2 u + e2 v + f2 w 8 = 7 0 8 =
0
d3 u + e3 v + f3 w
d1
7 d2
d3
5
e1
e2
e3
Next rewrite the left column vector as

6
5
6
5
6
5 6
e1
f1
0
d1
u 7 d2 8 + v 7 e2 8 + w 7 f2 8 = 7 0 8
0
d3
e3
f3
ua + vb + wc = 0=
5
Thus, the above are also equivalent to ua + vb + wc = 0 implies all u> v and w
must be zero. Such vectors are called linearly independent and will be further
studied in Section 5.1. The vector equation ua+vb+wc = 0 means the diagonal
of the parallelepiped formed by the vectors ua> vb and wc is the zero vector. See
Figure 2.2.4 where the volume of the larger parallelepiped, given by the scaled
vectors ua> vb and wc> is uvw times the volume of the smaller parallelepiped. This
65
Figure 2.2.4: Determinant and Volume

follows from the determinant calculation
5
d1 u
det([ua sb wc]) = det(7 d2 u
d3 u
6
e1 v f1 w
e2 v f2 w 8)
e3 v f3 w
= d1 u((e2 v)(f3 w) (f2 w)(e3 v)) +

= uvw det([a b c])=
Example 2.2.5. Compute the volume of the parallelepipeds given by the
b W = [6
vectors aW = [1 2 1]> bW = [2 0 1] and cW = [0 1 4]; b
aW = [2 4 2]> b
W
0 3] and b
c = [0 4 16]= Use determinants to nd the rst volume
5
6
1 2 0
(volume) = dev(det(7 2 0 1 8))
1 1 4
= dev(1(0(4) 1(1)) 2(2(4) 1(1)) + 0)
= dev(15) = 15=
The rst volume is 15= The second volume is 2(3)(4)15 = 360 because the new
vectors are scalar multiples of the previous three with u = 2> v = 3 and w = 4=
2.2.4
Cross and Box Product Using MATLAB
The MATLAB commands det(A) and cross(a,b) compute the determinant of a

square matrix D and the cross product of two vectors d and e. The following
computations illustrate some of the above identities.
AA a = [1 2 1]; b = [2 0 1]; c = [0 1 4]
c=
66
014
AA A = [a b c]
% convert rows to columns
A=
120
201
114
AA det(A)
ans =
-15
AA det(A)
% det(A) = det(A)
ans =
-15
AA newA = [c ; a; b]
% volume with base via a and b
newA =
014
121
201
AA det(newA)
ans =
-15
AA axb = cross(a,b)
axb =
2 1 -4
AA dot(c,axb)
% box product
ans =
-15
2.2.5
Exercises
$

$

$
$

$
Let $
d = [2 3 1] and e = [3 0 1]= Compute
d e and
d e =
$
$

$
$

$

$

$
$

$
$
2.
Let
d = 2 l + 4 m + n and e = l + 3 m n = Compute
d e and
$

d e =
$
$
$

$
$

$

$

$

$
3.
Let
d = l + 3 m + n and e = 0 l + 3 m n = Find the area of the
parallelogram formed by these vectors.
$

$
4.
Let
d = [2 0 1] and e = [3 1 1]= Find the area of the parallelogram
formed by these vectors.
$

$ $

$
5.
Show
d e =e
d=
$

$

$

$
$
$
6.
Show d ( e f ) may not be equal to (
d e)
f . Do both
$

$
$
computations for
d = [2 3 1]> e = [3 0 1] and
f = [1 2 3].
7.
Consider the wrench torque in Example 2.2.3. If the ends of the wrench
$
are given by
u = [1 3 1] and the force vector has components in the x,y and
$

$
z-directions equal to 2> 3 and 1> respectively, nd the torque vector
u i.
1.
2.3. LINES AND CURVES IN R3
67
8.
Consider the wrench torque in Example 2.2.3. If the ends of the wrench
$
$

$

$

$
are given by
u = l + 2 m + n and the force vector is i = [2 1 4]> compute
$

$
the torque vector
u i.
$
$
$

$

$
9.
A parallelepiped is given by three vectors
d = 4l +1m + n> e =
$
$

$

$

$

$

$
l + 5 m + n and
f =2 l + m +5n=
(a).
Use the box product to compute the volume.
(b).
Use a determinant to nd the volume.
$

$
10.
A parallelepiped is given by three vectors
d = [1 3 1]> e = [0 3 1]
$

and f = [1 2 4]=
(a).
Use the box product to compute the volume.
(b).
Use a determinant to nd the volume.
11.
Prove the determinant identities in equations (2.2.4) and (2.2.5).
12.
Use the MATLAB command det(A) and cross(a,b) to do the computations in exercise nine.
13.
Use the MATLAB command det(A) and cross(a,b) to do the computations in exercise ten.
2.3
Lines and Curves in R3
This section extends the study of lines in the plane to lines in space. Vector
and parametric representations of a line will be developed. Minimum distances
from a point to a line and from line to a line will be analyzed. The cross product
will be used to describe the minimum distance. More general curves including
the helix and path of a projectile will be presented.
2.3.1
Lines in Space
$
Let a given point in space be represented by the position vector
u 0 = [{0 |0
$

}0 ] and let a given direction vector be g = [g1 g2 g3 ]= As for a line in a plane
one can scale the direction vector and add it to the position vector whose end
point is on the line to obtain the position vector for any point on the line
$

$
$
u (w) =
u 0+wg=
This is called a vector equation for the line and is illustrated in Figure 2.3.1.
The direction vector may be found from two distinct points on the line by
$ $

$
u 0 = The parametric
subtracting the corresponding position vectors g =
u 1
equations can be derived from the three scalar equations that correspond to the
$
vector equation with
u (w) = [{(w) |(w) }(w)]
{(w) = {0 + wg1 > |(w) = |0 + wg2 and }(w) = }0 + wg3 =
Example 2.3.1. Find the vector and parametric representations of the line
$
$
u 1 = [2
containing the two points (1> 2> 4) and (2> 5> 3)= Let
u 0 = [1 2 4] and
68
Figure 2.3.1: Vector Equation and Minimum Distance

5 3] be the corresponding position vectors. A direction vector is computed by
$

$
$
u 0 = [2 5 3] [1 2 4] = [1 3 1]= Note, the choice for
u 0 and
g =$
u1
$

u 1 will give a dierent direction vector, but it will be a multiple of the above.
Also, one can choose either position vector to use in the vector representations
$

$
$
u (w) =
u0+wg
[{(w) |(w) }(w)] = [1 2 4] + w[1 3 1]=
The parametric representation is
{(w) = 1 + w> |(w) = 2 + 3w and }(w) = 4 w=
2.3.2
Minimum Distance to a Line

$
$
$
Consider a line given by
u (w) =
u 0 + w g > and let a point (s1 > s2 > s3 ) not be
$

on this line and have position vector s = [s1 s2 s3 ]= The problem is to nd
the point on the line, given by some w = w0 > so that the distance from the point
$
to the line is a minimum. The distance from the given end point of
s to the
$

$

$

$

$

$

end points of u (w) is the length of the vector u (w) s = ( u 0 s ) + w g =
$
$
$
$
Let i (w) (
u (w)
s ) (
u (w)
s ) so that one must solve the minimization
problem
i (w0 ) = min i (w)=
w
As in Section 1.5, this can be done by either calculus or by noting at w = w0 >

$
$
the vector
u (w)
s must be perpendicular to the direction vector of the line
69
$
$
(see Figure 2.3.1). This means the dot product of
u (w)
s with the direction
$

vector g must be zero
$

$
g ($
u (w)
s) = 0
$

$

$

$

g (( u 0 s ) + w g ) = 0
$

$
$

$
s ) + g g w = 0=
g ($
u 0
$ $

$

$
u 0 $
s )@( g g ). The minimum distance is the square root
Thus, w0 = g (
of i (w0 )=
The distance squared at w = w0 simplies to
$
$
$
$
i (w0 ) = (
u (w0 )
s ) (
u (w0 )
s)
$

$

$

$
$

s)
= (( u 0 s ) + w0 g ) ( u (w0 )
$

$

$

$

= ( u 0 s ) ( u (w0 ) s ) + 0
$

$
$
$
$
s ) ((
u 0
s ) + w0 g )=
= (
u 0
The computation of the minimum distance can be done by using the cross
product and the Lagrange identity in Theorem 2.2.1
$2
$

$

2
2
$
$
d k e (
d e )2 =
(2.3.1)
d e = k
$
$ $

$

$
u 0 $
s )@( g g ) into i (w0 ) and use the cross product
Next, insert w0 = g (
$

$
$
$
$
s and e = g
identity with
d =
u 0
$

$
$
$
$
$
$
i (w0 ) = (
u 0
s ) (
u 0
s ) + (
u0
s ) (w0 g )
$

$
$

$
$
$
$
$
$
(
u0
s ) (
u 0
s ) g g ((
u 0
s ) g )2
=
$

$
g g
2
$

$
u0
s) g
($
=
=
$2
g
$

$
$
Theorem 2.3.1 (Minimum Distance from Point to Line) Let
u (w) =
u 0 +w g
represent a line, and let a point (s1 > s2 > s3 ) not be on this line and have position
$

$ $

$

$
$
$
$
$
u (w0 ) =
u 0 + w0 g where w0 = g (
u 0
s )@ g g
vector
s = [s1 s2 s3 ]=
gives the point on the line that is closest to the point (s1 > s2 > s3 )= Moreover, the
minimum distance is
$

$
u 0
s) g
($
=
$
g
$

$
Example 2.3.2. Let a line be given by
u 0 = [1 1 1] and g = [1 2 2], and
nd the minimum distance from the point (4> 4> 1) to this line. According to
70
$

$
$
the above theorem the point on the line that is closest is
u (1) =
u 0 + 1 g = [1
1 1] + 1[1 2 2] = [2 3 3] where
w0
= [1 2 2] ([1 1 1] [4 4 1])@[1 2 2] [1 2 2]

= (9)@9 = 1=
$
One can compute the minimum distance directly by evaluating
u (1) and computing the distance, or by using the above cross product formula, which only
requires the two points and the direction vector. The direct calculation is

$
$
u (1)
s = [2 3 3] [4 4 1] = [2 1 2]
$

$

k u (1) s k = 3=
The cross product computation is
5 $
$ 6

$
l
m
n

$
$
$
(
u 0
s ) g = det(7 3 3 0 8)
1
2
2
$

$

$

= 6 l + 6 m 3 n
= [6 6 3]=
This distance is the same as the direct approach
$

$
u0
s) g
($
9
= = 3=
$
3
g
Another minimum distance problem is depicted in Figure 2.3.2, which involves two lines in space that are not parallel and do not intersect. We seek
to nd a point on one line and a point on the second line so that the distance
between them is the smallest. Use the following vector representations for the
two curves, where the parameter w locates a point on the rst line and the
parameter locates a point on the second line
$

$
$
u (w) =
u 0 + w g and
$

$

( ) = $
0+ =
$
$
An important observation is the vector
u (w)
( ) must be perpendicular to
both lines at the points where the minimum distance is attained. This means
$

$

this vector must be perpendicular to both the direction vectors g and
$

$
$
(
u (w)
( )) g = 0 and
$

$
$
(
u (w)
( )) = 0=
71
Figure 2.3.2: Distance between Two Lines

$

$

$
$
$
$
Since
u (w)
( ) = (
u 0
0 ) + w g > these two dot product equations
give two scalar equations for the two unknown parameters w and
$

$
$
$

$
$
$
(
u 0
0) g + g g w g
$

$
$
$

$
$
$
) + g w
(
u
0
= 0 and
= 0=
The matrix form of this is

"
"
$
$
$

$ #
$ #

$
$
g g g
w
(
u0
0) g
=
$

$
$

$
$ =

$
$

g
0)
(
u0
(2.3.2)
By Cramers rule as discussed in Section 2.4 and in Theorem 2.4.1, the solution
of this system will exist if the determinant of the 2 2 coe!cient matrix is not
zero
"
$
$
$

$ #

$2
$

$
g g g
$2
det(
)
=

g
+ ( g )2 =
$
$
$

$
g
If the direction vectors are not parallel, then the cross product will not be zero.
$

$

$

$
But, by the Lagrange identity in equation (2.3.1) with
d = g and e =
$2
$
this determinant is g 6= 0= Once equation (2.3.2) is solved for w = w0
$
and = 0 , the points on the two lines given by the position vectors
u (w0 ) and
$

$

$

( ) can be computed as well as the minimum distance k u (w ) ( )k =
0
The cross product can be used to compute the minimum distance without
$

$
having to solve the algebraic system in (2.3.2). Observe the cross product g
$

$

must be perpendicular to both g and and, hence, it must also be parallel
72
$

$
$
$
$
$
( 0 )= This means the angle between g and
u (w0 )
( 0 )
to
u (w0 )
must be either zero or 180 degrees so that by the cosine identity

$

$
$
$
$
$
$
$
(
u (w0 )
( 0 )) ( g ) = k
u (w0 )
( 0 )k g (1)

$

$

$

$
$
$
$
$
$
$
((
u 0
0 ) + g w0 0 ) ( g ) = k
u (w0 )
( 0 )k g (1)

$

$
$
$
$
$
$
$
0 ) ( g ) + 0 + 0 = k
u (w0 )
( 0 )k g (1)=
(
u 0
The minimum distance squared is
$

$
$
$
((
u 0
0 ) ( g ))2
2
$
$
k
u (w0 )
( 0 )k =
=
$2
$
g
(2.3.3)
Theorem 2.3.2 (Minimum Distance Between Lines) Let two lines be given
$

$

$
$
$
$
( ) =
0 + = If the two
by the vector equations
u (w) =
u 0 + w g and
$

$
lines are not parallel and do not intersect, then g is not the zero vector
and the solution of equation (2.3.2) gives the points on the lines with minimum
distance. Moreover, the minimum distance may be computed by (2.3.3).
$
Example 2.3.3. Let the two lines be given by
u (w) = [1 2 1] + w[2 0 1] and
$

( ) = [0 2 1] + [1 1 1]=
5
$ 6
$
$
l
m
n
$

$
g = det(7 2
0
1 8)
1 1 1
$

$

$

= 1 l 1m 2n
= [1 1 2]=
Equation (2.3.3) is used to compute the minimum distance squared
$

$
$
$
0 ) ( g ))2
((
u 0
([1 0 2] [1 1 2])2
3
=
= =
2
$
6
2
$
g
Thus, the minimum distance is
2.3.3
p
3@2=
Curves and Projectiles
Lines in space are given by three linear polynomials of the parameter w: {(w) =
{0 + g1 w> |(w) = |0 + g2 w and }(w) = }0 + g3 w= If the linear polynomials are
replaced by more complicated functions, then the curve will not be a straight
line. An example is the helix. Two illustrations are the outer edges of a spiral
staircase or a circular car ramp in a parking deck. The { and | components
must be projected onto a circle and the } component should be proportional to
73
Figure 2.3.3: Helix

the parameter. Consider a circle with radius u, and assume the } component
increases a height k for every revolution. One parametric equation for this is
{(w) = u cos(2w)>
|(w) = u sin(2w) and
}(w) = kw=
The MATLAB code helix.m was used to generate Figure 2.3.3.
Another interesting curve is a projectiles path as it moves through the air
and is subjected to gravity, air resistance and wind forces. The projectile may
be an artillery shell directed at a target or a sport ball moving towards a goal
post. In order to model the motion, we apply Newtons law of motion to each of
the three directions. Mass times acceleration is equal to the sum of the forces.
$
Suppose the wind velocity is given by speeds in the three directions
z = [z1
z2 z3 ]= Assume the force acting on the mass from air resistance is proportional
to the velocity relative to the air, and let f be the proportionality constant. (A
more realistic model for air resistance or drag is a quadratic function of the
$
velocity.) The velocity vector of the mass is
y = [{0 | 0 } 0 ] and is derived from
$

the position vector u = [{ | }]= The three equations are
p{00
p| 00
p} 00
= fz1 f{0
= fz2 f| 0 and
= fz3 f} 0 pj=
Let the initial position be at the origin so that {(0) = 0> |(0) = 0 and }(0) =
74
0 and for ease of notation consider a unit mass p = 1= The initial velocity
components are given {0 (0)> | 0 (0) and } 0 (0)=
Each of these equations has the form
y0 = g fy and y(0) = jlyhq=
If g and f are constants, then this can be solved by making the change of
variables z g fy and, thus, z0 = 0 fy 0 and
z0 = fz and z(0) = g fy(0)=
By the basic properties of exponential functions
z(w) = z(0)hfw
g fy(w) = (g fy(0))hfw
y(w) = g@f (g@f y(0))hfw =
(2.3.4)
The dierential equation for y = {0 uses p = 1> g = fz1 , {0 (0) given and
{(0) = 0= Then equation (2.3.4) gives
{0 (w) = z1 (z1 {0 (0))hfw and
{(w) = z1 w + ((z1 {0 (0))@f)hfw (z1 {0 (0))@f=
The solution for y = | 0 is similar
| 0 (w) = z2 (z2 | 0 (0))hfw and
|(w) = z2 w + ((z1 | 0 (0))@f)hfw (z2 | 0 (0))@f=
For y = } 0 use p = 1> g = fz3 pj = f0 32 = 32, } 0 (0) given and }(0) = 0
} 0 (w) = 32@f (32@f } 0 (0))hfw and
}(w) = (32@f)w + (((32@f) } 0 (0))@f)hfw + ((32@f) + } 0 (0))@f=
The MATLAB code proj3d.m was used to create the graph for this curve in
Figure 2.3.4.
2.3.4
Curves in Space Using MATLAB
The MATLAB command plot3(x,y,z) can be used to graph a curve in space

where x, y and z are lists (vectors) of the components on the three axes, respectively. The previous mentioned MATLAB codes helix.m and proj3d.m use
this command. The following illustrates how it can be used to create a graph
of one of the lines in Example 2.3.3.
AA t = -3:.1:3;
AA x = 1+2*t;
AA y = 2+0*t;
% creates row vector with 61 numbers

% computes 1 + 2w for all entries in w
75
Figure 2.3.4: Projectile in Space

AA z = 1+1*t;
AA plot3(x,y,z)
The command plot3() can be used to create more than one curve on the graph
by simply listing a second triple of coordinate points. A simple illustration is
the MATLAB code lines3d.m, which generates the lines in Example 2.3.3. Try
the above codes, experiment with the input data and be sure to "rotate" the
gures, that is, change perspectives or points of view.
2.3.5
Exercises
1.
Let a line be given by the two points (1> 0> 3) and (4> 1> 4)= Find the vector
equation for this line.
$

2.
Let a line be given by the point (1> 2> 1) and the direction vector g = [1
1 2]= Find the parametric equations for this line.
$
3.
Suppose a line has vector equation
u (w) = [1 1 2] + w[1 1 1]=
(a).
Find the point on the line that is closest to the point (2> 4> 7)=
(b).
Compute this distance.
(c).
Use the cross product as in Theorem 2.3.1 to compute the distance.
$
4.
Suppose a line has vector equation
u (w) = [1 2 2] + w[1 2 1]=
(a).
Find the point on the line that is closest to the point whose position
$
vector is
s = [1 5 3]=
(b).
Compute this distance.
(c).
Use the cross product as in Theorem 2.3.1 to compute the distance.
76
$
$
5.
Consider two lines given by
u (w) = [1 1 2] + w[1 1 1] and
(w) = [1
2 2] + [1 2 1]= Use the cross product as in Theorem 2.3.2 to compute the
minimum distance between the two lines.
$
$
6.
Consider two lines given by
u (w) = [1 0 1] + w[1 1 3] and
(w) = [1
-2 2] + [1 2 1]= Use the cross product as in Theorem 2.3.2 to compute the
minimum distance between the two lines.
7.
Consider the two lines given in Example 2.3.3.
(a).
Compute the components in the matrix equation (2.3.2).
(b).
Solve this system for w = w0 and = 0 =
$
$
(c).
Compute the minimum distance directly by k
u (w0 )
( 0 )k =
8.
Use the MATLAB lines3d.m.
(a).
Execute this code to view the two lines in Example 2.3.3. Use
"rotate" to obtain dierent perspectives.
(b).
Modify this code to graph the two lines in either exercise 5 or 6.
9.
Use the MATLAB helix.m.
(a).
Execute this code to view the "spiral staircase" with radius 6 and
height of 10 per revolution.
(b).
Modify the code to consider a "spiral ramp" with radius 60 and
height of 30 per revolution.
10.
Use the MATLAB proj3d.m.
$
$
(a).
Execute this code with the wind speeds
z = [5 10 0] and
z = [5
10 0]=
(b).
Execute this code with dierent air resistance coe!cients f = 0=01
and f = 0=005 (drier air!).
2.4
Planes in R3
This section will give four descriptions of a plane in space. The minimum
distance from a point to a plane with an application to curve tting to data will
be presented. Determinants will be used to solve resulting algebraic systems.
2.4.1
Four Plane Descriptions
The rst description uses a single equation with three variables {> | and }
d{ + e| + f} = g=
For example, if the four constants were positive, then the intercepts with the
coordinate axes will be positive and the graph would look similar to that given
in Figure 2.4.1. A point ({0 > |0 > }0 ) is in the plane if and only if it satises the
equation
d{0 + e|0 + f}0 = g=
Therefore, any point ({> |> }) in the plane must satisfy
d{ + e| + f} = d{0 + e|0 + f}0 or
d({ {0 ) + e(| |0 ) + f(} }0 ) = 0=
2.4. PLANES IN R3
77
Figure 2.4.1: Normal and Point

$
The second description of a plane requires a normal vector
q and a given
point ({0 > |0 > }0 )= Every vector in the plane must be perpendicular to the normal
vector, see Figure 2.4.1. Let any point in the plane be given by the position
$
$
vector
u = [{ | }] and let the given point have position vector
u 0 = [{0 |0
$

$

}0 ]= Since u u 0 is a vector in the plane,

$
$
$
q (
u
u 0 ) = 0=
The normal vector can be derived from algebraic equation formula from
d({ {0 ) + e(| |0 ) + f(} }0 ) = 0
[d e f] [{ {0 | |0 } }0 ] = 0=

Thus, one can choose $
q = [d e f]=
Example 2.4.1. Consider the algebraic equation 6{ + 2| + 3} = 6= The {> |
and } intercepts are { = 1> | = 3 and } = 2= If {0 = 1@2 and |0 = 1@2, then }0
$
must be (6 6(1@2) 2(1@2))@3 = 2@3= The normal vector is
q = [d e f] = [6
$

2 3] and we can use u 0 = [1@2 1@2 2@3] so that
[d e f] [{ {0 | |0 } }0 ] = 0
[6 2 3] [{ 1@2 | 1@2 } 2@3] = 0=
The third description uses three distinct points that are not on the same
$
$
$
line. Let the position vectors
u 0 = [{0 |0 }0 ]>
u 1 = [{1 |1 }1 ] and
u 2 = [{2
78
Figure 2.4.2: Three Points

$
$
$
|2 }2 ] be associated with these points. Then the vectors
d =
u1
u 0 and
$

$

$
e = u 2 u 0 beginning at the rst point and ending at the second and third
points are in the plane. Their cross product will be perpendicular to both, as is
illustrated in Figure 2.4.2. Since any vector in the plane is a linear combination
$

$
of
d and e > the normal vector can be chosen to be this cross product
$

$

$
$
$
$
$
q =
d e = (
u
u ) (
u
u )=
1
The fourth description requires a point and two non-parallel vectors in the
plane. The two non-parallel vectors could be calculated from three points as
above. Any point in the plane can be found by adding the position vector of
the given point, a scaled rst vector and a scaled second vector
$

$

$
$
u =
u + u
d +ve =
0
The scaling factors are real numbers u and v such that the linear combination
$

$
u
d + v e is a vector beginning at the point ({0 > |0 > }0 ) and ending at a desired
point in the plane ({> |> })= See Figure 2.4.3 for an illustration of the linear
combination.
Example 2.4.2. Find the plane that contains the points (1> 1> 3)> (2> 2> 1) and
$
$
$
(0> 3> 2)= Choose the position vectors
u 0 = [1 1 3]>
u 1 = [2 2 1] and
u 2 = [0
3 2] and compute
$

d = [2 2 1] [1 1 3] = [1 1 2] and
$

e = [0 3 2] [1 1 3] = [1 2 1]=
2.4. PLANES IN R3
79
Figure 2.4.3: Linear Combination of Vectors

The normal vector is computed by using the cross product
5
$ 6
$
$
l
m
n
$

$

$
q =
d e = det(7 1
1 2 8)
1 2 1
$

$

$

= 3 l + 3 m + 3 n = [3 3 3]=
Use this normal vector in the dot product
$

$
$
q (
u
u 0) = 0
[3 3 3] [{ 1 | 1 } 3] = 0
3({ 1) + 3(| 1) + 3(} 3) = 0=
Thus the algebraic description is { + | + } = 5= A linear combination representation of this plane is
$

$

$
$
u =
u 0 + u
d +ve
[{ | }] = [1 1 3] + u[1 1 2] + v[1 2 1]=
The column version of this is
5
6 5 6
5
6
5
6
{
1
1
1
7 | 8 = 7 1 8+ u7 1 8 + v7 2 8=
}
3
2
1
(2.4.1)
The three scalar equations are { = 1+1u1v> | = 1+1u+2v and } = 32u1v,

which satisfy { + | + } = 5=
80
2.4.2
Solving for Two Unknowns with Determinants
In Example 2.4.2 suppose a particular point on the plane is given and we want
to nd the parameters u and v so that the linear combination gives the point.
This will require solving for two unknowns. This type of problem also appeared
in the previous section where we were trying to nd the points on two lines
that were closest. A similar problem will be nding a point in a plane that is
closest to a given point not on the plane. For linear systems with small number
of unknowns, determinants give a compact formula for the solutions.
Let a 2 2 matrix D be given by two column vectors a and b> and let the
unknowns be u and v= With a given column vector f for the right side of the
vector equation we have

d1 e1
u
i1
=
or
(2.4.2)
v
d2 e2
i2

d1
e1
i1
u
+v
=
=
d2
e2
i2
The two equivalent scalar equations are
d1 u + e1 v = i1 and d2 u + e2 v = i2 =
Multiply the second equation by d1 > multiply the rst equation by d2 and subtract the two to get
(d1 e2 d2 e1 )v = d1 i2 d2 i1 =
A similar calculation eliminates u and gives
(d1 e2 d2 e1 )u = i1 e2 i2 e1 =
The coe!cient on the left side is the denition of the determinant of the 2 2
matrix D= By moving the column vector f into the rst or second column of D>
one can express the right sides using determinants. This is Cramers rule for
two unknowns.
Theorem 2.4.1 (Cramers Rule) Let D be the 22 matrix in equation (2.4.2).
If ghw(D) 6= 0> then the solution is

i1 e1
d1 i1
u = det(
)@ det(D) and v = det(
)@ det(D)=
i2 e2
d2 i2
Example 2.4.3. Consider Example 2.4.2 where the plane has representation
in equation (2.4.1) or { + | + } = 5= The point ({> |> }) = (4> 3> 2) is in this
plane. The problem is to nd u and v so that equation (2.4.1) holds. Since
(4> 3> 2) is in this plane, we only need to consider the rst two of the scalar
equations { = 4 = 1 + 1u 1v> | = 3 = 1 + 1u + 2v
uv = 3
u + 2v = 2=
2.4. PLANES IN R3
81
Figure 2.4.4: Minimum Distance to a Plane

The vector form of this is
1 1
1 2
u
v

=
3
2
The determinant of the matrix D is 1(2) (1)1 = 3 and the solution is

3 1
1 3
u = det(
)@3 = 8@3 and v = det(
)@3 = 1@3=
2 2
1 2
It is easy to see that all three scalar equations hold.
2.4.3
Minimum Distance to a Plane
This section extends the study of minimum distance from a given point to a
line in a plane to a given point in space to a plane in space. As in Section 1.4,
one can approach the problem either using calculus or geometry. The calculus
approach requires minimizing a function of two variables, and we will defer this
to the multivariable calculus course. The geometric approach gives the same
result and relies on the visualization of the problem as depicted in Figure 2.4.4.
Let the plane be given by a point and two non-parallel vectors in the plane
$

$

$
$
u =
u 0 + u
d +ve =
$
If the given point not in the plane is represented by the position vector
s > then
we must nd the real numbers u = u0 and v = v0 so that
$

$
$

$
d + v0 e
u min
u 0 + u0
82
$
$
$
$
$
is closest to the
s . In this case, the vectors
s
u min and
u
u min are
perpendicular for all points in the plane represented by the position vector.
$

$
$
$
$
Since
d and e are in the plane, we may choose either
u =
u min +
d or
$

$

$

$

$

$
u = u min + e = In these cases, s u min must be perpendicular to both
d
$

and e
$
$
$
d = 0 and
(
s
u min )
$

$

$

( s u min ) e = 0=
$

$
$
$
d + v0 e and using the properties of the dot product,
u 0 + u0
Since
u min
we obtain the following scalar equations
$ $

$
$
$
$
$
(
s
u 0)
d
d
d u0 e
d v0 = 0 and
$

$

$

$
$
$

$

( s u ) e d e u e e v = 0=
0
The matrix form of this

"
$

$
d
d
$

$

d e
is
#

$

u0
e $
d
=
$

$
v0
e e
$
$
$
(
s
u 0)
d
=
$

$
$
(
s
u 0) e
(2.4.3)
The determinant of the 2 2 coe!cient matrix is

$

$
$

$
$
$
(
d
d )( e e ) (
d e )2
$2
$ $2
2
$
= k
dk e ( e
d)
2
$
$
=
d e =
$

$
Since the vectors
d and e are required not to be parallel, this determinant is
not zero and equation (2.4.3) must have a solution. Once the solution is found,
$
$
$

s
u min k can be computed.
u min and the desired distance k
$

$
$
An alternative way to compute k s
u min k is to use a normal vector
q to
$

$

$

$

the plane. Then the cosine of the angle between s u min and u 0 s is
$
$

$
s)
q (
u 0
=
cos() =
$
$

$

kqkk u s k
0
$

The angle is indicated in Figure 2.4.4 with
u set equal to $
u 0 so that

$
$
$
u 0
s k |cos()|
k
s
u min k = k$
$

$
$
q (u0
s )
=
(2.4.4)
$
=
k
qk
$
Theorem 2.4.2 (Minimum Distance to a Plane) Let a plane be given by
u =
$

$

$

$

u 0 + u d + v e where u 0 is a position vector of a given point in the plane and
$

$

$
d and e are not parallel vectors in the plane. If
s is a position vector of
a point not in the plane, then the solution of equation (2.4.3) gives the point
$
in the plane closest to the point associated with
s = Moreover, the minimum
distance may also be calculated by equation (2.4.4).
2.4. PLANES IN R3
83
Example 2.4.4. Return to the Example 2.4.2 where the plane is given by
$

$

$
$
u =
u 0 + u
d +ve
[{ | }] = [1 1 3] + u[1 1 2] + v[1 2 1]=
$
Let the point not on this plane be given by the position vector
s = [10 11
$

$

12] and compute s u 0 = [10 11 12] [1 1 3] = [9 10 9]= Then equation
(2.4.3) is

6 3
u0
1
=
=
3 6
2
v0
$
$
u min
u0+
Apply Cramers rule to obtain u0 = 0@27 and v0 = 9@27= Then
s
$

$

$

$

k = 28@ 3= This is consistent
u d + v e = [2@3 5@3 8@3] and k s u
0
min
with equation (2.4.4)

$
$
k
s
u min k =
=
2.4.4

$
$
$
q (
u0
s)
$

kqk
s
[1 1 1] [9 10 9]
s
= 28@ 3=
3
Application to Curve Fitting to Data
A consumer would like to predict the price of a high denition television based
upon prices from the previous three months of $2000> $1950 and $1910 for the
rst, second and third months of the year. One way to view this problem is
to plot the prices on the vertical axis versus the times 1> 2 and 3. Then try to
draw a line "closest" to the data, estimate the slope p and the y-intercept f>
and use these in the formula | = p{ + f where { is the time in months and |
is the possible price of the television. Here we are trying to "t" a line to the
three points of price data.
A more precise way of doing this is to view p and f as unknowns in the
following three scalar equations
p1 + f = 2000
p2 + f = 1950 and
p3 + f = 1910=
This is an over determined system with three equations and two unknowns, and
in general does not have a solution to all three equations. In terms of matrices
this is
5
6
5
6
1 1
2000
7 2 1 8 p
= 7 1950 8
(2.4.5)
f
3 1
1910
5 6
5 6
5
6
1
1
2000
p 7 2 8 + f 7 1 8 = 7 1950 8 =
3
1
1910
84
Consider the plane in space determined by the linear combination of the

columns of the 3 2 matrix and nd the point in the plane that is closest to
the point given by the price column vector on the right side. Apply the results
$
$
$
in the previous section with
u 0 = [0 0 0]>
s = [2000 1950 1910]>
d = [1
$

2 3] and e = [1 1 1]= Equation (2.4.3) gives the solution of the minimum
distance problem for p = u0 and f = v0

14 6
p
11630
=
=
6 3
f
5860
Apply Cramers rule to solve for p = 270@6 = 45=00 and f = 12260@6
2043=33= Use these to predict the price in the fourth month
|
2.4.5
= p{ + f
(45=00)4 + 2043=33 = 1863=33=
Solving Systems Using MATLAB
The MATLAB command A\d can be used to solve D{ = g where the matrix D
has q rows and q columns. In the above curve tting problem q = 2 and the
following simple MATLAB computations produce the same results as Cramers
rule.
AA A =[14 6;6 3]
A=
14 6
6 3
AA d =[11630 5860]
d=
11630
5860
AA A\d
ans =
1.0e+003 *
-0.0450
2.0433
The MATLAB command A\d can also be used to solve certain over determined systems such as in equation (2.4.5).
AA A_big = [1 1;2 1;3 1]
A_big =
1 1
2 1
3 1
AA d_big = [2000; 1950; 1910]
d_big =
2.4. PLANES IN R3
85
2000
1950
1910
AA A_big\d_big
ans =
1.0e+003 *
-0.0450
2.0433
A more careful study of what is done to generate possible solutions will be given
in the next two chapters.
2.4.6
1.
2.
3.
4.
5.
Exercises
Consider the plane given by { + 2| + 4} = 12=

(a).
Verify the point (2> 1> 2) is in this plane.
(b).
Find the normal to the plane.
(c).
Find the normal equation representation of this plane.
Consider the plane given by 5{ + | + 4} = 20=
(a).
Verify the point (0> 4> 4) is in this plane.
(b).
Find the normal to the plane.
(c).
Find the normal equation representation of this plane.
Let a plane be given by the three points (1> 1> 1)> (0> 3> 0) and (3> 0> 1)=
(a).
Find a normal vector to this plane.
(b).
Compute the algebraic equation that describes this plane.
Let a plane be given by the three points (1> 0> 1)> (1> 3> 0) and (0> 1> 1)=
(a).
Find a normal vector to this plane.
(b).
Compute the algebraic equation that describes this plane.
Consider a plane given by
[{ | }] = [1 1 1] + u[2 3 0] + v[1 0 0]=
(a).
Find the algebraic system in equation (2.4.2) such that [{ | }] =
[2 3 1]=
(b).
Use Cramers rule to nd u and v.
6.
[{ | }] = [1 0 1] + u[4 0 1] + v[1 0 1]=
(a).
Find the algebraic system in equation (2.4.2) such that [{ | }] =
[1 0 2]=
(b).
Use Cramers rule to nd u and v.
7.
[{ | }] = [1 1 1] + u[2 3 1] + v[1 0 1]=
Find the point in the plane that is closest to the point (5> 4> 6)=
86
8.

[{ | }] = [1 0 1] + u[0 3 1] + v[1 10 1]=
Find the point in the plane that is closest to the point (5> 4> 6)=
9.
A consumer would like to predict the price of a laptop computer based
upon prices from the previous three months of $1000> $960 and $910 for the
rst, second and third months of the year. Model this by assuming the price
decreases as a linear function of time similar to equation (2.4.5).
(a).
Find the slope and the y-intercept of a straight line "closest" to this
data.
(b).
Use these to predict the price after four months.
10.
A consumer would like to predict the price of a laptop computer based
upon prices from the previous months one, two and four equal to $1000> $960
and $870, respectively. Model this by assuming the price decreases as a linear
function of time similar to equation (2.4.5).
(a).
Find the slope and the y-intercept of a straight line "closest" to this
data.
(b).
Use these to predict the price after ve months.
2.5
Extensions to Rq
This section is an introduction to vectors with more than three components and
to the remaining chapters in this book. There are sections with applications
to inventory and price vectors, larger algebraic systems of equations, curve
tting with more than three data points, higher dimensional spaces of vectors,
systems of ordinary dierential equations and matrices used in the context of
visualization. Even though we have mostly considered vectors with two or three
components, which may represent points in space or physical quantities such as
force and velocity vectors, we have used vectors with a number of components to
store information. Examples are the spiral and projectile curves in Subsection
2.3.3 and the lines as given by the MATLAB commands in Subsection 2.3.4.
In the line example, four vectors with 61 components were created to store
the values for the parameter and the {, | and } values of the points on the
line. Many of the operations on vectors have natural generalization to higher
dimensions, but some of the geometric aspects are lost as we move beyond
dimension two and three.
2.5.1
Inventory and Price Vectors
A store carries six dierent styles of television sets. There are a number of each
style in the inventory, and there is a price per set for each style. So, there are
a total of 12 numbers to be recorded. These can be stored in two vectors each
with six components. The inventory or stock numbers will be denoted by a
column vector s, and the price numbers will be stored in a column vector p=
2.5. EXTENSIONS TO RQ
Example 2.5.1.
5
9
9
9
s =9
9
9
7
87
2
4
5
3
7
1
:
9
:
9
:
9
: and p = 9
:
9
:
9
8
7
400
250
280
900
700
100
6
:
:
:
:=
:
:
8
If the stock is doubled and the price is decreased by

and price vectors are
6
5
5
360
4
9 225
9 8 :
:
9
9
9 252
9 10 :
:
9
9
2s = 9
: and .9p = 9 810
6
:
9
9
7 630
7 14 8
90
2
10%, then the new stock

6
:
:
:
:=
:
:
8
The total value of the original stock and price is given by a "dot product" of
the stock and price vectors
sW p = 2(400) + 4(250) + 5(280)
+3(900) + 7(700) + 1(100)
= 10900=
If there is a second store with a known stock vector b
s, then the total stock can
be computed by "adding" the vectors. For example, if the second store stock
vector is
5 6
1
9 2 :
9 :
9 2 :
:
b
s=9
9 5 :,
9 :
7 3 8
4
then the total stock is
5
9
9
9
b
s+s = 9
9
9
7
1+2
2+4
2+5
5+3
3+7
4+1
: 9
: 9
: 9
:=9
: 9
: 9
8 7
3
6
7
8
10
5
6
:
:
:
:=
:
:
8
The total value of the two stores can be computed in two ways
sW p + b
sW p = 10900 + 8460 or
W
(b
s + s) p = 19260=
88
Large department stores have thousands of products on their shelves, and their
inventory numbers and prices can be stored in vectors with large numbers of
components.
2.5.2
Algebraic Systems
In the previous sections algebraic systems with two unknowns and two equations
were solved using determinants and Cramers rule. In Subsection 2.2.3 the nonzero volume of a parallelepiped is generated by three column vectors a> b and c.
Any point in space can be represented by a linear combination of these vectors
(see Figure 2.2.4)
5
d1
7 d2
d3
e1
e2
e3
ua + vb + wc = r0
65 6
5
6
u
{0
f1
f2 8 7 v 8 = 7 |0 8 =
w
f3
}0
Given the three column vectors and the point, we want to nd u> v and w=
Determinants and a generalization of Cramers rule can be used to compute
these coe!cients. For three unknowns the rule is (see Section 3.7)
u = det( r0 b c )@ det( a b c )>
v = det( a r0 c )@ det( a b c ) and
w = det( a b r0 )@ det( a b c )=
W
W
Example 2.5.2. Consider the Example
2.2.5 whereW a = [1 2 1]> b = [2 0
W
1]> c = [0 1 4] and det( a b c ) = 15= If r0 = [5 6 4], then solve
65 6 5 6
1 2 0
u
5
7 2 0 1 87 v 8 = 7 6 8=
1 1 4
w
4
The three additional determinants
u = det( r0 b c
v = det( a r0 c
w = det( a b r0
are easy but messy to compute
)@(15) = 45@(15) = 3>
)@(15) = 15@(15) = 1 and
)@(15) = 0@(15) = 0=
For larger algebraic problems determinants are not used and usually row
operations are better suited for the solution. An example of a large system
would be from models of building whose support is from beams connected at a
number of nodes or joints. In order for the building to be in a static state, the
sum of the forces at each node must be zero. Each node has a small number of
equations, but the building usually will have a large number of nodes. Thus,
there are a large number of equations. This will be discussed in chapter three.
2.5.3
89
Curve Fitting
In Subsection 2.4.4 the minimum distance from a point in space to a plane

was used to t a line to three data points. If there are more than three data
points, then the minimization problem will be in a higher dimensional space.
For example, consider an additional data point to the data in Subsection 2.4.4
p1 + f
p2 + f
p3 + f
p4 + f
In terms of matrices this is
5
1
9 2
9
7 3
4
5
1
9 2
p9
7 3
4
=
=
=
=
6
1
1 :
: p
1 8 f
1
6
5
1
:
9 1
:+ f9
8
7 1
1
2000
1950
1910 and
1890=
6
:
:
8
6
2000
9 1950 :
:
= 9
7 1910 8
1890
5
6
2000
9 1950 :
:
= 9
7 1910 8 =
1890
The solution of this requires a notion of perpendicular vectors in four dimensional space! The solution has the same form and is called the normal equations.
Let aW = [1 2 3 4], bW = [1 1 1 1] and pW = [2000 1950 1910 1890]
W

W
a a aW b
p
a p
=
=
f
bW a bW b
bW p
These concepts will be developed in chapter four.
2.5.4
Multiple Solution of Algebraic Systems
Consider a single equation with three variables d{ + e| + f} = g= The points

in space that satisfy this equation form a plane, which may be represented by
a linear combination of two vectors plus a given vector. The plane is a two
dimension subset in a three dimensional space. If there are two equations and
the corresponding planes are not parallel, then they intersect and the set of
points in this intersection is a line, which is a one dimensional subset in a three
dimensional space.
Example 2.5.3. Find the line given by the intersection of the planes {+|+} =
1 and 6{ + 2| + 3} = 6= The direction vector of the line must be perpendicular
$
$
to both normal vectors, which are
q 1 = [1 1 1] and
q 2 = [6 2 3]= Since the
cross product of two vectors is perpendicular to both vectors, we can choose
90
$

the direction vector g to be

$
$
q1
q2
5 $
$ 6

$
l
m
n
= det(7 1 1 1 8)
6 2 3
$

$

$

= l (1) m (3) + n (4)
= [1 3 4]=
Find a point on both planes by setting } = 0 and solve the equations {+|+0 = 1
and 6{ + 2| + 0 = 6 to obtain { = 1 and | = 0= Then the vector equation for
$
the line is
u = [1 0 0] + w[1 3 4]=
If there are p equations and q variables with p ? q, then one would like
to be able to describe the set of points satisfying the p equations. Problems
of this nature arise as parts of models for structures, circuits and uids. These
problems will be discussed in chapter ve.
2.5.5
Systems of Dierential Equations
In Subsection 2.3.3 the movement of a projectile was given by three dierential

equations, which described Newtons law of motion in the three directions. Let
the velocity vector be given by three components y1 > y2 and y3 and assuming a
unit mass so that the three equations are
g
y1
gw
g
y2
gw
g
y3
gw
= dy1 + g1 >
= ey2 + g2 and
= fy3 + g3 =
The matrix form is

6 5
65
6 5
6
5
d 0 0
y1
g1
y
g 7 1 8 7
y2 = 0 e 0 8 7 y2 8 + 7 g2 8 =
gw y
0 0 f
y3
g3
3
A generalization of this is to replace the 3 3 diagonal matrix by any 3 3 matrix. Moreover, applications often have a number of unknowns and dierential
equations so that the matrix would be replaced by an q q matrix where q is
the number of unknown functions. Chapters six and seven will start the study
of these important time dependent models.
2.5.6
Images and Matrices
Grayscale images are associated with p q matrices whose components are

integers. For 8-bit images the integers range from 0 to 255 = 28 1. For
91
16-bit images they range from 0 to 65535 = 216 1. The black image pixel is
associated with 0, and the white image pixel is associated with 255 (8-bit) or
65535 (16-bit). If these matrices are converted to oating point numbers, then
one can use matrix operations to change the image. For example, if the image
is too bright, then multiply the corresponding matrix by a constant less than
one so that the resulting pixels have smaller values.
In MATLAB one can view the image in several ways. First, just look at the
matrix components. Second, use the MATLAB command mesh() to generate a
surface of the image where the indices of the matrix are on the xy-plane and
the intensity of the image is on the z-axis. Third, one can map the matrix into
a standard image le such as a *.jpg le. This can be done by the MATLAB
command imwrite(). The inverse of the imwrite() is imread(), which generates
a matrix from a *.jpg le.
The following MATLAB function le creates a 5040 matrix that represents
the letter N. The input parameter g in the function lettern(g) should be between
0 and 255 and will indicate how bright the letter will be. Other letter generating
functions are similar.
MATLAB function lettern()
function letn = lettern(g)
letn = zeros(50,40);
letn(10:40,4:8) = g;
letn(10:40,32:36) = g;
for j = 8:32
letn(48-j,j) = g;
letn(47-j,j) = g;
letn(46-j,j) = g;
letn(45-j,j) = g;
letn(44-j,j) = g;
letn(43-j,j) = g;
letn(42-j,j) = g;
end
The MATLAB code le imagncsu.m creates the letters NCSU and displays
them via mesh(), imwrite() as a *.jpg le and imwrite() for the negative image,
see Figures 2.5.1, 2.5.2 and 2.5.3.
MATLAB code le imagncsu.m
ncsu = [lettern(5) letterc(7) letters(9) letteru(11)];
mesh(ncsu);
newncsu = 20*ncsu;
newncsu = newncsu(50:-1:1,:);
negncsu = 255*ones(size(newncsu))-newncsu;
newncsu1 = uint8(newncsu);
imwrite(newncsu1, ncsu.jpg);
negncsu1 = uint8(negncsu);
imwrite(negncsu1, negncsu.jpg);
92
Figure 2.5.1: Mesh of Image Matrix
Figure 2.5.2: Imwrite of Image Matrix
Images and signals often have unwanted noise associated with the transmission and reception. Matrix manipulations can "lter" some of the unwanted
attributes. Chapters eight and nine will investigate space and frequency lters.
The frequency lters use the "fast Fourier transform." This is associated with
matrix products where the Fourier matrix has complex numbers as components.
Figure 2.5.3: Negative Image Matrix
2.5.7
93
Exercises
1.
A computer store stocks ve dierent laptops and has 5> 6> 2> 1 and 5 units
of the respective models. Suppose the prices per unit are 1000> 850> 1700> 2220
and 1150, respectively.
(a).
Find the stock and price vectors.
(b).
Compute this stores total value of the laptop stock by using the
dot product of these vectors.
2.
Consider the computer store in exercise one. The same owner wishes to
open a second store with the same ve laptop models and 4> 5> 3> 3 and 2 units
of the respective models.
(a).
Find the stock and price vectors for the second store.
(b).
Compute the total value of the stock in both stores by using the
dot product.
3.
Let ua + vb + wc = r0 where aW = [1 0 1]> bW = [2 1 1]> cW = [4 1 4]
and rW0 = [10 1 4]=
(a).
Write this as a matrix equation.
(b).
Use determinants and Cramers rule to solve the matrix equation.
4.
Let ua + vb + wc = r0 where aW = [1 0 1]> bW = [3 1 2]> cW = [2 1
4] and rW0 = [10 1 4]=
(a).
Write this as a matrix equation.
(b).
Use determinants and Cramers rule to solve the matrix equation.
5.
Two planes are given by 2{ + | + 3} = 12 and { + 3| + } = 6=
(a).
Find the normal vectors to each plane.
(b).
Find the vector equation for the line given by the intersection of the
two planes.
6.
Two planes are given by { + | + } = 3 and { + 3| + } = 12=
(a).
Find the normal vectors to each plane.
(b).
Find the vector equation for the line given by the intersection of the
two planes.
7.
Use the MATLAB command A\d to solve the matrix equation in exercise
three.
8.
Use the MATLAB command A\d to solve the matrix equation in exercise
four.
9.
Use the MATLAB function le lettern(g) to experiment with dierent
brightness levels. Use mesh(letn) to view the letter n, for example,
AA lettern(50)
AA mesh(letn)
10.
Use the MATLAB function le lettern(g) as a template to create letter
matrices for your rst name.
11.
Use the MATLAB code le imagncsu.m as a template to create a *.jpg
le for your rst name.
Chapter 3
Ax = d: Unique Solution
This and the next two chapters contain the basic methods for solving matrix
equations of the form Dx = d where D is a matrix and x and d are column vectors. A much more complete description of solution methods and linear algebra
can be found in [5]. The possibilities of a unique solution, no solution, multiple solutions and least square solutions are discussed. In this chapter most
of the algebraic systems are assumed to be square so that the number rows
(equations) and columns (variables) are equal. Row operations and elementary
matrices are used to do by-hand computations of the Gauss elimination, LU factorization and inverse matrix methods. These methods are also implemented in
MATLAB. Applications to steady state circuits, mixing tanks, heat conduction
and support trusses are given.
3.1
Matrix Models
The extension from vectors with two or three components and from 2 2 or
3 3 matrices to higher dimensions will be motivated by graphics and a variety
of applications. In this section the simplest model of cost, circuits and trusses
will be presented. We will return to these models in the remainder of the text
so as to increase their complexity, time dependence and accuracy.
3.1.1
Column Vectors and Rq
In Section 2.5 price, inventory and graphing vectors were introduced. Vectors
may either be row or column vectors and usually are an ordered list of real or
complex numbers. However, they could have more general components such as
a phone book where the components are triples having a name, address and
phone number. In the remainder of the text we will assume the vector is a
column and will use a bold lower case font to denote vectors.
95
CHAPTER 3. AX = D: UNIQUE SOLUTION
96
Denition 3.1.1. An q1 column vector is an ordered list of q real or complex

numbers. This will be described in two ways
5
6
d1
9
:
a = 7 ... 8 or a = [dl ] where l = 1> > q=
dq
The set of all q 1 column vectors with real components is denoted by Rq , and
Cq is the set of all q 1 column vectors with complex components.
Four basic operations with vectors are scalar product, addition, augmentation and transpose.
Denitions 3.1.2. Let a and b be q 1 column vectors and let v be a real or
complex number.
Scalar product va is another q 1 column vector whose components are vdl
6
5
vd1
:
9
va 7 ... 8 = [vdl ] where l = 1> > q=
vdq
Notation: va = [vdl ] = [dl v] = av=
Vector addition is a + b is another q 1 column vector whose components
are dl + el
6
5
d1 + e1
:
9
..
a + b 7
8 = [dl + el ] where l = 1> > q=
.
dq + eq
Vector augmentation of a and b denoted by [a b] is an q 2 matrix with
two column vectors where a is the rst column of the matrix
5
6
d1 e1
9
.. : = [d e ] where l = 1> > q=
[a b] = 7 ...
l
l
. 8
dq
eq
One can augment p q 1 column vectors to form an q p matrix, which will

be denoted by upper case fonts.
Transpose of a is a 1 q row vector aW
aW = [d1 dq ]=
Example 3.1.1. Consider the three 4 1 column vectors
5 6
5
6
5
6
1
3
0
9 6 :
9
:
9
:
: > b = 9 2 : and c = 9 1 : =
a =9
7 3 8
7 1 8
7 4 8
5
3
2
3.1. MATRIX MODELS
97
One can combine scalar multiplication and vector addition to form a linear
combination of these vectors
5 6
5
6 5
6
1
3
0
9 6 :
9
: 9
:
: + 39 2 : 9 1 :
2a+3b c = 2 9
7 3 8
7 1 8 7 4 8
5
3
2
6
6 5
5
11
2(1) + 3(3) 0
9 2(6) + 3(2) 1 : 9 17 :
:
: 9
= 9
7 2(3) + 3(1) 4 8 = 7 1 8 =
21
2(5) + 3(3) (2)
This is in contrast to the augmentation of the three column vectors, which is a
4 3 matrix or array
5
6
1 3
0
9 6 2
1 :
:
D=9
7 3 1 4 8 =
5 3 2
The set of q 1 column vectors Rq is called a vector space because it has
the properties listed in the next theorem.
Theorem 3.1.1 (Vector Space Properties of Rq ) Let a, b and c be in Rq and
let v and w be real numbers. Let a + b and va denote vector addition and scalar
product. Then the following hold:
a + b 5 Rq , a + 0 = a where 0 5 Rq has zeros as components, a + (a) = 0,
a + b = b + a, a + (b + c) = (a + b) + c;
va 5 Rq , 1a = a, v(a + b) = va + vb, (v + w)a = va + wa and v(wa) = (vw)a=
The denitions of dot product and norms for vectors in R2 and R3 , see Sections 1.4 and 2.1, have the following generalizations and analogous properties.
Denitions 3.1.3. Let a and b be given vectors in Rq = The dot product of a
and b is a real number dened by
a b aW b = d1 e1 + d2 e2 + + dq eq =
The vectors a and b are called orthogonal if and only if their dot product in
zero
a b = aW b = 0=
The Euclidean norm of a is a real number dened by
kak (a a)1@2 = (aW a)1@2 = (d21 + d22 + + d2q )1@2 =
In order to illustrate the dot product and norm, consider Example 3.1.1
where q = 4
a b = 1(3) + 6(2) + 3(1) + 5(3) = 27 and
s
kak = (12 + 62 + 32 + 52 )1@2 = 71=
98
Many properties of the dot product and norm, that hold for vectors in the plane
and space, also hold for vectors in Rq = The proofs of these are similar except
for the Cauchy-Schwarz inequality.
Theorem 3.1.2 (Properties of Dot Product) Let a> b and c be given vectors
in Rq and let v be a real number. Then the following hold
a b = b a> v(a b) = (va) b>
a (b + c) = a b + a c and
|a b| kak kbk (Cauchy-Schwarz inequality);
(3.1.1)
Theorem 3.1.3 (Properties of Norm) Let a and b be given vectors in Rq and

let v be a real number. Then the following hold
kak 0; a = 0 if and only if kak = 0>
kvak = |v| kak and
ka + bk kak + kbk (triangle inequality).
The Cauchy-Schwarz inequality in (3.1.1) for vectors in R2 and R3 follows
from the cosine identity in (2.1.2). The proof for vectors in Rq makes use of
the dot product properties and a clever observation. Let w be a real number
and dene the following function with b 6= 0
i (w) (a + wb) (a + wb) 0=
Note, it can be written as a quadratic function of w
i (w) a a + 2wa b + w2 b b=
The rst and second derivatives are
gi
gw
g2 i
gw2
= 0 + 2a b + 2wb b and
= 0 + 2b b=
Choose w0 so that i (w0 ) is a minimum, that is,

gi
= 0 + 2a b + 2w0 b b = 0=
gw
Thus,
a b
and
bb
i (w0 ) = a a + 2w0 a b + w20 b b
w0
= aa
(a b)
0=
bb
The inequality is equivalent to the Cauchy-Schwarz inequality.
3.1. MATRIX MODELS
3.1.2
99
Matrices
In Section 2.5 matrices were used to store the intensity at each point or pixel
in an image. Matrices can be used to store a variety of information, which
often requires visualizing or modications. There are four element-wise or array
operations that will be very useful.
Denitions 3.1.4. An augmentation of q p1 column vectors with 1 m q
of the form
5
6
d1m
9 .. :
7 . 8
dpm
is an p q matrix D where the rst index 1 l p is the row number, the
second index 1 m q is the column number and
6
5
d11 d12 d1q
9
..
.. : = [d ] =
D = 7 ...
lm
.
. 8
dp1
dq2
dpq
Let D and E be p q matrices. Dene the following four array operations,

which generate additional p q matrices:
i (D)
D= E
D=@E
D=n

[i (dlm )] >
[dlm elm ] >
[dlm @elm ] and
[dlm n] =
Example 3.1.2. Consider the 4 3 matrix in Example 3.1.1 formed by augmenting the three column vectors. The component in the third row and rst
column is d31 = 3> and d23 = 1 is the component in the second row and third
column. Array operations with exponents are not the same as matrix products
5 2
6 5
6
1
32
02
1 9 0
9 62
9
:
22
12 :
: = 9 36 4 1 : =
D=2 = 9
2
7 32 (1)2
8
7
9 1 16 8
4
2
2
2
25 9 4
5
3
(2)
Example 3.1.3. This example illustrates how array operations and matrices
can be used to create a graph in MATLAB of a surface given by } = i ({> |) =
100 {2 4| 2 = In order to graph the surface, a collection of points in the
xy-plane must be selected and then the values of i ({> |) at these points must
be computed and stored. Suppose three points in the x-direction are { = 0> 4
and 8, and four points in the y-direction are | = 0> 2> 4 and 6. There are
a total of 12 points in the xy-plane; the 12 values of { can be stored in a
4 3 matrix [, and the 12 values of | can be stored in a 4 3 matrix \= The
corresponding 12 values of i ({> |) can be computed and stored in a 43 matrix
100
I= The following MATLAB commands do this. Note, the row 4 and column 3
component of I is 100 82 4(62 ) = 108= By using smaller increments in
the MATLAB command meshgrid(), one can generate much larger matrices and
a more accurate depiction of the surface, which is graphed by the MATLAB
command mesh(X,Y,F).
AA [X Y] = meshgrid(0:4:8,0:2:6)
X=
048
048
048
048
Y=
000
222
444
666
AA F = 100 - X.^2 - 4*Y.^2
F=
100 84 36
084 68 20
036 20 -28
-44 -60 -108
AA mesh(X,Y,F)
3.1.3
Application to Visualization of Minimum Cost
The objective is to nd the dimensions of a 3D box with xed volume and with
minimum cost. In order to approximate the solution, the cost function will
be graphed, and we will inspect the graph for a minimum cost. Suppose the
volume must be 1000 ft3 , the bottom cost 3 $/ft2 , the four sides cost 1 $/ft2 and
there is no top. Let the bottom have edges equal to { and |, and let the height
of the box be }. This is illustrated by Figure 3.1.1. The cost of the bottom is
3{|> and the cost of the sides is 1(|} + |} + {} + {})= Since the volume must
be 1000, 1000 = {|} or } = 1000@({|)= Because the total cost equals the sum
of the cost of the bottom and the four sides,
F({> |) = 3{| + 2(|} + {})
= 3{| + 2(|1000@({|) + {1000@({|))
= 3{| + 2(1000@{ + 1000@|)=
The following MATLAB commands create the surface and contour plots of
this cost function as given in Figure 3.1.2. The three matrices [, \ and F are
20 20 and note how array operations are used to compute the possible costs
in the matrix F. An estimate of the minimum cost is given by { = 9 and | = 9
3.1. MATRIX MODELS
101
Figure 3.1.1: Box with Fixed Volume

so that F(9> 9) = 243 + 4000@9 687=4444= A better estimate is given by using
multivariable calculus where one sets the partial derivatives equal to zero and
solve for { and | to attain { = | = 8=7358 and minimum cost equal to 686=8285.
AA
AA
AA
AA
AA
AA
3.1.4
[X Y ] = meshgrid(1:1:20,1:1:20);
C = 3*X.*Y + 2000./Y + 2000./X;
subplot(1,2,1)
mesh(X,Y,C)
subplot(1,2,2)
contour(X,Y,C, 650:25:900)
Application to Two-bar Truss
Consider the two-bar truss in Figure 3.1.3. This truss is attached to a wall on
the left and can be used to support a mass on the node joining the two bars. In
the truss the bar ends are always pin connected. Consequently, the force in the
bars are always along the axis. Assume there is no deformation and the bars
are in a static state. The horizontal bar will have compression force i2 ? 0> and
the diagonal bar will have a tension force i1 A 0. At the joining node there are
three force vectors whose sum must be the zero vector to satisfy the equilibrium
equations
[i1 cos() i1 sin()] + [i2 0] + [0 z] = [0 0] or
[i1 cos() i2 i1 sin() z] = [0 0]=
By equating the rst and second components of the vector equation, we obtain
two scalar equations
i1 cos() i2 = 0 and
i1 sin() = z=
The matrix version of this is

0
cos() 1
i1
=
=
z
i2
sin()
0
102
Figure 3.1.2: Cost of a Box
Figure 3.1.3: Two-bar Truss
The solution is easy to compute. For example, if z = 100 and = @6> then

i1
i2

=
200s
100 3
If decreases, then the tension and compressive force magnitudes will increase.
If there are more than two bars, then the number of joints will increase as well
as the number of force vectors and force vector equations. This eventually leads
to much larger algebraic systems as demonstrated in Subsections 3.4.4.
3.1. MATRIX MODELS
103
Figure 3.1.4: Two-loop Circuit
3.1.5
Application to Two-loop Circuit
Consider the two-loop circuit in Figure 3.1.4 with three resistors and two batteries. The current going through resistors U1 > U2 and U3 will be denoted by l1 > l2
and l3 and will have directions indicated by the arrows in the gure. Kirchhos
current law requires the sum of the currents at any node be zero
l1 l2 + l3 = 0=
Ohms law states that the voltage drop across each resistor is the resistance
times the current. Kirchhos voltage law requires the sum of the voltage drops
in each loop be equal to zero
H1 U1 l1 + U3 l3
H2 U2 l2 U3 l3
The matrix version for the three unknown
equations is
5
65
1
1
1
7 U1
0
U3 8 7
0 U2 U3
= 0
= 0=
currents and the above three scalar

6 5
6
l1
0
l2 8 = 7 H1 8 =
l3
H2
The solution of this 3 3 algebraic system can be found in a number of

ways. For example, if U1 = 1> U2 = 2> U3 = 3, H1 = 10 and H2 = 20> then one
could use Cramers rule, see Sections 2.5 and 3.7, to nd the currents. Let
5
6
5
6
1 1 1
0
D = 7 1 0 3 8 and d = 7 10 8 and note
0 2 3
20
det(D) = 1(0 (3)(2)) (1)(1(3) 0) + 1(1(2) 0) = 11=
104
The solutions are
5
l1
l2
l3
6
0 1 1
= det(7 10 0 3 8)@ det(D) = 10@(11)>
20 2 3
5
6
1 0
1
= det(7 1 10 3 8)@ det(D) = 50@(11) and
0 20 3
5
6
1 1 0
= det(7 1 0 10 8)@ det(D) = 40@(11)=
0 2 20
Most circuits have many more loops so that the resulting algebraic systems are
very large. In these cases it is not practical to solve them using Cramers rule.
In the following sections viable alternative methods will be developed.
3.1.6
Exercises
Let aW = [2 5 7 1 1], bW = [0 1 2 7 4] and cW = [1 4 2 1 3]=

(a).
Find 3a 4b + 2c=
(b).
Find the augmentations of a, b and c=
(c).
Find a b=
(d).
Find kak =
2.
Let aW = [0 8 3 1 2], bW = [1 1 2 4 4] and cW = [2 1 5 1
3]=
(a).
Find 2a + 4b 3c=
(b).
Find the augmentations of a, b and c=
(c).
Find b c=
(d).
Find kck =
3.
Consider Example 3.1.2.
(a).
Find D=3.
(b).
Find a=@b where a and b are the rst and second columns of D.
(c).
Find vlq(D).
4.
(a).
Find D=(1@2).
(b).
Find b= c where b and c are the second and third columns of D.
(c).
Find b c=
5.
Consider the surface } = i ({> |) = 400 2{2 | 2 =
(a).
By hand compute the 2 3 matrix

i (1> 1) i (1> 2) i (1> 3)

I =
=
i (2> 1) i (2> 2) i (2> 3)
1.
6.
(b).
Use MATLAB and mesh(X,Y,F) to compute F and graph.
Consider the surface } = i ({> |) = 200 {2 3| 2 =
3.2. MATRIX PRODUCTS

(a).
105
By hand compute the 2 3 matrix

i (1> 1) i (1> 1=5) i (1> 2)

I =
=
i (2> 1) i (2> 1=5) i (2> 2)
(b).
Use MATLAB and mesh(X,Y,F) to compute F and graph.
7.
Consider the minimum cost of the box with no top, volume equals 200,
bottom costs 4 $/ft2 and sides cost 1 $/ft2 =
(a).
Find the cost function.
(b).
Use MATLAB and mesh(X,Y,C) to approximate the minimum cost.
8.
Consider the minimum cost of the box with a top costing 2 $/ft2 , volume
equals 200, bottom costs 4 $/ft2 and sides cost 1 $/ft2 =
(a).
Find the cost function.
(b).
Use MATLAB and mesh(X,Y,C) to approximate the minimum cost.
9.
Consider the two-bar truss with z = 100 and variable = @12> 2@12>
3@12 and 4@12= Find the forces on the two bars.
10.
Consider the two-bar truss with = @6 and variable z = 100> 200> 300
and 400= Find the forces on the two bars.
11.
Consider the two-loop circuit with H1 = 10> H2 = 20> U1 = 1> U2 = 2
and U3 = 2= Find the three currents.
12.
Consider the two-loop circuit with arbitrary batteries and resistors.
Find the three currents and show the center current must be negative.
3.2
Matrix Products
In the previous sections matrices were used to describe systems of equations

with two or three unknowns. Most realistic models have many more unknowns,
and the resulting matrices are much larger. The general matrix-vector and
matrix-matrix products will be dened and their properties will described. An
application to heat conduction in a thin wire will be introduced, and this model
will be enhanced in subsequent sections.
An algebraic system for three unknowns {1 > {2 and {3 can be listed either
as three scalar equations or as a vector equation
d11 {1 + d12 {2 + d13 {3
d21 {1 + d22 {2 + d23 {3
d31 {1 + d32 {2 + d33 {3
The vector equation has
5
d11 d12
7 d21 d22
d31 d32
= g1 >
= g2 and
= g3 =
the form
65
6
5
6
d13
{1
g1
d23 8 7 {2 8 = 7 g2 8 or
d33
{3
g3
Dx = d where D is 3 3=
106
If there were two equations with four unknowns, then

d11 {1 + d12 {2 + d13 {3 + d14 {4
d21 {1 + d22 {2 + d23 {3 + d24 {4
= g1 and
= g2 .
The vector form is

3.2.1
d11
d21
d12
d22
d13
d23
d14
d24
6
5
6
g1
{1
9 g2 :
9 {2 :
:
9
:
9
7 {3 8 = 7 g3 8 or
{4
g4
Dx = d where D is 2 4=
5
Matrix-vector Products
In the product of a 3 3 matrix times a 3 1 column vector we previously

noted that this can be done by either products of rows in the matrix and the
column vector, or by linear combinations of the columns of the matrix. The
row version is
5
65
6 5
6
d11 d12 d13
{1
d11 {1 + d12 {2 + d13 {3
7 d21 d22 d23 8 7 {2 8 = 7 d21 {1 + d22 {2 + d23 {3 8 =
d31 d32 d33
{3
d31 {1 + d32 {2 + d33 {3
The column version is
65
6
5
6
5
6
5
6
5
{1
d11
d12
d13
d11 d12 d13
7 d21 d22 d23 8 7 {2 8 = {1 7 d21 8 + {2 7 d22 8 + {3 7 d23 8 =
d31 d32 d33
{3
d31
d32
d33
This important observation also holds for general matrix-vector products.
Denitions 3.2.1. Let D and E be p q matrices, a and x be q 1 column
vectors and let v be a real number. The lm-component of D is dlm where l is the
row number and m is the column number.
Row vector aW times a column vector x is a real number equal to the sum
of the products of the components (also called the dot product of a and x)
aW x d1 {1 + + dq {q =
Matrix D times a column vector x is a q 1 column vector whose lwk
component is row l of D times the column vector x
Dx [dl1 {1 + + dlq {q ] where l = 1> > p=
The scalar v times a matrix D is another p q matrix whose lm-component
is v times the lm-component of D
vD [vdlm ]=
107
Notation: vD = [vdlm ] = [dlm v] = Dv=

The addition of two matrices D and E is another p q matrix whose
lm-component is the sum of the lm-components of D and E
D + E [dlm + elm ]=
Example 3.2.1. Let D
5
1
9 1
D=9
7 7
2
and E be 4 2 and x be a 2 1 column vector

6
5
6
2
1 3

9
:
0 :
: > E = 9 2 1 : and x = 3 =
7 0 5 8
4 8
4
3
2 4
Then Dx is dened to be a 4 1 column vector

5
6 5
1(3) + 2(4)
9 1(3) + 0(4) : 9
: 9
Dx = 9
7 7(3) + 4(4) 8 = 7
2(3) + 3(4)
6
11
3 :
:=
37 8
18
The matrix vector product could also have been computed by a linear combination of the two column vectors of D
6
6 5
6 5
6
5 6 5
5
11
8
3
2
1
9 0 : 9 3 : 9 0 : 9 3 :
9 1 :
:
: 9
: 9
:
9 : 9
Dx = 3 9
7 7 8 + 4 7 4 8 = 7 21 8 + 7 16 8 = 7 37 8 =
18
12
6
3
2
If

F=
2 4
1 2
>
then D + F is not dened because their row and column numbers are not equal.
If v = 7, then vD is dened to be the 4 2 matrix
5
6 5
6
7 14
7(1) 7(2)
9 7(1) 7(0) : 9 7 0 :
: 9
:
vD = 9
7 7(7) 7(4) 8 = 7 49 28 8 and
14 21
7(2) 7(3)
6
6 5
5
0 5
11 2+3
9 1 + 2 0 + 1 : 9 1 1 :
:
: 9
D+E = 9
7 7 + 0 4 + 5 8 = 7 7 9 8=
0 7
22 3+4
Theorem 3.2.1 (Matrix-vector Products Properties) Let D> E and F be pq
matrices. Let ] be a p q matrix with all components equal to zero, called the
zero matrix. Let x and y be q 1 column vectors. Then the following are true:
108
1.
D + (E + F) = (D + E) + F, D + E = E + F and D + ] = D=
2.
D(vx) = (vD)x where v is a real number, D(x + y) = Dx + Dy.
3.
The matrix-vector product can be computed as a linear combination

of the columns. Let column m of D be denoted by a:m
Dx = {1 a:1 + + {q a:q =
The proofs of the above six properties are routine. The last property is the
column version of a matrix-vector product whose proof is
5
6
d11 {1 + + d1p {p
9
:
..
Dx = 7
8
.
dq1 {1 + + dqp {p
5
6
5
6
d11 {1
d1p {q
9
:
9
:
..
..
= 7
8 + + 7
8
.
.
dq1 {1
dqp {p
5
6
5
6
d11
d1q
9
:
9
:
= {1 7 ... 8 + + {q 7 ... 8 =
dp1
dpq
Example 3.2.2. Verify the matrix addition properties in the above theorem
for
5
6
5
6
5
6
1 2
1 3
1 5
D = 7 2 0 8 > E = 7 2 1 8 and F = 7 2 3 8 =
4 6
2 5
4 3
6
5
6 5
6
1 2
1 3
1 5
D + (E + F) = 7 2 0 8 + (7 2 1 8 + 7 2 3 8)
4 6
2 5
4 3
5
6
1 + (1 + 1) 2 + (3 5)
= 7 2 + (2 + 2) 0 + (1 + 3) 8
4 + (2 + 4) 6 + (5 + 3)
5
6
(1 1) + 1 (2 + 3) 5
= 7 (2 + 2) + 2 (0 + 1) + 3 8 = (D + E) + F=
(4 + 2) + 4 (6 + 5) + 3
109
6 5
6
1 2
1 3
= 7 2 0 8+7 2 1 8
4 6
2 5
5
6
11 2+3
= 7 2+2 0+1 8
4+2 6+5
5
6
11 3+2
= 7 2 + 2 1 + 0 8 = E + D=
2+4 5+6
D+E
D+]
3.2.2
6 5
6
1 2
0 0
= 7 2 0 8+7 0 0 8
4 6
0 0
5
6
1+0 2+0
= 7 2 + 0 0 + 0 8 = D=
4+0 6+0
Matrix-matrix Products
Matrix-vector products may be extended to matrix-matrix products as long

as the number of columns in the left matrix is the same as the number of
rows in the right matrix. Another important fact is that the order of matrixmatrix product is important, that is, DE may not be the same as ED! Like
the matrix-vector product there is a row version and a column version. For
example, consider the product of two 2 2 matrices

2 3
4 5
6 7
8 9
2(6) + 3(8) 2(7) + 3(9)

4(6) + 5(8) 4(7) + 5(9)

2 3
6
2 3
7
=
+
=
4 5
8
4 5
9

This product is either computed by rows times columns or by matrix times

columns. Initially, we use the rows times columns approach for the general
denition of a matrix-matrix product.
Denition 3.2.2. Let D be an p q and let E be an q s matrices. The
product DE is an p s matrix whose lm-component is the product of row l of
D times column m of E
DE [dl1 e1m + + dlq eqm ]=
110
Example 3.2.3. First, let

1
D =
3

2
DE =
4
2
4
1
3
0 1
1 0
and E =
and note

3 4
6=
= ED=
1 2
Second, note DE may be dened while ED is not dened. Let D be 2 3 and

E be 3 3
5
6

2 1 0
2 4 7
D=
and E = 7 1 2 3 8 =
8 9 10
0 1 1

DE
2(2) + 4(1) + 7(0) 2(1) + 4(2) + 7(1) 2(0) + 4(3) + 7(1)

8(2) + 9(1) + 10(0) 8(1) + 9(2) + 10(1) 8(0) + 9(3) + 10(1)

8 17 19
=
=
25 36 37
This product also can be written as columns of the product given by D times
the column of E
Db1 Db2 Db3 where E = b1 b2 b3

DE =
5 6

2

2 4 7 7 8
8
1 =
Db1 =
8 9 10
25
0
5 6

1

2 4 7 7 8
17
2 =
Db2 =
8 9 10
36
1
5 6

0

2 4 7 7 8
19
3 =
Db3 =
=
8 9 10
37
1
b be p q, E
Theorem 3.2.2 (Matrix-matrix Products Properties) Let D> D
be q s, F be s t and v be a real number. Then the following are true:
1.
D(EF) = (DE)F, v(DE) = (vD)E = D(vE) and

DE = D b1 bs = Db1 Dbs =
b = DE + DE=
b
(D + D)E
b = DE + DE
b follows from the distributive property
The proof of (D + D)E
of real numbers. Let 1 l p and 1 m s
2.
b
(D + D)E
dl1 )e1m + + (dlq + b

dlq )eqm ]
= [(dl1 + b
= [(dl1 e1m + b
dl1 e1m ) + + (dlq eqm + b
dlq eqm )]
= [dl1 e1m + + dlq eqm ] + [b
dl1 e1m + + b
dlq eqm ]
b
= DE + DE=

Example 3.2.4.
3 3 matrices
5
1 0
D=7 0 1
0 2@3
Verify the associative rule D(EF) = (DE)F for the following

6
5
6
5
6
0
1 0 0
2 1 0
0 8 > E = 7 1@2 1 0 8 and F = 7 1 2 1 8 =
1
0 0 1
0 1 2
5
1 0
D(EF) = 7 0 1
0 2@3
5
1 0
= 7 0 1
0 2@3
5
(DE)F
3.2.3
111
6 5
0
0 8 (7
1
6 5
0
0 8 (7
1
1 0 0
= (7 0 1 0
0 2@3 1
5
1
0
= (7 1@2 1
1@3 2@3
65
1 0 0
2
1@2 1 0 8 7 1
0 0 1
0
6
5
2 1 0
0 3@2 1 8) = 7
0 1 2
65
6
1 0
2 1 8)
1 2
6
2 1
0
0 3@2 1 8 =
0 0 4@3
6 5
1 0 0
2 1
8 7 1@2 1 0 8) 7 1 2
0 0 1
0 1
6 5
6 5
2 1 0
0
2
0 8) 7 1 2 1 8 = 7 0
1
0 1 2
0
6
0
1 8
2
6
1
0
3@2 1 8 =
0 4@3
Application to Heat Conduction
The objective is to predict the temperature of a mass given an initial temperature and the temperature on the boundary. A very complicated example is
the temperature inside a spacecraft as it enters the earths atmosphere. Such
models have a long history of previous less sophisticated models. In this section
we restrict the mass to a long thin wire so that one may assume the diusion
of thermal energy is only along the length of the wire. For the present model,
we assume no heat is lost through the lateral surface area. Heat may enter the
wire through the left or right ends of the wire. Thus, the temperature will only
vary continuously with time and position along the wire. More complicated
heat models can be found in later sections of this text and in [12].
The model will be a discrete approximation of both time and space, that is,
the time and space variables will be broken into a nite number of segments
and the temperature will be assumed to be constant within each segment. This
is depicted in Figure 3.2.1 where there are four space segments with length
{ = O@4 and cross sectional area D= The temperatures in the three interior
segments are x1 > x2 and x3 ; the temperatures at the left x0 and right x4 are
given. The heat energy in the lwk segment is
fxl (D{)
where is the density, D{ is the volume and f is the specic heat of the wire.
112
Figure 3.2.1: Heat Conduction in a Wire

The Fourier heat law states that the heat ows from hot to cold, and the
amount of heat moving normal to a cross section is directly proportional to
the product of the time interval, cross section area and the derivative of the
temperature normal to the cross section area. Any change in the heat energy
of the segment is from diusion from left or right sides. The proportionality
constant is called the thermal conductivity N so that the heat entering from
the right is
xl+1 xl
w D N
{
and the heat entering from the left is
w D N
xl xl1
=
{
The new heat of the lwk segment is modeled or approximated by

)(D{) = fxl (D{) w D N
f(xqhz
l
xl xl1
xl+1 xl
+ w D N
=
{
{
Divide by f(D{) and let (w@{2 )(N@f) to get for l = 1> 2 and 3
= xl (xl1 + 2xl xl+1 ).
xqhz
l
One may write this as three scalar equations
xqhz
1
xqhz
2
xqhz
3
= x1 (x0 + 2x1 x2 )>

= x2 (x1 + 2x2 x3 ) and
= x3 (x2 + 2x2 x4 ).
113
Or, as one vector equation

5
6 5
65
6 5
6
5 qhz 6
x1
2 0
x1
x0
x1
8 = 7 x2 8 7 2 8 7 x2 8 + 7 0 8
7 xqhz
2
qhz
0 2
x4
x3
x3
x3
uqhz = u Du + d=
The initial temperature is given and one can compute the temperature at the
next time step by using the above.
In order for the discrete model to give accurate approximations of the continuous heat conduction process, the step sizes in both time and space must
be suitably small, and for the above model they must satisfy the stability constraint 1 2 A 0= Since (w@{2 )(N@f), changing either the step sizes
or the physical properties will alter 1 2= If diusion of heat occurs in all
three directions, then the discrete model can have a large number of unknown
temperatures. For example, if each direction has 100 unknown temperatures,
then there will be 1003 unknown temperatures in a three dimensional box!
Example 3.2.5. Consider the above with O = 10> { = 10@4> = f = 1>
N = 1@1000> w = 1000 so that (w@{2 )(N@f) = 16@100 = 0=16= If the
initial temperature is zero in the interior, the left side is 200 and the right side
is 70> then
x0 = 200> x1 = x2 = x3 = 0 and x4 = 70=
The approximation for the temperature at time equal to 1000 is u1 where the
superscript denotes the rst time step
u1
= u Du + d
5 6
5
65 6 5
6
0
2 1 0
0
(0=16)200
8
0
= 7 0 8 (0=16) 7 1 2 1 8 7 0 8 + 7
0
0 1 2
0
(0=16)70
5
6
32=0
= 7 0 8=
11=2
The approximate temperature at the second time step equal to 2000 is u2 where
the superscript denotes the second time step
u2
= u1 D u1 + d
5
6
5
65
6 5
6
32=0
2 1 0
32=0
(0=16)200
8
0
= 7 0 8 (0=16) 7 1 2 1 8 7 0 8 + 7
11=2
0 1 2
11=2
(0=16)70
5
6
53=7600
= 7 6=9120 8 =
18=8160
114
Figure 3.2.2: Steady State Heat Diusion

Note u2 can be represented by matrix-matrix products
u2
= (u Du + d) D (u Du + d) + d
= u 2Du + D2 u+2d Dd=
One can continue solving for the temperature at the next time step. This is illustrated in Figure 3.2.2, which was generated by the MATLAB code wire_time.m.
As the time steps increase, the temperature vector reaches a steady state, which
is a straight line. The means for larger time steps
u u Du + d=
Hence, the steady state solution
5
2 1
(0=16) 7 1 2
0 1
is dened to be the solution of Du = d

65
6 5
6
0
x1
(0=16)200
8=
1 8 7 x2 8 = 7
0
2
(0=16)70
x3
The solution is x1 = 167=5, x2 = 135=0 and x3 = 102=5=
3.2.4
Matrix Computations Using MATLAB
The calculations for the heat conduction application can easily be done by
MATLAB. Here we have computed the rst three time steps and the steady state
solution. The interested reader should examine the MATLAB code wire_time.m
to see how this can be done using for-loops.

AA A = .16*[2 -1 0;-1 2 -1;0 -1 2]
% creates a 3x3 matrix
A=
0.3200 -0.1600 0.0000
-0.1600 0.3200 -0.1600
0.0000 -0.1600 0.3200
AA B = eye(3) - A
% eye(3) is 3x3 identity matrix
B=
0.6800 0.1600 0.0000
0.1600 0.6800 0.1600
0.0000 0.1600 0.6800
AA d = [.16*200 0 .16*70]
d=
32.0000
0
11.2000
AA u = zeros(3,1)
u=
0
0
0
AA newu1 = B*u+d
% rst time step
newu1 =
32.0000
0
11.2000
AA newu2 = B*newu1+d
% second time step
newu2 =
53.7600
6.9120
18.8160
AA newu3 = B*newu2+d
% third time step
newu3 =
69.6627
16.3123
25.1008
AA SSu = A\d
% solves Au = d
SSu =
167.5000
135.0000
102.5000
115
116
3.2.5
Exercises
1.
Compute Dx (both row and column versions), D + E, D + F and vD
where

1 2 3 4
1 3 3 4
D =
> F=
>
4 1 2 5
2 1 12 5
5 6
2

9 1 :
0 22 2 4
:
E =
> x=9
7 5 8 and v = 9=
1 1 3 7
4
2.
Compute Dx (both row and column versions), D + E, D + F and vD
where

1 5 3
3 7
D =
> F=
> v = 7>
8 1 2
0 2
5
6

2
1 0 10
E =
and x = 7 11 8 =
2 3 1
5
3.
Verify D + (E + F) = (D + E) + F where D> E and F are the same as in
exercise one.
4.
Verify D(2x+3y) = 2Dx+3Dy where D and x are the same as in exercise
two and yW = [1 3 2 2].
5.
Compute DE (use both row and column versions) where
5
6

1 0 2 1
1 5 3
D=
and E = 7 5 4 7 1 8 =
8 1 2
1 2 0 1
6.
7.
8.
Compute ED (use both row and column versions)

6
5
5
1
2
1 0 2
9 5 2 :
: and E = 7 5 4 7
D=9
7 1 0 8
1 2 0
7
8
where
6
1
1 8=
1
Verify the D(EF) = (DE)F where

5
6

1 2
1 5 3
3 7
D=
> E = 7 3 0 8 and F =
=
8 1 2
0 2
1 3
b = DE + DE
b where
Verify the (D + D)E

D=
1 5 3
8 1 2
b=
> D
1
0
1
3 11 0
6
1 2
and E = 7 3 0 8 =
1 3
3.3. SPECIAL CASES OF AX = D
117
9.
Consider Example 3.2.5 of heat conduction. Change the thermal conductivity from N = 1@1000 to N = 2@1000 and leave the other parameters the
same. Compute > u1 > u2 and the steady state solution.
10.
Consider Example 3.2.5 of heat conduction. Change the number of
segments from q = 4 to q = 5 and leave the other parameters the same.
Compute > u1 > u2 and the steady state solution. Note there are now four
unknowns, the vectors have four components, the matrix is 4 4> the left
temperature is x0 = 200 and the right temperature is x5 = 70=
11.
Use MATLAB to do the computations in exercise 9.
12.
13.
Modify the MATLAB code wire_time.m to do exercise 9 and observe
convergence to the steady state solution.
14.
Modify the MATLAB code wire_time.m to do exercise 10 and observe
convergence to the steady state solution.
3.3
Special Cases of Ax = d
In the previous sections we have solved algebraic systems with two and three
unknowns. These could be written as a vector equation Dx = d where D is an
q q matrix with q = 2 or 3, d is a given q 1 column vector and x is an
unknown q 1 column vector. The overall objective is be able to systematically
solve problems with much larger q. In this section some important special
matrices will be considered and an application to heat conduction in a wire will
continue to be studied.
3.3.1
Four Possible Classes of "Solutions"
Algebraic systems may have a solution, no solution or multiple solutions. Moreover, any solution may be very sensitive to the data in D or d= These will be
illustrated by four simple 2 2 systems. One can gain insight to these examples
by viewing them from both an algebraic and geometric perspective. Consider
two equations
d11 {1 + d12 {2
d21 {1 + d22 {2
= g1 and
= g2 =
Or, view this as a single vector equation

d11 d12
{1
g1
=
=
d21 d22
{2
g2
Plot {1 on the horizontal axis and {2 on the vertical axis. The rst scalar
equation may be written in slope-intercept form of a line
{2 = (d11 @d12 ){1 + g1 @d12 > if d12 6= 0=
118
Do the same for the second scalar equation. The two lines are either not parallel,
parallel and do not overlap, parallel and overlap, or they are "nearly" parallel.
If they are not parallel, then there is exactly one solution. If they are parallel
and do not overlap, then there is no solution. If they are parallel and overlap,
then there are multiple solutions given by the points on the single line. If the
two lines are "nearly" parallel, then small changes in the data can cause large
changes in the intersection, that is, the solution. In the last case the small
changes in the data can be caused by measurement, roundo or human errors.
Case One. If the lines are not parallel, then

d11 d12
det(D) = det(
) 6= 0=
d21 d22
Cramers rule can be used to nd the unique solution

g1 d12
d11
)@ det(D) and {2 = det(
{1 = det(
g2 d22
d21
g1
g2
)@ det(D)=
Case Two. The lines are parallel and do not overlap. For example,
{1 + {2
2{1 + 2{2
= 1 and
= 4=
The second equation can be divided by 2 to get {1 + {2 = 2, which contradicts

the rst equation! So, there is no solution.
Case Three. The lines are parallel and do overlap. For example,
{1 + {2
2{1 + 2{2
= 1 and
= 2=
The second equation can be divided by 2 so that {1 + {2 = 1, which is the

same as the rst equation. Any point on this line will be a solution of algebraic
system.
Case Four. This example should be of great concern because it raises the
issue of sensitivity of the solution to the data. Let represent a small variation
in the data for g1 = 1 and consider the system
{1 + {2
(1 106 ){1 + {2
= 1 + and
= 1=
If = 0> then the unique solution is {1 = 0 and {2 = 1= If = 103 >

then the unique solution is extremely dierent and is {1 = 1000 and {2 =
998=999= Moreover, for small variations in the right side the residuals, r =
d Dx, appear to be small
1 ({1 + {2 ) = 1 (1000 998=999) = =001 and
1 ((1 106 ){1 + {2 ) = 1 ((1 106 )1000 998=999) = 0!
119
Problems whose solutions are very sensitive to small changes in the data are
called ill-conditioned.
The challenge is to detect these possibilities for large systems. This chapter
is restricted to the unique solution case, and chapter ve studies the multiple
solution case. Another possibility is having more equations than unknowns,
which will be investigated in the fourth chapter.
3.3.2
Triangular Matrices
The simplest system to solve is the diagonal system where the equations are for
l = 1> > q
dll {l = gl .
The matrix version of this uses the diagonal matrix D = G
65
5
6
5
d11 0
0
{1
g1
:
9
..
9 {2 :
9 g2
:
9 0 d22 . . .
. :9
:
9
9
.. : = 9 ..
:9
9 .
.
.
7
8
7 .
.
.
.
.
7 .
.
.
0 8
{
gq
q
0
0 dqq
Gx = d=
6
:
:
:
8
If each component on the diagonal is not zero (dll 6= 0 for all l), then the
solution is {l = gl @dll =
More general qq matrices are lower triangular D = O and upper triangular
D=X
5
65
6
5
6
d11
0
0
{1
g1
9
:
.
..
9 g2 :
.. : 9
{2 :
9 d21 d22
.
:
9
:
9
:9
=
9
:
9 .. :
.
9 .
:7 . 8
.
.
7 . 8
..
..
.
7 ..
0 8
{
gq
q
dq1 dq>q1 dqq
Ox = d and
5
d11
9
9 0
9
9 .
7 ..
0
d12
d22
..
.
..
.
..
.
0
d1q
..
.
dq1>q
dqq
65
:9
:9
:9
:7
8
{1
{2
..
.
:
9
:
9
: = 9
8
7
{q
g1
g2
..
.
6
:
:
:
8
gq
X x = d=
Both of these are relatively easy to solve provided all the diagonal components
dll are not zero. The lower triangular system is solved by rst nding {1 and
then {2 and so on until {q is found; this is called forward substitution= The
upper triangular system is solved in reverse order by rst nding {q and then
{q1 and so on until {1 is determined; this is called backward substitution.
120
Example 3.3.1. Let q = 3
5
1
7 2
4
and solve the following lower triangular system

6 5
6
65
11
0 0
{1
3 0 8 7 {2 8 = 7 28 8 =
60
5 6
{3
The system is equivalent to the following three scalar equations

1{1
2{1 + 3{2
4{1 + 5{2 + 6{3
= 11>
= 28 and
= 60=
The rst equation gives {1 = 11= Put this into the second equation 2(11)+3{2 =
28 to compute {2 = (28 2(11))@3 = 2= Finally, put {1 = 11 and {2 = 2 into
the third equation and nd {3 = (60 4(11) 5(2))@6 = 1=
Another way to solve this is by using the column version of the matrix-vector
product
5 6
5 6
5 6 5
6
1
0
0
11
{1 7 2 8 + {2 7 3 8 + {3 7 0 8 = 7 28 8 =
4
5
6
60
The rst equation gives {1 = 11= Now, put this into the left column on the left
side and move it to the right side
5 6
5 6 5
6
5 6 5
6
0
0
11
1
0
{2 7 3 8 + {3 7 0 8 = 7 28 8 11 7 2 8 = 7 6 8 =
5
6
60
4
16
The second equation gives {2 = 2= The last step is to
column and move it to the right side
5 6 5
6
5 6 5
0
0
0
{3 7 0 8 = 7 6 8 2 7 3 8 = 7
6
16
5
put {2 = 2 into the left

6
0
0 8=
6
The third equation yields {3 = 1=

Example 3.3.2. Let q = 4
5
2 3
9 0 1
9
7 0 0
0 0
and solve the

65
0
1
9
2 3 :
:9
8
3 1 7
0
2
following upper triangular system

6 5
6
25
{1
9
:
{2 :
: = 9 1 :=
8
7
9 8
{3
6
{4
The four equivalent scalar equations are

2{1 + 3{2 + 0{3 + 1{4
1{2 2{3 + 3{4
3{3 1{4
2{4
=
=
=
=
25>
1>
9 and
6=
121
The solution of the last equation is {4 = 6@2 = 3= Put this into the third
equation and solve for {3 = (9 + 1(3))@3 = 4= The second equation becomes
1{2 2(4) + 3(3) = 1 so that {2 = 0= The rst equation gives 2{1 + 3(0) + 0(4) +
1(3) = 25 so that {1 = 11=
This can also be solved by using the column version of the matrix-vector
product
5 6
5 6
5
6
5
6 5
6
2
3
0
1
25
9 0 :
9 1 :
9 2 :
9 3 : 9 1 :
:
9 :
9
:
9
: 9
:
{1 9
7 0 8 + {2 7 0 8 + {3 7 3 8 + {4 7 1 8 = 7 9 8 =
0
0
0
2
6
The last equation gives {4 = 3= Put this into the right column on the left side
and move it to the right side
6
6 5
6
5
6 5
5 6
5 6
5
22
1
25
2
3
0
9 3 : 9 8 :
9 0 :
9 1 :
9 2 : 9 1 :
:
: 9
:
9
:
: 9
9 :
9
{1 9
7 0 8 + {2 7 0 8 + {3 7 3 8 = 7 9 8 3 7 1 8 = 7 12 8 =
0
2
6
0
0
0
The third equation gives {3 = 4= Put {3 = 4 into
side and move it to the right side
5 6
5 6 5
6
5
2
3
22
9 0 :
9 1 : 9 8 :
9
:
9 : 9
:
9
{1 9
7 0 8 + {2 7 0 8 = 7 12 8 4 7
0
0
0
the right column on the left

6 5
0
22
9 0
2 :
:=9
3 8 7 0
0
0
6
:
:=
8
The second and rst equations give {2 = 0 and then {1 = 11=

The solution process in all of the above cases requires that the diagonal
components not be zero. In this case, there is one and only one solution, which
can be found by using either the row or column versions of the matrix-vector
product. For large systems there may be some advantages to using one of the
two matrix-vector products. This is often dependent on the computing tools
used to do the solves.
Theorem 3.3.1 (Triangular Solves) Consider a lower or an upper triangular
matrix D = O or X= If each diagonal component of the matrix is not zero, then
the triangular systems Ox = d or X x = d have one and only one solution.
3.3.3
A Has LU Factorization
If the given matrix can be written as a product of a lower and upper triangular
matrices, then one may be able to solve Dx = d= Suppose D = OX and both O
and X have non-zero diagonal components. By the associative property
Dx = d
(OX )x = d
O(X x) = d=
122
So, solve Oy = d by forward substitution, and then solve X x = y by backward

substitution=
Example 3.3.3. Consider the steady
where
5
65
2 1 0
7 1 2 1 8 7
0 1 2
state heat
6 5
{1
{2 8 = 7
{3
problem in Example 3.2.5

6
200
0 8=
70
Observe the matrix can be factored as follows

5
6
5
65
6
2 1 0
1
0
0
2 1
0
7 1 2 1 8 = 7 1@2
1
0 8 7 0 3@2 1 8
0 1 2
0
2@3 1
0 0 4@3
D = OX=
Later we will learn two methods for nding these factors. First, solve Oy = d
65
5
6 5
6
|1
1
0
0
200
7 1@2
1
0 8 7 |2 8 = 7 0 8 =
0
2@3 1
70
|3
The equation associated with the rst row gives |1 = 200= The equation associated with the second row is (1@2)200+1|2 = 0 implies |2 = 100= The equation
associated with the last row is (2@3)100 + 1|3 = 70 implies |3 = 410@3= Second, solve X x = y
5
65
6 5
6
2 1
0
{1
200
7 0 3@2 1 8 7 {2 8 = 7 100 8 =
0 0 4@3
410@3
{3
The last row implies {3 = (410@3)@(4@3) = 102=5. The second row gives
(3@2){2 102=5 = 100 or {2 = 202=5@(3@2) = 135=0= The top row gives
2{1 135 = 200 or {1 = 335@2 = 167=5=
If the matrix is tridiagonal, a matrix whose only non-zero components are
in the diagonal, sub-diagonal and super-diagonal, then one may be able to nd
the OX factors by a process similar to the following. Let D be 3 3 tridiagonal
matrix and assume the O and X factors have the special forms
5
6
5
6
65
d11 d12 0
1
0 0
1 d12 0
7 d21 d22 d23 8 = 7 1 1 0 8 7 0 2 d23 8
0 2 1
0 d32 d33
0
0
3
5
6
1
d12
0
8=
d23
= 7 1 1 1 d12 + 2
0
2 2
2 d23 + 3
The matrices are equal if and only if they have the same number of rows and
columns and their ij-components are all equal. This gives us ve non-trivial
123
scalar equations in the above 3 3 matrix equation: d11 = 1 > d21 = 1 1 ,

d22 = 1 d12 + 2 , d32 = 2 2 and d33 = 2 d23 + 3 = If each of the 1 , 2
and 3 are non-zero, then we can solve for 1 and 2 as well as do the upper
triangular solve. As an illustration consider Example 3.3.3 where the diagonal
components are all equal to 2 and the other non-zero components are equal to
1 :
2
2
1
2
3.3.4
=
=
=
=
1 > 1 = 1 1 = 1 2 and so 1 = 1@2>

1 + 2 = (1@2) + 2 and then 2 = 3@2>
2 2 so that 2 = 2@3 and
2 + 3 gives 3 = 4@3=
Application to Heat in Wire with Current
Consider heat conduction in a long thin wire as in the previous section. Assume
there are ve segments, not four, so that the unknown temperatures are x1 > x2 >
x3 and x4 with x0 = 70 at the left end and x5 = 70 at the right end. Also,
assume there is an electrical current, which generates heat at a rate of i per
unit volume, per unit time. The heat in a small lwk segment is now modeled or
approximated by
xl xl1
f(xqhz
)(D{) = fxl (D{) w D N
+
l
{
xl+1 xl
w D N
+ i w (D{)=
{
Divide by f(D{) and let (w@{2 )(N@f) to get for l = 1> 2> 3 and 4
xqhz
= xl (xl1 + 2xl xl+1 ) + i w@(f)=
l
One may write this as four scalar equations
xqhz
1
xqhz
2
xqhz
3
xqhz
4
=
=
=
=
x1 (x0 + 2x1 x2 ) + i w@(f)>

x2 (x1 + 2x2 x3 ) + i w@(f)>
x3 (x2 + 2x2 x4 ) + i w@(f) and
x4 (x3 + 2x4 x5 ) + i w@(f)=
Or, view it as one vector equation

5 qhz 6
5
6 5
x1
x1
2 0
0
9 xqhz
:
9
:
9

2

0
x
2
9 2qhz : = 9
: 9
7 x3
8
7 x3 8 7 0 2
0
0 2
xqhz
x4
4
6 5
6
5
i w@(f)
x0
9 0 : 9 i w@(f) :
: 9
:
9
7 0 8 + 7 i w@(f) 8
i w@(f)
x5
qhz
u
= u Du + d=
65
6
x1
: 9 x2 :
:9
:
8 7 x3 8 +
x4
124
The time dependent solution can be generated by the MATLAB code wire _time
_cur.m, and the graphical solution is illustrated in Figure 3.3.1. The steady
state solution is given by u = u Du + d so that one must solve Du = d
65
6
5
6 5
6 5
x1
2 1 0
0
x0
i w@(f)
9 1 2 1 0 : 9 x2 : 9 0 : 9 i w@(f) :
:9
:
: 9
: 9
9
7 0 1 2 1 8 7 x3 8 = 7 0 8 + 7 i w@(f) 8 =
0
0 1 2
i w@(f)
x5
x4
Example 3.3.4. Consider the above with O = 10> q = 5> { = 10@5> = f =
1> N = 1@1000> w = 1000 so that (w@{2 )(N@f) = 25@100 = 0=25= If
the electrical current in the wire generates heat at a rate equal to i = 0=1> then
i w@(f) = 100 and the above steady state model becomes
5
65
6 5
6 5
6
2 1 0
0
x1
70
400
9 1 2 1 0 : 9 x2 : 9 0 : 9 400 :
9
:9
: 9
: 9
:
7 0 1 2 1 8 7 x3 8 = 7 0 8 + 7 400 8 =
0
0 1 2
70
400
x3
The reader should verify D = OX
5
65
1
0
0
0
9 1@2
9
1
0
0 :
9
:9
7 0
8
2@3
1
0 7
0
0
3@4 1
where O and X are given below

65
6 5
2 1
0
0
x1
9
: 9
0 3@2 1
0 :
: 9 x2 : = 9
8
7
0 0 4@3 1
x3 8 7
0 0
0 5@4
x3
6
470
400 :
:=
400 8
470
One must solve Du = O(X u) = d= The solution of Oy = d is |1 = 470=0>

|2 = 635=0> |3 = 823=3 and |4 = 1087=5= The solution of X u = y is x1 = 870=0>
x2 = 1270=0> x3 = 1270=0 and x4 = 870=0= This is consistent with the graphical
solution for larger times steps in Figure 3.3.1.
3.3.5
Matrix Computations Using MATLAB
The calculations for the heat conduction with electrical current application
can easily be done by MATLAB. The steady state solution has been computed
several ways. The interested reader should examine the MATLAB code wire
_time _cur.m to see how the time dependent solution can be computed using
for-loops.
AA A = [ 2 -1 0 0;
-1 2 -1 0;
0 -1 2 -1;
0 0 -1 2];
AA d = [470 400 400 470];
AA A\d
ans =
% solves Au = d
125
Figure 3.3.1: Temperature in Wire with Current

1.0e+003 *
0.8700
1.2700
1.2700
0.8700
AA [L U] = lu(A)
L=
1.0000 0.0000
-0.5000 1.0000
0.0000 -0.6667
0.0000 0.0000
U=
2.0000 -1.0000
0.0000 1.5000
0.0000 0.0000
0.0000 0.0000
AA y = L\d
y=
1.0e+003 *
0.4700
0.6350
0.8233
1.0875
AA u = U\y
% nds the LU factorization of A

0.0000
0.0000
1.0000
-0.7500
0.0000
0.0000
0.0000
1.0000
0.0000 0.0000
-1.0000 0.0000
1.3333 -1.0000
0.0000 1.2500
% rst step using LU
% second step using LU
126
u=
1.0e+003 *
0.8700
1.2700
1.2700
0.8700
AA A3 = [A(:,1) A(:,2) d A(:,4)]
A3 =
2 -1 470 0
-1 2 400 0
0 -1 400 -1
0 0 470 2
AA u3 = det(A3)/det(A)
u3 =
1270
3.3.6
% use Cramers Rule to nd u3
Exercises
1.
Find the solution of

6 5
6
5
65
4
2 0 0
{1
7 1 3 0 8 7 {2 8 = 7 14 8 =
19
3 4 1
{3
2.

5
2 0
7 1
2 4
3.
4.
5.

5
1
9 0
9
7 0
0
6 5
6
65
h
0
{1
0 8 7 {2 8 = 7 2 8 =
10
{3
3
6
10
: 9 23 :
:
:=9
8 7 7 8=
3
7
2
0
0
8
3
1
0
65
{1
6
9 {2
4 :
:9
2 8 7 {3
3
{4

5
2 3
9 0 5
9
7 0 0
0 0
7
0
7
0
65
{1
6
9 {2
1 :
:9
2 8 7 {3
4
{4
6
s
: 9 2 :
:
:=9
8 7 15 8 =
16
5
Use the LU factors in Example 3.3.3 to solve

5
65
6 5
6
2 1 0
{1
210
7 1 2 1 8 7 {2 8 = 7 10 8 =
0 1 2
80
{3
3.4. ROW OPERATIONS AND GAUSS ELIMINATION

6.
127
Solve
6 5 6
65
65
1
1 0 0
2 7 8
{1
7 1 2 0 8 7 0 9 10 8 7 {2 8 = 7 2 8 =
3
2 0 3
0 0 1
{3
5
7.
Consider the heat conduction with current in Example 3.3 4. Verify the
solutions of Oy = d and X u = y=
8.
Consider the heat conduction with current in Example 3.3 4. Replace
i = 0=1 by i = 0=05, the left end temperature from 70 to 80 and the right end
temperature from 70 to 75. Find the steady state solution.
9.
10.
11.
Use the MATLAB code wire_time_cur.m to do the time dependent
computations in exercise 8.
3.4
Row Operations and Gauss Elimination
The objective of this section is to formulate a systematic way of transforming

b The solution of
Dx = d to an equivalent upper triangular system X x = d=
the upper triangular system can be found provided the diagonal components
of the upper triangular matrix are all not zero. This method is called Gauss
elimination and requires about q2 storage and q3 @3 operations for an q q
matrix D. This systematic approach can be coded so that a computer can do
these operations. An application to a six-bar truss will be given.
3.4.1
Introductory Illustration
Consider the following algebraic system for three unknowns

;
? {1 {2 + {3 = 5
{1 + {2 + {3 = 1
=
4{2 + {3 = 1=
The matrix version Dx = d
5
1
7 1
0
is not in upper triangular form

6 5
6
65
5
1 1
{1
1 1 8 7 {2 8 = 7 1 8 =
1
{3
4 1
If one adds equation one (row one) to equation two (row two), then the new
algebraic system is
;
? {1 {2 + {3 = 5
2{3 = 6
=
4{2 + {3 = 1=
128
This is equivalent to multiplying the matrix equation by an elementary matrix

H21 (1)
H21 (1) Dx = H21 (1) d
65
65
6
5
65
6
1 0 0
1 1 1
{1
1 0 0
5
7 1 1 0 8 7 1 1 1 8 7 {2 8 = 7 1 1 0 8 7 1 8
0 0 1
0
4 1
0 0 1
1
{3
5
65
6
5
6
1 1 1
{1
5
7 0 0 2 8 7 {2 8 = 7 6 8 =
0 4 1
1
{3
5
Next interchange equation two (row two) and equation three (row three)
;
? {1 {2 + {3 = 5
4{2 + {3 = 1
=
2{3 = 6=
The interchange is equivalent to multiplying the matrix equation by an elementary permutation matrix S23
5
65
1 0 0
1
7 0 0 1 87 0
0 1 0
0
5
1
7 0
0
S23 H21 (1) Dx = S23 H21 (1) d

65
6
5
65
6
1 1
{1
1 0 0
5
0 2 8 7 {2 8 = 7 0 0 1 8 7 6 8
4 1
0 1 0
1
{3
65
6
5
6
1 1
{1
5
4 1 8 7 {2 8 = 7 1 8 =
0 2
6
{3
b where X = S23 H21 (1) D and

Note this is in upper triangular form X x = d
b = S23 H21 (1) d= Since adding and interchanging equations can be reversed,
d
the solution of the upper triangular system must also be a solution of the
original system. The solution is {3 = 6@2 = 3> {2 = (1 3)@4 = 1 and
{1 = (5 3 + (1)) = 1=
3.4.2
Three Types of Row Operations
In order to minimize the repeated writing of the equations, the augmented

matrix notation will be used. Let D be an q q matrix and let d be an q 1
column vector. The augmented matrix [D d] is an q (q + 1) matrix with the
column vector augmented to the matrix. In the above example
5
6
1 1 1 5
[D d] = 7 1 1 1 1 8 =
0
4 1 1
The two row operations can be written as
b
S23 H21 (1) [D d] = [X d]=
129
There are three row operations, which are used to transform matrices to
upper triangular matrices. They can be used to solve algebraic systems, nd
the OX factors, evaluate determinants, nd multiple solutions and eigenvectors.
The row operations can be represented by elementary matrices Hlm (d), Slm and
Hl (f)=
Add d times (urz_m) to urz_l. This can be represented by an q q
elementary matrix Hlm (d) with the only non-zero components being ones on the
diagonal and d equal to the ij-component. For example, for q = 3> d = 2>
l = 3 and m = 2
5
6
1 0 0
H32 (2) = 7 0 1 0 8 =
0 2 1
Note, Hlm (d) is the inverse operation to Hlm (d),
For example,
5
65
1 0 0
1 0
H32 (2) H32 (2) = 7 0 1 0 8 7 0 1
0 2 1
0 2
that is, Hlm (d) Hlm (d) = L=

6 5
6
0
1 0 0
0 8 = 7 0 1 0 8=
1
0 0 1
Interchange urz_l and urz_m= This can be represented by an q q

matrix Slm with the only non-zero components being ones on the diagonal except
for urz_l and urz_m where the ij-component and ji-component equal one,
respectively. For example, for q = 4> l = 2 and m = 3
6
5
1 0 0 0
9 0 0 1 0 :
:
S23 = 9
7 0 1 0 0 8=
0 0 0 1
Note that Slm Slm = L=
Multiply urz_l by f= The is represented by an qq matrix Hl (f) with the
only non-zero components being ones on the diagonal except for urz_l where
the diagonal component is f= For example, for q = 4> f = 9 and l = 3
5
6
1 0 0 0
9 0 1 0 0 :
:
H3 (9) = 9
7 0 0 9 0 8=
0 0 0 1
It is easy to verify that Hl (1@f) Hl (f) = L=
Row operations are used, starting with the left columns in the augmented
matrix, to transform it to an upper triangular matrix. Once the upper triangular matrix is found, the solution may be computed by backward substitution as
130
long as the diagonal components of the upper triangular matrix are not zero.
The following 2 2 example illustrates this may not always be possible
{1 + {2
2{1 + 2{2
= 1 and
= 3=
The augmented matrix is

[D d] =
1 1 1
2 2 3
In order to obtain a zero in the 21-component add (2) times urz_1 to urz_2

1 0
1 1 1
1 1 1
b
=
= [X d]=
H21 (2) [D d] =
2 1
2 2 3
0 0 1
The 22-component of X is zero! Moreover, the second row is a shorthand way
of writing 0{1 + 0{2 = 1> which is impossible. Such algebraic systems are called
inconsistent and have no solution.
3.4.3
Gauss Elimination for Solving Dx = d
Assume Dx = d has a solution and it is unique. The Gauss elimination method

is dened so that it can be implemented for larger systems and using computers.
The reader will observe the by-hand calculations can become a little more than
tiresome, but the objective here is to illustrate how the method can be used to
solve larger systems.
Denition 3.4.1. Consider the q q algebraic system Dx = d= The Gauss
elimination method for solving this system has two stages:
b that is, transStage 1. Transform Dx = d to upper triangular form X x = d>
b
form the augmented matrix [D d] to an upper triangular matrix [X d]=
(a). start with the left column, column m = 1, and use row operations
to transform column m = 1 to zeros below row l = 1>
(b). move to the next column, column m = 2, and use row operations
to transform column m = 2 to zeros below row l = 2 and
(c). repeat this until column m = q 1 has been done.
b by backward substitution.
Stage 2. Solve the upper triangular system X x = d
(a). start with the bottom row, row l = q, and
solve the corresponding equation for {q >
(b). put {q into the equations corresponding to rows l = q 1> > 1>
(c). solve equation l = q 1 for {q1 and
(d). repeat this until {1 has been computed.
Example 3.4.1. Consider the 3 3 algebraic system
;
5{2 + 6{3 = 6
?
{1 + 3{2 + {3 = 2
=
2{1 + {2 + {3 = 3=
131

5
6
0 5 6 6
[D d] = 7 1 3 1 2 8 =
2 1 1 3
Stage 1. In order to get zeros in column m = 1, the 11-component must not be
zero. So, rst interchange urz_1 and urz_2
5
65
6 5
6
0 1 0
0 5 6 6
1 3 1 2
S12 [D d] = 7 1 0 0 8 7 1 3 1 2 8 = 7 0 5 6 6 8 =
0 0 1
2 1 1 3
2 1 1 3
Add (2) times urz_1 to urz_3
5
65
6 5
6
1 0 0
1 3 1 2
1 3
1
2
6
6 8=
H31 (2) S12 [D d] = 7 0 1 0 8 7 0 5 6 6 8 = 7 0 5
2 0 1
2 1 1 3
0 5 1 1
Move to the second column and add urz_2 to urz_3
5
65
6
1 0 0
1 3
1
2
6
6 8
H32 (1) H31 (2) S12 [D d] = 7 0 1 0 8 7 0 5
0 1 1
0 5 1 1
5
6
1 3 1 2
= 7 0 5 6 6 8=
0 0 5 5
Stage 2. The last row corresponds to the equation 5{3 = 5 and so {3 = 1= Put
this into the equations corresponding to the rows 1 and 2 to get
{1 + 3{2 + 1(1) = 2 and
5{2 + 6(1) = 6=
Then {2 = 0 and {1 = 1=
Example 3.4.2. Consider another 3 3 algebraic system
;
? 2{1 + 6{2 + 8{3 = 16
4{1 + 15{2 + 19{3 = 38
=
2{1 + 3{3 = 8=
5
6
2 6 8 16
[D d] = 7 4 15 19 38 8 =
2 0 3 8
132
Stage 1. Get zeros in the rst column by adding (2) times urz_1 to urz_2,
and then adding (1) times urz_1 to urz_3
5
6
2 6
8
16
3
6 8=
H31 (1) H21 (2) [D d] = 7 0 3
0 6 5 8
By adding (2) times urz_2 to urz_3 we get a zero
5
2
H32 (2) H31 (1) H21 (2) [D d] = 7 0
0
in the 32-component
6
6 8 16
3 3 6 8=
0 1 4
Stage 2. The last row corresponds to the equation 1{3 = 4 and so {3 = 4= Put
this into the equations corresponding to the rows 1 and 2 to get
2{1 + 6{2 + 8(4) = 16 and
3{2 + 3(4) = 6=
Then {2 = 2 and {1 = 2=
Example 3.4.3. Consider the two-loop circuit problem that was discussed in
Subsection 3.1.4
;
l1 l2 + l3 = 0
?
U1 l1 U3 l3 = H1
=
U2 l2 U3 l3 = H2 =
Let U1 = 1> U2 = 2, U3 = 3, H1 =
matrix is
5
1
[D d] = 7 1
0
10 and H2 = 20 so that the augmented

6
1 1
0
0 3 10 8 =
2 3 20
Stage 1. Obtain zeros in the 21-component and 32-component by using

5
6
1 1
1
0
4 10 8 =
H32 (2) H21 (1) [D d] = 7 0 1
0 0 11 40
Stage 2. The last row corresponds to the equation 11l3 = 40 and so l3 =
40@(11)= Put this into the equations corresponding to the rows 1 and 2 to get
l1 l2 + 40@(11) = 0 and
l2 4(40@(11)) = 10=
This gives l2 = 50@(11) and l1 = 10@(11)=
133
Example 3.4.4. Consider the steady state heat wire in Example 3.3.4 where
there were ve segments and electrical current generating heat within the wire.
The algebraic problem for the temperature in each interior segment is
65
5
6 5
6
2 1 0
0
x1
470
9 1 2 1 0 : 9 x2 : 9 400 :
9
:9
: 9
:
7 0 1 2 1 8 7 x3 8 = 7 400 8
0
0 1 2
470
x3
and the augmented matrix is
5
6
2 1 0
0 470
9 1 2 1 0 400 :
:
[D d] = 9
7 0 1 2 1 400 8 =
0
0 1 2 470
Stage 1. Obtain zeros in the subdiagonal by using

5
2 1
0
0
470=00
9 0 3@2 1
0
635=00
H43 (3@4) H32 (2@3) H21 (1@2) [D d] = 9
7 0 0 4@3 1 823=33
0 0
0 5@4 1087=50
6
:
:=
8
Stage 2. The last row corresponds to the equation (5@4)x4 = 1087=5 and so
x4 = 1087=50@1=25 = 870= Put this into the equations corresponding the rows
1, 2 and 3 to get x3 = x2 = 1270 and x1 = 870= The reader should note the
similarities of these calculations with the LU factorization method as given in
Subsection 3.3.4.
3.4.4
Application to Six-bar Truss
In Subsection 3.1.4 an application to a two-bar truss was presented. There was

one stationary node where a balance of 2D forces gave two scalar equations for
the forces within the two bars. If a bar is under tension, then the force within
the bar will be positive. If the bar is under compression, then the force will
be negative. The two-bar truss will be enhanced to a six-bar truss with three
stationary nodes, see Figure 3.4.1. Unlike the two-bar truss, it is not possible
to assume the bars are under compression or tension forces. Hence, all the
unknown bar forces in Figure 3.4.1 are initially displayed as under tension.
In order to derive an algebraic system for the six forces, use at each node
the equilibrium condition that the sum of the force vectors must be the zero
force vector. Let the angle between bars one and two be equal to and use the
short notation for sine and cosine v = sin() and f = cos()= Assume the mass
at the right end of the support structure has a weight equal to 10.
Node 1:
Node 2:
Node 3:
[i1
[i1 0] + [i2 f i2 v] + [0 10] = [0 0]>

[i2 f i2 v] + [0 i3 ] + [i4 0] = [0 0] and
0] + [0 i3 ] + [i5 f i5 v] + [i6 0] = [0 0]=
134
Figure 3.4.1: Six-bar Truss
Now equate the rst and second components in each vector equation to obtain
a system with six scalar equations
;
A
A
A
A
A
A
?
A
A
A
A
A
A
=
i1 i2 f = 0
i2 v = 10
i2 f i4 = 0
i2 v i3 = 0
i1 i5 f i6 = 0
i3 + i5 v = 0=
The vector version of these scalar equations is

5
9
9
9
9
9
9
7
1 f 0
0
0
0
0
v
0
0
0
0
0
f
0 1 0
0
0 v 1 0
0
0
1
0
0
0 f 1
0
0
1
0
v
0
65
:9
:9
:9
:9
:9
:9
87
i1
i2
i3
i4
i5
i6
: 9
: 9
: 9
:=9
: 9
: 9
8 7
0
10
0
0
0
0
6
:
:
:
:
:
:
8
and the augmented matrix is

5
9
9
9
[D d] = 9
9
9
7
6
1 f 0
0
0
0
0
0
v
0
0
0
0 10 :
:
0
f
0 1 0
0
0 :
:=
0 v 1 0
0
0
0 :
:
1
0
0
0 f 1 0 8
0
0
1
0
v
0
0
In order to do the rst stage of Gauss elimination, one must use row operation to transform the lower portion of the augmented matrix to zero components. Fortunately, this matrix has many zero components. The required row

operations are indicated by groups of operations on columns:
5
1 f 0
0
0
0
0
9 0
v
0
0
0
0
10
9
9 0
f
0 1 0
0
0
H51 (1) [D d] = 9
9 0 v 1 0
0
0
0
9
7 0 f 0
0 f 1 0
0
0
1
0
v
0
0
H52 (f@v) H42 (1)
5
1 f 0
9 0
v
0
9
9 0
0
0
9
9 0
0 1
9
7 0
0
0
0
0
1
135
6
:
:
:
:>
:
:
8
H32 (f@v) H51 (1) [D d] =

6
0
0
0
0
:
0
0
0
10
:
1 0
0 10f@v :
:>
:
0
0
0
10
:
0 f 1 10f@v 8
0
v
0
0
H63 (1) S43 H52 (f@v) H42 (1) H32 (f@v) H51 (1) [D d] =
5
9
9
9
9
9
9
7
6
1 f 0
0
0
0
0
:
0
v
0
0
0
0
10
:
:
0
0 1 0
0
0
10
: and
0
0
0 1 0
0 10f@v :
:
0
0
0
0 f 1 10f@v 8
0
0
0
0
v
0
10
H65 (v@f) H63 (1) S43 H52 (f@v) H42 (1) H32 (f@v) H51 (1) [D d] =
6
5
1 f 0
0
0
0
0
:
9 0
v
0
0
0
0
10
:
9
:
9 0
0 1 0
0
0
10
:=
9
9 0
0
0 1 0
0
10f@v :
:
9
7 0
0
0
0 f 1
10f@v 8
0
0
0
0
0 v@f
20
The second stage is a backwards substitution:
i6
i5
i4
i3
i2
i1
=
=
=
=
=
=
20f@v = 20 cos()@ sin()>

(10f@v 20f@v)@(f) = 10@v = 10@ sin()>
10f@v = 10 cos()@ sin()>
10>
10@v = 10@ sin() and
f(10@v) = 10f@v = 10 cos()@ sin()=
The negative signs indicate bars 1, 3 and 6 are under compression, and the other
bars 2, 4 and 5 are under tension. As the angle decreases, the magnitudes of
the forces increase, except for bar 3.
136
3.4.5
Gauss Elimination Using MATLAB
Row operations using MATLAB are illustrated in the MATLAB code gauss_el.m.
Both stages of Gauss elimination are combined in the single command MATLAB
command A\d. Consider the six-beam support structure with = @6=
AAc = cos(pi/6)
AA s = sin(pi/6);
AA A = [-1 -c 0 0 0 0 ;
0 s 0 00 0;
0 c 0 -1 0 0 ;
0 -s -1 0 0 0 ;
1 0 0 0 -c -1 ;
0 0 1 0 s 0];
AA d = [0 10 0 0 0 0];
AA A\d
ans =
-17.3205
20.0000
-10.0000
17.3205
20.0000
-34.6410
%
%
%
%
%
%
= i1
= i2
= i3
= i4
= i5
= i6
The reader will nd it interesting to experiment with dierent angles and

weights, which can easily be done using the MATLAB code support.m.
3.4.6
Exercises
1.
Find the augmented matrix for

;
? 2{1 2{2 + {3 = 7
4{1 + {2 + {3 = 0
=
3{2 + {3 = 2=
2.
Find the augmented matrix for

;
{1 2{2 + {3 5{4 = 1
A
A
?
{1 2{2 + 5{4 = 1
3{

2{
A
1
2 + {3 + 2{4 = 0
A
=
4{1 + {3 {4 = 5=
3.
4.
5.
Let q = 4 and nd the elementary matrices H43 (7)> S23 and H4 (10)=
Let q = 3 and nd the elementary matrices H21 (2)> S13 and H3 ()=
Consider the algebraic system
;
? {1 + 7{2 + {3 = 1
2{1 + 0{2 + 7{3 = 2
=
14{2 + {3 = 6=
137
(a).
Find the augmented matrix.
(b).
Use row operations to transform the augmented matrix to an upper
triangular matrix.
(c).
Find the solution by using the upper triangular matrix.
6.
As in exercise 5 use Gauss elimination to solve the system in exercise 1.
7.
Consider the algebraic system
;
{1 {2 + {4 = 6
A
A
?
{1 {3 {4 = 7
2{2 + 7{3 = 4
A
A
=
2{1 {4 = 8=
(a).
Find the augmented matrix.
(b).
Use row operations to transform the augmented matrix to an upper
triangular matrix.
(c).
Find the solution by using the upper triangular matrix.
8.
As in exercise 7 use Gauss elimination to solve the system in exercise 2.
9.
Consider the two-loop circuit in Example 3.4.3 with slightly dierent
U1 = 1> U2 = 2, U3 = 4, H1 = 20 and H2 = 10= Use Gauss elimination to nd
the solution.
10.
Consider the two-loop circuit in Example 3.4.3 with arbitrary symbols
for U1 > U2 , U3 , H1 and H2 = Use Gauss elimination to nd the solution.
11.
Consider the steady state heat conduction within a wire in Example
3.4.4. Verify the row operations to nd the indicated upper triangular matrix.
12.
Consider the steady state heat conduction within a wire in Example
3.4.4. Let the left end temperature be x0 = 80, the right end temperature be
x5 = 75 and the heat generated by the current be i = 0=05 per unit time per
unit volume. Use Gauss elimination to solve the steady state problem
6 5
6
5
65
280
2 1 0
0
x1
9 1 2 1 0 : 9 x2 : 9 200 :
: 9
:
9
:9
7 0 1 2 1 8 7 x3 8 = 7 200 8 =
275
0
0 1 2
x3
13.
Consider the six-bar truss.

(a).
Verify the operations and the solution.
(b).
Change the weight from z = 10 to z = 20 and compute the new
forces.
(c).
Show the solution for z = f10 is always ff where f is the solution
in the text.
14.
Consider the six-bar truss. Suppose the support structure has weights
at both nodes one [0 z1 ] and three [0 z2 ]=
(a).
Find the model for this new problem.
(b).
Find the solution when z1 = 10 and z2 = 20=
15.
Use MATLAB to conrm any of your calculations in exercises 5-14.
138
16.
Use MATLAB to nd the solutions in the six-bar truss problem for =
@6> @4 and 2@6= You may use the MATLAB code support.m.
17.
Use MATLAB to nd the solutions in the six-bar truss with two weights
as specied in exercise 14. You may modify the MATLAB code support.m, and
experiment with dierent weights and angles.
3.5
Inverse Matrices
The three types of elementary row operations can be reversed and be represented by elementary matrices. In terms of matrix operations this means for
each elementary matrix there is another elementary matrix such that the product is the identity matrix:
Hlm (d) Hlm (d) = L
Slm Slm = L and
Hl (1@f) Hl (f) = L=
The objective of this section is to be able to nd an q q matrix E such that
for a given q q matrix D we have DE = L= E is called an inverse matrix of D,
is often written as D1 = E. An important use is to nd the solution Dx = d
given by x = D1 d because
Dx =D(D1 d) = (DD1 )d = Ld = d=
Unfortunately, one can not always nd such matrices! We shall see in Section
3.7 D with a non-zero determinant implies that an inverse matrix exists.
3.5.1
Examples of Inverse Matrices
The identity q q matrix L has all zero components except on the diagonal
where the components are equal to one. The identity matrix can be written as
a matrix whose jwk column vector is the jwk unit column vector em
e1 e2 eq
L =
6
5
1 0 0
9 0 1 0 :
:
9
=
= 9 . . .
. . ... :
8
7 .. ..
0 0
Denition 3.5.1. Let D be an q q matrix. An q q matrix E is called an

inverse matrix of D if and only if DE = ED = L=
Notation. The inverse matrix is usually denoted by D1 . Here one should be
careful not to write this as 1@D or to have components equal to 1@dlm !
3.5. INVERSE MATRICES
139
Basic Examples.
1.
Elementary Matrices.
If D = Hlm (d)> then D1 = Hlm (d)= If D = Slm > then D1 = Slm = If D = Hl (f)>
then D1 = Hl (1@f)=
2.
Diagonal Matrices.
If D is a diagonal matrix with non-zero diagonal components dll 6= 0> then D1
is a diagonal matrix with diagonal components equal to 1@dll = For example, if
q = 3 and
5
6
4 0 0
D = 7 0 2 0 8,
0 0 3
then because DD1 = L
D1
6
1@4 0
0
= 7 0 1@2 0 8 =
0
0 1@3
3.
Block Diagonal Matrices.
If D is a block diagonal matrix with diagonal blocks that have inverses, then
D has an inverse matrix which is also a block diagonal matrix. For example,
consider a 5 5 matrix with two diagonal blocks
6
5
1 0 0 0 0

9 0 1 0 0 0 :
:
9
D11
0 4 1 0 0 :
D =
=9
: where
9
D22
7 0 0 0 2 0 8
0 0 0 0 3
5
6

1 0 0
2 0
D11 = H32 (4) = 7 0 1 0 8 and D22 =
=
0 3
0 4 1
Both D11 and D22 have inverses and, hence, the
5
1 0
9 0 1
1
9
D11
=9
D1 =
1
9 0 4
D22
7 0 0
0 0
inverse of D is
0 0
0
0 0
0
1 0
0
0 1@2 0
0 0 1@3
4.
D is a 2 2 Matrix with ghw(D) 6= 0=
The inverse matrix E must satisfy the matrix equation
D
b1
DE = L
b2
e1
=
e2
This is equivalent to two vector equations

Db1 = e1 and Db2 = e2 =
6
:
:
:.
:
8
140
One can solve these by either Cramers rule or Gauss elimination. For example,
use Cramers rule to nd the inverse of

d11 d12
D=
=
d21 d22
The rst column b1 of the inverse matrix must satisfy

d11
d21
d12
d22
Db1 = e1

e11
1
=
=
0
e21
Then
e11
e21
1 d12
= det(
)@ det(D) = d22 @ det(D) and
0 d22

d11 1
= det(
)@ det(D) = d21 @ det(D).
d21 0
The second column b2 of the inverse matrix must satisfy

d11
d21
d12
d22
Db2 = e2

e12
0
=
=
1
e22
Then

e12
= det(

e22
= det(
)@ det(D) = d12 @ det(D) and
0
)@ det(D) = d11 @ det(D).
1
0 d12
1 d22
d11
d21
Therefore, the inverse of D is

D1 = E =
b1
b2

=
d22
d21
d12
d11
@ det(D)=
The alternative is to use row operations and Gauss elimination. The GaussJordan method is a variation of Gauss elimination, which can be adapted to
larger matrices and to computers.
3.5.2
Gauss-Jordan Method to Find Inverse Matrices
As motivation consider a special case of the above 2 2 example where

2 1
2@3 1@3
1
D=
and D =
=
1 2
1@3 2@3
141
First, use row operations and the augmented matrix to solve Db1 = e1

2 1 1
[D e1 ] =
1 2 0

1 0
2 1 1
2 1
1
H21 (1@2) [D e1 ] =
=
=
1@2 1
1 2 0
0 3@2 1@2
At this point either use backward substitution or more row operation to obtain
an identity matrix in the left side of the augmented matrix. The row operations
include two row multiples and then adding (1@2) times urz_2 to urz_1 :

1 1@2 1@2
H2 (2@3) H1 (1@2) H21 (1@2) [D e1 ] =
0
1
1@3

1 0 2@3
H12 (1@2) H2 (2@3) H1 (1@2) H21 (1@2) [D e1 ] =
=
0 1 1@3
The right column in the transform augmented matrix is the rst column of the
inverse matrix. Let H be the product of the four elementary matrices and write
the above as
H [D e1 ] = [L b1 ]=
In order to nd the second column of the inverse matrix, the same row operations are needed to transform the augmented matrix from [D e2 ] to [L b2 ]
H [D e2 ] = [L b2 ]=
An e!cient way to do both these solves is to combine the two columns of the
inverse matrix into the right side of a larger augmented matrix. That is, use
the row operations to transform [D e1 e2 ] = [D L] to [L b1 b2 ] = [L E]
H [D e1 e2 ] = [L b2 b2 ]=
In order to nd the inverse of an q q matrix D> one must solve q vector
equations
DE = L
e1 e2 eq
D b1 b2 bq
=
Db1 Db2 Dbq

e1 e2 eq or
=
Dbm = em for m = 1> = = = > q=
Use row operations and elementary matrices to transform the augmented matrix
[D L] to [L D1 ]=
Denition 3.5.2. Consider the q q matrix D and nd the inverse matrix, if
it exists. The Gauss-Jordan method for solving this system has two stages:
Stage 1. Transform the augmented matrix [D L] to upper triangular form
[X O]=
142
(a). start with the left column, column m = 1, and use row operations
to transform column m = 1 to zeros below row l = 1>
(b). move to the next column, column m = 2, and use row operations
to transform column m = 2 to zeros below row l = 2 and
(c). repeat this until column m = q 1 has been done.
Stage 2. Transform the upper triangular matrix [X O] to [L D1 ]
(a). use elementary matrices so that the diagonal of U is transformed
to have ones on all of the diagonal components,
(b). start with the right column of X , column m = q, and use row operations
to transform column m = q to zeros above row l = q>
(c). move to the next left column, column m = q 1, and use row operations
to transform column m = q 1 to zeros above row l = q 1 and
(d). repeat this until column m = 2 has been done.
Example 3.5.1. Use the Gauss-Jordan method to nd the inverse of
5
6
1 2 1
D = 7 1 3 2 8=
1 0 1
5
6
1 2 1 1 0 0
[D L] = 7 1 3 2 0 1 0 8 =
1 0 1 0 0 1
Stage 1. Subtract urz_1 from urz_2 and urz_3
5
6
1 2 1 1 0 0
H31 (1) H21 (1) [D L] = 7 0 1 1 1 1 0 8 =
0 2 0 1 0 1
Add (2) times urz_2 to urz_3
5
6
1 2 1 1 0 0
H32 (2) H31 (1) H21 (1) [D L] = 7 0 1 1 1 1 0 8 =
0 0 2 3 2 1
Stage 2. Divide urz_3 by 2 and then subtract it from urz_2 and urz_1
H13 (1) H23 (1)
5
1
7 0
0
H3 (1@2) H32 (2) H31 (1) H21 (1) [D L] =

6
2 0 5@2 1 1@2
1 0 1@2
0 1@2 8 =
0 1 3@2 1
1@2
Finally, subtract (2) times urz_2 from urz_1

H12 (2) H13 (1) H23 (1) H3 (1@2) H32 (2) H31 (1) H21 (1) [D L] =
143
6
1 0 0 3@2 1 1@2
7 0 1 0 1@2
0 1@2 8 =
0 0 1 3@2 1
1@2
One can easily verify the inverse is given by the right side of the transformed
augmented matrix
5
6
3@2 1 1@2
0 1@2 8 =
D1 = 7 1@2
3@2 1
1@2
The solution of Dx = d = [1 2 3]W is easily computed by using the inverse
matrix
5
65 6 5
6
3@2 1 1@2
1
1
0 1@2 8 7 2 8 = 7 1 8 =
x = D1 d = 7 1@2
3@2 1
1@2
3
2
Example 3.5.2. Use the Gauss-Jordan method to nd the inverse of
5
6
0 5 6
D = 7 1 3 1 8=
2 1 1
5
6
0 5 6 1 0 0
[D L] = 7 1 3 1 0 1 0 8 =
2 1 1 0 0 1
Stage 1. Because the 11-component is zero,
must be interchanged
5
1 3 1
S12 [D L] = 7 0 5 6
2 1 1
urz_1 and urz_2 (or urz_3)

6
0 1 0
1 0 0 8
0 0 1
Subtract (2) times urz_1 from urz_3

5
6
1 3
1 0 1 0
6 1 0 0 8=
H31 (2) S12 [D L] = 7 0 5
0 5 1 0 2 1
Add urz_2 to urz_3
5
H32 (1) H31 (2) S12
6
1 3 1 0 1 0
[D L] = 7 0 5 6 1 0 0 8 =
0 0 5 1 2 1
Stage 2. Divide urz_2 and urz_3 by 2

H3 (1@2) H2 (1@2) H32 (1) H31 (2) S12 [D L] [D L] =
144
5
6
1 3 1
0
1
0
7 0 1 6@5 1@5
0
0 8=
0 0 1 1@5 2@5 1@5
In order to transform the left side to the identity, subtract (6@5) times urz_3
from urz_2> subtract urz_3 from urz_1 and last subtract (3) times urz_2
from urz_1
H12 (3) H13 (1) H23 (6@5) H3 (1@2) H2 (1@2) H32 (1) H31 (2) S12 [D L] =
5
6
1 0 0 2@25 1@25 13@25
7 0 1 0 1@25 12@25 6@25 8 =
0 0 1 5@25 10@25 5@25
The inverse is the right side of the transformed augmented matrix
5
6
2 1 13
D1 = (1@25) 7 1 12 6 8 =
5 10 5
The solution of Dx = d = [6 2 3]W is computed by using the inverse matrix
5
65 6 5 6
2 1 13
6
1
x = D1 d = (1@25) 7 1 12 6 8 7 2 8 = 7 0 8 =
5 10 5
3
1
Example 3.5.3. Consider the steady state heat wire problem in Sections 3.3
and 3.4 where there were ve segments and current generating heat within the
wire. The algebraic problem for the temperature in each interior segment is
65
5
6
6 5
x1
2 1 0
0
470
9 1 2 1 0 : 9 x2 : 9 400 :
:9
9
:
: 9
7 0 1 2 1 8 7 x3 8 = 7 400 8 =
0
0 1 2
470
x3
The augmented matrix for nding the inverse
5
2 1 0
0
9 1 2 1 0
9
[D L] = 7
0 1 2 1
0
0 1 2
matrix is
1
0
0
0
0
1
0
0
6
0
0 :
:=
0 8
1
0
0
1
0
Stage 1. Obtain zeros in the subdiagonal by using

H43 (3@4) H32 (2@3) H21 (1@2) [D L] =
5
2 1
0
0
1
0
0
9 0 3@2 1
0
1@2
1
0
9
7 0 0 4@3 1 1@3 2@3 1
0 0
0 5@4 1@4 2@4 3@4
6
0
0 :
:=
0 8
1
145
Stage 2. Multiply rows 1-4 by 1@2> 2@3> 3@4 and 4@5, respectively, to get for
G H4 (4@5) H3 (3@4) H2 (2@3) H1 (1@2)
G H43 (3@4) H32 (2@3) H21 (1@2) [D L] =
5
6
1 1@2
0
0
1@2 0
0
0
9 0
1
2@3
0
1@3 2@3 0
0 :
9
:=
7 0
0
1
3@4 1@4 2@4 3@4 0 8
0
0
0
1
1@5 2@5 3@5 4@5
In order to obtain zeros in the upper triangular part of the left side, use elementary matrices
H12 (1@2) H23 (2@3)
5
1 0
9 0 1
9
7 0 0
0 0
H34 (3@4) G H43 (3@4) H32 (2@3) H21 (1@2) [D L] =

6
0 0 8@10 6@10 4@10 2@10
0 0 6@10 12@10 8@10 4@10 :
:=
1 0 4@10 8@10 12@10 6@10 8
0 1 2@10 4@10 6@10 8@10
So, the inverse matrix is

5
D1
6
8 6 4 2
9 6 12 8 4 :
:
= (1@10) 9
7 4 8 12 6 8 >
2 4 6 8
and the solution of the above algebraic system is

5
65
8 6 4 2
9 6 12 8 4 : 9
:9
u = D1 d = (1@10) 9
7 4 8 12 6 8 7
2 4 6 8
3.5.3
6 5
470
870
9 1270
400 :
:=9
400 8 7 1270
470
870
6
:
:=
8
Properties of Inverse Matrices
Not all matrices have inverses! Any 2 2 with zero determinant cannot have
an inverse. For example, if

1 1
D=
>
2 2
then one cannot nd the rst column of the inverse matrix because of the
following contradiction

1 1
2 2
Db1 = e1

e11
1
=
0
e21
e11 + e21 = 1 and 2e11 + 2e21 = 0!
146
In the last section of this chapter determinants of qq matrices will be dened,

and any matrix with a non-zero determinant will be shown to have an inverse
matrix.
One very important property of inverse matrices is the solution of Dx = d
is D1 d. Other properties are summarized in the following theorem.
Theorem 3.5.1 (Properties of Inverse Matrices) Let D, D1 and D2 be q q
matrices that have inverses. Then the following statements hold.
1.
2.
3.
4.
5.
If D1 = E> then D times froxpq m of E is hm .

D1 g is the unique solution of Dx = d=
1
(note the reverse order).
(D1 D2 )1 = D1
2 D1
There is only one inverse matrix.
The following are equivalent:
(i). D has an inverse,
(ii). the determinant of D is not zero and
(iii). Dx = 0 implies x = 0.
1
The proof of (D1 D2 )1 = D1
2 D1 follows from the associative property of
matrix products
(D1 D2 )1 (D1 D2 ) =

=
=
=
1
(D1
2 D1 )(D1 D2 )
1
D2 (D1
1 (D1 D2 ))
1
D2 ((D1
1 D1 )D2 )
1
D2 ((L)D2 ) = L=
The proof of the fth property will be given in Section 3.7.

Example 3.5.4. This illustrates the third property
Find the inverse matrix of D = D1 D2 where
5
6 5
65
4 0 0
4 0 0
1 0
D = 7 4 2 0 8 = 7 0 2 0 8 7 2 1
0 0 1
0 0 1
0 0
D1 is a diagonal matrix and
5
1@4
7
0
D1
=
1
0
in the above theorem.
D2 is an elementary matrix
6
5
0 0
1
7
1@2 0 8 and D1
2
=
2
0 1
0
6
0
0 8 = D1 D2 =
1
with inverses
6
0 0
1 0 8=
0 1
The inverse of A is
5
1
D1 = D1
2 D1
65
6 5
6
1 0 0
1@4 0 0
1@4 0 0
= 7 2 1 0 8 7 0 1@2 0 8 = 7 2@4 1@2 0 8 =
0 0 1
0
0 1
0
0 1
147
In the Gauss elimination method the augmented matrix was transformed by

a product of elementary matrices, which have inverses. Therefore, the product
of the elementary matrices, H, has an inverse and
b
H[D d] = [X d]=
If all the diagonal components of X are not zero, the solutions of Dx = d and
b are the same.
X x = HDx = Hd = d
3.5.4
Inverse Matrices and MATLAB
The by-hand computation of inverse matrices can be an extremely tedious

process. Even using computing tools, the number of calculations is of order
2q3 @3 arithmetic operations for an q q matrix! The MATLAB command
inv(A) will compute the inverse of a matrix D= Some illustrations of this and
row operations are given in the MATLAB code inv_mat.m. Example 3.5.2 done
using MATLAB is
AA A = [0 5 6;1 3 1;2 1 1]
A=
056
131
211
AA B = inv(A)
% computes the inverse matrix
B=
-0.0800 -0.0400 0.5200
-0.0400 0.4800 -0.2400
0.2000 -0.4000 0.2000
AA A*B
ans =
1.0000 0.0000 0.0000
-0.0000 1.0000 -0.0000
0.0000 0.0000 1.0000
AA d = [6 2 3]
d=
6
2
3
AA solution1 = inv(A)*d
% uses inverse matrix
solution1 =
1.0000
-0.0000
1.0000
AA solutions2 = A\d
% uses Gauss elimination
solutions2 =
1
148
0
1
In the previous section an application to a six-bar truss generated a 6 6

matrix. The inverse of the matrix can be found using the MATLAB code support.m, and the interested reader should examine this application and exercises
10 and 11.
3.5.5
Exercises
3 1
1.
Let D =
=
1 3
(a).
Use Cramers rule to nd the inverse of D=
(b).
Use Gauss-Jordan method to nd the inverse.
(c).
Find the inverse of
6
5
3 1 0 0
9 1 3 0 0 :
:
9
7 0 0 3 0 8=
0 0 0 4

2.
3.
4.
5.
6.
7.
6
1 1 1
Let D = 7 1 0 3 8 =
0 2 3
(a).
Use the Gauss-Jordan method to nd the inverse matrix.
W
(b).
Use5the inverse to solve
6 Dx = d = [0 10 20] =
3 1 0
Let D = 7 1 3 1 8 =
0 1 3
(a).
(b).
Use5the inverse 6to solve Dx = d = [1 2 3]W =
0 2 1
Let D = 7 1 3 5 8 =
2 2 0
(a).
(b).
Use the inverse to solve Dx = d = [4 3 5]W .
Verify the5 row operations
6 used in Example 3.5.3.
1 2 0 1
9 2 1 0 0 :
:
Let D = 9
7 0 0 2 0 8=
1 2 0 2
(a).
W
(b).
Use the
5 inverse to 6solve Dx = d
5 = [2 1 4 61] =
1 0 0
1 2 0
Let D1 = 7 0 1 0 8 and D2 = 7 0 1 0 8 =
4 0 1
0 0 1
3.6. OX FACTORIZATION
149
Figure 3.5.1: Five-bar Truss

(a).
Find the inverses of both matrices.
(b).
Find the inverses of D1 D2 and D2 D1 =
8.
Prove: if D has an inverse, then there is only one inverse matrix.
9.
Use MATLAB to conrm any by-hand calculations done in exercises 2-4
or 6.
10.
Consider the six-bar truss in Section 3.4, see Figure 3.4.1.
(a).
Use the MATLAB command inv(A) to nd the inverse of the 6 6
matrix.
(b).
Find the forces for = @6> @4 and 2@6=
11.
Consider the ve-bar truss in Figure 3.5.1, see the MATLAB code bridge.m
(a).
Formulate an algebraic system for the eight forces.
(b).
Use the MATLAB commands A\d and inv(A) to solve this when
z = 10000 and variable = @6> @4 and 2@6=
3.6
OX Factorization
If the given matrix can be written as a product of a lower and upper triangular
matrices, then one may be able to solve Dx = d= Suppose D = OX and both O
and X have non-zero diagonal components. By the associative property
Dx = (OX )x = O(X x) = d=
So, solve by forward substitution Oy = d and then solve by backward substitution X x = y= An example was given in Section 3.3
6 5
6
5
65
200
2 1 0
{1
7 1 2 1 8 7 {2 8 = 7 0 8 =
70
0 1 2
{3
150
By using the tridiagonal structure of the matrix we were able to

5
6
5
65
2 1 0
1
0
0
2 1
7 1 2 1 8 = 7 1@2
1
0 8 7 0 3@2
0 1 2
0
2@3 1
0 0
D = OX=
nd factors
6
0
1 8
4@3
The objective of this section is to show how one can use row operations and
elementary matrices to possibly nd the OX factors for more general matrices.
Block versions of the OX factors will be described and applied to a three-loop
circuit problem.
3.6.1
Attributes of OX Factorization
Before we present the formal denition, the following three 2 2 matrices

reveal some important facts. First, not all matrices have OX factors, but the
interchange of some rows may allow us to nd the factors

0 1
D =
1 1

1 1
1 0
1 1
S12 D =
=
= OX=
0 1
0 1
0 1
Second, one can use row operations and elementary matrices to nd OX factors

2 1
D =
1 2

2 1
H21 (1@2) D =
= X=
0 3@2
Now let O be the inverse of the elementary matrix H21 (1@2) giving

2 1
1
1
H21 (1@2) (H21 (1@2) D) = H21 (1@2)
= OX
0 3@2

1
0
2 1
D =
= OX=
1@2 1
0 3@2
Third, the lower and upper matrices may not have an inverse

1 1
D =
1 1

1 1
H21 (1) D =
=X
0 0

1 0
1 1
D =
= OX=
1 1
0 0
151
Denition 3.6.1. An q q matrix D has an OX factorization if and only if

D = OX where O is a lower triangular q q matrix (ij-components are zero for
m A l), X is an upper triangular q q matrix (ij-components are zero for m ? l)
and both O and X have inverse matrices (their ii-components are not zero).
The following theorem can be proved by using mathematical induction on
the dimension of the matrix.
Theorem 3.6.1 (OX Factorization) If the q q matrix D has an inverse,
then for some permutation of the rows S D = OX= Moreover, Dx = d has a
unique solution, which can be found via
S Dx = S d
O(X x) = S d=
Denition 3.6.2. The OX factorization method uses row operations and inverses of the elementary matrices.
Stage 1. Use row operations to transform the matrix from D to X=
Stage 2. Let H be the product of the elementary matrices such that HD = X=
Dene O = H 1 and note D = H 1 (HD) = H 1 X = OX=
Example 3.6.1. This example does not require the interchange of rows to nd
the factorization of the 3 3 matrix
5
6
2 1 0
D = 7 1 2 1 8 =
0 1 2
Stage 1. Two elementary matrices are required to transform D to upper
triangular form
5
6
2 1
0
HD = H32 (2@3) H21 (1@2) D = 7 0 3@2 1 8 =
0 0 4@3
Stage 2. Multiply both sides from the left by H32 (2@3)1 = H32 (2@3)> then
multiply from the left by H21 (1@2)1 = H21 (1@2)
5
6
2 1
0
D = H21 (1@2) H32 (2@3) 7 0 3@2 1 8
0 0 4@3
5
65
6
1
0
0
2 1
0
1
0 8 7 0 3@2 1 8 = OX=
= 7 1@2
0
2@3 1
0 0 4@3
Notice the inverse of H is easy to compute as are the matrix products
H 1
= (H32 (2@3) H21 (1@2))1

= H21 (1@2)1 H32 (2@3)1
= H21 (1@2) H32 (2@3)=
152
Example 3.6.2. This example does require the interchange of rows to nd the
factorization of the 3 3 matrix
5
6
0 5 6
D = 7 1 3 1 8=
2 1 1
Stage 1. Interchange rows
5
0 1
S12 D = 7 1 0
0 0
one and two (or three)

65
6 5
6
0
0 5 6
1 3 1
0 87 1 3 1 8 = 7 0 5 6 8=
1
2 1 1
2 1 1
Two elementary matrices are required to transform S12 D to upper triangular

form
5
6
1 3 1
H32 (1) H31 (2) S12 D = 7 0 5 6 8 =
0 0 5
Stage 2. Multiply both sides from the left by H32 (1)1 = H32 (1)> and then
multiply from the left by H31 (2)1 = H31 (2)
5
6
1 3 1
S12 D = H31 (2) H32 (1) 7 0 5 6 8
0 0 5
5
65
6
1 0 0
1 3 1
= 7 0 1 0 8 7 0 5 6 8 = OX=
2 1 1
0 0 5
In order to solve the algebraic system Dx = d = [6 2 3]W , use this OX
factorization in S12 Dx = S12 d=
O(X x) = S12 d
6 5
65
6
5 6
1 0 0
1 3 1
{1
2
7 0 1 0 8 (7 0 5 6 8 7 {2 8) = 7 6 8 =
2 1 1
0 0 5
3
{3
5
The solution of Oy = S12 d is y = [2 6 5]> and the solution of X x = y is x = [1

0 1]=
3.6.2
Block OX Factors
Often it is useful to represent an q q matrix D as a 2 2 block matrix where

q = q1 + q2 , D11 is q1 q1 , D22 is q2 q2 , D12 is q1 q2 and D21 is q2 q1

D11 D12
D=
=
D21 D22
153
An algebraic system Dx = d may be written with xW = [xW1 xW2 ], dW = [dW1

dW2 ] where x1 and d1 are q1 1 column vectors, x2 and d2 are q2 1 column
vectors and

D11
D21
D12
D22
Dx
= d

x1
d1
=
=
x2
d2
An example is for q = 5 = 2 + 3 = q1 + q2
diagonal matrix
5
2 1 1
9 1 2 0
9
9 1
0 2
9
7 0
1 0
1
0 0
(3.6.1)
with D11 is 2 2 and D22 is a 3 3

0
1
0
2
0
1
0
0
0
2
6
:
:
:=
:
8
In many applications the blocks are parts of models for physical applications
such as velocity and pressure or voltages and currents. Moreover, one may have
some previous knowledge about the square blocks and their inverse matrices.
Block matrix products can be dened as long as number of columns in the
left submatrix is equal to the number of rows in the right submatrix. Block
elementary matrices can be used to transform the block matrix into a block
upper triangular matrix. Here, we must assume the 11-block has an inverse D1
11 =
1
1
b
b
When L1 is q1 q1 , L2 is q2 q2 , D22 D22 D21 D11 D12 , d2 d2 D21 D11 d1 ,
] is an q1 q2 zero matrix, then

L1
L1
]
]
D11 D12
x1
d1
=
D21 D22
x2
d2
D21 D1
D21 D1
L2
L2
11
11

d1
D11 D12
x1
=
b2 =
b22
x2
]W D
d
b22 also has an
b22 is called the Schur complement of D11 in D. If D
The matrix D
inverse, then one can solve for x2 and then x1
b22 x2
D
D11 x1
b 2 and
= d
= d1 D12 x2 =
An alternative view point is to use block OX factors

by the block elementary matrix

D11
]
L1
D11 D12
=
D
D
D21 D1
L
]W
21
22
2
11
(3.6.2)
(3.6.3)
of D. Multiplying D
D12
b22
D
The block elementary matrix has an inverse, which is very easy to nd

1

]
]
L1
L1
=
=
D21 D1
D21 D1
L2
L2
11
11
154
Multiply both sides by this inverse to get a block OX factorization of D

D11 D12
L1
]
D11 D12
=
(3.6.4)
b22 =
D21 D22
D21 D1
L2
]W D
11
Both the block lower and block upper triangular matrices will have inverses
provided the diagonal blocks have inverses.
Example 3.6.3. Consider the above 5 5 matrix with q1 = 2, q1 = 3>

2 1
1 0 1
D11 =
, D12 =
1 2
0 1 0
5
6
5
6
1 0
2 0 0
D21 = 7 0 1 8 and D22 = 7 0 2 0 8 =
1 0
0 0 2
b22 D22 D21 D1 D12
D
11
5
2 0
b22 7 0 2
D
0 0
5
b1
D
22
is easy to
6 5
0
0 87
2
compute and has an inverse

6

1 0
2@3 1@3
1 0 1
8
0 1
1@3 2@3
0 1 0
1 0
6
4 1 2
= 1@3 7 1 4 1 8 and
2 1 4
5
6
5 2 3
= 1@4 7 2 4 2 8 =
3 2 5
The solution of Dx = d = [1 2 3 4 5]W can be computed by using equations

(3.6.2) and (3.6.3)
x2
1
b
b1 d
b1
= D
22 2 = D22 (d2 D21 D11 d1 )
5
6 5 6 5
6

5 2 3
3
1 0
2@3 1@3
1
7
8
7
8
7
8
= 1@4 2 4 2 ( 4 0 1
)
1@3 2@3
2
3 2 5
5
1 0
5 6
6
= 7 5 8
7
and
x1
= D1
11 (d1 D12 x2 )
5 6

6
2@3 1@3
1
1 0 1 7 8
5 )
=
(

1@3 2@3
2
0 1 0
7

9
=
=
6
155
Figure 3.6.1: Three-loop Circuit

The block approach to solving systems is applicable to many important
b22 matrix has an
problems, but one point of concern is whether or not the D
inverse. If D11 has an inverse, then

D11 D12
]
L1
D11 D12
=
and
b22
D21 D22
D21 D1
L2
]W D
11

D11 ]
D11 D12
L1 D1
11 D12
=
b22
b22 =
]W
L2
]W D
]W D
If D and D11 have inverses, then the left side is a product of three matrices with
inverses. Consequently, the product on the left side must have an inverse, and
the 2 2 block diagonal matrix on the right side must have an inverse. This
b22 has an inverse. The following theorem summarizes the
means the 22-block D
above results.
Theorem 3.6.2 (Block OX Factors) Consider the 2 2 block system in equab22 has an inverse. Moreover,
tion (3.6.1). If D and D11 have inverses, then D
the solution of (3.6.1) can be found by solving equations (3.6.2) and (3.6.3).
3.6.3
Application to Three-loop Circuit
In Subsection 3.1.5 a two-loop circuit was considered. By using the Kirchho

voltage and current laws an algebraic system for the three unknown currents
was formulated and solved by using determinants. In circuits with more loops
there will be a larger number of unknown currents and potentials. This requires
more sophisticated solution methods such as Gauss elimination or block factorizations. In this section a three-loop circuit will be studied, see Figure 3.6.1.
There will be ve unknown currents and two unknown potentials at nodes one
and two.
In order to further understand the potential within a circuit, consider a
single loop circuit with a battery and two resistors as illustrated in the top half
of Figure 3.6.2. We choose point d to be the ground potential set equal to zero.
156
Figure 3.6.2: Potential in a Single-loop Circuit
The battery increases the potential from zero to E. Within the battery there is
a small resistance r so that by Ohms law there is a drop in potential of lu from
H to H lu, which is depicted by the segment from points d to e= The segment
from e to f has very small resistance and is neglected. The larger resistance
U between points f and g has a potential drop equal to lU from H lu to
H lu lU = 0=
The derivation of the ve voltage equations and two current equations at
nodes one and two assumes the potentials at nodes one and two are given by {1
and {2 and the potential at the bottoms nodes is set to zero. The voltage drops
across resistors Un for n = 1> > 5 are plus or minus ln Un , which depends on
the direction of the current.
Voltage from node one to ground:
{1 + H1 l1 U1 = 0 and
{1 l3 U3 = 0=
Voltage from node one to node two:
{1 + H2 (l2 )U2 = {2 .
Voltage from node two to ground:
{2 + H3 (l5 )U5 = 0 and
{2 l4 U4 = 0=
At each node the sum of the currents must be zero:
l1 + l2 l3 = 0 and
l2 l4 + l5 = 0=
The matrix version of these seven equations is
5
65
U1 0
0
0
0 1 0
9
9 0 U2 0
0
0
1 1 :
9
:9
9 0
:
0 U3 0
0 1 0 : 9
9
9
9
9 0
0
0 U4 0
0 1 :
9
:9
9 0
:
0
0
0 U5 0
1 :9
9
9
7 1 1 1 0
0
0
0 87
0 1 0 1 1
0
0
157
l1
l2
l3
l4
l5
{1
{2
: 9
: 9
: 9
: 9
:=9
: 9
: 9
: 9
8 7
H1
H2
0
0
H3
0
0
6
:
:
:
:
:=
:
:
:
8
One can view the coe!cient matrix as either a 7 7 matrix or as 2 2 block

matrix where the 11-block is a 5 5 diagonal matrix with the resistances on the
diagonal. The block OX factorization method reduces to solving a 2 2 matrix
b 2 where the unknown vector has the two potentials, and
b22 x2 = d
equation D
solving a 5 5 diagonal matrix equation D11 x1 = d1 D12 x2 = This reduction in
the computational complexity illustrates the merits of using appropriate block
structure.
Example 3.6.4. The MATLAB code circuit3.m can be used to solve this circuit
problem. It uses three methods: Gauss elimination command A\d, the inverse
matrix command inv(A)*d and block OX factorization as in equation (3.6.4).
The following resistor and battery values were used: U1 = 1> U2 = 0=1> U3 = 3,
U4 = 2, U5 = 4 and H1 = 10, H2 = 20, H3 = 30= The code generated the
solution
6
5
6 5
5=2290
l1
9 l2 : 9 10=3053 :
:
9
: 9
9 l3 : 9 5=0763 :
:
9
: 9
9 l4 : = 9 1=8702 : =
:
9
: 9
9 l5 : 9 8=4351 :
:
9
: 9
7 {1 8 7 15=2290 8
3=7405
{2
The negative currents indicate the current ow is in an opposite direction as
indicated in Figure 3.6.1. The potentials are relative to the bottom nodes.
3.6.4
LU Factorization Using MATLAB
The MATLAB command lu(A) can be used to factor an q q matrix D= The

OX factor of Example 3.6.1 is
AA A = [2 -1 0;-1 2 -1;0 -1 2]
A=
2 -1 0
-1 2 -1
0 -1 2
AA [L U] = lu(A)
L=
1.0000 0.0000 0.0000
158
-0.5000 1.0000 0.0000

0.0000 -0.6667 1.0000
U=
2.0000 -1.0000 0.0000
0.0000 1.5000 -1.0000
0.0000 0.0000 1.3333
The OX factor of Example 5.6.2 is a little more confusing because rows one
and three are interchanged and not rows one and two
AA [L U P] = lu(A)
L=
1.0000
0.0000
0.5000
U=
2.0000
0.0000
0.0000
P=
0 0 1
1 0 0
0 1 0
3.6.5
1.
0.0000 0.0000
1.0000 0.0000
0.5000 1.0000
1.0000 1.0000
5.0000 6.0000
0.0000 -2.5000
Exercises
Let D be the 4 4 matrix

5
6
2 1 0
0
9 1 2 1 0 :
:
D=9
7 0 1 2 1 8 =
0
0 1 2
(a).
Use row operations and elementary matrices to nd the OX factors.
(b).
Use these to solve Dx = d = [470 400 400 470]W =
2.
5
6
1 2 1
D = 7 1 3 6 8=
1 0 1
(a).
(b).
Use these to solve Dx = d = [1 2 3]W =
3.
Let D be the 3 3 matrix in Example 3.6.2. Interchange row one and
three, that is, use S13 and not S12 .
(a).
(b).
3.7. DETERMINANTS AND CRAMERS RULE

4.
159

5
6
1 2 3
D = 7 1 2 6 8=
2 3 9
(a).
(b).
5.
Let D be the 4 4 matrix in exercise 1.
(a).
View this as a block 2 2 matrix where q = 4 = 2 + 2 = q1 + q2 =
b22 =
Find the four blocks and the Schur complement matrix D
(b).
Use equations (3.6.2) and (3.6.3) to solve Dx = d = [470 400 400
470]W =
6.
6
5
2 0 1
0
1
9 0 3 0
2
0 :
:
9
9 1 0 3 1 0 : =
:
9
7 0 1 1 3 1 8
1 1 0 1 3
(a).
View this as a block 2 2 matrix where q = 5 = 2 + 3 = q1 + q2 =
b22 =
Find the four blocks and the Schur complement matrix D
(b).
Use equations (3.6.2) and (3.6.3) to solve Dx = d = [5 6 7 8 9]W =
7.
Use MATLAB to conrm any calculations in exercises 1-6.
8.
Consider the 7 7 matrix in Example 3.6.4. By-hand calculations and
Gauss elimination solve this problem.
9.
OX factorization solve this problem.
10.
block OX factorization solve this problem.
11.
Consider the 7 7 matrix in Example 3.6.4.
(a).
Use the MATLAB code circuit3.m to conrm any calculations in
exercises 8-10.
(b).
Use this code to experiment with dierent resistors and batteries.
(c).
What happens when U2 is near or is zero?
3.7
Determinants and Cramers Rule
In Sections 1.4 and 2.2 determinants of 22 and 3 3 matrices were introduced

as natural ways to compute areas and volumes between vectors, which are
columns in the matrix. Also, the unique solution of algebraic systems with
two or three unknowns is characterized by the determinant of the 2 2 or
3 3 coe!cient matrix not being zero. In this section these topics will be
reviewed and extended to q q matrices. Basic properties of determinants will
160
be presented, inverse matrix computation via determinants will be formulated

and an application to steady state three-tanking mixing will illustrate the use
of Cramers rule.
3.7.1
Determinants for 2 2 and 3 3 Matrices
Consider a 2 2 matrix with column vectors a and b

d11 d12
D=
= [a b]=
d21 d22
The area of the parallelogram formed by the vectors, which are not parallel, is
plus or minus the determinant of D
det(D) d11 d22 d12 d21 =
As indicated in Section 2.4 determinants can be used to solve 2 2 algebraic
systems Dx = d by Cramers rule when det(D) 6= 0

g1 d12
d11 g1
)@ det(D) and {2 = det(
)@ det(D)=
{1 = det(
g2 d22
d21 g2
Notice the det(D) does change if the rows and columns are interchanged

d11 d21
) = det(DW ) = det(D)=
det(
d12 d22
And, det(D) changes sign if the rows are interchanged

d21 d22
) = det(D)=
det(
d11 d12
The extension of determinants to 3 3 matrices can be done in an inductive
way. Consider a 3 3 matrix with column vectors a, b and c
6
5
d11 d12 d13
D = 7 d21 d22 d23 8 = [a b c]=
d31 d32 d33
The volume of the parallelepiped formed by the three vectors is plus or minus
the dot product of a with b c, which can be written as the determinant of D
det(D) d11 (d22 d33 d23 d32 )
d12 (d21 d33 d23 d31 )
+d13 (d21 d32 d22 d31 )
= d11 (d22 d33 d23 d32 )
d21 (d12 d33 d13 d32 )
+d31 (d12 d23 d13 d22 )
= a (b c)=
161
Here we have interchanged rows and columns and the determinant does not
change. That is, det(D) = det(DW ) where DW is called the transpose of A and
is dened as
6 5 W 6
5
a
d11 d21 d31
DW 7 d12 d22 d32 8 = 7 bW 8 =
d13 d23 d33
cW
Cramers rule, for det(D) 6= 0> allows one to easily solve the 3 3 algebraic
systems Dx = d
{1
{2
{2
= det([d b c])@ det(D)>

= det([a d c])@ det(D) and
= det([a b d])@ det(D)=
The justication of the formula for the rst unknown follows from the properties
of the cross product and the column version of a matrix-vector product
Dx = d
{1 a + {2 b + {3 c = d
{1 a = d {2 b {3 c=
Recall the cross product b c is perpendicular to both b and c= Use this fact
and take the dot product of both sides with b c
({1 a) (b c) = (d {2 b {3 c) (b c)
(a (b c)){1 = d (b c) b (b c){2 c (b c){3 )
det(D){1 = det([d b c]) 0{2 0{3 =
The derivations of the equations for the other two unknowns are similar. Furthermore, these concepts generalize to algebraic systems with q unknowns.
3.7.2
Determinant of an q q Matrix
The determinant of an q q matrix is dened inductively from determinant of

(q 1) (q 1) matrices in much the same way as for 3 3 matrices. Let Plm
be the determinant of the (q 1) (q 1) matrix formed by deleting row l and
column m of D. The determinant of D may be computed by expanding about
any row of column. Here we use the rst row of the matrix.
Denition 3.7.1. The row expansion of a determinant of an q q matrix D
is along the rst row
det(D) d11 (1)1+1 P11 + d12 (1)1+2 P12 + + d1q (1)1+q P1q =
The coe!cients of dlm > (1)l+m Plm > are called cofactors.
For q = 3 it is easy to expand the 2 2 determinants to reveal the following
seven rules:
162
The determinant of the transpose matrix is det(DW ) = det(D)=

The determinant of the matrix formed by an interchange of
two rows is det(Slm D) = det(D).
3.
The determinant of the matrix formed by adding a
multiple of a row m to row l is unchanged
det(Hlm (d)D) = det(D)=
4.
The determinant of the matrix formed by multiplying a row by
f is det(Hl (f)D) = f det(D)=
5.
det(DE) = det(D) det(E)=
6.
det(D) = dl1 (1)l+1 Pl1 + dl2 (1)l+2 Pl2 + dl3 (1)l+3 Pl3
(row l expansion).
7.
det(D) = d1m (1)1+m P1m + d2m (1)2+m P2m + d3m (1)3+m P3m
(column m expansion).
These seven rules extend to q q matrices. The following example illustrates
how they may be used to reduce the computational burden.
1.
2.
Example 3.7.1. Consider the matrix in Sections 3.1 and 3.4 (Example 3.4.3)
from the two-loop circuit. First, the determinant is evaluated by expanding
down column one, which has only two non-zero components,
5
6
1 1 1
det(D) = det(7 1 0 3 8)
0 2 3

0 3
1 1
= (1)2 (1) det(
) + (1)2+1 (1) det(
)
2 3
2 3
= 6 5 = 11=
One can also use the following elementary row operations to transform the
matrix to upper triangular form
5
6
1 1
1
4 8 =
H32 (2) H11 (1) D = X = 7 0 1
0 0 11
Apply rule three twice to get det(H32 (2)(H11 (1) D)) = det(H11 (1) D) = det(
D) = det(X )= The determinant of an upper triangular matrix can be expanded
by the left column so that the determinant is the product of the diagonal components. Therefore, det( D) = det(X ) = (1)(1)(11) = 11=
Example 3.7.2. Consider the 4 4 matrix for the heat conduction problem in
Sections 3.3 and 3.4 (Example 3.4.4) and expand it by column one using rule
163
one
5
2 1
9 1 2
det(D) = det(9
7 0 1
0
0
5
2 1
= 2 det(7 1 2
0 1
= 2(4) 3 = 5=
6
0
0
1 0 :
:)
2 1 8
1 2
6
5
6
1 0
0
0
1 8) (1) det(7 1 2 1 8)
2
0 1 2
This determinant may also be computed by elementary matrix

to obtain an upper triangular form
5
2 1
0
9 0 3@2 1
9
H43 (3@4) H32 (2@3) H21 (1@2) D = X = 7
0 0 4@3
0 0
0
transformation
6
0
0 :
:=
1 8
5@4
Use the third rule three times and expand the upper triangular matrix by the
left column to get
det(H43 (3@4) H32 (2@3) H21 (1@2) D) = det(D) = det(X ) = 1(3@2)(4@3)(5@4) = 5=
3.7.3
Cramers Rule and Inverses
In order to generalize Cramers rule from 3 3 to q q matrices, we must

generalize perpendicular three dimensional vectors to n-dimensional vectors.
Two n-dimensional column vectors a = [d1 dq ]W and b = [e1 eq ]W are
called perpendicular or orthogonal if and only if the dot product is zero, that is,
aW b = ab d1 e1 + +dq eq = 0. Let D be an qq matrix with components
dlm , and denote its column m by a:m so that D = [a:1 a:q ]= The matrix-vector
product Dx can be written as a linear combination of the column vectors, see
Theorem 3.2.1,
Dx = d
{1 a:1 + + {q a:q = d=
In order to nd the rst unknown {1 , we must solve
{1 a:1 = d ({2 a:2 + + {q a:q ) =
In the case q = 3, we computed the dot product of both sides with the cross
product of the second and third columns. Since the cross product of two vectors
is perpendicular to both the vectors, the dot product on the right side simplied
to the det([d a:2 a:3 ]) and, consequently, {1 = det([d a:2 a:3 ])@ det(D)=
164
The generalization of the cross product to higher dimensions requires an

n-dimensional vector that will be perpendicular to q 1 given n-dimensional
vectors. Let the q 1 given column vectors m = 2> > q in the matrix D be
a:2 > > a:q = We now claim the column vector c of cofactors with m = 1 will be
perpendicular to each of these vectors
6
5
(1)1+1 P11
9 (1)2+1 P21 :
:
9
c=9
:=
..
8
7
.
q+1
(1)
Pq1
To see how this is established, consider the reduced notational burden of the
case q = 4= We must nd {1 in
{1 a:1 = d ({2 a:2 + {3 a:3 + {4 a:4 ) =
Compute the dot product of both sides with c
({1 a:1 ) c = (d ({2 a:2 + {3 a:3 + {4 a:4 )) c
(3.7.1)
(a:1 c){1 = d c (a:2 c){2 (a:3 c){3 (a:4 c){4 =
All ve of the dot products can be expressed as determinants.
a:1 c = d11 (1)1+1 P11 + d21 (1)2+1 P21 +
d31 (1)3+1 P31 + d41 (1)4+1 P41
= det(D) (expansion by column one) and similarly
d c = det([d a:2 a:3 a:4 ])=
The other three determinants are also expanded by column one and are equal
to zero because they have two identical columns.
(a:2 c) = d12 (1)1+1 P11 + d22 (1)2+1 P21 +
d32 (1)3+1 P31 + d42 (1)4+1 P41
6
5
d12 d12 d13 d14
9 d22 d22 d23 d24 :
:
= det(9
7 d32 d32 d33 d34 8) = det([a:2 a:2 a:3 a:4 ]) = 0>
d42 d42 d43 d44
(a:3 c) = det([a:3 a:2 a:3 a:4 ]) = 0 and
(a:4 c) = det([a:4 a:2 a:3 a:4 ]) = 0=
Then equation (3.7.1) becomes
det(D){1 = det([d a:2 a:3 a:4 ])=
By using other columns of cofactors one can solve for the other unknowns.
165
Theorem 3.7.1 (Cramers Rule) Let D be an q q matrix, and let Dl be an

q q matrix with the same columns of D except column l is replaced by column
d. If det(D) 6= 0, then the solution of Dx = d is given by
{l = det(Dl )@ det(D).
Cramers rule is useful for smaller dimensional matrices, special calculations
where only a few of the unknown components are required and some theoretical
considerations. However, it does allow one to give an explicit formula for the
inverse matrix
DD1 = L
D[b1 bq ] = [e1 eq ] or equivalently
Db1 = e1 > > Dbq = eq =
In order to nd the rst column b1 of the inverse matrix, apply Cramers rule
with d = e1
el1
e11
e21
eq1
= det(Dl )@ det(D) where

= det([e1 a:2 a:q ])@ det(D) = (1)1+1 P11 @ det(D)
= det([a:1 e1 a:q ])@ det(D) = (1)1+2 P12 @ det(D)
..
.
= det([a:1 a:2 e1 ])@ det(D) = (1)1+q P1q @ det(D)=
The other columns of the inverse matrix are similar.

Theorem 3.7.2 (Inverse Matrix via Determinants) Let D be an q q matrix
with non-zero determinant. Then the inverse matrix exists and column m in the
inverse matrix is
6
6 5
5
(1)m+1 Pm1 @ det(D)
e1m
9 e2m : 9 (1)m+2 Pm2 @ det(D) :
:
: 9
9
bm = 9 . : = 9
:=
..
8
7 .. 8 7
.
eqm
(1)m+q Pmq @ det(D)
Example 3.7.3. Use determinants to nd

5
1 2
D=7 1 3
1 0
the inverse of
6
1
2 8=
1
Since det(D) = 2 6= 0, the inverse matrix exists. The rst column in the inverse
matrix D1 = E = [b1 b2 b3 ] is
6

5
3 2
6
6 5
5
6 9 det( 0 1 )@2 : 5
:
9

3@2
(1)1+1 P11 @2
e11
:
9
1
2
: 7 2@2 8 =
b1 = 7 e21 8 = 7 (1)1+2 P12 @2 8 = 9
9 det( 1 1 )@2 : =
1+3
:
9

3@2
(1) P13 @2
e31
8
7
1 3
det(
)@2
1 0
166
The other two columns in the inverse matrix may be computed in a similar
fashion to yield
5
6
3 2 1
0 1 8 =
D1 = (1@2) 7 1
3 2
1
If any column of an qq matrix is a linear combination of the other columns,
then the determinant of the matrix must be zero. Equivalently, if the determinant of the matrix is not zero, then no column of the matrix D is a linear
combination of the other columns (the columns are linearly independent, see
Section 5.1 for a more general discussion)
Dx = {1 a:1 + + {q a:q = 0 implies x = 0=
The bold font 0 represents a column vector of zeros. In the cases q = 2 or
q = 3, this condition requires the vectors not be parallel for q = 2> and not
be in the same plane for q = 3. Each column of such matrices must have at
least one non-zero component so that row operations can be done to transform
the matrix to upper triangular form. This eventually leads to the following
theorem, which gives a characterization of matrices that have inverses.
Theorem 3.7.3 (Inverse Matrix Equivalence) Let D be an q q matrix. The
following are equivalent:
(i). D has an inverse,
(ii). Dx = 0 implies x = 0 and
(iii). det(D) =
6 0=
3.7.4
Application to Three-tank Mixing
Mixing chemicals occurs in many industrial and biological systems and include
hot tubs, lead poisoning and brine solutions. Here we consider three wellstirred tanks each with 24 liters (L) with ow rates (L/min) such that the tanks
remain full, see Figure 3.7.1. The amounts of a chemical, say salt, are given
by the unknown functions {(w)> |(w) and }(w) (Kg) so that the concentrations
are {(w)@24> |(w)@24 and }(w)@24 (Kg/L). The ow rates in and out of the tanks
are the numbers (L/min) adjacent to the horizontal arrows in the gure. Also,
the chemical is directly added to the tanks as indicated by the rate numbers
(Kg/min) above the vertical arrows in the gure.
The model is derived by equating the rate of change for the amount to
the rate of change incoming minus the rate of change outgoing. The ow rate
(L/min) times the concentration (Kg/L) is the rate of change in or out of a
tank from an adjacent tank. For the tank on the left, pure water is owing in
at a rate equal to 6 (L/min) and the mixture from the center tank is owing in
at a rate equal to 4 (L/min) with a concentration equal to |@24 (Kg/L). The
rate of change for the amount incoming is 1 + 4(|@24) and the rate of change
of amount outgoing is 10({@24)= This gives the dierential equation for the left
167
Figure 3.7.1: Three-tank Mixing

tank, and the dierential equations for the center and right tanks are similar,
generating a system of three dierential equations
; 0 g{
? { = gw = (1 + 4(|@24)) 10({@24)
| 0 = g|
gw = (2 + 10({@24) + 2(}@24)) (4(|@24) + 8(|@24))
= 0 g}
} = gw = 8(|@24) (2(}@24) + 6(}@24))=
The solution of the system of dierential equations is more complicated than
the system for the projectile problem in Subsection 2.3.3. The time dependent
solution will be discussed in more detail in chapters six and seven.
Presently, consider the steady state solution, which requires all amounts
to be independent of time so that their time derivatives are zero. The above
system of dierential equations becomes an algebraic system
;
? 0 = (1 + 4(|@24)) 10({@24)
0 = (2 + 10({@24) + 2(}@24)) (4(|@24) + 8(|@24))
=
0 = 8(|@24) (2(}@24) + 6(}@24))=
The equivalent vector equation is
5
65
6 5
6
10@24
4@24
0
{
1
7 10@24 12@24 2@24 8 7 | 8 = 7 2 8 =
0
8@24
8@24
}
0
This has a solution because the determinant of the matrix is not zero
5
6

10@24
4@24
0
12@24 2@24
det(7 10@24 12@24 2@24 8) = (10@24)
8@24
8@24
0
8@24
8@24

10@24 2@24
(2@24)
0
8@24
= 5@144=
Cramers rule can easily be used to nd the steady state solutions { = 7=2>
| = 12=0 and } = 12=0. In chapter seven we will show that the time dependent
solutions will approach the steady state solutions as time goes to innity.
168
3.7.5
Determinants Using MATLAB
Return to Example 3.7.2 where the matrix is 4 4 and we want to solve Du =

d = [470 400 400 470]W using Cramers rule. The rst two unknowns are as
follows.
AA A =[2 -1 0 0;
-1 2 -1 0;
0 -1 2 -1;
0 0 -1 2];
AA d = [470 400 400 470];
AA det(A)
ans =
5
AA A1 = [d A(:,2) A(:,3) A(:,4)]
A1 =
470 -1 0 0
400 2 -1 0
400 -1 2 -1
470 0 -1 2
AA u1 = det(A1)/5
u1 =
870
AA A2 = [A(:,1) d A(:,3) A(:,4)]
A2 =
2 470 0 0
-1 400 -1 0
0 400 2 -1
0 470 -1 2
AA u2 = det(A2)/5
u2 =
1270.
Reconsider the application to the three-tank mixing where the right tank
now has the chemical being directly inserted from the top at a rate equal to 3
(Kg/min). The dierential equation for the right tank in Figure 3.7.1 becomes
}0 =
g}
= (3 + 8(|@24)) (2(}@24) + 6(}@24))=
gw
The steady state solution must satisfy

5
65
6 5
6
10@24
4@24
0
{
1
7 10@24 12@24 2@24 8 7 | 8 = 7 2 8 =
0
8@24
8@24
}
3
Multiplying both sides by 24 and use Cramers rule to nd the solution.
AA A = [-10 4 0;
AA
AA
AA
AA
AA
AA
AA
AA
AA
3.7.6
10 -12 2;
0 8 -8];
det(A)
ans =
-480
d = [-24 -48 -72];
A\d
ans =
8.4000
15.0000
24.0000
A1 = [d A(:,2) A(:,3)];
A2 = [A(:,1) d A(:,3)];
A3 = [A(:,1) A(:,2) d];
x = det(A1)/-480
x=
8.4000
y = det(A2)/-480
y=
15
z = det(A3)/-480
z=
24.
169
% uses Gauss elimination
% uses Cramers rule
Exercises
1.
Consider Example 3.7.1 and verify

(a).
det(DW ) = det(D) and
(b).
det(S23 D) = det(D), that is, interchange rows 2 and 3.
2.
Consider Example 3.7.1 and verify
(a).
det(H32 (10) D) = det(D)> that is, multiply row 2 by 10 and add to
row 3 and
(b).
det(H2 (4)D) = 4 det(D), that is, multiply row 2 by 4.
3.
Consider Example 3.7.1 and compute the determinant
(a).
by expanding row l = 3 and
(b).
by expanding column m = 2.
4.
6
5
3 1 0
0
9 1 3 1 0 :
:
D=9
7 0 1 3 1 8 =
0
0 1 3
(a).
Find the determinant by expanding row one.
(b).
Use elementary matrices to transform the matrix to upper triangular form and then compute the determinant.
170
5.

5
6
1 2 1 0
9 0 1 0 0 :
:
D=9
7 8 16 11 0 8 =
1 2 3 7
(a).
Find the determinant by expanding row two.
(b).
Use elementary matrices to transform the matrix to upper triangular form and then compute the determinant.
6.
In Example 3.7.3 use determinants to compute the other two columns of
the inverse matrix.
7.
In Example 3.7.1 use determinants to compute inverse matrix.
8.
Consider the derivation of Cramers rule for q = 3 (see the end of Subsection 3.7.1) and justify the equations for {2 and {3 =
9.
Consider the derivation of Cramers rule for q = 4 (see Subsection 3.7.3)
and justify the equation for {4 =
10.
Use Cramers rule to compute the solution to the three-tank mixing
problem.
11.
Consider the three-tank mixing problem where the chemical is now directly inserted from the top at rates 2, 3 and 4 (Kg/min) for the left, center
and right tanks, respectively. Assume the ow rates between the 24 liter tanks
remain as in Figure 3.7.1.
(a).
Find the three dierential equations.
(b).
Find the matrix equation for the steady state solution.
(c).
Solve the matrix equation.
12.
Use MATLAB to conrm any calculations done in exercises 4-7.
13.
Use MATLAB to conrm any calculations done in exercises 10 or 11.
14.
Use MATLAB and determinants to compute the other two solutions of
Du = d = [470 400 400 470]W where D is from Example 3.7.2.
Chapter 4
Ax = d: Least Squares
Solution
This chapter contains methods for "solving" matrix equations of the form
Dx = d where D is a matrix with more rows (equations) than columns (variables). The normal equations will be developed and used to solve these over
determined algebraic systems. Applications include data tting such as sales
predictions, radioactive decay, world population models, ow rate data and
parameter identication. By-hand calculations will be done as well as implementations in MATLAB for larger and nonlinear problems.
4.1
Curve Fitting to Data
This section is a continuation of the price prediction problem introduced in

Subsection 2.4.4 where a person is trying to predict the price of a new product
such as high denition televisions based on past observations of the price as a
function of time. There were three observations made at months 1, 2 and 3
with prices $2000, $1950 and $1910. In an attempt to "model" this data by
a line | = pw + f, the slope p and the intercept f were to be found from the
three algebraic equations in (2.4.5) or in matrix form
5
6
5
6
1 1
2000
7 2 1 8 p
= 7 1950 8
f
3 1
1910
Dx = d=
In this case D in 3 2> x = [p f]W is 2 1 and there are more equations
than unknowns. So, it is a good possibility that there is no single straight line
containing the three data points. This was viewed geometrically where we chose
p and f so that the point in the plane given by the linear combination of the
two column vectors of D would be closest to the point d = [2000 1950 1910]W =
171
CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
172
Another way given by the residual vector r d Dx to describe this is to

choose p and f so that the least squares function rW r is a minimum.
The objective is to extend both the geometric and residual approaches to
additional data points (rows in D) and to additional model parameters (columns
in D) so that more complicated and accurate data modeling can be done. This
section will focus on more data points with two model parameters. Applications
will be given to sales prediction, radioactive decay and population models.
4.1.1
The Least Squares Problem
Consider D = [a b] with two q 1 columns where q may be larger than 3.

The possible solution of the system
Dx
= d
p
[a b]
= d
f
pa + fb = d

is the p and f that forces the residual vector r d (pa + fb) = [u1 u2 uq ]
to be as "close" as possible to the zero vector. If the sum of the squares of the
residual vectors components is a minimum, then this is called the least squares
solution. The objective is to nd p and f that minimize
rW r = u12 + u22 + + uq2 =
The least squares function of p and f can be explicitly computed as
I (p> f) rW r = (d (pa + fb))W (d (pa + fb))
= dW d2dW ap 2dW bf +
2aW bpf + aW ap2 + bW bf2 =
(4.1.1)
This problem will be solved three ways: graphically, calculus and normal equations (in Section 4.2).
For the above high denition television data a = [1 2 3]W , b = [1 1 1]W
and d = [2000 1950 1910]W and gives
I (p> f) = 114506002(11630)p 2(5860)f + 2(6)pf + 14p2 + 3f2 =
In Subsection 2.4.4 a geometric argument required the minimum to be the
solution of the following algebraic system
W
a r = aW (d (pa + fb)) = aW d (aT ap + aW bf) = 0
(4.1.2)
bW r = bW (d (pa + fb)) = bW d (bT ap + bW bf) = 0=
This is a special case of the normal equations, and as we shall see they give a
solution to the least squares problem even if the number of data points is larger
than q = 3.
4.1. CURVE FITTING TO DATA
173
Table 4.1.1: Computer Sales Data

Months Computers Sold
1
78
2
85
3
90
4
95
5
104
6
113
Example 4.1.1. Suppose a new computer company has had increased sales
and would like to make a prediction of future sales based on past sales. During
the past six months sales have been 78, 85, 90, 95, 104 and 113 computers, see
Table 4.1.1.
A rst look at this data suggests an increase of about 8 computers each
month. A more precise way to view this is to determine the slope and intercept
so that | = pw + f is "closest" to the data points as is depicted in the left graph
in Figure 4.1.1, which will be discussed in detail in the next section. Each of
the data points can be inserted into the linear equation and gives a system with
six equations
;
p1 + f = 78
A
A
A
A
p2 + f = 85
A
A
?
p3 + f = 90
p4 + f = 96
A
A
A
A
p5 + f = 104
A
A
=
p6 + f = 113=
The matrix form is
5
9
9
9
9
9
9
7
1
2
3
4
5
6
1
1
1
1
1
1
:
9
:
9
: p
9
:
=9
: f
9
:
9
8
7
78
85
90
96
104
113
6
:
:
:
:=
:
:
8
The column vectors now have six components, but the least squares function
I (p> f) is a function of two variables. The least squares solution will be determined at the end of this section and will give the straight line indicated in
Figure 4.1.1.
Example 4.1.2. Estimates of the world population over the past hundred years
are in Table 4.1.2, see [9]. Careful inspection of the populations reveals larger
increases in the populations as time progresses. So, an attempt to model this
data by a straight line may not yield very accurate predictions about the world
population. One possible model, among many, is to use a quadratic function
174
Figure 4.1.1: Sales Data
Table 4.1.2: World Population Data

Year World Pop. (billions)
1900
1.65
1910
1.75
1920
1.86
1930
2.07
1940
2.30
1950
2.52
1960
3.02
1970
3.70
1980
4.44
1990
5.27
2000
6.06
175
of time where the three coe!cients are chosen so that the quadratic function
is increasing and concave up for times near the data. Let | = dw2 + ew + f
where time starts in 1900, w = 0, and ends in 2000, w = 100= There are q = 11
equations giving the algebraic system
;
d02 + e0 + 1f = 1=65
A
A
A
2
A
A
? d102 + e10 + 1f = 1=75
d20 + e20 + 1f = 1=86
A
..
A
A
.
A
A
=
d1002 + e100 + f = 6=06=
The equivalent matrix equation uses an 11 3 matrix D and has the form
5
9
9
9
9
9
7
0
102
202
..
.
1002
6
5
1=65
5
6
:
9 1=75 :
:
: d
9
:
:7
9
: e 8 = 9 1=86 : =
:
9 .. :
8 f
7 . 8
6=06
100 1
0
10
20
..
.
1
1
1
..
.
In this case the least square function I (d> e> f) depends on the data and the
three unknown coe!cients d> e and f. The normal equations as described in
the Section 4.2 will be used to solve this system.
4.1.2
Least Squares Solution and Graphing
Consider the least squares problem for tting data to a straight line with q data
points so that the matrix is q 2 and D = [a b] with least squares function
F(m,c) as given in equation (4.1.1). Since the least squares function depends
on two variables, we may graph the value } = I (p> f) on the z-axis with the
independent variables { = p and | = f. Then upon doing the graphing for
appropriate choices of p and f, the minimum of the least squares function can
be estimated and the corresponding values of p and f approximated.
The MATLAB code ls_graph.m has been used to generate the 3D graph
on the left in Figure 4.1.2 and the contour graph on the right of the same
gure for the least squares function I (p> f) in Example 4.1.1. Some trial and
error was required to locate the minimum of the least squares function, which
is approximately I (p> f) 10 where (p> f) (6=6> 71)= MATLAB commands
mesh() and contour() were used to generate the left and right graphs in the
Figure 4.1.2, respectively. This method is not very accurate and not applicable
when more than two parameters are to be found. But, it does give geometric
interpretation and can be useful when examining the sensitivity of the least
squares solution to variations in the data. In Figure 4.1.2 notice the solution is
located in a long thin "valley" of the graph. This suggests small changes in the
data might cause signicant changes in the solution!
176
Figure 4.1.2: Least Squares Function for Sales Data
4.1.3
Least Squares Solution and Calculus
The least square function I (p> f) in equation (4.1.1) is a quadratic function of

two variables. By xing one of the variables, it becomes a function of the other
variable. The left graph in Figure 4.1.2 is formed from the trace curves in both
the f and p directions. The minimum of the least squares function I (p> f)
must be located at the intersection of trace curves in the p and f directions so
that both trace curves are at a minimum. This allows one to use calculus of a
single variable, but applied twice: once to the trace curve in the p direction
and once to the trace curve in the f direction.
Fix f = f0 and consider I (p> f0 ) as a function of p. Dene i (p)
i (p) I (p> f0 ) = dW d2dW ap 2dW bf0 + 2aW bpf0 + aW ap2 + bW bf20 =
Compute the rst derivative with respect to p and set it equal to zero
gi
= 0 = 0 2dW a 0 + 2aW bf0 + aW a2p + 0=
gp
This derivative is also called the partial derivative of F(m,c) with respect to the
CI
=
m variable and is written as Cp
Fix p = p0 and consider I (p> f) as a function of f. Dene the j(f)
j(f) I (p0 > f) = dW d2dW ap0 2dW bf + 2aW bp0 f + aW ap20 + bW bf2 =
177
Compute the rst derivative with respect to f and set it equal to zero
gj
= 0 = 0 02dW b + 2aW bp0 + bW b2f + 0=
gf
This derivative is also called the partial derivative of F(m,c) with respect to the
c variable and is written as CI
Cf =
These two equations can be written in vector form
W

W
a d
p0
a a aW b
=
=
bW a bW b
bW d
f0
Note they are the same as in equations (4.1.2), but now the length of the vectors
is not restricted to q = 3! The matrix equation can also be written as
DW Dx = DW d where
aW
p0
W
D = [a b]> D =
and x =
=
f0
bW
This is called the normal equations corresponding to the least squares problem
Dx = d=
Return to the sales prediction problem in Example 4.1.1 where
a = [1 2 3 4 5 6]W
b = [1 1 1 1 1 1]W and
d = [78 85 90 96 104 113]W >
giving aW a = 91> aW b = 21 = bW a, bW b = 6, aW d = 2100 and bW d = 566= The
normal equation is

91 21
p0
2100
=
>
21 6
566
f0
and the solution is easily computed to be p0 = 34@5 = 6=8000 and f0 =
1058@15 70=5333 with I (p0 > f0 ) 8=1333= These numbers are consistent
with the estimates given by the graphical approach in Figure 4.1.2. Now use
the equation | = p0 w + f0 to predict any future sales. For example, at the
month w = 15 the predicted sales is
|(15) = p0 15 + f0 = 6=8000(15) + 70=5333 172=
The data, straight line model | = p0 w + f0 and prediction are illustrated in
Figure 4.1.1, which was generated by the MATLAB code market.m.
4.1.4
Application to Radioactive Decay
The model for the amount of radioactive material, x(w), is

gx
gw
= ux where
u is the decay rate and x(0) is the initial amount.
178

Table 4.1.3: Radioactive Decay Data
Times Amounts
10
101
11
95
12
90
13
86.5
14
82
15
79
16
78.2
The solution of this simple dierential equation is x(w) = x(0)huw = If both u

and x(0) are known, then it is easy to make predictions about the amount of
radioactive material by simply evaluating this function at the desired time.
Often these constants are not known! By making observations at dierent
times one can formulate a least squares problem for u and x(0). In order to
convert this to a linear model, compute the natural log of both sides of the
model to get
ln(x(w)) = ln(x(0)huw ) = ln(x(0)) + ln(huw ) = ln(x(0)) + (u)w=
Use the linear model | = pw + f with | = ln(x(w)), p = u and f = ln(x(0))=
Consider following data taken at times t = [10 11 12 13 14 15 16]W and
measured amounts u = [101 95 90 86=5 82 79 78=2]W , see Table 4.1.3. There
q = 7 algebraic equations give the system
;
p10 + f = ln(101)
A
A
A
?
p11 + f = ln(95)
..
A
.
A
A
=
p16 + f = ln(78=2)=
The equivalent 7 2 matrix
5
10
9 11
9
9 ..
7 .
equation is
6
5
ln(101)
1

9 ln(95)
1 :
p
:
9
=9
..
.. : f
7
.
. 8
ln(78=2)
16 1
6
:
:
:=
8
The normal equations are solved, see the MATLAB code decay.m, to give p =
0=0439 and f = 5=0372= This implies u = p = 0=0439 and x(0) = hln(x(0) =
hf = h5=0372 = 154=0352 giving
x(w) = x(0)huw = 154=0352h0=0439w =
The predicted amount at time w = 60 is 11=5172. The data and exponential
model are illustrated in Figure 4.1.3
Figure 4.1.3: Radioactive Decay
4.1.5
Least Squares Computations Using MATLAB
Consider Example 4.1.1 with the sales data in Table 4.1.1.

AA
AA
AA
AA
AA
AA
AA
AA
AA
AA
AA
t = [1 2 3 4 5 6];
sales = [ 78 85 90 96 104 113 ];
a = t;
b = ones(6,1);
d = sales;
A=[a b];
% 6x2 matrix
a*a
% compute the components
ans =
91
a*b
ans =
21
b*a
ans =
21
b*b
ans =
6
a*d
ans =
2100
179
180
AA b*d
ans =
566
AA A*A
% matrix product for normal equations
ans =
91 21
21 6
AA A*d
ans =
2100
566
AA lssol=(A*A)\A*d
% solution via Gauss elimination
lssol =
6.8000
70.5333
AA lssol2 = A\d
% better method...uses QR factors
lssol2 =
6.8000
70.5333
The reader should examine the MATLAB codes market.m and decay.m for
additional least squares computations.
4.1.6
1.
Exercises
Consider the least squares solution of the system

;
? 1p + f = 1
2p + f = 3
=
3p + f = 7=
(a).
Find the least squares function in equation (4.1.1).
(b).
Solve the system in equation (4.1.2).
(c).
Compute | = pw + f at time w = 5.
2.
Consider the price of high denition television after four months of price
watching
Months
1
2
3
4
(a).
(b).
(c).
HDTV Prices
2000
1950
1910
1875

Compute | = pw + f at time w = 10 months.
181
3.
Consider the computer sales problem in Example 4.1.1 after two additional months of sales: month 7 there were 120 computers sold and month 8
there were 131 computers sold. Consider all 8 data points, see Table 4.1.1.
(a).
(b).
(c).
Compute | = pw + f at time w = 15 months.
4.
Consider tuition, fees and books (lumped as costs) at University Expensive. These have risen over the past four years and one would like to predict
what they will be four years from now. Suppose
Years
1
2
3
4
Costs
11,300
11,900
12,700
13,400
(a).
(b).
(c).
Compute | = pw + f at time w = 8 years.
5.
Consider the radioactive decay problem, see Table 4.1.3. Suppose there
was a data entry error at time w = 13 where the measured amount should have
been 85.6 and not 86.5.
(a).
Find the least squares solution.
(b).
Predict the amount at time w = 60.
6.
Consider a population such as sh in a large lake, which for a limited
time interval, can be modeled by S 0 = (e g)S where S (w) is the population
size at time w, e is the birth rate and g is the death rate. The solution of this
dierential equation is
S (w) = S (0)h(eg)w =
These rates and the initial population are usually not known and must be
determined from observations. Use the radioactive decay model as a template
and the following population data
Years
1
2
3
4
Population Estimates
1000
1300
1750
2400
(a).
(b).
Compute S (w) at time w = 8 years.
7.
Use MATLAB to conrm any calculations done in exercises 1-6.
8.
Use the MATLAB codes market.m and ls_graph.m to examine sensitively
of the least squares solution when there is a 10% error or uncertainty in the
data for Example 4.1.1.
182
9.
Use the MATLAB codes decay.m to examine sensitively of the least squares
solution when there is a 5% error or uncertainty in the data for the radioactive
decay problem.
4.2
Normal Equations
The normal equations will be developed for the general least squares solution of
Dx = d where D is p q and p A q (more rows or equations than columns or
unknowns). In order to achieve this, the transpose of a matrix and its properties
will be studied. The normal equations DW Dx = DW d will be shown to solve the
least squares problem, provided DW D has an inverse. These concepts will be
related to the calculus and geometric approaches that were discussed in previous
sections. An application to population predictions, based on past population
data, will be discussed.
4.2.1
Transpose Matrix
The transpose of a single p 1 column vector a is a 1 p row vector, which we

have been denoting by aW = [d1 d2 dp ]. Likewise, the transpose of a 1 p
row vector will be p 1 column vector. As in the past, we will continue to
assume lower case bold font means a column vector. The transpose operation
may be applied to matrices.
Denition 4.2.1. Let D = [a1 a2 aq ] be an p q matrix whose m column
p 1 vectors are am = The transpose of D, DW > is an q p matrix whose m row
vectors are the transpose of the m column vectors of D, aWm ,
5
9
9
DW 9
7
aW1
aW2
..
.
6
:
:
:=
8
aWq
Another way of dening the transpose is DW [dml ] where D = [dlm ], 1 l p
and 1 m q=
Example 4.2.1. Consider the following matrices

3
2 7
1 3 4
x=
> D=
and E =
=
4
8 1
7 8 9
The transpose of these are
xW =
3 4

> DW =
2 8
7 1
6
1 7
and E W = 7 3 8 8 =
4 9
4.2. NORMAL EQUATIONS

Note the following identity
183

2 7
3 W
)
8 1
4

W
34
=
= 34 20
20

2 8
3 4
=
7 1

(Dx)W
= (
= xW DW =
The transpose of a matrix-vector product may also be extended to a matrixmatrix product

2 7
1 3 4
51 62 71
DE =
=
8 1
7 8 9
1 16 23
5
6 5
6
51 1
1 7
2 8
(DE)W = 7 62 16 8 = 7 3 8 8
7 1
71 23
4 9
= E W DW =
Theorem 4.2.1 (Properties of Transpose) Let D and E be p q matrices,
F be an q s matrix, x and y be q 1 column vectors and v be a real number.
1.
2.
3.
(vD)W = v(DW ) and (D + E)W = DW + E W =

xW y = yW x> (Dx)W = xW DW and (DF)W = F W DW .
If D is p p and has an inverse, then (D1 )W = (DW )1 =
The proof of (Dx)W = xW DW follows from the column version of a matrixvector product, see Theorem 3.2.1,
(Dx)W
= ({1 a1 + {2 a2 + + {q aq )W
= ({1 a1 )W + ({2 a2 )W + + ({q aq )W
= {1 aW1 + {2 aW2 + + {q aWq
5 W 6
a1
9
9 aW2 :
:
{1 {2 {q 9 . :
=
.
7 . 8
aWp
= xW DW =
The proofs of the other properties are also direct consequences of the denitions.
4.2.2
Normal Equations and Least Squares
If the exact solution of Dx = d cannot be found, then perhaps one can nd

the x such that the residual r(x) d Dx is as "small" as possible. As in the
184
previous section we shall dene "small" in terms of the least squares function
W
I (x) r(x) r(x), which is a function from n-dimensional space into the real
numbers, that is, I : Rq $ R=
Denition 4.2.2. Consider Dx = d where D is p q and p A q. The
least squares solution is an q 1 column vectors x such that the least squares
function evaluated at this x is less than or equal to all other evaluations of the
least squares function
I (x) I (y) for all q 1 column vectors y=
In the previous section with two variables it was not too complicated to
solve the minimization problem. For larger number of columns q A 2, it is
useful to take advantage of the properties of the transpose matrix to derive the
normal equations. Let y = x + (y x) where x is the least squares solution.
Then
r(y) =
=
=
=
d Dy
dD(x + (y x))
dDxD(y x)
r(x)D(y x)=
Put this into the least squares function and use (D(y x))W (D(y x)) 0
I (y) = r(y)W r(y)
= (r(x)D(y x))W (r(x)D(y x))
= (r(x)W (D(y x))W )(r(x)D(y x))
= r(x)W r(x) 2(D(y x))W r(x) + (D(y x))W (D(y x))
I (x) 2(y x)W DW r(x)+0=
(4.2.1)
If DW r(x) = 0> then for all y I (y) I (x)= The vector equation DW r(x) = 0
can be written as
DW r(x) = 0
D (d Dx) = 0
DW Dx = DW d=
W
If the q q matrix DW D has an inverse, then the solution of this equation will
also be the solution of the least squares problem.
Denition 4.2.3. Consider Dx = d where D is p q and p A q. The normal
equation uses the q q matrix DW D and is
DW Dx = DW d=
In order to ensure the q q matrix DW D has an inverse, see Theorem 3.7.3,
we must show DW Dz = 0 implies z = 0. This leads to important concept of
185
linearly independent vectors, which will be more completely studied in Section

5.1.
Denition 4.2.4. Consider q p 1 column vectors. They are called linearly
independent if and only if no column vector is a linear combination of the other
column vectors. This is equivalent to Dz = 0 implies z = 0 where D is the
p q matrix formed by the column vectors.
Example 4.2.2. Consider the HDTV price data problem in Subsection 2.4.2
where {1 = p, {2 = f and
5
6
5
6
2000
1 1
7 2 1 8 {1 = 7 1950 8 =
{2
1910
3 1
The normal equations are
5

1
1 2 3 7
2
1 1 1
3

14
6
6
1
1 8
1

6
2
{1
{2
1 2 3
1 1 1
{1
11630
=
>
5860
{2
=
6
2000
7 1950 8
1910
and the solution is {1 = 45=00 and {2 1863=33= The column vectors in the
3 2 matrix are linearly independent because Dz = 0 implies z = 0 as follows
from
Dz
6
1 1
7 2 1 8 }1
}2
3 1
}1 + }2
2}1 + }2
3}1 + }2
5
= 5
0
=
=
=
=
6
0
7 0 8 or equivalently
0
0
0 and
0=
The only way the three equations can hold is if }1 = }2 = 0, that is, z = 0=
In order to show DW D has an inverse, suppose DW Dz = 0 and show z = 0=
Use the associative and the transpose properties
zW (DW Dz) = zW 0
(zW DW )Dz = 0
(Dz)W (Dz) = 0=
This means Dz must be the zero vector. Since Dz is a linear combination of
the columns of D, it is su!cient to require that no column of D can be written
as a linear combination of the other columns. This proves the next theorem.
186
Theorem 4.2.2 (Normal Equation Solution) Consider Dx = d where D is

p q and p A q. If the columns of D are linearly independent, then DW D
has an inverse and the normal equation has a unique solution, which is also the
least squares solution of Dx = d=
One di!culty in solving the normal equations is that they are often "illconditioned" problems. This means small variations in the data or numerical
approximations can result in large deviations from the exact solution. For such
problems, other methods that use the QR factorization of the matrix should be
used, see [5]. The MATLAB command A\d is an implementation of these more
robust methods.
4.2.3
Normal Equations and Calculus

W
The minimization of the least squares function I (x) = r(x) r(x) has q independent variables in the vector x= Because of the special structure of the least
squares function, this can be simplied to a function of a single real variable w=
Replace x by x+wy where x is the least square solution and dene
W
i (w) r(x + wy) r(x+wy)=

This function must have a minimum at w = 0 and is a quadratic function of w
(again use the properties of the transpose)
i (w) = r(x)W r(x)2w(Dy)W r(x) + w2 (Dy)W (Dy)=
(4.2.2)
The rst and second derivatives are

gi
g2 i
(w) = 0 2(Dy)W r(x) + 2w(Dy)W (Dy) and 2 (w) = 2(Dy)W (Dy)=
gw
gw
If the columns of D are linearly independent and y 6= 0, then Dy 6= 0= At w = 0
the rst derivative must be zero and the second derivative must be positive
gi
(0) = 0 = 2(Dy)W r(x)=
gw
Choose y to be the m unit column vector so that Dy = Dem = a:m is the m
column vector of D. Thus, for all columns of D
aW:m r(x) = 0>
which means the normal equations must hold because all rows of the transpose
matrix times the residual vector are zero.
In Subsection 2.4.4 a geometric interpretation was given for the special case
p = 3 and q = 2. The least squares solution is the point on the plane, given by
the linear combination of the two column vectors, that is closest to the point
associated with the vector d= The column vectors must be perpendicular to the
residual vector, and these two resulting equations are just the normal equations.
187
When p A 3 or q A 2> one can generalize this approach. The plane will
now be replaced by the range of the matrix, which is the linear combination of
q p 1 column vectors and can be written as Dy. The concept of distance
between two vectors d and Dy in m-dimensional space will be
((d Dy)W (d Dy))1@2
and, hence, the distance squared between d and Dy is r(y)W r(y)= The least
squares solution is the point in the range of D that is closest to d. Dene
perpendicular or orthogonal vectors a and r in m-dimensional space to mean
aW r = 0= The normal equations now simply require the columns of D to be
orthogonal to the residual vector.
4.2.4
Application to World Population Modeling
Consider the world population data in Table 4.1.2, which was initially presented
in Example 4.1 2 of the previous section. There were 11 data points from 1900 to
2000 in decade increments, and examination of the data indicated an increasing
and concave upward population estimates. The objective is to nd a quadratic
curve | = dw2 + ew + f so that the data will be "close" to this curve. For a
limited time interval this function can be used to make predictions about the
world population based on just the past observations. This is perhaps one of
the most primitive population models and ignores changing birth and death
rates and variable population densities.
This model is the least square problem with p = 11, q = 3 and {1 = d,
{2 = e and {3 = f= The 11 3 matrix and 11 equations that correspond to the
data (where time equals zero is 1900) is derived from the equations
dw2l + ewl + f = population estimates at times wl =
The p = 11 equations can be written in vector form
5
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
7
0
102
202
302
402
502
602
702
802
902
1002
0
10
20
30
40
50
60
70
80
90
100
1
1
1
1
1
1
1
1
1
1
1
9
:
9
:
9
:
9
:
9
:
:5
6 9
9
: d
9
:
:7 e 8 = 9
9
:
9
: f
9
:
9
:
9
:
9
:
9
:
7
8
1=65
1=75
1=86
2=07
2=30
2=52
3=02
3=70
4=44
5=27
6=06
6
:
:
:
:
:
:
:
:
:=
:
:
:
:
:
:
:
8
188
Figure 4.2.1: World Population Prediction

The normal equation requires one to solve the 3 3 algebraic system
W
DW Dx
65
6 = D d5
6
25333=00 302=50 3=85
d
17=347
3=85 =0550 8 7 e 8 = 104 7 =221 8
104 7 302=50
3=85
=0550 =0011
f
=003
whose solution is d = 0=0005, e = 0=0075 and f = 1=7393=

Figure 4.2.1 illustrates the data and a linear and quadratic least squares
approximation, which was done using the MATLAB code pop.m. The left graph
in Figure 4.2.1 clearly indicates the data are more quadratic than linear. Once
the two least squares problems have been solved, the two least squares functions
are evaluated and are uhvolq = 2=2911 and uhvtxdg = =0583, which conrms
our visual observations. The right graph in Figure 4.2.1 extends the time axis
from 100 (year 2000) to 150 (year 2050) so that any predictions can be made.
For example, the predicted population at w = 130 (year 2030) based on the
quadratic least squares function is
|
4.2.5
= dw2 + ew + f
= (0=0005)1302 + (0=0075) 130 + 1=7393 9=38=
Least Squares Using MATLAB
The following is the MATLAB code pop.m used to generate Figure 4.2.1. It
uses the MATLAB command A\d to accurately solve the least squares problem
189
for the linear and quadratic t to the world population data.

MATLAB code pop.m
% This code nds the linear and quadratic least squares curve
% t to the world population data from 1900 to 2000.
% Time data where 1900 starts at 0 in increments of 10.
t = [0 10 20 30 40 50 60 70 80 90 100];
% Population data in billions.
popw = [1.65 1.75 1.86 2.07 2.30 2.52 3.02 3.70 4.44 5.27 6.06];
one = ones(11,1);
Alin = [t one];
lin= Alin\popw
% Solves normal equation for linear t.
poplin = lin(1)*t + lin(2);
Aquad = [t.^2 t one];
quad = Aquad\popw
% Solves normal equation for quadratic t.
popquad = quad(1)*t.^2 + quad(2)*t + quad(3);
reslin = (popw - poplin)*(popw - poplin)
resquad = (popw - popquad)*(popw - popquad)
subplot(1,2,1)
plot(t,popw,*, t,poplin, t,popquad)
newt = 0:10:150;
poplin = lin(1)*newt + lin(2);
popquad = quad(1)*newt.^2 + quad(2)*newt + quad(3);
subplot(1,2,2)
plot(t,popw,*, newt,poplin, newt,popquad)
The quadratic function can be evaluated at w = 130 by executing quad(1)*130^2
+ quad(2)*130 + quad(3) where the 3 1 vector quad is the solution of the
normal equations given by quad = Aquad\popw. The less robust method for
solving the normal equations is the Gauss elimination method that is implemented by the MATLAB command (Aquad*Aquad)\(Aquad*popw).
4.2.6
1.
Exercises
Consider the following matrices

x=
3
4

> D=
2 0 1
7 1 2
Find xW , DW and E W =
W
W W
Verify
5 (DE) =6E D =
1 2 1
2.
Let D = 7 1 3 2 8 =
1 0 1
(a).
Find DW and D1 =
(b).
Verify (DW )1 = (D1 )W =
(a).
(b).
6
1 3 4
and E = 7 1 0 5 8 =
3 1 2
190
Prove: (DW )1 = (D1 )W when the q q matrix D has an inverse.

Use the properties of the transpose.
(a).
Justify each step in the derivation of the inequality in line (4.2.1).
(b).
Derive equation (4.2.2).
5.
Consider the four data points in the HDTV price problem and the resulting linear least squares problem with {1 = p and {2 = f
3.
4.
1
9 2
9
7 3
4
6
5
1
2000
9 1950
{
1 :
1
:
=9
7 1910
1 8 {2
1
1875
6
:
:=
8
(a).
Find the normal equations.
(b).
Show the columns in the 4 2 matrix are linearly independent.
(c).
Solve the normal equations.
(d).
Use the above to predict the HDTV price at time w = 8.
6.
Consider the rst four data points in the world population problem and
the resulting quadratic least squares problem with {1 = d> {2 = e and {3 = f
5
6
5
6
0
0 1 5
1=65
{
1
9 100 10 1 :
9 1=75
9
:7
8 9
7 400 20 1 8 {2 = 7 1=86
{3
900 30 1
2=07
6
:
:=
8
(a).
Find the normal equations.
(b).
Show the columns in the 4 3 matrix are linearly independent.
(c).
Solve the normal equations.
(d).
Use the above to predict the world population at time w = 130, and
compare with the prediction based on 11 data points.
7.
Consider the world population problem with 11 data points.
(a).
Find the normal equations for the linear least squares problem.
(b).
Conrm the normal equations for the quadratic least squares problem.
(c).
Compare the predicted world population at time w = 120 for the
linear and quadratic models.
8.
Use MATLAB to conrm any calculations in exercises 1, 2, 5, 6 or 7.
9.
Use the MATLAB code pop.m.
(a).
Conrm the calculations and predict the population at w = 160.
(b).
Use the Gauss elimination method, via (A*A)\(A*d), to solve the
normal equations for both the linear and quadratic models.
(c).
Suppose the population estimates for the last two data points were
too low and should have been 5% higher. Solve the new normal equations and
make a new prediction of the population at w = 160.
4.3. MULTILINEAR DATA FITTING
191
Table 4.3.1: Multlinear Data

i xl yl }l (dependent)
1 1
1
100
2 2
1
98
3 1
2
95
4 2
2
94
4.3
Multilinear Data Fitting
Data may be dependent on more than one variable. In this section the focus will
be on multilinear data approximation such as } = i ({> |) = p1 {+p2 |+f where
the parameters for the slopes in the { and | directions and the } intercept must
be found so that the data }l is "closest" to the surface points i ({l > |l )= Problems
like this are easily formulated as least squares problems whose solution will be
given by the normal equations. Applications will include price data for sales
levels in several markets, real estate appraisal as function of several attributes,
and identication of ow rates in "mixing tank" models.
4.3.1
Multilinear in Two Variables
Consider the following four data points in Table 4.3.1 where the measured data
is }l with l = 1> > 4 and may depend in a linear way on the independent
variables { and |. The objective is to nd p1 > p2 and f so that }l p1 {l +
p2 |l + f= This means the residuals ul = }l (p1 {l + p2 |l + f) should be as
small as possible. From inspection of the data points it appears the two slopes
should be negative and the intercept should be positive.
In order to formulate the problem in terms of the normal equations, one
must list the possible equations in the system
;
p1 1 + p2 1 + f = 100
A
A
?
p1 2 + p2 1 + f = 98
p1 1 + p2 2 + f = 95
A
A
=
p1 2 + p2 2 + f = 94=
The vector version of this,
D is 4 3> is
5
1
9 2
9
7 1
2
where x = [p1 p2 f]W , d = [100 98 95 94]W and

1
1
2
2
6
5
6
1 5
100
p1
9 98
1 :
: 7 p2 8 = 9
7 95
1 8
f
1
94
6
:
:=
8
192
The least squares solution is determined by the normal equations

5
6 1
1 2 1 2 9
7 1 1 2 2 89 2
7 1
1 1 1 1
2
5
6
7 9
6
1
1
2
2
1
1
1
1
9 6
10 6
6 4
DW Dx = DW d
5
5
6
5
6 100
p
1
2
1
2
1
:
9
: 7 p2 8 = 7 1 1 2 2 8 9 98
8
7 95
f
1 1 1 1
94
65
6
5
6
p1
579
8 7 p2 8 = 7 576 8 =
f
387
6
6
:
:
8
The solution is p1 = 1=50, p2 = 4=50 and f = 105=75= The residual vector

is r = d Dx = [=25 =25 =25 =25]W so that rW r = 0=25= The predicted value
of } for ({> |) = (3> 2) is
p1 3 + p2 2 + f = (1=50)3 + (4=50)2 + 105=75 = 92=25=
Generally, there should be a much larger set of data points than four for the
three unknowns.
4.3.2
Application to Price Data in Multiple Markets
Investors are always interested in the price per unit of an item. As the number
of units sold increases, usually the price per unit decreases. The price per unit
in one market will be dierent than the price per unit in another market. In
the data in Table 4.3.1 you might consider the } values to be a price per unit in
either country A or country B. Let { be the units sold in country A and | be the
units sold in country B. The price function for country A i ({> |) = p1 {+p2 |+f
is a function of the units sold in the two markets. The two markets have some
interaction so that the number of units sold in one market does have an impact
on the price in the other market. A similar price function for the other country
could also be modeled.
Consider three adjacent countries A, B and C with {, | and } units sold,
respectively. Table 4.3.2 has 9 data points for the price per unit for country A
given by sl where l = 1> > 9= The objective is to nd p1 > p2 > p3 and f so
that sl p1 {l + p2 |l + p3 }l + f= The corresponding equations give the system
;
p1 2 + p2 3 + p3 1 + f = 20> 000
A
A
A
? p1 3 + p2 4 + p3 2 + f = 19> 900
..
A
.
A
A
=
p1 27 + p2 13 + p3 9 + f = 16> 500=
i
1
2
3
4
5
6
7
8
9
Table
xl
2
3
5
8
12
15
19
23
27
The matrix equation is

= [p1 p2 p3 f]W
5
2
9 3
9
9 ..
7 .
4.3.2:
yl
3
4
4
5
6
8
10
11
13
193
Price Data for Three Markets

zl sl (price for country A)
1
20,000
2
19,900
3
19,300
3
19,100
4
18,900
6
18,400
7
17,800
7
17,200
9
16,500
Dx = d where D is 9 4 and there four unknowns x
6
6
5
1 1 5 p1 6
20> 000
9 19> 900 :
2 1 :
p2 :
:9
:
9
9
:
=
:
:=
.. .. 7 p 8 9
..
8
8
7
3
. .
.
f
27 13 9 1
16> 500
3
4
..
.
The normal equations reduce to a 4 4 matrix solve, which can easily be done
using MATLAB command A\d.
AA [A d]
2 3 1 1 20000
3 4 2 1 19900
5 4 3 1 19300
8 5 3 1 19100
12 6 4 1 18900
15 8 6 1 18400
19 10 7 1 17800
23 11 7 1 17200
27 13 9 1 16500
AA A*A
2090 1064 722 114
1064 556 374 64
722 374 254 42
114 64 42 9
AA A*d
2031100
1154600
754500
167100
AA A\d
% (A*A)\(A*d) is a less robust solver
194

1.0e+004 *
-0.00934343434343
-0.00636363636364
-0.00474747474747
2.04242424242424
This means p1 93=43> p2 63=63> p3 47=47 and f 20> 424=24=

The predicted price in country A at a sales level ({> |> }) = (40> 16> 12) is
p1 40 + p2 16 + p3 12 + f 15> 098=98=
The accuracy of the model depends on the data and the choice of the linear
approximation to the data. In this case the price data is fairly large and the least
squares function rW r should be compared with dW d such as in rW r@dW d = 6=03
105 =
4.3.3
Application to Real Estate Appraisal
Consider the appraisal of a home based on its location and ve attributes: age,
number of bathrooms, number of bedrooms, heated square feet and general
condition. Several homes in this location have recently been sold for a known
price and known values of the ve attributes. The home being appraised will
have known values of the ve attributes, but not the price. The objective is to
estimate the price of the home based on the recent sales of comparable homes.
Let the independent variables be {1 (age)> {2 (bathrooms)> {3 (bedrooms)>
{4 (square feet) and {5 (condition)= Suppose there are seven comparable homes
in this location, and they have the following attributes and values y given in
Table 4.3.3. The model will assume the value of the property depends linearly
on the values of the attributes and has the form
y p1 {1 + p2 {2 + p3 {3 + p4 {4 + p5 {5 + f=
The equations for the seven comparable homes yield the system
;
p1 10 + p2 2=5 + p3 3 + p4 2000 + p5 9 + f = 260
A
A
A
? p1 12 + p2 2=0 + p3 5 + p4 2500 + p5 8 + f = 270
..
A
.
A
A
=
p1 10 + p2 2=0 + p3 4 + p4 2400 + p5 9 + f = 280=
The corresponding vector equation has a 7 6 matrix, ve unknown slopes and
an unknown intercept
5
6
6
6 p1
5
5
260
10 2=5 3 2000 9 1 9 p2 :
: 9
:
9 12 2=0 5 2500 8 1 : 9
p3 :
:9
9 270 :
9
9
= 9 . :=
9 ..
..
..
..
.. .. : 9 p :
:
7 .
.
.
.
. . 8 9 4 : 7 .. 8
7 p5 8
280
10 2=0 4 2400 9 1
f
l (home)
1
2
3
4
5
6
7
Table
xl1
10
12
15
8
11
9
10
4.3.3:
xl2
2.5
2
2
3
2.5
2
2
195
Home Appraisal Data

xl3
xl4
xl5 yl (thousand)
3
2000
9
260
5
2500
8
270
4
1900
6
230
3
1600
9
250
4
2300
7
260
3
2000
8
250
4
2400
9
280
As in the price data application, the normal equation is easy to solve using
MATLAB.
AA [A d]
AA
AA
AA
AA
1.0e+003 *
0.0100 0.0025 0.0030 2.0000 0.0090 0.0010 0.2600
0.0120 0.0020 0.0050 2.5000 0.0080 0.0010 0.2700
0.0150 0.0020 0.0040 1.9000 0.0060 0.0010 0.2300
0.0080 0.0030 0.0030 1.6000 0.0090 0.0010 0.2500
0.0110 0.0025 0.0040 2.3000 0.0070 0.0010 0.2600
0.0090 0.0020 0.0030 2.0000 0.0080 0.0010 0.2500
0.0100 0.0020 0.0040 2.4000 0.0090 0.0010 0.2800
lssol = A\d
lssol =
-0.5489
% equals p1
6.7198
% equals p2
0.8193
% equals p3
0.0425
% equals p4
7.7542
% equals p5
93.3308
% equals f
r = d - A*lssol
r*r/(d*d)
5.7814e-005
lssol(1)*9 + lssol(2)*2.5 + lssol(3)*4 +
lssol(4)*1900 + lssol(5)*9 + lssol(6)
259.0114
% equals the appraised value of the home
Assume the home being appraised has the following attributes: 9 years old, 2=5
bathrooms, 4 bedrooms, 1900 heated square feet and general condition equal
to 9. Once the ve slopes and intercept have been computed, the home can be
appraised by evaluating
appraised value
= p1 {1 + p2 {2 + p3 {3 + p4 {4 + p5 {5 + f
= p1 9 + p2 2=5 + p3 4 + p4 1900 + p5 9 + f
259=0=
196
Notice the rst slope, p1 , is a negative number, which indicates the value of
the home decreases as the age increases. The other four attributes add to the
value of the home and so their slopes should be positive.
4.3.4
Application to Flow Rate Identication
In Subsection 3.7.4 the three-tank mixing of a chemical was modeled provided

one knew the ow rates between the tanks. The objective was to nd the
steady state concentrations of the chemical in each of the tanks. A related
application is to nd the concentration of a chemical such as a medicine or
toxin in an animal. A simplistic model is to view the three "tanks" as blood,
bones or tissue. The chemical can pass in and out of these, but the ow rates
and volumes may not be directly found. The following is one possible way of
estimating the coe!cients based on past observations of the concentrations in
the three "tanks."
Consider three tanks whose concentrations are {(w), |(w) and }(w). Suppose
the model for the rst tank has the form
g{
= d{ + e| + f}=
gw
This is a simplication because often the coe!cients must be related such as by
d = ef. The coe!cients d, e and f are to be found from the past observations
{(wl )> |(wl ) and }(wl )= Let {l = {(wl )> |l = |(wl ), }l = }(wl ) and approximate the
derivative by centered nite dierences
{l+1 {l1
g{
gl =
(wl )
gw
wl+1 wl1
Then the above dierential equation gives q algebraic equations where there
were q + 2 observations
d{l + e|l + f}l = gl =
For example, consider the 10 observations recorded in Table 4.3.4. The algebraic
equations give the system
;
d3 + e9 + f12 = 1=5
A
A
A
? d5 + e8 + f13 = 2=0
..
A
.
A
A
=
d15 + e1 + f29 = 1=5=
The corresponding 8 3 vector
5
3 9
9 5 8
9
9 .. ..
7 . .
equation Dx = d
6
5
12 5
6
d
9
13 :
:
9
.. : 7 e 8 = 9
8
7
.
f
15 1 29
where x = [d e f]W is
6
1=5
2=0 :
:
.. : =
. 8
1=5
The least squares solution is easy to nd using the MATLAB command A\d.

Table
tl
1
2
3
4
5
6
7
8
9
10
197
4.3.4: Three-tank Mixing Data

{l1
xl yl zl dl = {wl+1
l+1 wl1
2 10 11
3
9 12
1.5
5
8 13
2.0
7
7 16
2.0
9
5 18
2.0
11 4 21
1.5
12 3 23
1.5
14 2 26
1.5
15 1 29
1.5
17 1 33
AA [A d]
3.0000 9.0000 12.0000 1.5000
5.0000 8.0000 13.0000 2.0000
7.0000 7.0000 16.0000 2.0000
9.0000 5.0000 18.0000 2.0000
11.0000 4.0000 21.0000 1.5000
12.0000 3.0000 23.0000 1.5000
14.0000 2.0000 26.0000 1.5000
15.0000 1.0000 29.0000 1.5000
AA lssol = A\d
lssol =
0.2849
% equals d
0.2361
% equals e
-0.1099
% equals f
AA r = d - A*lssol;
AA r*r
0.2313
The coe!cients in the other two dierential equations
g|
gw
g}
gw
= b
d{ + be| + b
f} and
= e
d{ + ee| + e
f}
may be computed in a similar way by replacing gl with gbl and gel > respectively,
g|
(wl )
gw
g}
(wl )
gw
|l+1 |l1
gbl and
wl+1 wl1
}l+1 }l1
gel =
wl+1 wl1
198
4.3.5
Exercises
1.
Consider the multilinear two variable data in Table 4.3.1.

(a).
By-hand calculations conrm the normal equations.
(b).
Suppose the data for { = 2 and | = 2 is 105 and not 94. Is it
appropriate to model the data by p1 { + p2 | + f?
(c).
Do the calculations with 105 and compare the new residual vector
with the old residual vector.
2.
Consider the multilinear two variable data
i
1
2
3
4
5
6
7
8
9
xl
1
2
1
2
3
1
3
2
3
yl
1
1
2
2
1
3
2
3
3
}l (dependent)
100
98
95
94
=
95
89
91
86
84
(a).
Find 9 3 matrix D.
(b).
Use MATLAB to solve the least squares problem.
(c).
Use the least squares solution to predict the value of p1 { + p2 | + f
when { = 4 and | = 6.
3.
Consider the price data application in three markets as given for country
A in Table 4.3.2. Suppose country B has price data p = [25000 24200 23900
23200 22600 22100 21500 20500 20100]W and the other data in the table
remain the same.
(a).
Find 11 4 matrix D.
(b).
(c).
Use the least squares solution to predict the unit price in country
B from p1 { + p2 | + p3 } + f when { = 40> | = 16 and } = 12.
4.
Consider the real estate appraisal data in Table 4.3.3. Suppose value of
the comparable homes is v = [260 270 230 250 260 250 295]W and the other
data in the table remain the same.
(a).
Find 7 6 matrix D.
(b).
(c).
Use the least squares solution to predict value of the same home.
5.
Consider the three-tank data in Table 4.3.4. Find the coe!cients in
g|
=b
d{ + be| + b
f}=
gw
(a).
(b).
b
Find 8 3 matrix D and the 8 1 column vector d.
4.4. PARAMETER IDENTIFICATION
4.4
199
Parameter Identication
Models evolving from dierential equations have parameters, which must be approximated. An example is radioactive decay as described in Section 4.1. The
model leads to x(0)huw where the parameters are the initial amount x(0) and
the decay rate u= We were able to convert the exponential function into linear
curve t by using the natural logarithm. However, this does not always simplify
all models! An alternative is to use a search algorithm to locate the minimum
of the more complicated nonlinear least squares function. The MATLAB command fminsearch() is an implementation of the Nelder-Mead search algorithm.
Applications to radioactive decay, logistics population and steady state heat
conduction parameter identications will be given.
4.4.1
Search Algorithm for Minimization
Minimization problems that occur in parameter identication are from a nonlinear least squares function
I (s1 > s2 > > sq ) = (g1 x(w1 > s1 > s2 > > sq ))2 +
+(gp x(wp > s1 > s2 > > sq ))2
where x(w> s1 > s2 > > sq ) is a solution of a dierential equation and q is the
number of parameters and p is the number of data points. In the radioactive decay problem there are two parameters s1 = x(0) and s2 = u> and the
dierential equation is
gx
= ux with solution x(w> s1 > s2 ) = s1 hs2 w = x(0)huw =
gw
The Nelder-Mead simplex method searches for the minimum of a real-valued
function I (p) of p variables. In the case of three variables a sketchy outline
of the method is as follows.
Step 1.
Step 2.
Step 3.
Step 4.
Pick four points p0 > p1 > p2 and p3 to form a tetrahedron in three

dimensional space.
Reorder the points so that I (p0 ) I (p1 ) I (p2 ) I (p3 )=
Choose a new fourth point, qhz_p3 , by locating it on the line
containing p3 and (p0 + p1 + p2 )@3=
Repeat steps two and three until values of I (p) in step two are
"nearly" the same.
In step two (p0 + p1 + p2 )@3 is the center of the face (triangle) in the
tetrahedron with vertices p0 > p1 and p2 and
qhz_p3 = p3 + ((p0 + p1 + p2 )@3 p3 )=
If = 0> then qhz_p3 = p3 = If = 1> then qhz_p3 = (p0 + p1 + p2 )@3. The
tricky part of the algorithm is a "good" choice of > where the objective is to
200
choose so that the new set of four points is "closer" to the minimum of I (p).
We shall rely on the implementation given by MATLAB.
The MATLAB implementation of the Nelder-Mead method has the form
fminsearch(Function, guess_ vector). Function is a function le and guess_
vector is an initial guess for the parameters of the function I (p) such that it is
a minimum. For example, use fminsearch() to nd the minimum of
I (s1 > s2 > s3 ) = 100 + (s1 2)2 + (s2 3)2 + (s3 4)2 =
The function le and execution of fminsearch() are
function Fnm = Fnm(p)
Fnm = 100 + (p(1)-2)^2 + (p(2)-3)^2 + (p(3)-4)^2;
AA [p Fval] = fminsearch(Fnm, [10 10 10])
p=
1.9999 3.0000 4.0000
Fval =
100.0000.
4.4.2
Application to Radioactive Decay
Return to the radioactive decay data in Section 4.1 where at times t = [10
11 12 13 14 15 16] the measured amounts of a radioactive substance were
u = [101 95 90 86=2 82 79 78=2]= The exponential function with unknown
initial amount and decay rate must be t to these seven data points. So, p = 7,
q = 2 and the nonlinear least squares function is
I (x(0)> u) = (x1 x(0)huw1 )2 +
+(x7 x(0)huw7 )2 =
The following MATLAB implementation has two function les and a code le.
The function le rad_sol.m has the exponential function and the le ls_rad.m
has the above least squares function. The code le rad_fmin.m has the call to
the MATLAB command fminsearch(), and it also lists the two function les.
MATLAB Code rad_fmin.m
% function r = rad_sol(x1,x2,t)
% r = x1*exp(-x2*t);
%
%function lsrad = ls_rad(xx)
% t = 10:16;
% a = [101 95 90 86.5 82 79 78.2 ];
% lsrad = 0;
% for i = 1:7
% lsrad = lsrad + (a(i)-rad_sol(xx(1),xx(2),t(i)))^2;
% end
%
201
format long
[xx lsval] = fminsearch(ls_rad,[120 .03])
rad_sol(xx(1),xx(2),60)
for i = 0:100
amount(i+1) = rad_sol(xx(1),xx(2),i);
end
plot(amount)
The two function les must be in your directory as well as the above code le
rad_fmin.m. Execution of this code le gives
AA rad_fmin
xx =
1.0e+002 *
1.55946611533598 0.00044865208356
lsval =
9.73565029005973
rad_sol(xx(1),xx(2),60)
10.56557646847834.
This means the initial amount is 155.95 and the decay rate is 0.04486. The
predicted amount at time equal to 60 is 10.5655. These values are very close
to the output from using the linear t to ln(x(w)) = ln(x(0)) + (u)w> where
the initial amount is 154.0352, decay rate is 0.0439 and amount at time 60 is
11.5172.
4.4.3
Application to United States Population Data
In Section 4.2 we t a linear and quadratic function to world population data.

In this section population data from the United States will be examined. Four
models of the data will be considered: linear, quadratic, exponential and logistic
curves. Variation in population data and the methods for solving the least
squares problem will be discussed.
The population data in Table 4.4.1, see [10], has a number of inconsistencies
in how and who were included. Before 1950 the populations in Hawaii and
Alaska were not counted. There has been some variation in how resident and
nonresident citizens were counted. Also, note the 2000 entry, which was 275.13
million before the 2000 census and 282.12 after the census. The four models for
the population are
Model
linear
quadratic
exponential
logistic
Function
pw + f
dw2 + ew + f
x(0)h(eg)w
PF
F+hPfw
Dierential Equation
none
none
=
gx
gw = (e g)x
gx
gw = f(P x)x
The least square problems for the rst three can be solved by the normal equations. The dierential equations for the population x(w) are based on the birth
202

Table 4.4.1: US Population Data
Year
U.S. Pop. (millions)
1900
76.09
1910
92.41
1920
106.46
1930
123.08
1940
132.12
1950
152.27
1960
180.67
1970
205.05
1980
227.22
1990
249.46
2000
275.12
2000 (after census)
282.12
and death rates, e and g. For small time intervals they may be approximated
by constants and in this case we get the exponential function. In the logistic
model the birth and death rates are allowed to vary linearly with the population
so that e g = f(P x)= One can view P as the maximum population, and f
will determine how fast the maximum population will be attained.
The parameters P and f can be determined from a least squares approximation of the data derived by using centered nite dierences to approximate
the derivative
gx
1
(wl )
gw
x(wl )
x(wl+1 ) x(wl1 ) 1
wl+1 wl1
x(wl )
= f(P x(wl ))
= fP fx(wl )=
An implementation is in the MATLAB code pop_us.m where the minimum of

the least squares functions is computed uhvolq = 825=37, uhvtxdg = 108=81,
uhvh{sr = 278=22 and uhvorjlvwlf = 144=08. The least squares function for the
quadratic curve t is the smallest, but the model for larger time intervals does
not have any other way to track births, deaths and emigration.
Another way of determining the constants in the logistic model is to apply
the nonlinear least squares approach to the logistic function
x(w) =
PF
=
F + hP fw
There are eleven data points and three parameters s1 = P> s2 = F and s3 = f
so that p = 11 and q = 3. The second parameter reects the initial population
because
PF
x(0)
x(0) =
giving F =
=
F +1
P x(0)
203
The nonlinear least squares function is

PF
)2 +
F + hPfw1
PF
)2 =
+(x11
F + hPfw11
I (P> F> f) = (x1
The following MATLAB implementation has two function les and a code le.
The function le log_sol.m has the logistic function and the le ls_log.m has
the above least squares function. The code le pop_us_fmin.m has the call to
the MATLAB command fminsearch(), and it also lists the two function les.
MATLAB Code pop_us_fmin.m
% function r = log_sol(x1,x2,x3,t)
% r = x1*x2/(x2+exp(-x1*x3*t));
%
%function ls = ls_log(xx)
% t = 0:10:100;
% p = [76.09 92.41 106.46 123.08 132.12 152.27 180.67
205.05 227.22 249.46 275.13];
% ls = 0;
% for i = 1:11
% ls = ls + (p(i)-log_sol(xx(1),xx(2),xx(3),t(i)))^2;
% end
%
format long
[xx lsval] = fminsearch(ls_log,[500 .2 .0001])
log_sol(xx(1),xx(2),xx(3),150)
x2 = 282.12*exp(-xx(1)*xx(3)*100)/(xx(1)-282.12);
log_sol(xx(1),x2,xx(3),150)
t = 0:10:100;
p = [76.09 92.41 106.46 123.08 132.12 152.27 180.67
205.05 227.22 249.46 275.13];
for i = 0:30
newt(i+1) = i*10+1900;
popus(i+1) = log_sol(xx(1),x2,xx(3),i*10);
end
plot(t+1900,p,*,newt, popus)
The two function les must be in your directory as well as the above code le
pop_us_fmin.m and execution of this code le gives
AA pop_us_fmin
xx =
1.0e+002 *
5.90373049898959 0.00150901875126 0.00000029836503
lsval =
204
Figure 4.4.1: US Population and Logistic Model

1.051582719560504e+002
% predicted population at 2050 using 273.13 at 2000
4.011070115341667e+002
% predicted population at 2050 using 282.12 to restart at 2000.
4.063488610657162e+002
The computed values for the parameters are P = 590=37, F = 0=15092 and
f = 0=00002983= The least squares function evaluated at these parameters is
equal to 105=04. Figure 4.4.1 is a graph of this logistic function and the data
used to nd above parameters.
The following table indicates dierent interpretations of the data depending
on the data and the method used.
Method
pop_us.m
pop_us.m
pop_us_fmin.m
pop_us_fmin.m
United States?
4.4.4
2000 Data
275.13
282.12
275.13
282.12
additional
P
586
676
590
744
*
F
.1613
.1328
.1509
.1175
*
f
.00002982
.00002489
.00002983
.00002208
*
2050 Pred.
404
422
406
433
419
Consider a long thin rod of length O and rectangular cross section Z K=

The left end is attached to a hot mass. Heat diuses down the rod and is lost
through the surface of the rod to a cooler surrounding region whose temperature
xl
10
14
18
22
26
30
34
38
205
Table 4.4.2: Temperature Data

xl (temperature) xl xl (temperature)
115.95
42
48.21
101.94
46
44.48
90.18
50
40.66
80.01
54
38.51
71.37
58
37.81
63.75
62
36.11
56.94
66
35.18
51.97
is xvxu = Let the temperature be independent of time but be a function of { from

the left end where x({) is the temperature. The steady state model is derived
from the Fourier heat law. Let N be the thermal conductivity and f be the
proportionality constant that controls heat loss from the rod to the surrounding
region.
change is heat
diusion from left and right + loss through the surface

g2 x
0 = N(Z K) 2 + f(2K + 2Z )(xvxu x)
g{
f(2K + 2Z )
g2 x
+ F(xvxu x) where F
0 =
=
2
g{
N(Z K)
gx
The unknown parameters are F, gx
g{ (0) = T1 and g{ (O) = T2 =
The solution of the ordinary dierential equation can be shown to have the
form
x({) = F1 sinh(F 1@2 {) + F2 cosh(F 1@2 {) + xvxu =
The constants are determined from the left and right boundary conditions
gx
gx
g{ (0) = T1 and g{ (O) = T2 and are
F1
= T1 @F 1@2 and
F2
T2 T1 cosh(F 1@2 O)
=
F 1@2 sinh(F 1@2 O)
This means the temperature x({> F> T1 > T2 ) is a function of the three parameters as well as {=
The temperature data was taken in rod with length O = 70 and at points
starting at { = 10 in increments of 4 until { = 66 is recorded in Table 4.4.2,
which was taken at the Math Instructional and Research Laboratory [3]. The
surrounding temperature was xvxu = 21=47. The nonlinear least squares function has p = 15 data points and q = 3 parameters and is
I (F> T1 > T2 ) = (x1 x({1 > F> T1 > T2 ))2 +
+(x15 x({15 > F> T1 > T2 ))2 =
206
Figure 4.4.2 was generated by the MATLAB code heat_fmin.m and illustrates
the temperature data and the temperature function with the parameters
s1 = F = 0=0018447> s2 = T1 = 6=257308 and s3 = T2 = 0=00000769=
The value of the least squares function was 7=2725.
MATLAB Code heat_fmin.m
% function r = heat_sol(x1,x2,x3,t)
% r = 2147/100+x2/x1^(1/2)*sinh(x1^(1/2)*t)(x2*cosh(70*x1^(1/2))-x3)/x1^(1/2)/sinh(70*x1^(1/2))
*cosh(x1^(1/2)*t);
%
% function lsr1 = ls_heat(xx)
% x = [10 14 18 22 26 30 34 38 42 46 50 54 58 62 66];
% temp_data = [ 115.95 101.94 90.18 80.01 71.37 63.75 56.94
51.97 48.21 44.48 40.66 38.51 37.81 36.11 35.18];
% lsr1 = 0;
% for i = 1:15
%
lsr1 = lsr1 + (temp(i)-sol1(xx(1),xx(2),xx(3),x(i)))^2;
% end
%
format long
[xx fctval] = fminsearch(ls_heat, [.001 -4.3 .0001])
for i = 1:57
xnew(i) = i+9;
temperature(i) = heat_sol(xx(1),xx(2),xx(3),xnew(i));
end
x = [10 14 18 22 26 30 34 38 42 46 50 54 58 62 66];
temp_data = [ 115.95 101.94 90.18 80.01 71.37 63.75 56.94
51.97 48.21 44.48 40.66 38.51 37.81 36.11 35.18];
plot(x,temp_data,*,xnew,temperature)
4.4.5
1.
Exercises
Use the MATLAB command fminsearch() to nd the minimum of

I (s1 > s2 > s3 ) = 100 + 2(s1 2)2 + 4(s2 3)2 + 10(s3 4)2 =
2.
Use the MATLAB command fminsearch() to nd some of the many minimum points of
I (s1 > s2 > s3 ) = 100 sin(s1 ) sin(=5s2 ) sin(=25s3 )=
3.
Use the MATLAB command fminsearch() to solve the radioactive decay
problem in exercise 5 in Section 4.1.
207
Figure 4.4.2: Temperature Data and Curve Fit

4.
Use the MATLAB command fminsearch() to solve the population problem
in exercise 6 in Section 4.1.
5.
Verify the computation for the United States population, which was done
by the MATLAB code pop_us_fmin.m.
6.
Consider the world population data given in Example 4.1.2, see Table
4.1.2. Use a variation of the MATLAB code pop_us_fmin.m to nd a logistic
curve t to this data, see pop_world_fmin.m.
7.
Consider the heat conduction model. Derive the equations for F1 and
F2 =
8.
Consider the heat conduction model. Suppose there was an error in
collecting the temperature data, which should have been 5% less than recorded
in Table 4.4.2. Find the new solution to the curve tting problem.
Chapter 5
Ax = d: Multiple Solutions
This chapter contains methods for solving algebraic equations Dx = d where D
is a matrix with more columns than rows, that is, more variables than equations.
Row operations are used to nd the basis of the nullspace solution set and a
particular solution. Applications to circuits, trusses and uid ow are studied
in the context of equilibrium equations where the block Gauss elimination and
nullspace methods are introduced.
5.1
Subspaces and Solutions in R3
In this section the foundation is prepared for the systematic representation of

multiple solutions when there are p equations, q unknowns, q is larger than
p and they are large numbers. First, we consider subspaces in Rq and the
general form of the solutions of Dx = d= Then we focus on the cases with three
variables (q = 3) and one or two equations (p = 1 or 2). The bridging concepts
of subspaces, spanning sets and basis will be illustrated. They will be used to
represent solutions of Dx = d> and an application to a two-loop circuit will be
given.
5.1.1
Subspaces in Rq
Let a, b and c be vectors in Rq and let v and w be real numbers. Let a + b and
va denote vector addition and scalar multiplication. Theorem 3.1.1 listed the
following vector space properties for the set of all vectors in Rq :
a + b 5 Rq , a + 0 = a where 0 5 Rq has zeros as components, a + (a) = 0>
a + b = b + a, a + (b + c) = (a + b) + c;
va 5 Rq , 1a = a, v(a + b) = va + vb, (v + w)a = va + wa and v(wa) = (vw)a=
Denition 5.1.1. A subset V of Rq is called a vector subspace of Rq if and
only if for all a, b and c in V
a + b 5 V, a + 0 = a where 0 5 V has zeros as components, a + (a) = 0
209
210
CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
where a 5 S, a + b = b + a, a + (b + c) = (a + b) + c;
va 5 V, 1a = a, v(a + b) = va + vb, (v + w)a = va + wa and v(wa) = (vw)a=
The lines and planes that contain the origin are examples of subspaces in
R3 = The following characterization of a subspace in Rq easily follows from the
vector space properties of Rq =
Theorem 5.1.1 (Vector Subspace) A subset V in Rq is subspace if and only if
(i). the scalar multiple of any vector in V must also be in V and
(ii). the sum of any two vectors in V is also in V.
Example 5.1.1. In Section 2.3 a line in R3 was described by a given point
$

$
$
and direction vector as a row vector
u =
u 0 + w g where the parameter w is a
free real number. For the position of particle moving in a straight line w could
represent time. It will be convenient to change to column vector notation so
that a line in R3 is the set of vectors
x = x0 + wd
where x0 represents the given point in R3 > w is a free real number and d is the
given direction vector in R3 = If the given point is the origin, then the line is
called a one-dimensional subspace of R3 and
V = {x 5R3 : x = wd and w 5 R}=
The set of all points in a line, x = 0 + wd = wd> has the two properties in
Theorem 5.1.1. Any scalar multiple of a point is in the line, which follows from
vx = v(wd) = (vw)d where v is a real number. The sum of any two points in
the line is also in the line, which follows from x = w1 d and y = w2 d giving
x + y = w1 d + w2 d = (w1 + w2 )d. If a line in R3 does not contain the origin, then
it is called an a!ne subset of R3 =
Example 5.1.2. Planes in R3 were described in Section 2.4 by a given point
$

$
$
$
and two vectors that are not parallel as
u =
u 0 + u
d + v e where the two
parameters u and v are free real numbers. The column vector notation for a
plane in R3 is
x = x0 + ua + vb
where the two non-parallel vectors are a and b with a 6= bn for any real number
n. If the given point is the origin, then the plane is called a two-dimensional
subspace of R3 and
V = {x 5R3 : x = ua + vb and u> v 5 R}=
Any point in such a plane is represented by x = 0 + ua + vb = ua + vb and
has the two properties in Theorem 5.1.1. Any scalar multiple of a point in the
plane is also in the plane, which follows from w(ua + vb) = (wu)a + (wv)b where w
5.1. SUBSPACES AND SOLUTIONS IN R3
211
is a real number. Let x = u1 a + v1 b and y = u2 a + v2 b be in a plane containing

the origin. Then the sum is clearly in this plane because
x + y = (u1 a + v1 b) + (u2 a + v2 b) = (u1 + u2 )a + (v1 + v2 )b=
If the given point is not the origin, then the plane is also called an a!ne subset
of R3 =
Example 5.1.3. Any linear combination of vectors is also a subspace such as
the linear combination of the two vector in R4 = Let
a = [2@6 1@3 1 0]W and b = [1@6 8@3 0 1]W
be given vectors in R4 . The set of all linear combination is a subspace of R4
V = {x 5 R4 : x = ua + vb and u> v 5 R}=
The two vectors a and b will be derived in Example 5.2.3 from the set of vectors
in R4 such that
5 6
5
6
0
2 1 1 3
9 0 :
:
Dx = 7 2 4 2 11 8 x = 9
7 0 8=
0 3 1 8
0
Denitions 5.1.2. Let D be an p q matrix. The nullspace of D is subspace
qxoo(D) Rq
qxoo(D) {x 5 Rq : Dx = 0}=
In other words, the nullspace of D is the set of homogeneous solutions. The
range space of A is subspace udqjh(D) Rp
udqjh(D) {y 5 Rp : Dx = y for some x 5 Rq }=
In other words, the range of D is the set of all linear combinations of its column
vectors.
Example 5.1.4. Let
6
2 1 1 3
D = 7 2 4 2 11 8 =
0 3 1 8
The range space of D is the linear combination of the four column vectors in
D= By subtracting row one from row two and then row two from row three we
have
5
6
2 1 1 3
HD = H32 (1) H21 (1) D = 7 0 3 1 8 8 =
0 0 0 0
212
Because the third and fourth columns can be written as a linear combination
of the rst two columns, the range space of HD may be described by the linear
combination of the rst two column vectors in HD.
Denition 5.1.3. A set of vectors y1 > > yn in Rp is called linearly independent if and only if
f1 y1 + + fn yn = 0 implies f1 = f2 = = fn = 0=
This may be stated as \ c = 0 implies c = 0 where \ is the p n matrix of
column vectors and c and 0 are n 1 column vectors.
Example 5.1.1 (continued). In a line R3 containing the origin with one
non-zero vector y1 = d is an example of linearly independent set.
Example 5.1.2 (continued). In a plane R3 containing the origin can be
described by a linear combination of two vectors y1 = a and y2 = b that are
not parallel. Thus, this is an example of a subspace in R3
Example 5.1.3 (continued). Here a subspace in R4 is dened by a linear
combination of two vectors in R4 where y1 = a = [2@6 1@3 1 0]W and
y2 = b = [1@6 8@3 0 1]W = They are linearly independent because the third
and fourth equations in
5
6
5
6 5 6
2@6
1@6
0
9 1@3 :
9 8@3 : 9 0 :
:
9
: 9 :
f1 9
7 1 8 + f2 7 0 8 = 7 0 8
0
1
0
imply f1 = 0 and f2 = 0=
Example 5.1.4 (continued). The rst two column vectors in the new matrix
HD are linearly independent because
5 6
5 6 5 6
2
1
0
f1 7 0 8 + f2 7 3 8 = 7 0 8
0
0
0
implies f1 2 + f2 1 = 0 and f1 0 + f2 3 = 0 and, hence, f1 = 0 and f2 = 0=
An important application is the case where one has n = p linearly independent vectors in Rp . Let D be the p p matrix formed by the columns of
these vectors
D = [y1 y2 yp ]=
Dc = 0 implies c = 0 is equivalent to linearly independent vectors, and in this
case it is characterized by det(D) 6= 0> see Theorem 3.7.3.
Another signicant case is from the least squares problem as described in
Section 4.2 where the p n coe!cient matrix D has more rows than columns
p A n= If the columns are linearly independent, then the n n matrix DW D in
the normal equation has an inverse, see Theorem 4.2.2.
213
Denition 5.1.4. Let V Rp be a subspace. A set of vectors y1 , ,yn in V

is a basis of V if it has following two properties:
(i). any vector in V is a linear combination of the basis vectors and
(ii). the basis vectors are linearly independent.
The dimension of the subspace is n where n is the number of vectors in the
basis.
In R2 and R3 the Cartesian unit vectors are the classical examples of a basis.
In Examples 5.1.3 and 5.2.3 (in the next section) the two vectors in R4 will be
shown to be a basis for a nullspace of the 3 4 matrix. In Example 5.1.4 the
range space of HD has a basis given by y1 = [2 0 0]W and y2 = [1 3 0]W and
has dimension equal to two.
The set of all solutions Dx = d can be broken into the homogeneous and
particular parts x = xk + xs = The homogeneous part must satisfy Dxk = 0>
and the particular part must satisfy Dxs = d= For any p q matrix where
p ? q the general solution of Dx = d is x = xk + xs because
Dx = D(xk + xs ) = Dxk + Dxs = 0 + d = d=
Thus, if one can nd all homogeneous solutions and at least one particular
solution, then the general form of all solutions is known. The homogeneous
solutions are elements of the nullspace. If the nullspace has a basis given by
y1 , ,yn , then the general homogeneous solution is a linear combination of
these vectors
xk = f1 y1 + + fn yn =
If d is in the range space of D, then there is a particular solution Dxs = d=
Theorem 5.1.2 (General Solution of Dx = d) If D is an p q matrix where
p ? q> y1 , ,yn is a basis for the nullspace of D and xs is a particular solution
of Dxs = d, then the general solution of Dx = d is
x = f1 y1 + + fn yn + xs =
Moreover, if H is an p p matrix with an inverse matrix, then the solution
sets of Dx = d and HDx = Hd are equal. In this case, the algebraic systems
are often called equivalent.
5.1.2
Solving One Equation in R3
Let a plane containing the origin in R3 be represented by all linear combinations

of two non-parallel vectors y1 = a and y2 = b= This is signicant because for
each vector in such a plane there is only one linear combination of a and b that
represents of the vector. In order to see this, let x = u1 a+v1 b and x = u2 a+v2 b
giving
0 = x x = (u1 a + v1 b) (u2 a + v2 b) = (u1 u2 )a + (v1 v2 )b=
214
The linear independence implies u1 u2 = 0 and v1 v2 = 0= The following example shows there may be more that one basis for a particular two-dimensional
subspace, which in Example 5.1.6 is a nullspace of a 1 3 matrix D = [2 2
2].
Example 5.1.5. Consider the two-dimensional subspace in R3 given by the
linear combinations of two vectors a = [1 2 3]W and b = [1 0 1]W . This is
a plane containing the origin whose vector equation is n (x 0) = 0 where
n = a b = [2 2 2]W = This gives the traditional algebraic equation
2{1 + 2{2 2{3 = 0.
Two other vectors that satisfy this equation and are not parallel are b
a = [1
W
W
b
1 0] and b = [0 1 1] . They also describe the original plane because
their vector equation is equivalent to the above algebraic equation. That is,
b = [1 1 1]W and so n
b (x 0) = 0 where n
b =a
bb
b = (1@2)n and
n
b is a linear combination of a and
{1 {2 + {3 = 0. One can explicitly show a
b
ua + vb = b
a
5 6
5
6
1
1
1
u 7 2 8 + v 7 0 8 = 7 1 8 =
3
1
0
5
This vector equation is equivalent to three scalar equations

;
? 1u + 1v = 1
2u + 0v = 1
=
3u + 1v = 0=
The solution of the rst two equations easily gives u = 1@2 and v = 3@2= Note,
they also satisfy the third equation.
The following examples illustrate how one can nd the general solution for
a single equation
d11 {1 + d12 {2 + d13 {3 = g1
6
5
{1
d11 d12 d13 7 {2 8 = [g1 ] =
{3
Example 5.1.6. Find a particular solution of the equation {1 +2{2 +3{3 = 10
by letting {2 and {3 equal to zero. Then {1 must be equal to 10 and one
particular solution vector is xs = [10 0 0]W = In order to nd the homogeneous
solutions, let {2 and {3 be "free variables" with {2 = u and {3 = v= Then the
215
homogeneous equation {1 + 2{2 + 3{3 = 0 gives a formula for {1 = 2u 3v=

The vector form of these three equations is
5
6
5
6
5
6
2u 3v
2
3
8 = u7 1 8 + v7 0 8=
u
xk = 7
v
0
1
This suggests y1 = a = [2 1 0]W and y2 = b = [3 0 1]W form a basis for
the nullspace of D = [1 2 3]. Clearly, both vectors satisfy the homogeneous
equation and are not parallel and, so, in R3 they are linearly independent. By
Theorem 5.1.2 the general solution is
5
6
5
6 5
6
2
3
10
x = xk + xs = u 7 1 8 + v 7 0 8 + 7 0 8 =
0
1
0
Example 5.1.7. Find a particular solution of the equation 2{2 + {3 = 10 by

letting {1 = 0 and {3 = 0 so that {2 = 10@2 = 5 and xs = [0 5 0]W = The
homogeneous equation is 2{2 + {3 = 0, and by letting {1 = u and {3 = v be
the "free variables" we can solve for {2 = v@2= The vector form of this is
5
6
5 6
5
6
u
1
0
xk = 7 v@2 8 = u 7 0 8 + v 7 1@2 8 =
v
0
1
Both column vectors on the right side are homogeneous solutions and are not
parallel and, hence, they form a basis for the nullspace of D = [0 2 1]. By
Theorem 5.1.2 the general solution is
5
6
5
6 5 6
1
0
0
x = xk + xs = u 7 0 8 + v 7 1@2 8 + 7 5 8 =
0
1
0
5.1.3
Solving Two Equations in R3
The set of points in R3 that satisfy two equations is equal to the possible
intersection of two planes in R3 = Either the planes never intersect, intersect in a
line or the two planes are the same. Given a collection of algebraic equations it is
not always evident which one of the three cases one has. If the planes are parallel
and are not the same, then there is no solution and the algebraic equations are
called inconsistent. For example, consider the following two equations
{1 + 2{2 + 3{3 = 1
2{1 + 4{2 + 6{3 = 3=
216
By subtracting two times equation one from equation two we have a contradiction 0 = 3 2! In terms of elementary matrices and the augmented matrix this
is

1 0
1 2 3 1
H21 (2) [D d] =
2 1
2 4 6 3

1 2 3 1
=
=
(5.1.1)
0 0 0 1
The last row is equivalent to 0{1 + 0{2 + 0{3 = 1! This means d = [1 3]W is
not in the range space of the 2 3 matrix D=
In the next example the two planes are not parallel and, so, the solution
set will be a line in R3 = Example 2.5.3 illustrates that the direction vector of
the line given by the intersection of the two planes could be found by taking
the cross product of the two normal vectors. The following example gives an
alternative way, which will be useful for higher dimensional systems.
Example 5.1.8. Find all the solutions of
{1 + 2{2 + 3{3 = 4
4{1 + 5{2 + 6{3 = 7=

[D g] =
1 2 3 4
4 5 6 7
Use an elementary row operation to generate an equivalent system

1 0
1 2 3 4
H21 (4) [D g] =
4 1
4 5 6 7

1 2
3
4
=
=
0 3 6 9
(5.1.2)
Let {3 = w be the "free variable" so that H21 (4) [D g] represents an equivalent

system

1 2
{1
4
3
=
w
=
0 3
9
6
{2
Find a particular solution by setting {3 = w = 0 and solving for {2 = 9@(3) =
3 and {1 = 4 2(3) = 2 and, hence, xs = [2 3 0]W = In order to nd the
homogeneous solution, replace the right side of the algebraic system by the zero
vector d = 0 and repeat the row operation to obtain

1 2
{1
0
3
=
w
=
0 3
0
6
{2
217
Solve for {2 = 6w@(3) = 2w and {1 = 3w 2(2w) = w so that xk = [w 2w

w]W = Since H = H21 (4) has an inverse matrix, one can use Theorem 5.1.2 to
nd the general solution
5
6 5
6
1
2
x = xk + xs = w 7 2 8 + 7 3 8 =
1
0
Note, the direction vector [1 2 1]W is a multiple of the cross product of the
b = [4 5 6]W , which is n n
b = [3 6
two normal vectors n = [1 2 3]W and n
W
3] =
Example 5.1.9. Find all the solutions of
{1 + 2{2 + 3{3 = 4
{1 + 3{2 = 5=
Use an elementary row operation to generate an equivalent system

1 0
1 1
1 2 3 4
H21 (1) [D g] =
1 3 0 5

1 2 3 4
=
=
0 1 3 1
(5.1.3)
In this example let {3 = w be the "free variable" so that H21 (1) [D g] represents
an equivalent system

1 2
0 1
{1
{2

=
4
1

w
3
3
A particular solution can be found by setting {3 = w = 0 and solving for {2 = 1

and {1 = 4 2(1) = 2 so that xs = [2 1 0]W = In order to nd the homogeneous
solutions, replace the right side of the algebraic system by the zero vector d = 0
and repeat the row operation to obtain

1 2
0 1
{1
{2

=
0
0

w
3
3
Solve for {2 = 3w and {1 = 3w 2(3w) = 9w so that xk = [9w 3w w]W = Since
H = H21 (1) has an inverse matrix and by Theorem 5.1.2, the general solution
is
5
6 5 6
9
2
x = xk + xs = w 7 3 8 + 7 1 8 =
1
0
218
5.1.4
Application to Two-loop Circuit
The two-loop circuit in Subsection 3.1.5, see Figure 3.1.4, will be used to illustrate how the representation of the solution for the current equation is used.
Let {1 be the potential at the top node relative to the bottom node. The Kirchho voltage law gives three equations for the voltage drops in the three vertical
portions of the loops
;
{1 + H1 U1 l1 = 0
?
{1 U3 l3 = 0
=
{1 (l2 )U2 + H2 = 0=
The Kirchho current law requires the three currents to satisfy
l1 + l2 l3 = 0=
The vector form of the four equations
5
U1 0
0 1
9 0 U2 0
1
9
7 0
0 U3 1
1 1 1 0
is
65
6 5
l1
H1
: 9 l2 : 9 H2
:9
: 9
8 7 l3 8 = 7 0
0
{1
6
:
:=
8
Systems of this form, called the equilibrium equations, also appear in structure,
uid ow and optimizations models, and they will be discussed in more detail
in Section 5.3.
The representation of the current solution to the fourth equation leads to a
reduction in size of the larger algebraic problem. In order to nd the homogeneous solution, let l2 = u and l3 = v be the "free variables" so that l1 = u v=
6 5
6
5 6
5
6
5
uv
1
1
l1
ik = 7 l2 8 = 7 u 8 = u 7 1 8 + v 7 0 8 =
v
0
1
l3
The two column vectors on the right side are homogeneous solutions.
The two column vectors are nullspace basis vectors, which can be recorded
in a 3 2 matrix
5
6
1 1
Q = 7 1 0 8=
0 1
This means [1 1 1]Q = [0 0] or equivalently
W
0 0
( 1 1 1 Q )W =
5
6

1
0
=
QW 7 1 8 =
0
1
219
In order to reduce the size of the algebraic problem from

knowns, put ik into the top three equations to obtain
6 5 6
5
6
5
6
5
1
1
1
0
U1 0
7 0 U2 0 8 (u 7 1 8 + v 7 0 8) + 7 1 8 {1 =
0
1
1
0
0 U3
5
65
6
5
6
U1 0
1 1
1
0
7 0 U2 0 8 7 1 0 8 u + 7 1 8 {1 =
v
0 1
1
0
0 U3
Multiply both sides by the transpose of the basis vectors
that they are homogeneous solutions
5
6
5
6

U1 0
1
0
u
+ 7 1 8 {1 ) =
Q W (7 0 U2 0 8 Q
v
1
0
0 U3
5
6
5
6

U1 0
1
0
u
(Q W 7 0 U2 0 8 Q )
+ Q W 7 1 8 {1 =
v
1
0
0 U3

U1 + U2
u
0
U1
+
{1 =
v
0
U1
U1 + U3

U1 + U2
u
U1
=
v
U1
U1 + U3
four to two un5
6
H1
7 H2 8
0
5
6
H1
7 H2 8 =
0
Q and use the fact

5
6
H1
Q W 7 H2 8
0
5
6
H1
Q W 7 H2 8
0

H1 H2
H1

H1 H2
=
H1
The solution of this 2 2 system is easily found by Cramers rule to be

u
v =
H1 U3 H2 (U1 + U3 )
and
U1 U2 + U1 U3 + U2 U3
H2 U1 H1 U2
=
U1 U2 + U1 U3 + U2 U3
The current vector is

6
5 6
5
6
5
1
1
l1
7 l2 8 = u 7 1 8 + v 7 0 8
0
1
l3
5
6
H1 (U3 + U2 ) H2 U3
1
= 7 H1 U3 H2 (U1 + U3 ) 8
=
U
U
+
U
1 2
1 U3 + U2 U3
U2 H1 U1 H2
5.1.5
1.
Exercises

(a).
Use the denition of cross product to compute the two normal vecb.
tors n and n
b
(b).
Find u and v such that ua + vb = b.
220
2.
Let a plane containing the origin be a linear combination of a = [1 1
1]W and b = [2 0 1]W =
(a).
Find the vector and algebraic equations that describe this plane.
b = [0 2 1]W and
(b).
Show this plane is also a linear combination of a
W
b = [1 1 0] =
b
3.
Find the general solution in R3 of 3{1 + 2{2 + {3 = 6=
4.
Find the general solution in R3 of 4{1 + {3 = 4=
5.
Find the general solution in R3 of 2{1 + {2 = 10=
6.
Find the general solution in R3 of 2{1 + 3{2 + 5{3 = 30=
7.
Use an elementary matrix and an augmented matrix to show the following
equations are inconsistent (have no solution)
3{1 + 2{2 + {3 = 2
9{1 + 6{2 + 3{3 = 5=
8.
Find the general solution in R3 of
3{1 + 2{2 + {3 = 2
9{1 + 6{2 + 3{3 = 6=
9.
{1 + 2{2 + {3 = 2
{1 + 3{3 = 3=
10.
{1 + 2{2 + {3 = 2
{1 + 3{2 = 1=
11.
{1 + 2{2 + {3 = 2
2{1 + 5{2 + 3{3 = 7=
12.
(a).
(b).
5.2
Consider the application to the two-loop circuit.

Justify the 2 2 matrix and the vector equation=
Use Cramers rule to nd the indicated solutions for u and v.
Row Echelon Form
This section presents a systematic way of representing solutions of Dx = d

where D is an p q matrix with p ? q= The main step is to use elementary
row operations to transform the system to an equivalent system in row echelon
form. The examples in the previous section illustrated this for p = 1 or 2
equations and q = 3 variables. This will allow us to identify the homogeneous
and particular solutions. A number of interesting examples will be done byhand calculations as well as by using the MATLAB commands rref() and null().
5.2. ROW ECHELON FORM
5.2.1
221
Solutions in R4
The solution in R4 of {1 + 2{2 + 3{3 = 10 is dierent from the solution of the

same equation in R3 . In Example 5.1.5 we derived the solution in R3 to be
5
6
5
6 5
6
2
3
10
x = u7 1 8+ v7 0 8+ 7 0 8=
0
1
0
In R4 the equation is {1 + 2{2 + 3{3 + 0{4 = 10 and there is a third free variable
{4 = w. Hence, the homogeneous solution in R4 must satisfy {1 + 2u + 3v = 0>
{2 = u> {3 = v and {4 = w= The vector form of the homogeneous solution is
6
5 6
6
5
5
0
3
2
9 0 :
9 0 :
9 1 :
:
9 :
:
9
xk = u 9
7 0 8 + v7 1 8 + w7 0 8=
1
0
0
The three vectors in R4 form a basis for the nullspace of D = [1 2 3 0]= A
particular solution is now a vector in R4 and is xs = [10 0 0 0]W and, so, the
general solution in R4 is
5
6
5
6
5 6 5
6
2
3
0
10
9 1 :
9 0 :
9 0 : 9 0 :
:
9
:
9 : 9
:
x = u9
7 0 8 + v7 1 8 + w7 0 8 + 7 0 8=
0
0
1
0
The next example illustrates that increasing the number of variables can
convert an inconsistent smaller variable problem into a problem with multiple
solutions! The following system with two equations in R3 is inconsistent (has
no solution)
{1 + 2{2 + 3{3 = 1
2{1 + 4{2 + 6{3 = 3=
Modify these two equations by introducing a fourth variable
{1 + 2{2 + 3{3 + 4{4 = 1

2{1 + 4{2 + 6{3 + 9{4 = 3=
Use an elementary matrix transformation on the
the representation of an equivalent system

1 0
1
H21 (2) [D d] =
2 1
2

1 2 3 4
=
0 0 0 1
(5.2.1)
augmented matrix to obtain

2 3 4 1
4 6 9 3
1
=
1
(5.2.2)
222
Choose two "free variables" to be {2 = u and {3 = v. Then in R4 an equivalent

system is

1 4
{1
1
2
3
=
u
v
=
0 1
1
0
0
{4
Set u and v equal to zero to nd a particular solution xs = [3 0 0 1]W = In
order to nd the homogeneous solution, consider

1 4
{1
0
2
3
=
u
v
=
0 1
0
0
0
{4
Then {4 = 0> {2 = u> {3 = v and {1 = 2u 3v or in vector notation the
homogeneous solution is
6
6
5
5
3
2
9 0 :
9 1 :
:
:
9
xk = u 9
7 0 8 + v7 1 8.
0
0
Use Theorem 5.1.2 with H = H21 (2) to get the general solution in R4
5
6
5
6 5
6
2
3
3
9 1 :
9 0 : 9 0 :
:
9
: 9
:
x = u9
7 0 8 + v7 1 8 + 7 0 8.
0
0
1
5.2.2
Row Echelon Form
In the previous examples we used row operations to transform the augmented

matrix into another matrix with all zeros in the "lower left" portion of the
matrix. The "free variables" were dened to be the variables not located at a
"corner" (called a pivot) of the matrix. These were illustrated by the transformed matrices in (5.1.1), (5.1.2), (5.1.3) and (5.2.2). A more precise denition
of the transformed matrix and one that generalizes beyond two equations is row
echelon form, which is found by a sequence of row operations.
Denitions 5.2.1. The p (q + 1) augmented matrix [D d] has been transformed into a row echelon form matrix by a product of elementary matrices H
if and only if
H [D d] = E where
(i). any non-zero row in E has more leading zero components
than the previous row and
(ii). any row below a zero row must also be a zero row=
In terms of the components elm of E this means elm = 0 for m ? ql , elql 6= 0 and
ql1 ? ql = The elql are called the pivots, the variables {ql are called xed and
the remaining variables are called free. The number of pivots in E is called the
rank of [D d], and the number of pivots in the rst q columns of B is called
the rank of D.
223
Example 5.2.1. In the above system in (5.2.1) with two equations and four
variables the augmented matrix is

1 2 3 4 1
[D d] =
.
2 4 6 9 3
The row echelon form matrix is
H21 (2) [D d] =
1 2 3 4 1
0 0 0 1 1
where
the two pivots are e1q1 = 1 with q1 = 1 and e2q2 = 1 with q2 = 4= The xed
variables are {1 and {4 > and the free variables are {2 and {3 = The rank of the
2 4 matrix D is two.
Example 5.2.2. Consider the following system with three equations and four
unknowns
;
? {1 + 2{2 + {3 + 3{4 = 1
2{1 + 4{2 + {3 + 7{4 = 3
=
{3 + {4 = 7=
5
6
1 2 1 3 1
[D d] = 7 2 4 1 7 3 8 =
0 0 1 1 7
Subtract 2 times row 1 from row 2 and then
5
1
H32 (1) H21 (2) [D d] = 7 0
0
add row 2 to row 3 to get

6
2 1 3 1
0 1 1 1 8 =
0 0 2 8
The three pivots are e1q1 = 1 with q1 = 1, e2q2 = 1 with q2 = 3 and e3q3 = 2
with q3 = 4= The xed variables are {1 > {3 and {4 > and the free variable is
{2 = u= The rank of the 3 4 matrix D is three.
The row echelon form matrix is a short way of writing the equivalent system
6 5 6
5 6
5
65
1
2
1 2 3
{1
7 0 1 1 8 7 {3 8 = 7 1 8 u 7 0 8 =
8
0
0 0 2
{4
A particular solution is found by setting the free variable u = 0 and solving the
upper triangular system for the xed variables. Since the diagonal components
are the pivots, the upper triangular solve can be done to yield a particular
solution xs = [14 0 3 4]W = In order to nd the homogeneous solution,
replace the right side d by the zero vector and solve
6
5 6
5
65
2
1 2 3
{1
7 0 1 1 8 7 {3 8 = u 7 0 8 =
0
0 0 2
{4
224
This gives {4 = 0> {3 = 0> {2 = u and {1

homogeneous solution is
5
2
9 1
xk = u 9
7 0
0
= 2u or in vector notation the

6
:
:.
8
By Theorem 5.1.2 with H32 (1) H21 (2) the general solution in R4 is
5
6 5
6
2
14
9 1 : 9 0 :
: 9
:
x = u9
7 0 8 + 7 3 8.
0
4
Example 5.2.3. Consider the following system with three equations and four
unknowns
;
2{1 + {2 + {3 + 3{4 = 4
?
2{1 + 4{2 + 2{3 + 11{4 = 10
=
3{2 + {3 + 8{4 = 6=
5
6
2 1 1 3 4
[D d] = 7 2 4 2 11 10 8 =
0 3 1 8 6
Subtract row 1 from row 2 and then subtract row
5
2 1
H32 (1) H21 (1) [D d] = 7 0 3
0 0
2 from row 3 to get

6
1 3 4
1 8 6 8=
0 0 0
The two pivots are e1q1 = 2 with q1 = 1 and e2q2 = 3 with q2 = 2. The xed
variables are {1 and {2 > and the free variables are {3 = u and {4 = v= The rank
of the 3 4 matrix D is two.
The row echelon form matrix is a short way of writing the equivalent system

4
1
3
2 1
{1
=
u
v
=
6
1
8
0 3
{2
A particular solution is found by setting the free variables u = 0 and v = 0
and solving the upper triangular system for the xed variables. A particular
solution is xs = [1 2 0 0]W = In order to nd the homogeneous solution, replace
the right side d by the zero vector and solve

1
3
2 1
{1
= u
v
=
1
8
0 3
{2
225
This gives {4 = v> {3 = u> {2 = (1@3)u + (3@8)v and {1 = (2@6)u + (1@6)v

so that the homogeneous solution is
6
6
5
5
1@6
2@6
9 8@3 :
9 1@3 :
:
:
9
xk = u 9
7 1 8 + v7 0 8.
1
0
Use Theorem 5.1.2 with H = H32 (1) H21 (1) to obtain the general solution
in R4
6 5 6
6
5
5
1
1@6
2@6
9 8@3 : 9 2 :
9 1@3 :
: 9 :
:
9
x = u9
7 1 8 + v7 0 8 + 7 0 8=
0
1
0
5.2.3
General Solution of Dx = d
Consider the algebraic system Dx = d where D is an p q matrix with p ? q=

Let H be a product of elementary matrices such that
H [D d] = [HD Hd] = E
is in row echelon form. Since H has an inverse, the solution sets of the algebraic
problems Dx = d and HDx = Hd are equal. However, there may not be a
solution. Reconsider Example 5.2.3 where the third equation has been changed
to 3{2 + {3 + 8{4 = 7= Then H [D d] = [HD Hd] = E becomes
5
6
2 1 1 3 4
H32 (1) H21 (1) [D d] = E = 7 0 3 1 8 6 8 =
0 0 0 0 1
The last row implies a contradiction
0{1 + 0{2 + 0{3 + 0{4 = 1!
The rank of the 3 4 matrix D is two, and the rank of the 3 5 augmented
matrix [D d] is three.
Theorem 5.2.1 The algebraic system Dx = d is inconsistent (has no solution)
if and only if any row, say row l, in the row echelon form matrix E has all zeros
to the left of component el>q+1 and el>q+1 is not zero.
The existence of particular solution requires that there be some vector xs
such that Dxs = d= This means d must be a linear combination of the column
vectors of D= The homogeneous solutions of Dxk = 0 are linear combinations
226
of column vectors obtained from the free variables given by the row echelon
matrix. The multiple solutions are all homogeneous solutions plus a particular
solution.
Outline
1.
2.
3.
4.
5.
6.
7.
for Solving Dx = d=
Find the augmented matrix [D d]=
Use elementary matrices to nd the row echelon matrix H[D d]=
There is no solution if the row echelon form matrix is inconsistent.
Use the pivots to nd the xed and free variables.
Find a particular solution of HDxs = Hd=
Set the free variables equal to zero and
solve for the xed variables.
Find the homogeneous solutions of HDxk = H0 = 0=
Replace Hd by the zero vector and
solve for the xed variables in terms of the free variables.
The general solution is the linear combination of the homogeneous
solution plus a particular solution.
Example 5.2.4. This example is more complicated since it has four equations
and six unknowns
;
{1 + {2 + {3 + {4 + {5 + {6 = 1
A
A
?
{1 + 2{2 + {4 + {6 = 1
{
+
{3 + 2{4 + {5 + {6 = 2
A
2
A
=
2{5 + {6 = 4=
The following elementary row operations and
the row echelon form matrix
5
1 1
9 1 2
[D d] = 9
7 0 1
0 0
5
1 1
9 0 1
H32 (1) H21 (1) [D d] = 9
7 0 0
0 0
The free variables
following system
5
1 1
9 0 1
9
7 0 0
0 0
the augmented matrix generate

1
0
1
0
1
1
2
0
1
1
2
0
1
0
1
2
1
1
1
1
6
1
1 :
:
2 8
4
6
1 1 1 1
0 1 0 0 :
:=
2 2 1 2 8
0 2 1 4
are {4 = u and {6 = v, and the row echelon matrix gives the

65
{1
1
1
9 {2
1 1 :
:9
2
2 8 7 {3
0
2
{5
6
5
1
1
9 0
: 9 0 :
: = 9 :u9
7 2
8 7 2 8
0
4
6
6
1
:
9 :
: v9 0 :=
8
7 1 8
1
6
By setting the two free variables to zero and solving the upper triangular system,
we nd a particular solution xs = [1 1 1 0 2 0]W = Set the vector d equal

to the zero vector
5
1
9 0
9
7 0
0
227
and solve the following for the homogeneous solutions

65
5 6
6
5 6
{1
1
1 1
1
1
9 {2 :
9 0 :
9 0 :
1 1 1 :
:9
: = u 9 : v 9 : =
7 1 8
7 2 8
0 2
2 8 7 {3 8
1
0 0
2
0
{5
Use Theorem 5.1.2 with H = H32 (1) H21 (1) to

R6
6
5
5
0
1
9 1@2
9 1 :
:
9
9
9 0
9 1 :
:
9
9
x = xk + xs = u 9
: + v9 0
1
:
9
9
7 1@2
7 0 8
1
0
nd the general solution in

6
6 5
1
: 9 1 :
:
: 9
: 9 1 :
:
:+9
: 9 0 :=
:
: 9
8 7 2 8
0
The nullspace of D has dimension equal to two, and the dimension of the range
space is four.
5.2.4
Reduced Row Echelon Form and MATLAB
Given an augmented matrix [D d] the row echelon matrix is not unique! As

an illustration consider the algebraic system
{1 + 2{2 + 3{3 = 1
4{1 + 5{2 + 6{3 = 3=

[D d] =
One row echelon form matrix is
H21 (4) [D d] =
1 2 3 1
4 5 6 3

1 2
3
1
0 3 6 1
Another row echelon form matrix is obtained by dividing row 2 by 3

1 2 3 1
=
H2 (1@(3)) H21 (4) [D d] =
0 1 2 1@3
A third one is obtained by using row 2 to get a zero in the 12-component

1 0 1 1@3
=
H12 (2) H2 (1@(3)) H21 (4) [D d] =
0 1 2 1@3
This is an example of a reduced row echelon matrix, which can be shown to be
unique. Moreover, the general solution to the algebraic solution of the original
system is easy to construct from

1 0
{1
1@3
1
=
u
where {3 = u.
0 1
1@3
2
{2
228
In this case the general solution is

5
6 5
6
5
6
{1
1@3
1
7 {2 8 = 7 1@3 8 + u 7 2 8 =
0
1
{3
Denition 5.2.2. The p (q + 1) augmented matrix [D d] has been transformed into reduced row echelon form matrix by a product of elementary matrices H if and only if
H [D d] = E where
(i). H [D d] has row echelon form,
(ii). the pivots are all equal to one (elql = 1) and
(iii). zeros are above the pivots (enql = 0 for all n ? l).
The MATLAB commands rref() and null() will easily compute the reduced
row echelon matrix and the nullspace basis, respectively.
AA [A d]
1231
4563
AA rref([A d])
1.0000 0.0000 -1.0000 0.3333
0.0000 1.0000 2.0000 0.3333
AA N = null(A,r)
N=
1
-2
1
AA A*N
0
0
The by-hand computations for Example 5.2.4 are easy to verify.
AA [A d]
1111111
1201011
0112112
0000214
AA rref([A d])
1 0 0 -1 0 0
0 1 0 1 0 .5
0 0 1 1 0 0
0 0 0 0 1 .5
AA N = null(A, r)
N=
-1
1
-1
2

1
-1
-1
1
0
0
229
0
-.5
0
0
-.5
1
AA A*N
0
0
0
0
5.2.5
0
0
0
0
Exercises
Consider the solution of 3{1 + 2{2 + {3 = 12 in R4 =

(a).
Find the particular solution in R4 =
(b).
Find the homogeneous solutions in R4 =
(c).
Find the general solution in R4 =
2.
Consider the row echelon form for some [D d]
6
5
1 2 3 6 1 1
9 0 0 1 2 1 2 :
:
9
7 0 0 0 0 2 3 8=
0 0 0 0 0 1
1.
(a).
Find the pivots, xed and free variables, and the ranks of D and [D
d].
3.
(b).
Does the algebraic system Dx = d have a solution?
Consider the solution in R4 of
3{1 + 2{2 + {3 + {4 = 6
9{1 + 6{2 + 3{3 + 4{4 = 19=
(a).
Find a row echelon matrix.
(b).
(c).
(d).
4.
;
? 2{1 + {2 + {3 + 3{4 = 1
2{1 + 3{2 + 2{3 + 3{4 = 4
=
4{2 + {4 = 7=
(a).
(b).
(c).
(d).

230
5.

;
2{1 + {2 + {3 + 3{4 = 4
?
2{1 + 4{2 + 2{3 + 11{4 = 10
=
6{2 + 2{3 + 16{4 = 12=
(a).
(b).
Find the reduced row echelon matrix.
(c).
6.
;
? {1 + {2 + {3 + {4 = 1
2{1 + 3{3 + 4{4 = 2
=
2{2 3{3 + 2{4 = 6=
(a).
(b).
Find the reduced row echelon matrix.
(c).
7.
Consider the solution Example 5.2.4. By-hand calculations nd the reduced row echelon matrix.
8.
Use the MATLAB commands rref() and null() to conrm any calculations
done in exercises 2-7.
5.3
Nullspaces and Equilibrium Equations
In Subsection 5.1.4 an application was given to a two-loop circuit, and the solution was obtained by using the nullspace associated with the current equation
at the top node in Figure 3.1.4. The resulting equation has the form
5
6 5
6
65
U1 0
i1
0 1
{1
9 0 U2 0
: 9
:
9
1 :
9
: 9 {2 : = 9 i2 :
7 0
8
7
8
7
0 U3 1
{3
i3 8
1 1 1 0
|1
j1
where the rst three unknowns are the currents and the fourth unknown is the
potential at the top node relative to the ground, see Figure 3.1.4. This system
is a special case of the equilibrium equations. In this section the general equilibrium equations will be dened and solved by both block Gauss elimination
and nullspace methods. Applications to more complicated circuits, structures
and uid ows models will be outlined. Additional material on equilibrium
equations can be found in chapter two of [8].
Denition 5.3.1. Let D be an q q matrix and H be an p q matrix with
p ? q= The equilibrium equations can be written as
Dx + H W y = f and
Hx = g
5.3. NULLSPACES AND EQUILIBRIUM EQUATIONS
231
or using 0 as an p p matrix of zeros

D HW
x
f
=
=
H
0
y
g
In the above two-loop circuit algebraic system D is the 33 diagonal matrix
with the three resistor values, and H is the 1 3 row matrix with coe!cients
given by the Kirchho current law. Typically, Hx = g represents some sort of
balance or conservation of quantities in x such as currents, mass or forces. The
top equation Dx + H W y = f may be an implementation of an empirical law
such as voltage drops, momentum or deformation from forces.
5.3.1
Block Gauss Elimination Method
In this section assume D is symmetric (D = DW ) positive denite (x 6= 0 implies

xW Dx A 0), and H has full row rank (the row echelon form has exactly p nonzero pivots). In other words, the columns of H W are linearly independent and,
hence, H W y = 0 implies y = 0= By combining these two assumptions we can
conclude by Theorem 3.7.3 that both D and HD1 H W have inverse matrices.
Solve the rst equation for x = D1 (f H W y) and then put this into the
second equation
Hx = g
HD1 (f H W y) = g=
This gives an equation for y where HD1 H W is an p p matrix
HD1 H W y = HD1 f g=
Once y is known, x is computed from the rst equation
x = D1 (f H W y)=
The block Gauss elimination method may also be described by a block row
operation, see Section 3.6,

Lq
D HW
f
0
0
x
Lq
=
g
HD1 Lp
HD1 Lp
y
H
0

W
D
H
x
f
=
=
y
HD1 f + g
0 HD1 H W
Block Gauss Elimination Method for Equilibrium Equations.
1.
Solve Dz = f and D] = H W so that z = D1 f and ] = D1 H W =
2.
Compute Hz g and H]=
3.
Solve H]y = Hz g so that y = (HD1 H W )1 (HD1 f g)=
4.
Let x = z ]y = D1 f D1 H W y=
232
Example 5.3.1. Consider a particular case of the two-loop circuit

65
6
5
6 5
{1
2 0 0 1
9
9
:
9 0 3 0
: 9
1 :
: 9 {2 : 9 9 :
9
7 0 0 4 1 8 7 {3 8 = 7 0 8 =
1 1 1 0
0
|1
Since D is a diagonal matrix, the solves for z and ] are easy
5
6
5
6
9@2
1@2
z = 7 9@3 8 and ] = 7 1@3 8 =
0@4
1@4
Hz g = 1(9@2) + 1(9@3) 1(0@4) 0 = 15@2 and H] = 1(1@2) +
1(1@3) 1(1@4) = 13@12= Then the solution of H]y = Hz g is also easy to
compute y = (15@2)@(13@12) = 90@13= The last step is
x = z ]y
5
6 5
6
5
6
9@2
1@2
27@26
= 7 9@3 8 7 1@3 8 (90@13) = 7 18@26 8 =
0@4
1@4
45@26
Example 5.3.2. If there are more rows in H, then the solves become more
complicated as in
5
65
6 5 6
{1
2 1 0
1 0
1
9 1 2 1 1 1 : 9 {2 : 9 2 :
9
:9
: 9 :
9
9 0 1 2
: 9 :
0 1 :
9
: 9 {3 : = 9 3 : =
7 1 1 0
0 0 8 7 |1 8 7 4 8
5
0
1
1
0 0
|2
Here D is tridiagonal and H has two rows
5
6

2 1 0
1 1 0
D = 7 1 2 1 8 and H =
=
0 1 1
0 1 2
The inverse of D is
D1
6
3 2 1
= (1@4) 7 2 4 2 8
1 2 3
and, hence, } = D1 f and ] = D1 H W are

5
6
5
6
10
1 3
} = (1@4) 7 16 8 and ] = (1@4) 7 2 6 8 =
14
1 5
233
Now H]y = Hz g becomes

5=5
=75 =75
|1
=
2=5
=75 2=75
|2
whose solution is |1 = 8=8333 and |2 = 1=500= The nal step requires the
computation of x = z ]y
6 5
6 5
6
5
6
5

2=5
=25
=75
5=8333
{1
7 {2 8 = 7 4=0 8 + 7 =5 1=5 8 8=3333 = 7 1=8333 8 =
1=5000
3=5
=25 1=25
3=1667
{3
5.3.2
Nullspace Method for Equilibrium Equations
The nullspace method is patterned on how we solved the two-loop circuit problem in Section 5.1. Again, assume D is symmetric positive denite and H has
full row rank. The full row rank assumption gives the existence of a particular
solution xs so that Hxs = g= Moreover, there is a basis for the nullspace with
q p vectors. Both these can be constructed as illustrated in the previous
sections from the row echelon form of the augmented matrix [H g].
Let Q be the q(qp) matrix formed by the basis vectors for the nullspace
of H. This implies H times each column in Q is a zero vector and, consequently,
HQ = 0. The homogeneous solutions are linear combinations of the columns in
Q , which may be written as a matrix-vector product Q x0 where the coe!cients
in the linear combination are the components of x0 . The general solution of
Hxs = g is
x = xk + xs = Q x0 + xs =
The next step is to put this representation for x into the rst equation
D (Q x0 + xs ) + H W y = f =
Now multiply both sides by Q W and use HQ = 0 giving (HQ )W = 0W > Q W H W =
0 and
Q W (D (Q x0 + xs ) + H W y) = Q W f
Q W DQ x0 + Q W Dxs + Q W H W y = Q W f
Q W DQ x0 + Q W Dxs + 0 = Q W f =
Thus, one must solve
Q W DQ x0 = Q W f Q W Dxs =
The above two assumptions imply the (q p) (q p) matrix Q W DQ has
an inverse.
Finally one can solve for y by multiplying the rst equation by H
H(D (Q x0 + xs ) + H W y) = Hf =
234
This requires the solution of

HH W y = Hf H(D (Q x0 + xs ))=
The p p matrix HH W has an inverse because the columns in H W are linearly
independent.
Nullspace Method for Solving the Equilibrium Equations.
1.
Find a row echelon form of the augmented matrix [H g]=
2.
Find a particular solution xs =
3.
Find a basis for the nullspace and Q .
4.
Compute w = Q W (f Dxs ) and Z = Q W D=
5.
Solve Z Q x0 = z so that x0 = (Q W DQ )1 (Q W f Q W Dxs )=
6.
Solve HH W y = H(f D (Q x0 + xs ))=
Example 5.3.3. Reconsider the two-loop circuit problem in Example 5.3.1.
The 13 matrix H = [1 1 1] already is in row echelon form. Since g is zero,
the particular solution is zero. The nullspace basis vectors are the columns in
5
6
1 1
Q = 7 1 0 8=
0 1
Next compute w and Z
5
1
= Q W (f Dxs ) = 7 1
0
5
6W
1 1
= QW D = 7 1 0 8
0 1
1
0
1
5
2
7 0
0
6W 5
8
0
3
0
6

9
0
7
8
( 9 D0) =
9
0
6

0
2 3 0
0 8=
=
2 0 4
4
The solution of

Z Q x0 = z
6

1 1 0
2 3 0 7
{1
0
1 0 8
=
2 0 4
9
{02
0 1

0

5 2
{1
0
=
=
2 6
9
{02
is {01 = 18@26 and {02 = 45@26= The x vector is

x = Q x0 + xs
5
6
5 6 5
6
1 1
0
27@26
18@26
= 7 1 0 8
+ 7 0 8 = 7 18@26 8 =
45@26
0 1
0
45@26
235
Example 5.3.4. Reconsider Example 5.3.2 where the augmented matrix is in

row echelon form

1 1 0 4
[H g] =
=
0 1 1 5
The free variable is {3 = u and this is equivalent to

4
0
1 1
{1
=
u
=
5
1
0 1
{2
Set u = 0 and solve for
homogeneous solution by

1
0
the particular solution xs = [9 5 0]W . Find the

solving

1
{1
0
0
=
u
=
1
0
1
{2
This gives {2 = u> {1 = u and {3 = u or in vector form

5
6
5
6
1
1
xk = 7 1 8 u and Q = 7 1 8 =
1
1
Next compute w = Q W (f Dxs ) and Z = Q W D=
w
= Q W (f Dxs )
65 6
5
6W 5 6 5
1
2 1 0
9
1
= 7 1 8 (7 2 8 7 1 2 1 8 7 5 8)
3
0 1 2
0
1
= 1(12) 1(1) + 1(8) = 19 and
= QW D
6
5
6W 5
2 1 0
1
= 7 1 8 7 1 2 1 8 = 1 2 3 =
0 1 2
1
Solve Z Q x0 = z
1 2 3
6
1
7 1 8 x0 = 19 so that x0 = 19@6=
1
Then x =Q x0 + xs is as follows
5
6
5 6 5
6
1
9
35@6
x = 7 1 8 (19@6) + 7 5 8 = 7 11@6 8 =
1
0
19@6
236
Find y by solving HH W y = H W (f D (Q x0 + xs ))
5
6
5
6

53@6

1 0
1 1 0 7
1 1 0 7
|1
44@6 8
1 1 8
=
0 1 1
|2
0 1 1
9@6
0 1

2 1
|1
97@6
=
1 2
35@6
|2
whose solution is |1 = 159@18 and |2 = 27@18=
5.3.3
Application to Three-loop Circuit
Consider the three-loop circuit in Subsection 3.6.3, see Figure 3.6.1. The matrix
equation for the ve currents x and two potentials at the top nodes y is
5
65
6
6 5
{1
U1 0
H1
0
0
0 1 0
9
9 0 U2 0
:
: 9
0
0
1 1 :
9
: 9 {2 : 9 H2 :
9
9 0
:
:
:
9
0
U
0
0
1
0
{
0
3
9
:9 3 : 9
:
9
9 0
:
:=
:
9
0
0
U
0
0
0
1
{
=
4
9
:9 4 : 9
:
9
9 0
:
:
:
9
H
0
0
0
U
0
1
{
5
3 :
9
:9 5 : 9
7 1 1 1 0
0
0
0 8 7 |1 8 7 0 8
0 1 0 1 1
0
0
0
|2
This is an equilibrium equation with D being a 5 5 diagonal matrix and the
augmented matrix being

1 1 1 0 0 0
[H g] =
=
0 1 0 1 1 0
Here we outline the nullspace method, which can be viewed as a dimension
reduction scheme where the size of the matrix solves gets smaller. The following
row operations transform the augmented matrix to reduced row echelon form

1 0 1 1 1 0
H12 (1) H2 (1) H1 (1) [H g] =
=
0 1 0 1 1 0
There are three free variables {3 = u> {4 = v and {5 = w> and this implies

1 0
{1
0
1
1
1
=
u
v
w
=
0 1
0
0
1
1
{2
Then the homogeneous solutions
5
1
9 0
9
xk = u 9
9 1
7 0
0
are
6
:
9
:
9
: + v9
:
9
8
7
1
1
0
1
0
:
9
:
9
: + w9
:
9
8
7
1
1
0
0
1
6
:
:
:=
:
8
237
Figure 5.3.1: Bar e with Four Forces

The nullspace matrix is
5
9
9
Q =9
9
7
6
1 1 1
0 1 1 :
:
1
0 0 :
:=
0
1 0 8
0
0 1
The matrix for the solve in step ve is only 3 3> which is smaller than the
7 7 matrix in the equilibrium equation,
5
6
U1 + U3
U1
U1
U1
U1 + U2 + U4
U1 U2 8 =
Q W DQ = 7
U1
U1 U2
U1 + U2 + U5
5.3.4
Application to Six-bar Truss
The six-bar truss in Subsection 3.4.4, see Figure 3.4.1, assumed there was no
deformation of the bars when an external force is applied to one or more nodes.
If the force is large, then there may be some signicant movement of the nodes
resulting in extension or compression of the connecting bars. The model of this
has the form of the equilibrium equations. The rst equation is an enhanced
version of Hookes law, which we will not go into. The second equation manages
the assembly of the local forces and requires the sum of the forces at each node
to be a zero force vector.
Assume the structure deforms only in one plane so that at each end of
the bar (often called an element) there is a two-dimensional force vector, see
Figure 5.3.1. Let the superscript h denote the bar or element number, list the
force components in the horizontal direction by odd number subscripts and
238
the vertical forces by the even subscripts so that for each bar there is a fourdimensional force vector
W
f h = i1h i2h i3h i4h
=
Using the same labels as in Figure 3.4.1, the sum of the forces at each node
gives
Node 1:
Node 2:
Node 3:
[i11
[i31 i41 ] + [i32 i42 ] + [0 z] = [0 0]>

[i12 i22 ] + [i13 i23 ] + [i34 i44 ] = [0 0] and
1
i2 ] + [i33 i43 ] + [i35 i45 ] + [i36 i46 ] = [0 0]=
Now there are still three two-dimensional vector equations, but there are 18
local force components. This can be written as Hx = g where H is an 6 18
matrix and g is a external force vector with six components. In Hx = g the two
equations from the two-dimensional vector equation at node one are in rows one
and two, and the remaining four rows are obtained from the vector equations
at nodes two and three where
6
5
0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 :
:
9
9 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 :
:>
9
H = 9
:
9 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 :
7 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 1 0 8
0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 1
g = [0 z 0 0 0 0]W and
x = [i11 i21 i31 i41 i12 i22 i32 i42 i13 i23 i33 i43 i34 i44 i35 i45 i36 i46 ]W =
The corresponding augmented matrix may be transformed to a row echelon
form matrix by simply moving rows ve and six to rows one and two. The
development of the rst equation in the equilibrium equations is beyond the
scope of this course, but this illustration should give the reader an appreciation
for the role of row echelon forms and nullspaces.
5.3.5
Application to Fluid Flow
Models for uid ow have two basic parts. The rst part describes the momentum in the directions of the ow, and the second part tracks the change in
mass. When a volume is broken into union of small cubes, then these parts form
the rst and second equations in the equilibrium equations. In the case of an
incompressible liquid such as water, any mass moving into a xed volume must
also leave the volume. The incompressibility condition is similar to Kirchhos
current law and to the balance of forces at each node of a truss.
In order to keep the discussion as simple as possible, consider uid ow in
a plane, and the rate of change of mass for four adjacent squares or cells as
indicated in Figure 5.3.2. Each cell has a horizontal and vertical component
239
Figure 5.3.2: Fluid Flow in Four Cells
that are proportional to the velocity components in the uid. Let cell l have
horizontal and vertical components given by xl and yl where l = 1> 2> 3 and 4.
For each cell the mass in minus the mass out yields four equations for the eight
unknown components. For the four cells depicted in Figure 5.3.2 with incoming
mass for cells 1-3, we have the following four equations
Cell
Cell
Cell
Cell
1:
2:
3:
4:
v1 x1 + y2 y1 = 0>
v2 x2 + v2 y2 = 0>
x2 x3 + v3 y3 = 0 and
x1 x4 + y3 y4 = 0=
This can be written as Hx = g where H is an 48 matrix and x has components

that are proportional to the velocity components
5
5
1
0 0
9 0
1 0
9
7 0 1 1
1 0 0
0
0
0
1
1 1 0
0 1
0
0 0
1
0 0 1
9
69
0 9
9
9
0 :
:9
0 89
9
1 9
9
7
x1
x2
x3
x4
y1
y2
y3
y4
6
:
: 5
:
v1
:
: 9 v2 + v2
:=9
: 7
v3
:
:
0
:
8
6
:
:=
8
The augmented matrix [H g] can be transformed into a reduced row echelon

form by adding row 1 to row 4 and then row 2 to row 3. The vertical components
will be the free variables, and the resulting representation of the unknowns can
be used in the momentum equations for the uid ow. This does indicate an
important role for nullspaces in computational uid ow modeling.
240
5.3.6
Solving Equilibrium Equations Using MATLAB
The MATLAB codes E_eq_bge.m and E_eq_null.m implement the block Gauss
elimination and nullspace methods for the two-loop circuit problem. They can
easily be modied to do more complicated problems. The following solves the
three-loop circuit problem with the same parameters as in Subsection 3.6.3,
Example 3.6.4.
AA A =
1. 0
0 .1
0 0
0 0
0 0
0
0
3.
0
0
0
0
0
2.
0
0
0
0
0
4.
AA E =
-1 1 -1 0 0
0 -1 0 -1 1
AA f =
10
-20
0
0
-30
AA g =
AA
AA
AA
AA
AA
AA
0
0
N = null(E,r)
N=
-1 -1 1
0 -1 1
1 0 0
0 1 0
0 0 1
xp = E\g;
z = (N*A*N)\(N*(f - A*xp));
x = xp + N*z;
y = (E*E)\(E*(f - A*x));
sol_null = [x;y]
sol_null =
-5.2290
-10.3053
-5.0763
1.8702
-8.4351
-15.2290
3.7405
5.3.7
241
Exercises
1.
Find the solution of the following equilibrium equation using the block
Gauss elimination method
65
6
5
6 5
{1
1 0 0
2
1
9
:
9 0 2 0
: 9
1 :
: 9 {2 : 9 6 :
9
7 0 0 3 1 8 7 {3 8 = 7 12 8 =
2 1 1 0
0
|1
2.
Find the solution of the following
Gauss elimination method
5
3 1 0 1 0
9 1 3 1 1 2
9
9 0 1 3 0 1
9
7 1
1
0 0 0
0
2
1 0 0
equilibrium equation using the block

65
:9
:9
:9
:9
87
{1
{2
{3
|1
|2
: 9
: 9
:=9
: 9
8 7
1
2
3
4
5
6
:
:
:=
:
8
3.
Find the solution of the equilibrium equation in exercise 1 using the
nullspace method.
4.
Find the solution of the equilibrium equation in exercise 2 using the
nullspace method.
5.
Use MATLAB to conrm any calculations in exercises 1-4.
6.
Modify the six-bar truss problem to have external forces at both nodes
one and three. Find the new equation [H g] and its reduced row echelon form.
7.
In the uid ow problem consider six cells by attaching two cells to the
right of cells three and four. Find the new equation [H g] and its reduced row
echelon form.
Chapter 6
Linear Initial Value

Problems
First and second order linear initial value problems are studied. Initial value
problems require nding a function of time that satises an equation with derivatives and an initial condition. Examples include the motion of projectiles in
Subsection 2.3.3, mixing tanks in Subsection 3.7.4 and population models in
Subsection 4.4.3. Additional applications include time dependent models of
tuned circuits and resonant mass-spring systems. Particular solutions are found
by both undetermined coe!cients and variation of parameters. The system formulations are developed for the second order problems, and numerical solutions
are given by the MATLAB dierential equation solvers. Chapters six and seven
are an introduction to dierential equations, and this topic is covered in more
detail in [1] and [7]
6.1
First Order Linear
A rst order linear initial value problem for an unknown function { = {(w) with
given i (w) is
g{
= {0 = d{ + i (w) and {(0) = {0 =
gw
This can be written in dierential operator form where O({) {0 d{ and
O({) = i (w)= The operator O is called linear because
O({ + |) = ({ + |)0 d({ + |)
= {0 d{ + | 0 d|
= O({) + O(|)
243
244
CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
and for all real numbers f

O(f{) = (f{)0 d(f{)
= f({0 d{)
= fO({)=
The objective is to nd {(w) given d, an initial condition {(0) and the function
i (w).
When i (w) is a constant, the solution is easy to nd by dening } d{ + i
and noting } 0 = d{0 + 0 and so that {0 = d{ + i becomes (}@d)0 = }= In this
case }(w) = }(0)hdw and, hence,
d{(w) + i = (d{(0) + i )hdw
{(w) = ({(0) + i @d)hdw i @d=
If i (w) is not a constant, then the solution process is more complicated.
6.1.1
Motivating Applications
A simple model for the vertical velocity y of a projectile is derived from Newtons
law of motion
py 0 = fy pj
where p is the mass, j is the gravitational constant and fy is the air resistance.
In this case { = y, d = f@p, i (w) = j and {(0) = y(0) is the initial velocity.
The velocity as a function of these parameters is
y(w) = (y(0) + pj@f)h(f@p)w pj@f=
Another application is to circuits. Suppose the circuit has two components
in series and an imposed voltage Y (w) = 110 sin(120w)= Let one component be
a resistor U with voltage drop LU and the other component be a capacitor F
with voltage drop T@F where T is the charge and T0 = L. Kirchhos voltage
law gives
T0 U + (1@F)T = Y (w)
where { = T, d = 1@(UF), i (w) = Y (w)@U and {(0) = T(0) is the initial
charge. Since i (w) is now a function of time, the solution is a little more di!cult
to nd, see Example 6.1.4.
A third application is modeling the amount D(w) of a chemical in a single
mixing tank with xed volume Y with incoming concentration F(w) with ow
rate U and outgoing concentration D(w)@Y and ow rate U. The rate of change
of D(w) is the rate incoming minus the rate outgoing
D0 = UF(w) U(D@Y )=
Here { = D, d = D@Y , i (w) = UF(w) and {(0) = D(0) is the initial amount.
If the incoming concentration F(w) varies with time, then the solution step will
be more challenging.
6.1. FIRST ORDER LINEAR
245
Example 6.1.1. Consider the single mixing tank model with U = 6, F = =1,
Y = 1000 and zero initial amount. The initial value problem is
D0 = =6 =006D and D(0) = 0=
We wish to verify D(w) = 100h=006w + 100 is the solution. The left and right
sides of the dierential equation must be equal and the solution evaluated at
time w = 0 must also be zero:
left side = D0 = (100h=006w +100)0 = 100(h=006w )0 = 100h=006w (=006)>
right side = =6 =006(100h=006w + 100) = =006(100h=006w ) and
D(0) = 100h=006(0) + 100 = 100(1) + 100 = 0=
It is always a good idea to check or verify a proposed solution is truly a solution.
6.1.2
Homogeneous Solutions O({k ) = 0
The solution of {0 = d{ + 0 is called the homogeneous solution of {0 = d{ + i (w)=

From the properties of the exponential function, the homogeneous solution must
have the form
{k (w) = Fhdw =
Any particular solution of {0 = d{ + i (w) is denoted by {s = {s (w) so that
O({s ) = i and O({k ) = 0= Since O is a linear operator,
O({k + {s ) = O(Fhdw + {s )
= FO(hdw ) + O({s )
= 0 + i=
Thus, the general solution of {0 = d{ + i (w) has the form
{(w) = Fhdw + {s (w) =
The constant F in the homogeneous solution is determined from the initial condition and the particular solution. In Example 6.1.1 the homogeneous solution
is 100h=006w and the particular solution is 100= If i (w) is not a constant, then
the particular solution will not be a constant and will be harder to nd.
6.1.3
Solutions of O({s ) = i by Undetermined Coe!cients
The method of undetermined coe!cients works for some classes of i (w)> and it
is an "educated" guess for the form of the solution. The simplest class is if i (w)
is a polynomial of w, and in this case one speculates that the particular solution
may also be a polynomial. The coe!cients in the polynomial particular solution
must be chosen so that the dierential equation holds for all values of w. Other
classes of i (w) are sine, cosine and exponential functions, and the particular
solution for these are illustrated in the following examples.
Example 6.1.2. Consider the initial value problem
{0 = 3{ + 2w + 1 and {(0) = 10=
246
Here i (w) = 2w + 1 and d = 3. Then the homogeneous and particular solutions

have the form
{k = Fh3w and {s = f1 w + f0 =
First, nd the coe!cients in the particular solution by requiring {s to satisfy
the dierential equation
(f1 w + f0 )0 = 3 (f1 w + f0 ) + 2w + 1
(0)w + (f1 1 + 0)1 = (3f1 + 2)w + (3f0 + 1)1=
Since this must hold for all time w, the coe!cients of w and 1 must be equal
0 = 3f1 + 2 and
f1 = 3f0 + 1=
The solution is f1 = 2@3 and f0 = 5@9 giving {s = (2@3)w 5@9= The
general solution is
{ = {k + {s = Fh3w (2@3)w 5@9=
Second, nd the constant F in the homogeneous solution by requiring
3(0)
Fh
{(0) = 10
(2@3)0 5@9 = 10
F 5@9 = 10=
Then F = 95@9 and { = (95@9)h3w (2@3)w 5@9= If i (w) were a quadratic

polynomial, then the particular solution must have the form
{s = f2 w2 + f1 w + f0 =
Example 6.1.3. Consider the initial value problem
{0 = d{ + 4h2w and {(0) = 10=
Since i (w) is an exponential function, it is reasonable to speculate that the
particular solution should have the form
{s = fh2w =
Choose the coe!cient so that the dierential equation holds
(fh2w )0 = d(fh2w ) + 4h2w
fh2w 2 = d(fh2w ) + 4h2w =
Since this must hold for all time w> f2 = df + 4= Provided d 6= 2, one can solve
for f to get f = 4@(2 d)= The general solution is
{ = {k + {s = Fhdw + (4@(2 d))h2w =
247
The initial condition gives
Fh
d(0)
{(0) = 10
+ (4@(2 d))h2(0) = 10
F + 4@(2 d) = 10=
If d = 3, then the solution of the initial value problem is { = 14h3w + (4)h2w = If

d = 2, then one must make a more elaborate guess for the particular solution
such as {s = fwh2w =
Example 6.1.4. If i (w) is a sine or cosine function, then the particular solution
might be a linear combination of the sine and cosine functions. Consider the
initial value problem
{0 = d{ + 10 sin(w) and {(0) = 20=
Assume the particular solution has the form
{s = f sin(w) + g cos(w)=
Choose the coe!cients so that the dierential equation holds
(f sin(w) + g cos(w))0 = d (f sin(w) + g cos(w)) + 10 sin(w)
f cos(w) g sin(w) = (df + 10) sin(w) + dg cos(w)=
Match the coe!cients of cos(w) and sin(w) to obtain
f = dg and
g = df + 10=
Solve these to get g = 10@(d2 + 1) and f = 10d@(d2 + 1)= If d = 2, then the
general solution is
{ = Fh2w 4 sin(w) 2 cos(w)
and the initial condition gives
{(0) = 20
Fh2(0) 4 sin(0) 2 cos(0) = 20
F 0 2 = 20=
The solution of the initial value problem is { = 22h2w 4 sin(w) 2 cos(w)=
6.1.4
Solutions of O({s ) = i by Variation of Parameters
The particular solution will be assumed to have of the form

{s (w) = hdw y(w)
248
where the function y(w) (the variation of parameter) will be chosen so that the
dierential equation holds. Use the product rule to compute the derivative
dw
0
h y(w)
= d hdw y(w) + i (w)
dw 0
h
y(w) + hdw (y(w))0 = d hdw y(w) + i (w)
hdw dy(w) + hdw y0 (w) = d hdw y(w) + i (w)=

Solve for the derivative of y(w)
y 0 (w) = hdw i (w)=
Integrate this to solve for y(w) and the particular solution
Z w
hd i ( )g =
{s (w) = hdw y(w) = hdw
This gives a compact formula for the particular solution. The integral exists if
the i (w) is continuous. In many cases the integral may exist, but it may not be
easy to compute.
Example 6.1.5. Reconsider Example 6.1.2 where d = 3 and i (w) = 2w + 1 and
use integration by parts to compute the integral
Z w
{s (w) = hdw
hd i ( )g
Z w
= hdw
hd (2 + 1)g
Z w
Z w
= hdw
hd 2 g + hdw
hd g
= hdw (2@d2 (dw 1))hdw + hdw (1@d)hdw
= (2@d)w (2 + d)@d2 =
This agrees with the method of undetermined coe!cients that was used in
Example 6.1.2. Note, the lower limit in the integral can be any constant, because
the dierence in the integral is part of the homogeneous solution.
The results of the above discussion are summarized in the following theorem.
Theorem 6.1.1 (First Order Linear Initial Value Problem) Consider the initial value problem O({) = {0 d{ = i (w) and {(0) = {0 . If i (w) is continuous
on the interval [0 W ], then for w in this interval the solution of the dierential
equation is
Z w
dw
dw
{(w) = Fh + h
hd i ( )g
Z w
dw
= h (F +
hd i ( )g )=
249
The solution of the initial value problem can be found by requiring F to be

chosen so that {(0) = {0 = Moreover, this is the only solution to the initial value
problem.
6.1.5
Solutions Using int() and dsolve()
MATLAB can do symbolic operations, which is in contrast to numerical computation. The MATLAB command int() does symbolic integration such as in
Example 6.1.5 where integration by parts was required.
AA syms t a
AA sol = int(exp(-a*t)*(2*t+1),t)
AA
sol = -1/a*(-2/a*(-exp(-a*t)*a*t-exp(-a*t))+exp(-a*t))
AA simple(sol)
AA
-exp(-a*t)*(2*a*t + 2 + a)/a^2
The MATLAB command dsolve() nds the symbolic solution of many initial
value problems such as in Example 6.1.5.
AA syms t a
AA solde = dsolve(Dx = a*x + 2*t + 1, x(0) = 10)
AA
solde = -2/a*t - 2/a^2 - 1/a + (2 + a + 10*a^2)/a^2*exp(a*t)
Both the above symbolic calculations agree with the by-hand calculations done
in Example 6.1.5.
6.1.6
Exercises
1.
As in Example 6.1.1 verify { = (95@9)h3w (2@3)w 5@9 is a solution to
Example 6.1.2.
2.
As in Example 6.1.1 verify { = 14h3w + (4)h2w is a solution to Example
6.1.3.
3.
Consider the initial value problem {0 = 5{ + 3w + 2 and {(0) = 5=
(a).
Find the homogeneous solution.
(b).
Use undetermined coe!cients to nd the particular solution.
(c).
Find the solution to the initial value problem.
4.
Consider the initial value problem {0 = 3{ + 3w2 + 2w + 1 and {(0) = 10=
(a).
(b).
Use undetermined coe!cients to nd the particular solution
(see the remark at the end of Example 6.1.2).
(c).
5.
Consider the initial value problem {0 = 2{ + 10h3w and {(0) = 5=
(a).
(b).
(c).
6.
Consider the initial value problem {0 = 2{ + 4h2w and {(0) = 12=
(a).
250
(b).
Use undetermined coe!cients to nd the particular solution

(see the remark at the end of Example 6.1.3).
(c).
7.
Consider the initial value problem {0 = 5{ + 10 cos(w) and {(0) = 1=
(a).
(b).
(c).
8.
Consider the initial value problem {0 = 5{ + 10 cos(2w) + 5 sin(2w) and
{(0) = 1=
(a).
(b).
(c).
9.
Use variation of parameters as in Theorem 6.1.1 to nd the solution to
the initial value problem in exercise 3.
10.
Use variation of parameters as in Theorem 6.1.1 to nd the solution to
the initial value problem in exercise 5.
11.
Use the MATLAB commands int() and dsolve() to conrm any by-hand
calculations in exercises 3-10.
12.
Let {(w) and |(w) satisfy the same initial value problem
{0
|0
= d{ + i (w) and {(0) = {0

= d| + i (w) and |(0) = {0 =
(a). Show } { | must satisfy } 0 = d} and }(0) = 0=

(b). Show there is only one solution to the initial value problem, that is,
{ = |.
6.2
Second Order Linear
Second order initial value problems require one to nd a function of time {(w)
that satises a dierential equation with two time derivatives and two initial
conditions. We will focus on second order linear initial value problems with
constant coe!cients
d{00 + e{0 + f{ = i (w) with {(0) = {0 and {0 (0) = y0 =
The motion of a projectile as illustrated in Subsection 2.3.3 is an example where
{(w) is the position, {0 (w) is the velocity in one direction, {00 (w) is the acceleration
and the dierential equation is derived from Newtons law of motion. Two
important applications are the motion of mass attached to a spring and the
electrical current in a series LRC circuit. These are precursors to models of
cars and trucks with several springs and shock absorbers and to models of
tuning circuits for a radio or television.
6.2. SECOND ORDER LINEAR
251
Figure 6.2.1: Mass-Spring System
6.2.1
Motivating Application to Mass-Spring
Consider a mass, which is attached to a spring as indicated in Figure 6.2.1. Let

{ = {(w) be the displacement from equilibrium,which is the position with no
motion and no external force. The mass moves in the horizontal direction and
is subject to three forces: n{ from the spring for "small values" of {(w), e{0
due the friction force given by gravity and the solid surface, and an external
force i . By Newtons law of motion the mass times acceleration is the sum of
the three forces
p{00 = i n{ e{0 =
If the initial position {(0) and initial velocity {0 (0) are given, then the objective
is to nd {(w) for all time w A 0= A similar system is a model for a car or truck,
but in this case the mass is the vehicle, the system is vertical and the friction
force is replaced by a force from a shock absorber. If either the shock absorber
is too small or too woren, then the coe!cient e will be relatively small.
Example 6.2.1. Assume there are no friction and external forces so that the
model is
p{00 = 0 n{ 0{0 and {(0) = {0 and {0 (0) = y0 =
If the initial position or the initial velocity is not zero, then the mass will have
a periodic motion. In fact, one can easily verify from the rst and second
derivatives and the chain rule that two possible solutions are
p
p
{1 (w) sin( n@pw) and {2 (w) cos( n@pw)=
The rst and second derivatives of {1 (w) are
{01 (w) =
{001 (w) =
p
p
p
p
g
gp
n@pw = cos( n@pw) n@p
sin( n@pw) = cos( n@pw)
gw
gw
p
p
p
p
p
g
cos( n@pw) n@p = sin( n@pw) n@p n@p=
gw
This means p{001 = n{1 = The verication for {2 (w) is similar. Because the
derivative operation is linear, the most general solution of p{00 + n{ = 0 is a
252
linear combination of {1 (w) and {2 (w)

p
p
{(w) = f1 sin( n@pw) + f2 cos( n@pw)=
The initial conditions can be used to determine
pthe values of the coe!cients.
As an illustration suppose p = 1, n = 4 so that n@p = 2, and with {(0) = 2
and {0 (0) = 1= The initial conditions mean at time equal to zero the mass is two
units the right of the equilibrium position and is set in motion with a velocity
of unit one moving to the right. In order to solve for the coe!cients, we must
use the two initial conditions
{(0) =
=
0
{ (0) =
=
f1 sin(2(0)) + f2 cos(2(0)) = 2
f1 0 + f2 1 = 2 and
f1 cos(2(0)) 2 f2 sin(2()) 2 = 1
f1 2 + f2 0 = 1=
Note, {0 (0) means rst compute the derivative and then evaluate the derivative
at time equal to zero. This algebraic system in matrix form is

0 1
f1
2
=
2 0
1
f2
and is easy to solve. So, the solution of the second order initial value problem
is
{(w) = (1@2) sin(2w) + 2 cos(2w)=
6.2.2
Solution of the Homogeneous System: i (w) = 0
In Example 6.1.1 the solution of p{00 + n{ = 0 was simply stated, and we

veried that it was a linear combination of sine and cosine functions. In the
remainder of this and the next sections we will present a method for discovering
the form of the solutions to the homogeneous second order dierential equation
d{00 + e{0 + f{ = i (w) = 0=
When the coe!cients are constants, we assume the solution has the form of an
exponential function
{ = huw where u is possibly a complex number.
In order to nd u> we put this exponential function into the dierential equation
0
00
d huw + e huw + fhuw = 0
dhuw u2 + ehuw u + fhuw
(du2 + eu + f)huw
= 0
= 0=
253
Whether or not u is real or complex, the exponential function is

huw = hRe(u)w+l Im(u)w = hRe(u)w hl Im(u)w
is dened and not zero, see Section 1.2. Thus, the quadratic factor must be
zero, which is often called the auxiliary or characteristic equation,
du2 + eu + f = 0=
By the quadratic formula there are three cases to consider: two distinct real
roots, repeated real roots and complex roots. As we shall see, in all three cases
this leads to two homogeneous solutions {1 (w) and {2 (w) that are not multiples
of each other.
Denition 6.2.1. The dierential operator O({) is linear if and only if
O({ + |) = O({) + O(|) and O(b
f{) = b
fO({)=
The second order dierential operator O({) d{00 +e{0 +f{ is linear because
O({ + |) = d({ + |)00 + e({ + |)0 + f({ + |)
= d({00 + | 00 ) + e({0 + | 0 ) + f({ + |)
= O({) + O(|)
and for all real numbers b
f
O(b
f{) = d(b
f{)00 + e(b
f{)0 + f(b
f{)
00
0
= db
f{ + eb
f{ + fb
f{
= b
fO({)=
Because of the linear property, any linear combination of the two homogeneous
solutions will also be a homogeneous solution
O(f1 {1 + f2 {2 ) = O(f1 {1 ) + O(f2 {2 )
= f1 O({1 ) + f2 O({2 )
= f1 0 + f2 0 = 0=
Example 6.2.1 illustrates this where the characteristic equation is 1u2 +0u+4 = 0
and has conjugate imaginary roots u = l2> which leads to the sine and cosine
functions as outlined in the Section 6.3. The next two subsections discuss the
distinct real and repeated real root cases.
6.2.3
Homogeneous Solution: Distinct Real Roots
Consider the distinct real solutions of du2 + eu + f = 0 where e2 4df A 0 and

p
u1 = (e + e2 4df)@2d and
p
u2 = (e e2 4df)@2d=
254
The two homogeneous solutions are

{1 (w) = hu1 w and {2 (w) = hu2 w =
Because u1 6= u2 , they are not multiples of each other, and {(w) = f1 hu1 w + f1 hu2 w
is the general homogeneous solution.
Example 6.2.2. The second order dierential equation {00 + 5{0 6{ = 0 has
the characteristic equation
u2 + 5u 6 = 0
whose solutions are
u = (5
p
s
52 4(1)(6))@2 = (5 49)@2 = 1 or 6=
Therefore, the solution of the dierential equation is

{(w) = f1 hw + f2 h6w =
Suppose the initial conditions are {(0) = 1 and {0 (0) = 2 so that the coe!cient must satisfy
{(0) = f1 h0 + f2 h6(0) = f1 + f2 = 1 and
{0 (0) = f1 h0 + f2 h6(0) (6) = f1 6f1 = 2=
Note, this algebraic system may be written in matrix form

1 1
f1
1
=
>
1 6
2
f2
and the solution is f1 = 8@(7) and f2 = 1@(7)= The solution of the initial
value problem is
{(w) = (8@7)hw (1@7)h6w =
6.2.4
Homogeneous Solution: Repeated Real Roots
Consider the repeated real solutions of du2 + eu + f = 0 where e2 4df = 0

so that u1 = u2 = e@2d= Using the rst order dierential equation as a guide,
choose two possible homogeneous solutions to be
{1 (w) = h(e@2d)w and {2 (w) = wh(e@2d)w =
We need to verify {2 (w) = wh(e@2d)w is really a homogeneous solution of
d{00 + e{0 + f{ = 0
when e2 4df = 0= Insert {2 (w) = wh(e@2d)w into this dierential equation
00
O(wh(e@2d)w ) = d wh(e@2d)w + e(wh(e@2d)w )0 + f(wh(e@2d)w )
= d(wh(e@2d)w (e@2d)2 + 2h(e@2d)w (e@2d))
e(1h(e@2d)w + wh(e@2d)w (e@2d)) + f(wh(e@2d)w )
= h(e@2d)w [(d2(e@2d) + e) + w(d(e@2d)2 + e(e@2d) + f)]
= h(e@2d)w [0 + w(0)] = 0=
255
Example 6.2.3. The second order dierential equation {00 + 4{0 + 4{ = 0 has
the characteristic equation
u2 + 4u + 4 = 0
whose solution is
u = (4
p
s
42 4(1)(4))@2 = (4 0)@2 = 2 and 2=
The solution of the dierential equation is

{(w) = f1 h2w + f2 wh2w =
If the initial conditions are {(0) = 1 and {0 (0) = 3, then
{(0) = f1 h2(0) + f2 (0)h2(0) = f1 + f2 0 = 1 and
{0 (0) = f1 h2(0) (2) + f2 (1h2(0) + (0)h2(0) (2))
= f1 (2) + f2 = 3=
The matrix version of the algebraic system is

1 0
f1
1
=
2 1
3
f2
whose solution is f1 = 1 and f2 = 5= The solution of the initial value problem is
{(w) = h2w + 5wh2w =
6.2.5
Homogeneous Initial Value Problem
Let {1 (w) and {2 (w) be homogeneous solutions so that {(w) = f1 {1 (w) + f2 {2 (w)
is also a homogeneous solution. We want to be able to choose the coe!cients
such that the initial conditions {(0) = {0 and {0 (0) = y0 hold. This results in
the following algebraic problem
{(0) = f1 {1 (0) + f2 {2 (0) and
{0 (0) = f1 {01 (0) + f2 {02 (0)=
The matrix form is
{1 (0) {2 (0)
{01 (0) {02 (0)
The 2 2 matrix

Z (0)
f1
f2

=
{1 (0) {2 (0)
{01 (0) {02 (0)
{0
y0
is called the Wronskian matrix associated with the dierential equation. If

det(Z (0)) 6= 0, then the solution of algebraic equation exists and is unique, see
Theorem 3.7.3. For the distinct real case

1 1
det(Z (0)) = det(
) = u2 u1 6= 0=
u1 u2
256
For the repeated real case

det(Z (0)) = det(
1
0
e@2d 1
) = 1 6= 0=
Thus, one can solve the initial value problem for the both the distinct and
repeated real solutions of the characteristic equation. These results are summarized in the following theorem
Theorem 6.2.1 (Homogeneous Linear Initial Value Problem) Let {1 (w) and
{2 (w) be homogeneous solutions of the dierential equation O({) = d{00 + e{0 +
f{ = 0. Consider the initial value problem given by the dierential equation
and the initial conditions {(0) = {0 and {0 (0) = y0 = L is a linear operator
and, so, f1 {1 (w) + f2 {2 (w) is also a homogeneous solution. If det(Z (0)) =
{1 (0){02 (0) {2 (0){01 (0) 6= 0> then one can nd the coe!cients such that the
initial conditions hold, and the solution of the initial value problem is unique.
6.2.6
Solutions Using MATLAB Command dsolve()
The MATLAB command dsolve() can also be used to solve linear second order
initial value problems. The following illustrate how Examples 6.2.1-6.2.3 can
be done using dsolve(), and note the results agree with the above by-hand
calculations.
AA syms m k t
% example 1
AA dsolve(m*D2x + k*x = 0,t)
C1*sin((k/m)^1/2*t) + C2*cos((k/m)^1/2*t)
% example 1 with initial conditions
AA dsolve(D2x + 4*x = 0,x(0) = 2,Dx(0) = 1,t)
2*cos(2*t) + 1/2*sin(2*t)
AA dsolve(D2x + 5*Dx - 6*x = 0,x(0) = 1,Dx(0) = 2,t)
8/7*exp(t) - 1/7*exp(-6*t)
AA dsolve(D2x + 4*Dx +4*x = 0,x(0) = 1,Dx(0) = 3,t)
exp(-2*t) + 5*exp(-2*t)*t
6.2.7
1.
Exercises

(a).
Show {2 (w) is a homogeneous solution.
(b).
Verify {(w) = (1@2) sin(2w) + 2 cos(2w) is the solution of the initial
value problem.
2.
Verify {(w) = (8@7)hw (1@7)h6w is the solution of the initial value problem in Example 6.2.2.
6.3. HOMOGENEOUS AND COMPLEX SOLUTION
257
3.
Verify {(w) = h2w + 5wh2w is the solution of the initial value problem in
Example 6.2.3.
4.
Consider the initial value problem {00 3{0 + 2{ = 0 with {(0) = 1 and
0
{ (0) = 2=
(a).
Find the homogeneous solutions.
(b).
5.
Consider the initial value problem 2{00 + 2{0 12{ = 0 with {(0) = 1
and {0 (0) = 1=
(a).
(b).
6.
Consider the initial value problem {00 + 2{0 + { = 0 with {(0) = 2 and
{0 (0) = 1=
(a).
(b).
7.
Consider the initial value problem 3{00 24{0 + 48{ = 0 with {(0) = 2
and {0 (0) = 3=
(a).
(b).
8.
Use the MATLAB command dsolve() to conrm any by-hand calculations
in exercises 4-7.
9.
Let {(w) and |(w) be solutions of the initial value problem
d{00 + e{0 + f{ = 0 with {(0) = {0 and {0 (0) = y0
d| 00 + e| 0 + f| = 0 with |(0) = {0 and | 0 (0) = y0 =
(a).
Dene }(w) {(w) |(w) and show

d} 00 + e} 0 + f} = 0 with }(0) = 0 and } 0 (0) = 0=
(b).
Let Z (0) be the Wronskian associated with the dierential equation. Assume det(Z (0)) 6= 0 and show }(w) must be zero and, hence, the
solution of the initial value problem is unique.
6.3
Homogeneous and Complex Solution
As in the preceding section the homogeneous solution of d{00 + e{0 + f{ = 0

has the form huw = The u may be a complex number and must be chosen so
that the dierential equation holds. This leads to the characteristic equation
du2 + eu + f = 0> which can be solved by the quadratic formula. The distinct
and repeated real root cases were considered in the previous section. Here we
assume e2 4df ? 0 where the roots are complex numbers, and complex roots
will lead to homogeneous solutions that are products of exponential and sine
or cosine functions. An application to damped (non-zero friction or resistive
forces) mass-springs will illustrate the importance of this topic.
258
6.3.1
Homogeneous Solutions: Complex Roots
In this section we use Eulers formula for the complex exponential function, see
Section 1.2,
hl|
h}
= cos(|) + l sin(|) and

= h{+l| = h{ hl| = h{ (cos(|) + l sin(|))=
The sine and cosine functions can be expressed as complex linear combinations
of hl| and hl|
cos(|) =
sin(|) =
1 l| 1 l|
and
h + h
2
2
1 l|
1
h hl| =
2l
2l
We will use these when { is replaced by the real part uw, and | is replaced by
the imaginary part of uw.
When e2 4df ? 0> the solutions of the characteristic equation can be
expressed as
u
= l where
e@2d and
p
e2 + 4df@2d=
Eulers formula then gives two possible homogeneous solutions

{
b1 (w) = huw = h(+l)w = hw (cos(w) + l sin(w)) and
{
b2 (w) = huw = h(l)w = hw (cos(w) l sin(w))=
These are complex valued functions, but appropriate linear combinations generate two real valued homogeneous solutions
{1 (w) = (1@2)b
{1 (w) + (1@2)b
{2 (w)) = hw cos(w) and
{2 (w) = (1@(2l))b
{1 (w) (1@(2l))b
{2 (w)) = hw sin(w).
One can easily show the Wronskian at time equal to zero has non-zero determinant

{1 (0) {2 (0)
1 0
det(Z (0)) = det(
)
=
det(
) = 6= 0>
{01 (0) {02 (0)

and, therefore, one can nd f1 and f2 such that f1 {1 (w) + f2 {2 (w) will satisfy
given initial values {(0) and {0 (0).
Example 6.3.1. Again consider the undamped mass-spring system p{00 +n{ =
2
0 where ds= p> e = 0 and
p f = n so that e 4df = 0 4pn ? 0= Then = 0
and = 4pn@2p = n@p and
p
p
{1 (w) = h0w cos( n@pw) = cos( n@pw) and
p
p
{2 (w) = h0w sin( n@pw) = sin( n@pw).
259
Suppose the mass is initially at rest and is one unit to the left of the equilibrium
position, that is, {(0) = 1 and {0 (0) = 0= Find f1 and f2 so that f1 {1 (w) +
f2 {2 (w) satises these initial conditions
p
p
f1 cos( n@p0) + f2 sin( n@p0) = f1 1 + f2 0 = 1 and
p
p
p
p
p
f1 sin( n@p0)( n@p) + f2 cos( p@n0) n@p = f1 0 + f2 n@p = 0=
The matrix version of this algebraic system is

1 p0
1
f1
=
0
f2
0
n@p
whose solution is f1 = 1 and f2 = 0= The solution of the corresponding initial
value problem is
p
p
p
{(w) = (1) cos( n@pw) + (0) sin( n@pw) = cos( n@pw)=
Example 6.3.2. Consider the initial value problem {00 + 6{0 + 25{ = 0 with
{(0) = 2 and {0 (0) = 3= First, the characteristic equation is u2 + 6u + 25 = 0
and the solution is
p
6 62 4(1)25
u=
= 3 4l=
2(1)
Second, note this is the complex root case where = 3 and = 4 and, thus,
the homogeneous solutions are
{1 (w) = h3w cos(4w) and
{2 (w) = h3w sin(4w).
Third, nd f1 and f2 so that {(w) = f1 {1 (w) + f2 {2 (w) satises the given initial
conditions. Here we must compute the derivative of {(w) by using the product
rule
{0
= (f1 h3w cos(4w) + f2 h3w sin(4w))0

= f1 (h3w (3) cos(4w) + h3w sin(4w)(4)) +
f2 (h3w (3) sin(4w) + h3w cos(4w)(4))=
The following algebraic system must be solved

f1 h3(0) cos(4(0)) + f2 h3(0) sin(4(0)) =
f1 1 + f2 0 = 2 and
f1 (3 + 0) + f2 (0 + 4) = 3=

1 0
3 4
f1
f2

=
2
3
260
The algebraic solution is f1 = 2 and f2 = 9@4> and the solution to the initial
value problem is
{(w) = 2h3w cos(4w) + (9@4)h3w sin(4w)=
Note, the mass-spring oscillates, but amplitudes of the oscillations rapidly decrease to zero. This is a result of the friction forces, which remove energy from
the mass-spring system.
6.3.2
Application to the Damped Mass-Spring System
Consider the model given by p{00 + e{0 + n{ = 0 where the mass p and spring
constant n are xed and the friction (damping) constant e is positive with small
to large values. As e goes from small to large, the nature of the homogeneous
solution will change due to changing characteristic equation roots from complex,
repeated to distinct real roots. The three types of homogeneous solutions are
e2 4pn
e2 4pn
e2 4pn
? 0 (underdamped) :
p
p
h(e@2p)w cos( 4pn e2 w) and h(e@2p)w sin( 4pn e2 w)
= 0 (critically damped ) :
h(e@2p)w and wh(e@2p)w
A 0 (overdamped) :
s
e2 4pn@2p)w
h(e@2p
s
e2 4pn@2p)w
and h(e@2p+
Because p> e and n are all positive, the coe!cients of time w in the above
exponential function are all negative. This implies the homogeneous solution
will approach zero as time increases, which is not the case if e is zero.
Example 6.3.3. Let p = 1, n = 1 and vary e = 1> 2 and 4= Consider the initial
value problem
{00 + e{0 + { = 0 with {(0) = 2 and {0 (0) = 0=
The characteristic equation is u2 + eu + 1 = 0 whose solutions are
s
e e2 4
u=
=
2
s Let e = 1 so that the characteristic equation has complex roots u = 1@2
l 3@2 and the homogeneous solutions have the form
s
s
{(w) = f1 h(1@2)w cos(( 3@2)w) + f2 h(1@2)w sin(( 3@2)w)=
As in Example 6.3.2 the initial conditions are used to nd the coe!cients
f1 h(1@2)(0) cos(0) + f2 h(1@2)(0) sin(0) = f1 1 + f2 0 = 2 and
s
f1 (1@2 + 0) + f2 (0 + 3@2) = 0=


1
s0
1@2
3@2
f1
f2
2
0
261
s
The algebraic solution is f1 = 2 and f2 = 2@ 3> and the solution to the initial
value problem is
s
s
s
{(w) = 2h(1@2)w cos(( 3@2)w) + (2@ 3)h(1@2)w sin(( 3@2)w)=
Let e = 2 and in this case the characteristic equation has repeated real roots
u = 1 and 1 and the homogeneous solution is
{(w) = f1 hw + f2 whw =
Imposing the initial conditions gives
f1 1 + f2 0 = 2 and
f1 (1) + f2 (1 + 0) = 0
whose matrix form is

1 0
1 1
f1
f2

=
2
0
The algebraic solution is f1 = 2 and f2 = 2> and the solution to the initial value
problem is
{(w) = 2hw + 2whw =
s
Let e = 4> which illustrates the distinct real roots case with u = 2 3>
and the general homogeneous solution is
s
3)w
{(w) = f1 h(2+
s
3)w
+ f2 h(2
The initial conditions are used to nd the coe!cients as follows

f1 1 + f2 1 = 2 and
s
s
f1 (2 + 3) + f2 (2 3) = 0=
Or, in terms of matrices

1s
1s
2 + 3 2 3
f1
f2

=
2
0
s
s
s
By Cramers rule
is f1 s
= 2(2 3)@(2 3) = 2@ 3 + 1 and
s the solution
s
f2 = 2(2 + 3)@(2 3) = 2@ 3 + 1= The solution to the initial value
problem is
s
s
s
s
{(w) = (2@ 3 + 1)h(2+ 3)w + (2@ 3 + 1)h(2 3)w =
262
6.3.3
MATLAB Commands dsolve() and ezplot()
The MATLAB command dsolve() is used to nd the symbolic solutions to Example 6.3.3 for e = 0> 1> 2 and 4= Then the MATLAB command ezplot() is used
to generate Figure 6.3.1.
AA % b = 0
AA solb0 = dsolve(D2x + 0*Dx + x = 0,x(0) = 2,Dx(0) = 0, t)
solb0 = 2*cos(t)
AA % b =1
solb1 = 2*exp(-1/2*t)*cos(1/2*3^(1/2)*t)
+ 2/3*3^(1/2)*exp(-1/2*t)*sin(1/2*3^(1/2)*t)
AA % b = 2
solb2 = 2*exp(-*t) + 2*exp(-t)*t
AA % b = 4
solb4 = (2/3*3^(1/2) + 1)*exp((-2 + 3^(1/2))*t)
+ 1/3*(-2 + 3^(1/2))*3^(1/2)*exp(-(2 + 3^(1/2))*t)
AA % use ezplot to create the curves on the same graph
AA ezplot(solb4, [0 8])
AA hold on
6.3.4
Exercises
1.
Consider the homogeneous solutions {1 (w) = hw cos(w) and {2 (w) =
hw sin(w)=
(a).
Compute their derivatives.
(b).
Find the Wronskian matrix Z (0)=
(c).
Show det(Z (0))
p= =
2.
Verify {(w) = cos( n@pw) is the solution for Example 6.3.1.
3.
Consider the initial value problem 3{00 + 27{ = 0 with {(0) = 2 and
0
{ (0) = 1=
(a).
Find the homogeneous solutions of the dierential equation.
(b).
4.
Consider the initial value problem {00 + 2{0 + 2{ = 0 with {(0) = 1 and
{0 (0) = 1=
(a).
(b).
5.
Consider the initial value problem 2{00 + 4{0 + 6{ = 0 with {(0) = 1 and
{0 (0) = 2=
(a).
6.4. NONHOMOGENEOUS DIFFERENTIAL EQUATIONS
263
Figure 6.3.1: Variable Damped Mass-Spring
(b).
6.
Consider Example 6.3.3. Verify the proposed functions are really solutions for the three choices of e=
7.
Consider the initial value problem {00 + 4{0 + n{ = 0 with {(0) = 3 and
0
{ (0) = 0= Find the solutions for variable n = 1> 4 and 10=
8.
Consider the initial value problem p{00 + 4{0 + 1{ = 0 with {(0) = 3 and
0
{ (0) = 0= Find the solutions for variable p = 1> 4 and 10=
9.
Use MATLABs command dsolve() to conrm any calculations done in
exercises 3 5> 7 and 8=
6.4
Nonhomogeneous Dierential Equations
In this section we focus on nding particular solutions of second order linear

dierential equations O({) = d{00 + e{0 + f{ = i (w) 6= 0= As in the rst order
linear problems discussed in Section 6.1, this can be done by either the undetermined coe!cients or variation of parameters methods. The undetermined
coe!cients method works for i (w) of a certain type and will be illustrated by
the rst ve examples. The variation of parameters method works for any continuous function i (w) and will be presented in the Section 6.5. An important
application to resonance of a forced mass-spring systems will be given.
264
6.4.1
Particular Solutions by Undetermined Coe!cients
Any function {(w) = {s (w) that satises the second order dierential equation
O({) = d{00 + e{ + f{ = i (w)
is called a particular solution. Because O is a linear operator, see Denition
6.2.1, any homogeneous solution plus a particular solution will also satisfy
O({) = i (w)
O(f1 {1 + f2 {2 + {s ) = O(f1 {1 ) + O(f2 {2 ) + O({s )
= f1 O({1 ) + f2 O({2 ) + O({s )
= f1 0 + f2 0 + i (w) = i (w)=
The general solution is denoted by f1 {1 + f2 {2 + {s where {1 is not a multiple
of {2 =
Theorem 6.4.1 (Properties of L(x)) Let O({) d{00 + e{ + f{ where d> e and
f are constants. If O({1 ) = 0> O({2 ) = 0 and O({s ) = i (w)> then
1.
2.
3.
4.
O({) is linear operator,

O(f1 {1 + f2 {2 ) = 0,
O(f1 {1 + f2 {2 + {s ) = i (w) and
O({ + |) = i + j when O({) = i and O(|) = j=
In this section we will focus on using the undetermined coe!cient method

for nding particular solutions. The undetermined coe!cient method involves
making an educated guess about the form of a particular solution. This discussion will be limited to cases where the right side of the dierential equation
is a polynomial, an exponential, sine or cosine function. If i (w) is a linear or
quadratic polynomial, then one might expect a particular solution to be polynomial where the coe!cients must be chosen so that the dierential equation
holds for all time w= See Examples 6.4.1 and 6.4.2 for the polynomial case and
Examples 6.4.3 and 6.4.4 for exponential functions. In Example 6.4.5 and the
application to the forced mass-spring problem we will nd particular solutions
when the sine and cosine functions are in the right side of the dierential equation.
Example 6.4.1. Consider the dierential equation {00 +3{0 +2{ = 3w+1 where
the homogeneous solutions are hw and h2w = Since the right side is a rst order
polynomial, assume the particular solution is also linear polynomial
{s = d1 w + d0>
and choose d1 and d0 so that the dierential equation holds. This means
(d1 w + d1 )00 + 3(d1 w + d0 )0 + 2(d1 w + d0 ) = 3w + 1
0 + 3d1 + 2(d1 w + d0 ) = 3w + 1
(2d1 )w + (3d1 + 2d0 )1 = 3w + 1=
265
So, match the coe!cients of w and 1 to get

2d1 = 3 and 3d1 + 2d0 = 1=
The matrix form is
0 2
2 3
d0
d1

=
3
1
whose solution is d1 = 3@2 and d0 = 7@4= The particular solution of the

dierential equation is {s = (3@2)w + (7@4), and the general solution is the
homogeneous plus a particular solution
{ = {k + {s = f1 hw + f2 h2w + (3@2)w + (7@4)=
Example 6.4.2. Consider the dierential equation {00 + 2{0 + 2{ = 2w2 + 3w +
2 where the homogeneous solutions are hw cos(w) and hw sin(w)= Assume the
particular solution is also a quadratic polynomial
{s = d2 w2 + d1 w + d0 =
Choose d2 > d1 and d0 so that the dierential equation holds
00
0
2
d2 w + d1 w + d0 + 2 d2 w2 + d1 w + d0 + 2 d2 w2 + d1 w + d0 = 2w2 + 3w + 2
2d2 + 4d2 w + 2d1 + 2 d2 w2 + d1 w + d0 = 2w2 + 3w + 2

(2d2 )w2 + (4d2 + 2d1 )w + (2d2 + 2d1 + 2d0 )1 = 2w2 + 3w + 2=
Match the coe!cients of w2 , w and 1 to get three algebraic equations, which may
be written in matrix form as
6 5 6
5
65
2
2 2 2
d0
7 0 2 4 8 7 d1 8 = 7 3 8 =
2
d2
0 0 2
The algebraic solution is easily found and gives a particular solution {s =
w2 + (1@2)w + (1@2)= The general solution of the dierential equations is
{ = f1 hw cos(w) + f2 hw sin(w) + w2 + (1@2)w + (1@2)=
Example 6.4.3. Consider the same dierential operator as in Example 6.4.1,
but suppose it has a dierent right side {00 + 3{0 + 2{ = 10h3w . Assume a
particular solution has the form
{s = dh3w >
and choose the coe!cient so that the dierential equation holds
(dh3w )00 + 3(dh3w )0 + 2(dh3w ) = 10h3w
dh3w 9 + 3dh3w 3 + 2(dh3w ) = 10h3w
(20d)h3w = 10h3w =
266
Match the coe!cient of the exponential function to nd d = 10@20 = 1@2= The

particular solution of the dierential equation is {s = (1@2)h3w , and the general
solution is the homogeneous plus a particular solution
{ = {k + {s = f1 hw + f2 h2w + (1@2)h3w =
At this point observe Examples 6.4.1 and 6.4.3 can be combined to generate
more particular solutions by making use of the linear property of the dierential
operator O({) = d{00 + e{0 + f{= If O({) = i and O(|) = j, then O({ + |) =
O({) + O(|) = i + j so that a particular solution of O(}) = i + j is } = { + |=
In Example 6.4.1 i = 3w + 1 with { = (3@2)w + (7@4) and in Example 6.4.3
j = 10h3w with | = (1@2)h3w = Therefore, a particular solution of
{00 + 3{0 + 2{ = 3w + 1 + 10h3w
is { + | = (3@2)w + (7@4) + (1@2)h3w =
Example 6.4.4. Consider the same dierential operator as in Example 6.4.1,
but suppose the right side is one of the homogeneous solutions {00 + 3{0 + 2{ =
5hw . In this case the particular solution can not be a multiple of hw , and
another guess is to assume a particular has the form
{s = dwhw =
Find the coe!cient by requiring this to satisfy the dierential equation
w
(d(2h
(dwhw )00 + 3(dwhw )0 + 2(dwhw ) = 5hw

+ wh )) + 3(d(1hw + whw (1))) + 2(dwhw ) = 5hw
dhw + (0)whw = 5hw =
w
Thus, d = 5, a particular solution is {s = 5whw and the general solution is

{ = {k + {s = f1 hw + f2 h2w + 5whw =
Example 6.4.5. Consider sine and cosine functions in the right side {00 +
3{0 + 2{ = 2 cos(w) + 3 sin(w) = i (w)= Assume the particular solution is a linear
combination of cos(w) and sin(w)
{s = d cos(w) + e sin(w)=
Put this into the dierential equation and match the coe!cients of cos(w) and
sin(w)
0
(d cos(w) + e sin(w))00 + 3 (d cos(w) + e sin(w)) + 2 (d cos(w) + e sin(w)) = i (w)

d cos(w) e sin(w) 3d sin(w) + 3e cos(w) + 2 (d cos(w) + e sin(w)) = i (w)
(d + 3e + 2d) cos(w) + (e 3d + 2e) sin(w) = i (w)=
267
This gives two algebraic equations whose matrix form is

1 3
d
2
=
=
3 1
e
3
Cramers rule can be used to solve for d = 7@10 and e = 9@10= A particular
solution is {s = (7@10) cos(w) + (9@10) sin(w)> and the general solution is
{ = {k + {s = f1 hw + f2 h2w + (7@10) cos(w) + (9@10) sin(w)=
6.4.2
Nonhomogeneous Initial Value Problems
Let O({) = d{00 + e{0 + f{ where d, e and f are constants. Homogeneous

solutions of O({k ) = 0, {1 (w) and {2 (w)> were derived in the previous two
sections, and some particular solutions of O({s ) = i (w) were found via undetermined coe!cients for some classes of i (w)= Because O is a linear operator,
O({k + {s ) = O({k ) + O({s ) = 0 + i (w)= The general solution is the linear
combination of the two homogeneous solutions plus a particular solution
{(w) = f1 {1 (w) + f2 {2 (w) + {s (w)=
(6.4.1)
The solution of the initial value problem (IVP) O({) = i (w) with {(0) = {0 and
{0 (0) = y0 is found by choosing f1 and f2 in the general solution so that the
initial conditions are satised. This reduces to two algebraic equations
{(0) = f1 {1 (0) + f2 {2 (0) + {s (0) and
{0 (0) = f1 {01 (0) + f2 {02 (0) + {0s (0)=
The matrix form of this system is

{0 {s (0)
{1 (0) {2 (0)
f1
=
=
y0 {0s (0)
{01 (0) {02 (0)
f2
(6.4.2)
The 2 2 matrix is the Wronskian

{1 (0) {2 (0)
>
Z (0)
{01 (0) {02 (0)
and in order to solve for the coe!cients, it must have non-zero determinant.
Using Theorem 3.7.3 and the discussion in Section 5.1, this is equivalent to the
column vectors in Z (0) being linearly independent. These results are summarized in the following theorem.
Theorem 6.4.2 (Second Order Linear Initial Value Problem) Consider the
nonhomogeneous initial value problem O({) = d{00 + e{0 + f{ = i (w) with {(0) =
{0 and {0 (0) = y0 = Let {1 (w) and {2 (w) be homogeneous solutions and {s (w) be
a particular solution. If det(Z (0)) 6= 0> then the nonhomogeneous initial value
problem has a solution given by (6.4.1) and the algebraic solution of (6.4.2).
268
Outline
1.
2.
3.
4.

for Solving Nonhomogeneous Second Order Linear IVP.
Find two homogeneous solutions.
Find a particular solution.
Form the general solution {(w) = f1 {1 (w) + f2 {2 (w) + {s (w)=
Find the coe!cients by solving equation (6.4.2).
Example 6.4.2 with initial conditions {(0) = 2 and {0 (0) = 1= Steps 1-3
have been done and yield
{ = f1 hw cos(w) + f2 hw sin(w) + w2 + (1@2)w + (1@2)=
The particular solution is {s (w) = w2 + (1@2)w + (1@2) so that {s (0) = 1@2 and
{0s (0) = 1@2= The algebraic system in (6.4.2) is

1 0
1 1
f1
f2

=
2 1@2
1 (1@2)
whose solution is f1 = 3@2 and f2 = 3= Thus, the solution to the initial value
problem is
{ = (3@2)hw cos(w) + 3hw sin(w) + w2 + (1@2)w + (1@2)=
Example 6.4.4 with initial conditions {(0) = 1 and {0 (0) = 2= Steps 1-3
have been done and yield
{ = f1 hw + f2 h2w + 5whw =
The particular solution is {s (w) = 5whw giving {s (0) = 0 and {0s (0) = 5= The
algebraic system in (6.4.2) is

1
1
f1
10
=
=
1 2
25
f2
The solution is found by using Cramers rule to be f1 = 1@(1) = 1 and
f2 = 2@(1) = 2= Thus, the solution to the initial value problem is
{ = hw + 2h2w + 5whw =
6.4.3
Application to Forced Mass-Spring
The forced mass-spring model has an external force i (w) 6= 0 as well as forces
from the spring and from friction. The corresponding initial value problem is
p{00 + e{0 + n{ = i (w) with {(0) = {0 and {0 (0) = y0 = In order to keep the
discussion as clear as possible, let p = 1, e = 0, n = 1 and i (w) = sin(zw)
with {(0) = 0 and {0 (0) = p
0= The natural frequency, which
is given by the
p
homogeneous
solutions
cos(
n@pw)
=
cos(w)
and
sin(
n@pw)
= sin(w)> is
p
n@p@(2) = 1@(2)> and the frequency of the external force is z@(2)= The
269
objective of this subsection is to understand what happens to the motion of the

mass as z approaches 1=
Find the particular solution when z 6= 1= In this case i (w) = sin(zw) is not
a homogeneous solution, and we speculate the particular solution has the form
{s (w) = d cos(zw) + e sin(zw)=
Put this into the dierential equation {00 + { = sin(zw) and match coe!cients
of the sine and cosine function
(d cos(zw) + e sin(zw))00 + d cos(zw) + e sin(zw) = 0 cos(zw) + 1 sin(zw)
z d cos(zw) z2 e sin(zw) + d cos(zw) + e sin(zw) = 0 cos(zw) + 1 sin(zw)
(1 z2 )d cos(zw) + (1 z2 )e sin(zw) = 0 cos(zw) + 1 sin(zw)=
2
This implies (1 z2 )d = 0 and (1 z2 )e = 1 so that the particular solution is

{s (w) =
1
sin(zw)=
1 z2
Since the denominator goes to zero as z approaches one, this indicates some
special event!
Next impose the initial conditions on the general solution
{(w) = f1 cos(w) + f2 sin(w) +
1
sin(zw)=
1 z2
Use the zero initial position and velocity

1
sin(z0) = f1 = 0 and
1 z2
1
z
{0 (0) = f1 ( sin(0)) + f2 cos(0) +
cos(z0)z = f2 +
= 0=
2
1z
1 z2
{(0) = f1 cos(0) + f2 sin(0) +
Thus, the solution to the initial value problem is

{(w) =
z
1
sin(w) +
sin(zw)=
2
1z
1 z2
In order to investigate what happens as z approaches one, note the general

solution has an indeterminate form with respect to z
{(w) =
z sin(w) + sin(zw)
=
1 z2
Write this solution as a function of both w and z> and compute the limit with
respect to z using LHpitals rule
lim {(w> z) =
z$1
=
=
g
gz (z sin(w) + sin(zw))
g
2
z$1
gz (1 z )
lim
sin(w) + cos(zw) w
2z
1
w cos(w)
sin(w) +
=
2
2
lim
z$1
270
Since the amplitude of cos(w) is w@(2)> the spring eventually will be deformed
so much that the model is no longer valid. This is known as pure resonance,
and it can be controlled by inserting a suitable amount of resistive or damping
force, as is illustrated in the next subsection.
The solution of the above limiting case can also be derived directly by letting
z = 1 and nding the solution to {00 + { = sin(w)= Since i (w) = sin(w) is also a
particular solution, we speculate the particular solution has the form
{s (w) = w(d cos(w) + e sin(w))=
By placing this into the dierential equation and collecting like terms we have
(0)w sin(w) + (0)w cos(w)(e + e) 2e sin(w) 2d cos(w) = 1 sin(w) + 0 cos(w)=
This gives the same particular solution as well as the same solution to the initial
value problem.
6.4.4
Damped Forced Mass-Spring Using MATLAB
Consider the forced mass-spring when the resistive or damping force is not zero.
Use the MATLAB command dsolve() to nd the solutions for e = 0> 1> 2 and 4
of the nonhomogeneous initial value problem
{00 + e{0 + { = sin(w) with {(0) = 0 and {0 (0) = 0=
The MATLAB command ezplot is then used to graph the four curves, see Figure
6.4.1. Note, for large values of time and positive e the homogeneous solutions
have essentially decayed to zero so that the curves in this gure reect the
particular solutions (1@2)w cos(w) for e = 0, frv(w) for e = 1, (1@2)frv(w) for
e = 2 and (1@4)frv(w) for e = 4
AA % let b = 0
AA solb0 = dsolve(D2x+0*Dx+x = sin(t), x(0) = 0,
AA simple(solb0)
-1/2*cos(t)*t + 1/2*sin(t)
AA % let b = 1
AA simple(solb1)
exp(-1/2*t)*cos(1/2*3^(1/2)*t) - cos(t) +
1/3*exp(-1/2*t)*cos(1/2*3^(1/2)*t)*3^(1/2
AA % let b = 2
AA simple(solb2)
-1/2*cos(t) + 1/2*exp(-t) + 1/2*exp(-t)*t
AA % let b = 4
AA simple(solb4)
Dx(0) = 0,t)
Dx(0) = 0,t)
Dx(0) = 0,t)
Dx(0) = 0,t)
271
Figure 6.4.1: Forced Mass-Spring
AA
AA
AA
AA
AA
AA
6.4.5
-1/4*cos(t) + (1/8 + 1/12*3^(1/2))*exp((3^(1/2) - 2)*t) +

(1/24*(3^(1/2)-2)*3^(1/2))*exp(-(2+3^(1/2))*t)
use ezplot to create the curves on the same graph
ezplot(solb4, [0 20])
hold on
Exercises
1.
Find the homogeneous and particular solutions for {00 + 5{0 + 4{ = 3w + 4=
2.
Find the homogeneous and particular solutions for {00 + 2{0 + 2{ = w2 +
2w + 3=
3.
Find the homogeneous and particular solutions for {00 + 5{0 + 4{ = 3h2w =
4.
Find the homogeneous and particular solutions for {00 + 5{0 + 4{ = 2h4w =
5.
Find the homogeneous and particular solutions for {00 + 4{ = sin(w) +
3 cos(w)=
6.
Find the homogeneous and particular solutions for {00 + 4{ = sin(2w)=
7.
Solve the initial value problem for the dierential equation in Example
6.4.1 using the initial conditions {(0) = 1 and {0 (0) = 2=
8.
Solve the initial value problem for the dierential equation in Example
6.4.3 using the initial conditions {(0) = 0 and {0 (0) = 1=
272
9.
Find the solution to the initial value problem {00 +5{0 +4{ = 6h2w +3w +4
with {(0) = 0 and {0 (0) = 0=
10.
Find the solution to the initial value problem {00 + 4{ = 2h2w + w + 4
with {(0) = 0 and {0 (0) = 1=
11.
Consider the force mass-spring problem {00 + { = cos(zw) with {(0) = 0
0
and { (0) = 0=
(a).
Find the particular solution when z 6= 1=
(b).
Solve the initial value problem when z 6= 1=
(c).
Solve the initial value problem when z = 1=
12.
Conrm any by-hand calculations in exercises 1-11 using MATLABs
command dsolve().
13.
Use MATLABs commands dsolve() and ezplot() to solve and graph the
{00 + e{0 + { = cos(w) with {(0) = 0 and {0 (0) = 0 for e = 0> 1> 2 and 4=
6.5
System Form of Linear Second Order
This section begins with an application to LRC series circuits where a model for
the current is also a second order linear dierential equation. The right side can
be a very complicated function, and the method of undetermined coe!cients
may not yield a particular solution. Possible alternatives are to use the variation of parameters method to nd the particular solution or to use a suitable
numerical method. Both these alternatives require the second order linear differential equation to be written as a rst order system of dierential equations.
The numerical method that is implemented in the MATLAB command ode45()
is a variation of the Runge-Kutta scheme and uses variable time steps.
6.5.1
Application to a Series LRC Circuit
In the model for the forced mass-spring the damping or friction coe!cient e
of the velocity {0 was used to control the magnitude of the oscillations of the
mass. In a car or truck this is similar to having eective shock absorbers. The
model for the current in a series LRC circuit with an external source is also a
second order dierential equation with sine and cosine functions of time in the
right side. In a tuning circuit one wants to maximize the current by adjusting
the inductors or the capacitors. The series LRC circuit is a primitive tuning
circuit, and for a simple external source one can derive an exact formula for the
maximum current. In more complicated external sources, such as from an AM
(amplitude modulation) radio station, the initial value problem must be solved
numerically.
A series LRC circuit has three components, an inductor O, resistor U and
capacitor F, and an external source H(w). In an AM radio the external source
is from the antenna and is a function with terms equal to products of sine
and cosine functions. Figure 6.5.1 is a depiction of the series circuit and the
corresponding dierential equation model. The voltage drop across the resistor
is YU = UL, the voltage drop across the inductor is YO = OL 0 and the voltage
6.5. SYSTEM FORM OF LINEAR SECOND ORDER
273
Figure 6.5.1: Series LRC Circuit

drop across the capacitor is given by T = FYF where T is the charge and
T0 = L is the current. The model is derived from Kirchho voltage law, which
requires the sum of the voltage drops be equal to the imposed voltage
YO + YU + YF
OL 0 + UL + T@F
= H(w)
= H(w)=
Compute the derivative of both sides with respect to time to obtain a model
for the current
OL 00 + UL 0 + L@F = H 0 (w)=
The objective is to x the resistance U, and to vary the inductance O or the
capacitance F so that the current L will have a maximum amplitude. In tuning
a radio the amplitudes will vary with the frequency of the various stations. The
desired station will have a large amplitude relative to other stations.
Example 6.5.1. Let H(w) = H0 sin(w) where H0 is a constant (unmodulated
carrier wave) and = iv 2 with iv equals the frequency of the carrier wave.
The model for the current is
OL 00 + UL 0 + L@F = H 0 (w) = H0 cos(w)
where the coe!cients are positive constants. The homogeneous solutions L1 (w)
and L2 (w) are derived as in the previous sections from the characteristic algebraic equation. Because the O, U and F are positive, regardless of which of the
three cases (distinct, repeated or complex roots), the homogeneous solutions
will decay to zero and are called the transient solutions. The signal that remains signicant is the particular solution Ls (w)= The method of undetermined
274
coe!cients suggest that the particular solution should be a linear combination

of the cos(w) and sin(w)
Ls (w) = d cos(w) + e sin(w)=
By placing this into the dierential equation, computing the derivatives and
collecting like terms we have for D Od 2 + Ue + d@F and E Oe 2
Ud + e@F
D cos(w) + E sin(w) = H0 cos(w) + 0 sin(w)=
Match the coe!cients of cos(w) and sin(w) to obtain two algebraic equations
whose matrix form is

U
d
H0
O 2 + 1@F
=
=
U
O 2 + 1@F
e
0
Cramers rule easily gives the solution
O 2 + 1@F
and
+ 1@F)2 + (U)2
U
e = H0
=
2
(O + 1@F)2 + (U)2
d = H0
(O 2
Let be such that tan() = (O 2 + 1@F)@ (U) and write the particular
solution as
O 2 + 1@F
cos(w) +
(O 2 + 1@F)2 + (U)2
U
sin(w)]
(O 2 + 1@F)2 + (U)2
H0
p
[sin() cos(w) + cos() sin(w)]
2
(O + 1@F)2 + (U)2
H0
p
sin( + w)=
(O 2 + 1@F)2 + (U)2
Ls (w) = H0 [
=
=
The amplitude is the coe!cient of sin( + w) and will be a maximum if the
denominator is a minimum. Since the resistance is xed, the denominator will
be a minimum when O 2 + 1@F = 0= Thus, the current will have a maximum
amplitude when
O 2 + 1@F

= 0 or
1
=
= s
OF
If = iv 2 and iv is the
s desired frequency, then O and F should be adjusted
so that = iv 2 = 1@ OF= As will be illustrated by the MATLAB numerical calculations at the end of this section, these results also approximate the
maximum current for the modulated signal.
6.5.2
275
Systems and Variation of Parameters
If the right side of the second order dierential equation is too complicated, the
undetermined coe!cient method may not be able to nd a particular solution.
The variation of parameters method, which was used in Section 6.1 for rst
order dierential equations, can be generalized to nd a particular solution for
second order dierential equations. However, they will need to be reformulated
as a rst order system where there are two rst order dierential equations and
two unknowns for the function and its derivative.
Consider the linear second order dierential equation with constant coe!cients and d 6= 0
d{00 + e{0 + f{ = i (w)=
(6.5.1)
Let | be the derivative of { and, thus,
{0
|0
= | and
= {00 = (i (w) e{0 f{)@d
= i (w)@d e|@d f{@d=
This can be written in vector form as a rst order system of dierential equations

0 0

{
{
0
1
{
0

=
+
=
(6.5.2)
|
f@d e@d
|
i (w)@d
|0
Let {1 (w) and {2 (w) be homogeneous solutions (i (w) = 0) of the second order
problem in (6.5.1). In terms of vector equation in (6.5.2) this means

0
{1
0
1
{1
=
and
f@d e@d
{01
{01

0

0
1
{2
{2
=
=
f@d e@d
{02
{02
Use the vector notation

{1
{2
0
0
1

x1
,
x
,
f

and
D

=
2
i (w)@d
f@d e@d
{01
{02
The two vectors x1 and x2 satisfy
x01 = Dx1 and x02 = Dx2 =
(6.5.3)
The augmentation of the vectors x1 and x2 is the Wronskian at time w

{1 {2
Z (w) [x1 x2 ] =
=
{01 {02
The two vector equations in (6.5.3) may be written as a 2 2 matrix equation
Z (w)0 [x01 x02 ] = [Dx1 Dx2 ] = D[x1 x2 ] = DZ (w)=
(6.5.4)
276
The general homogeneous vector solution of (6.5.2) can be written as

{1 {2
f1
xk = x1 f1 + x2 f2 =
= Z (w)c=
{01 {02
f2
Following the rst order dierential equation variation of parameters method,
we speculate that a vector particular solution is also a linear combination of
the column vectors in Z (w), but now the coe!cients depend on time and the
vector particular solution has the form

{1 {2
y1 (w)
xs = x1 y1 (w) + x2 y2 (w) =
= Z (w)v(w)=
y2 (w)
{01 {02
Choose the components of the v(w) so that the nonhomogeneous system (6.5.2)
is satised
x0s
(Z (w)v(w))0

{1
{2
(
y1 (w) +
y2 (w))0
{01
{02
= Dxs + f
= D(Z (w)v(w)) + f
= (DZ (w))v(w) + f =
Now apply the product rule to each component of the two dimensional vectors
x01 y1 (w) + x1 y10 (w) + x02 y2 (w) + x2 y20 (w) = (DZ (w))v(w) + f
Z (w)0 v(w) + Z (w)v0 (w) = (DZ (w))v(w) + f =
Apply the matrix equation in (6.5.4) to get
(DZ (w))v(w) + Z (w)v0 (w) = (DZ (w))v(w) + f =
This means the derivatives of the variable coe!cients must be a solution of

Z (w)v0 (w) = f
0
y1 (w)
{2
0
=
=
y20 (w)
{02
i (w)@d
{1
{01
Cramers rule easily gives
0
{2
i (w)@d {02
(i (w)@d){2
=
and
det(Z (w))
det(Z (w))

0
{1
det(
{01 i (w)@d
(i (w)@d){1
=
=
det(Z (w))
det(Z (w))

det(
y10 (w) =
y20 (w) =
The vector particular solution is found by integrating both of these to give the
vector version of the variation of parameters particular solution formula
Z
Z
(i (w)@d){2
(i (w)@d){1
+ x2
=
(6.5.5)
xs = x1 y1 (w) + x2 y2 (w) = x1
det(Z (w))
det(Z (w))
277
The rst component of this vector equation is denoted by {s = {1 y1 (w)+{2 y2 (w)=

The integrations can be di!cult, but they could be approximated by numerical
integrations.
Example 6.5.2. Find the particular solution {00 4{ = w+1 using the variation
of parameters formula in (6.5.5). The two homogeneous solutions have the form
huw where u solves the characteristic equation u2 4 = 0= This gives {1 = h2w
and {2 = h2w so that
2w
h
h2w
det(Z (w)) = det( 2w
) = 4=
h 2 h2w (2)
Use integration by part to nd
Z
Z
(i (w)@d){2
((w + 1)@1)h2w
y1 (w) =
=
= h2w (w@8 3@16) and
det(Z (w))
4
Z
Z
(i (w)@d){1
(w + 1)@1h2w
y2 (w) =
=
= h2w (w@8 1@16)=
det(Z (w))
4
The particular solution is
{s
= {1 y1 (w) + {2 y2 (w)
= h2w (h2w (w@8 3@16)) + h2w (h2w (w@8 1@16))
= (w + 1)@(4)=
The integrations seem to make variation of parameter approach for this

problem more di!cult; however, the variation of parameters formula in (6.5.5)
is more general as long as the integrals exist, as is the case if the integrands are
continuous. Moreover, this approach generalizes to more complicated systems
of dierential equations and will be further studied in Subsection 7.3.3.
6.5.3
Numerical Solution Using MATLABs ode45()
In this section we will numerically approximate the current in the series LRC
circuit where there is a modulated imposed signal
H(w) = 100 sin(id 2w) sin(iv 2w) where
audio frequency = id ?? iv = station frequency.
In the rst part of this section we were able to nd the exact particular solution
for the unmodulated signal
H(w) = 100 sin(iv 2w)>
and in this case the maximum amplitude of the current was determined to be
100
p
2
(O + 1@F)2 + (U)2
278
where = iv 2= If = 710, U = 400> and L and C are chosen so that O 2 +
1@F 0> then the maximum amplitude is approximately
100
100
p
=
= 0=25=
2
2
400
(0) + (400)
p
s
If O = 2 and F = 106 > then the tuned station is = 1@ 2(106 ) = 103 @ 2
707=1= A signicant variation from O = 2 and F = 106 causes the amplitude
to decrease as long as is at this tuned signal and the signal is unmodulated.
The numerical calculations will illustrate the tuning formula O 2 + 1@F = 0
giving an approximation for tuning even if the signal is modulated.
The MATLAB command ode45() is a variation of the Runge-Kutta numerical
method and uses variable time steps. It requires three input parameters: the
character representation of the function le containing the right side of the
system, the row vector with the initial and nal times and the row vector with
the initial charge and current. The call to ode45() is contained in the MATLAB
code le lrc.m. The MATLAB function le yplrc.m contains the right side of
the vector equation (6.5.2). In the le yplrc.m the charge is depicted by the
four character symbol y(1), and the current is depicted by the four character
symbol y(2). In yplrc.m the rst and second components of the right side of
the vector equation (6.5.2) are denoted by the eight character symbols yplrc(1)
and yplrc(2), respectively.
MATLAB Code lrc.m
% This le has a call to ode45().
% It requires a second le called yplrc.m,
% which models a series LRC tuning circuit.
clear;
t0 = 0;
tf = .3;
% nal scaled time
y0 = [0 0];
% initial charge and current
[t y] = ode45(yplrc, [t0 tf], y0);
plot(t, y(:,2))
title( Series LRC Circuit)
xlabel(time)
ylabel(current)
MATLAB Code yplrc.m
function yplrc = yplrc(t,y)
L = 20; R = 400; C = 10^(-6);
fa = 63/(2*pi);
% scaled audio frequency
fs = 710/(2*pi);
% scaled station frequency
E0 = 100*sin(2*pi*fa.*t);
% modulated signal
% E0 = 100;
% unmodulated signal
yplrc(1) = y(2);
yplrc(2) = (E0*sin(2*pi*fs.*t) - R*y(2) - (1/C)*y(1))/L;
279
Figure 6.5.2: Tuned Circuit with Modulated Signal

yplrc = [yplrc(1) yplrc(2)];
The code was executed three times with three values of inductance O = 0=2>
2 and 20= The currents are given in Figure 6.5.2, where the largest maximum
amplitude corresponds to O = 2 and is consistent with the unmodulated analysis.
6.5.4
Exercises
1.
In Example 6.5.1 derive the algebraic system and solution for the coe!cients d and e=
2.
Use Example 6.5.1 as a template to solve equation (6.5.2) with H(w) =
H0 cos(w)=
3.
Use variation of parameters to nd the particular solution of {00 9{ =
2w + 3=
4.
Use variation of parameters to nd the particular solution of {00 + { =
sin(w)=
5.
Use MATLAB codes lrc.m and yplrc.m to experiment with dierent F =
(0=1)106 , (1=0)106 and (10=0)106 and keep O = 2=
cos(1500w)=
6.
Consider equation (6.5.2) with
s H(w) = H0 sin(63w)
6
(a).
Use the formula = 1@ OF and F = 10 to estimate the O that
will give the maximum amplitude.
(b).
Conrm this by executing the MATLAB code le lrc.m with a modied yplrc.m le.
280
7.
Use a variation of the MATLAB codes lrc.m and yplrc.m to solve and
graph the solutions of {00 + e{0 + { = cos(w) with {(0) = 0 and {0 (0) = 0 for
e = 0> 1> 2 and 4=
Chapter 7
Eigenvalues and Dierential

Equations
This chapter contains the eigenvalue and eigenvector approach to solving linear
systems of dierential equations. This is a generalization of the simple single
dierential equation {0 = d{ + i to a system x0 = Dx + f where D is a matrix
and x a column vector of unknown functions of time. Applications will include
time dependent mixing tanks and heat conduction.
7.1
Solution of x0 = Dx by Elimination
In Section 6.5 a single second order linear dierential equation was expressed
as a system of two rst order dierential equations

0

{
0
1
{
0
{(0)
{0
=
+
with
=
=
|
f@d e@d
|
i @d
|(0)
|0
(7.1.1)
This is a special case of the more general initial value problem

0

{
d e
{
i
{(0)
{0
=
+
with
=
=
(7.1.2)
|
f g
|
j
|(0)
|0
Furthermore, there can be more than two unknowns and two dierential equations. The objective of this section is to present some examples that will indicate
how one can approach these problems so that larger systems can be solved in a
systematic way.
7.1.1
Overdamped Mass-Spring System
The solution of the homogeneous second order dierential equation d{00 + e{0 +
f{ = 0 has the exponential form huw where u is the solution of the characteristic
281
282
CHAPTER 7. EIGENVALUES AND DIFFERENTIAL EQUATIONS
equations du2 + eu + f = 0. There are three possibilitie, either u is distinct

real, repeated real or complex root. Consider the case where u has distinct real
values and, hence, the general solution is {(w) = f1 hu1 w +f2 hu2 w = When converting
the second order equation to a system, we dened | = {0 . Thus, the general
homogeneous solution to the system in equation (7.1.1) with i = 0 is

{
f1 hu1 w + f2 hu2 w
=
|
(f1 hu1 w + f2 hu2 w )0

f1 hu1 w + f2 hu2 w
=
f1 hu1 w u1 + f2 hu2 w u2

1
1
= f1
hu1 w + f2
hu2 w =
u1
u2
The two column vectors are examples of eigenvectors associated with the system
matrix in (7.1.1), and u1 and u2 are the corresponding eigenvalues, which will
be studied in more detail in Section 7.2. If initial conditions are required, then
one must choose the coe!cients f1 and f2 so that {(0) = {0 and |(0) = |0 =
Example 7.1.1. Reconsider overdamped mass-spring in Example 6.3.3, where
{00 + 4{0 + { = 0=
The equivalent system is

{
|

=
0
1
1 4
{
|
whose solution is

{
|

= f1
1
u1

hu1 w + f2
1
u2
hu2 w
s
s
where u1 = 2 + 3 and u2 = 2 3= Find f1 and f2 so that {(0) = 1 and
|(0) = 2. This requires the solution of the algebraic system

1
1
{(0)
u1 0
h + f2
hu2 0
= f1
u1
u2
|(0)

1
1 1
f1
=
=
2
u1 u2
f2
Since the roots are distinct, this has unique solution given by Cramers rule
s
s
u2 2
4 3
2 u1
4 3
s
s =
=
=
f1 =
and f2 =
u2 u1
u2 u1
2 3
2 3
7.1. SOLUTION OF X0 = DX BY ELIMINATION
7.1.2
283
Two-tank Mixing System
As a second illustration consider a two-tank mixing problem similar to mixing

application in Subsection 3.7.4. Suppose the rst and second tanks have xed
volumes equal to 100 (L) and have {(w) (kg) and |(w) (kg) amounts of salt so
that the concentrations are {(w)@100 (L/kg) and |(w)@100 (L/kg), respectively.
Let the rst tank have incoming pure water with ow rate 6 (L/min) and
concentration 0 (L/kg), and incoming from the second tank with ow rate
2 (L/min) with concentration |(w)@100. The rst tank feeds into the second
tank with ow rate 8 (L/min) with concentration {(w)@100, and these yield the
dierential equation for the rst tank
{0 = 6(0) + 2|@100 8{@100=
In order for the second tank to have xed volume, the ow rate out and not
into the rst tank must be 6 (L/min) with concentration |(w)@100 (L/kg). This
gives the dierential equation for the second tank
| 0 = 8{@100 2|@100 6|@100=
These two dierential equations may be written as a system

0
4@50 1@50
{
{
=
=
4@50 4@50
|
|
In subsequent sections we shall derive the solutions by elimination and by eigenvalues giving
{ = f1 h(1@25)w + f2 h(3@25)w and | = f1 2h(1@25)w + f2 (2)h(3@25)w =

1
1
{
(1@25)w
+ f2
h
h(3@25)w =
= f1
2
2
|
Example 7.1.2. Suppose the initial amounts in the tanks are {(0) = 2 and
|(0) = 3= Find the coe!cients so that

1
1
{(0)
h(1@25)0 + f2
h(3@25)0
= f1
2
2
|(0)

2
1 1
f1
=
=
3
2 2
f2
The algebraic solution is f1 = (2(2) 1(3))@(4) = 7@4 and f2 = (1(3)
2(2))@(4) = 1@4= The solution to the homogeneous rst order system is

{
1
1
= (7@4)
h(1@25)w + (1@4)
h(3@25)w =
|
2
2
284
7.1.3
Homogeneous Solutions by Elimination
For small systems of dierential equations this method is practical, and requires
one to write the rst order system as a single higher order dierential equation.
In the case of two rst order homogeneous equations (7.1.2) with i = j = 0 we
have
{0 = d{ + e| and | 0 = f{ + g|=
If e 6= 0, then the rst dierential equation can be written as
| = ({0 d{)@e=
Put this into the second dierential equation to yield a single second order
dierential equation
(({0 d{)@e)0 = f{ + g(({0 d{)@e)
({00 d{0 )@e = f{ + g(({0 d{)@e)=
As an illustration reconsider Example 7.1.2 and derive the solution using the
elimination method.
Example 7.1.2 (continued). The two dierential equations are
{0 = (4@50){ + (1@50)| and | 0 = (4@50){ + (4@50)|=
In the rst equation solve for |
| = 50{0 + 4{=
Insert this into the second equation
(50{0 + 4{)0
50{00 + 4{0
= (4@50){ + (4@50)(50{0 + 4{)

= (4@50){ 4{0 + (16@50){=
This is a second order dierential equation for {

50{00 + 8{0 + (12@50){ = 0=
The solution has the exponential form huw where u is the solution of the characteristic equation
50u2 + 8u + 12@50 = 0
whose solution is u = 1@25 and u = 3@25= This means the solution of the
second order dierential equation is
{ = f1 h(1@25)w + f2 h(3@25)w =
285
The rst order system requires the solution of both { and |= Find | by using
| = 50{0 + 4{
|
= 50(f1 h(1@25)w + f2 h(3@25)w )0 + 4(f1 h(1@25)w + f2 h(3@25)w )

= 50(f1 h(1@25)w (1@25) + f2 h(3@25)w (3@25)) +
4(f1 h(1@25)w + f2 h(3@25)w )
= f1 h(1@25)w (50@25 + 4) + f2 h(3@25)w (150@25 + 4)
= f1 h(1@25)w 2 + f2 h(3@25)w (2)=
Example 7.1.3. Use elimination to nd the general homogeneous solution of

0
2 3
{
{
=
=
1 2
|
|
The rst dierential equation {0 = 2{ 3| can be solved for | in terms of {
| = ({0 2{)@(3)=
Then put this into the second dierential equation | 0 = { 2|
(({0 2{)@(3))0 = { 2(({0 2{)@(3))=
This yields the single second order dierential equation {00 { = 0 whose general
solution is
{ = f1 hw + f2 hw =
Use | = ({0 2{)@(3) to nd the general solution for |
|
= ((f1 hw + f2 hw )0 2(f1 hw + f2 hw ))@(3)

= (f1 hw (1) + f2 hw (1) 2(f1 hw + f2 hw ))@(3)
= f1 hw (1@3) + f2 hw (1)=
The vector form of the system solution is

1
1
{
hw + f2
hw =
= f1
1@3
1
|
Find the coe!cients so that {(0) = 6 and |(0) = 4> that is,

1
1
{(0)
0
h + f2
h0
= f1
1@3
1
|(0)

6
1 1
f1
=
=
4
1@3 1
f2
The algebraic solution is f1 = 3 and f2 = 3, and so the solution of the initial
value problem is

{
1
1
=3
hw + 3
hw =
|
1@3
1
286
7.1.4
This is an extension of the heat conduction model in a wire in Subsections

3.2.3 and 3.3.4. Let the wire be broken into four parts and let xl (w) be the
approximation of the temperature at time w and at location l{ where { =
O@4, O is the length of the thin wire, and x0 and x4 are given at the left and
right ends of the wire. The objective is to nd x1 (w), x2 (w) and x3 (w) given the
initial temperatures x1 (0), x2 (0) and x3 (0)= The time dependent model is
60
6
5
65
5
2 1 0
x1
x1
f 7 x2 8 = (N@({)2 ) 7 1 2 1 8 7 x2 8 +
0 1 2
x3
x3
5
6
2
i + (N@({) )x0
7
8=
i
2
i + (N@({) )x4
This presents a larger system of dierential equations as well as some additional
terms on the right sides. For the moment, consider the following simplication.
Example 7.1.4.
60
6
5
65
x1
2 1 0
x1
7 x2 8 = 7 1 2 1 8 7 x2 8 =
0 1 2
x3
x3
5
The elimination method still works, but one can clearly see the additional complications. Solve the rst dierential equation for x2 in terms of x1
x2 = x01 + 2x1 =
(7.1.3)
Put x2 into the second equation and solve for x3

x3 = x001 + 4x01 + 3x1 =
(7.1.4)
Insert x3 and x2 = x01 + 2x1 into the third equation and collect like terms to
obtain a single third order dierential equation
00
0
x000
1 + 6x1 + 10x1 + 4x1 = 0=
(7.1.5)
Assume the solution of (7.1.5) is an exponential huw to get a third order characteristic equation
u3 + 6u2 + 10u + 4 = 0
(u + 2)(u2 + 4u + 2) = 0=
s
s
The algebraic solutions are u = 2, 2 + 2 and 2 2= Therefore, the
general solution to the third order dierential equation is
s
2)w
x1 (w) = f1 h2w + f2 h(2+
s
2)w
+ f3 h(2
287
Use equations (7.1.3) and (7.1.4) to nd x2 (w) and x3 (w), and write the general
solution to the homogeneous system in vector form
6
5
6
5
6
5
1
x1
1
s
s
7 x2 8 = f1 7 0 8 h2w + f2 7 2 8 h(2+ 2)w +
1
x3
1
6
5
1
s
s
f3 7 2 8 h(2 2)w =
1
Find the three coe!cients so that the initial temperatures are x1 (0) = 20,
x2 (0) = 30 and x3 (0) = 20= Insert w = 0 into the above general solution to
obtain the algebraic system
65
5
6 5
6
1 s1
1
f1
20
s
7 30 8 = 7 0
2 2 8 7 f2 8 =
20
f3
1 1
1
The algebraic solution is f1 0=0000, f2 20=6066 and f3 0=6066 and so
the solution to the system of dierential equations is
6
6
5
6
5
5
1
x1
s
s
s1
s
7 x2 8 20=6066 7 2 8 h(2+ 2)w 0=6066 7 2 8 h(2 2)w =
x3
1
1
7.1.5
Solution of Systems Using MATLAB
The MATLAB command dsolve() can be used to solve simple linear systems of
dierential equations. The solution of the two-tank mixing in Example 7.1.2 is
found as follows.
AA soltank = dsolve(Dx = (-4/50)*x + (1/50)*y,
Dy = (4/50)*x + (-4/50)*y,
x(0) = 2, y(0) = 3, t)
soltank =
x: [1x1 sym]
y: [1x1 sym]
AA soltank.x
1/4*exp(-3/25*t) + 7/4*exp(-1/25*t)
AA soltank.y
7/2*exp(-1/25*t) - 1/2*exp(-3/25*t)
The solution to the heat conduction in Example 7.1.4 with three equations
can also be easily done. But, one can observe the increased symbolic output. For
more complicated problems numerical calculations such as done by MATLABs
command ode45() are a preferred method.
288

AA solheat = dsolve(Du1 = -2*u1 + u2, Du2 = u1 -2*u2 + u3,
Du3 = u2 -2*u3, u1(0) = 20,
u2(0) = 30, u3(0) = 20, t)
solheat =
u1: [1x1 sym]
u2: [1x1 sym]
u3: [1x1 sym]
AA solheat.u1
-15/2*2^(1/2)*exp(-(2+2^(1/2))*t) +
15/2*2^(1/2)*exp((-2+2^(1/2))*t) +
10*exp(-(2+2^(1/2))*t) + 10*exp((-2+2^(1/2))*t)
AA solheat.u2
15*exp(-(2+2^(1/2))*t) + 15*exp((-2+2^(1/2))*t) 10*2^(1/2)*exp(-(2+2^(1/2))*t) +
10*2^(1/2)*exp((-2+2^(1/2))*t)
AA solheat.u3
-15/2*2^(1/2)*exp(-(2+2^(1/2))*t) +
15/2*2^(1/2)*exp((-2+2^(1/2))*t) +
10*exp(-(2+2^(1/2))*t) + 10*exp((-2+2^(1/2))*t)
7.1.6
Exercises
Consider the overdamped mass-spring model {00 + 8{0 + { = 0.

(a).
Dene | = {0 and write this as a rst order system for { and |=
(b).
Find the vector form of the system solution.
(c).
Find the coe!cients so that {(0) = 2 and |(0) = 0=
2.
Consider dierential equation {00 3{0 + 2{ = 0.
(a).
Dene | = {0 and write this as a rst order system for { and |=
(b).
Find the vector form of the system solution.
(c).
Find the coe!cients so that {(0) = 1 and |(0) = 2=
3.
Consider the two-tank mixing model where both tanks have xed volume
equal to 50 (L).
(a).
Derive the model to show it has the form
1.
{
|

=
4@25 1@25
4@25 4@25
{
|
(b).
Use elimination to nd the vector form of the general solution.
(c).
Find the solution of the initial value problem {(0) = 2 and |(0) = 3=
4.
Use elimination to solve the initial value problem

{
|

=
1 1
2 4
{
|

with
{(0)
|(0)

=
1
4
7.2. REAL EIGENVALUES AND EIGENVECTORS

5.

0

{
3 1
{
{(0)
2
=
with
=
=
|
1 3
|
|(0)
4
6.

0

{
4 1
{
{(0)
1
=
with
=
=
|
2 1
|
|(0)
3
289
7.
Consider the heat conduction model and Example 7.1.4.

(a).
Derive equations (7.1.4) and (7.1.5).
(b).
Derive the expressions for x2 (w) and x3 (w)=
8.
Solve the initial value problem
5
60 5
65
6
5
6 5
6
x1
3 1
0
x1
x1 (0)
10
7 x2 8 = 7 1 3 1 8 7 x2 8 with 7 x2 (0) 8 = 7 30 8 =
0
1 3
x3 (0)
x3
x3
5
9.
Use MATLAB to conrm any calculations in exercises 3-6 and 8.
7.2
Real Eigenvalues and Eigenvectors
The examples in the previous section suggest that the solutions of x0 = Dx

are linear combination vector functions of the form uhuw . The exact expression for the vector functions was derived from the elimination method, which
transforms the system into a single higher order dierential equation. Because
the elimination method becomes complicated for larger systems, this is not a
preferred method. An alternative is to nd the vectors u and the associated
numbers u, called eigenvalues, by noting that they must satisfy Du = uu= In
the discussion of the eigenvalue method we restrict the examples to the cases
where the eigenvalues are distinct and real. The computation of the required
eigenvalues and eigenvectors for larger systems can be done by software such as
MATLAB.
7.2.1
Eigenvalue and Eigenvector Denitions
Assume a solution of x0 = Dx has the form x = uhuw where u is a nonzero

constant vector with possibly complex components and u is possibly a complex
number. We must choose u and u so that x0 = Dx holds
(uhuw )0
uhuw u
= D(uhuw )
= (Du)huw =
Since huw is never zero,

Du = uu
Du = uLu
(D uL)u = 0=
290
Because u is not a zero vector, D uL must not have an inverse and, thus, the
determinant of D uL must be zero. This gives a single polynomial equation
that u must satisfy. Once u is found, solve algebraic system (D uL)u = 0 for
the unknown constant vector u that is associated with this u=
Denition 7.2.1. Let D be an q q matrix. An eigenvalue is a number (real
or complex) u such that there is a nonzero vector u and Du = uu= The vector
u is called the eigenvector of D corresponding to the eigenvalue u=
Note, any constant times u is also an eigenvector, because D(fu) = f(Du) =
f(ux) = u(fu)= Examples 7.1.1-7.1.4 in the previous section give the following
examples of eigenvectors and eigenvectors .

0
1
1
Example 7.2.1. Let D =
and consider u =
and u = u1 =
1 4
u1
s
2 + 3=

1s
0
1
Du =
1 4
2 + 3
s

0(1) + 1(2 + s3)

=
1(1) 4(2 + 3)

s
1s
= (2 + 3)
=
2 + 3

4@50 1@50
4@50 4@50
u = u1 = 1@25=

Du =

4@50 1@50
4@50 4@50

and consider u =

(4@50)1 + (1@50)2
(4@50)1 + (4@50)2

1
= (1@25)
=
2
1
2
1
2
and

2 3
1 2

and consider u =
1=

Du =

2 3
1 2
2(1) 3(1)
1(1) 2(1)

1
= (1)
=
1
1
1
1
1
and u = u2 =
291
6
6
5
1
2 1
0
s
Example 7.2.4. Let D = 7 1 2 1 8 and consider u = 7 2 8 and
0
1 2
1
s
u = u3 = 2 2=
6
5
65
1
2 1
0
s
Du = 7 1 2 1 8 7 2 8
0
1 2
1
s
5
6
2(1) +s
1( 2)
8
= 7 1(1) 2(
s 2) + 1(1)
1( 2) 2(1)
6
5
1
s
s
= (2 2) 7 2 8 =
1
5
7.2.2
Finding Eigenvalues by Determinants
For small matrices a simple way to nd eigenvalues u is to solve

det(D uL) = 0=
In general this is a polynomial equation of degree q and, so, there are q possible
roots or eigenvalues. For each eigenvalue there is an eigenvector u that satises
the algebraic system
(D uL)u = 0=
A simple example is the system associated with the second order dierential
equation d{00 + e{0 + f{ = 0 where

0
1
D=
=
f@d e@d
In this case the determinant is easy to evaluate and gives the characteristic
algebraic equation

0
1
1 0
det(D uL) = det(
u
)
f@d e@d
0 1

0u
1
= det(
)
f@d e@d u
= u(e@d u) 1(f@d)
= u2 + (e@d)u + f@d = 0=
The solutions may be complex numbers, but we will focus on systems where
the roots of det(D uL) = 0 are real and distinct.
292
Example 7.2.1 (continued). Find the eigenvalues using determinants by

solving

0u
1
det(D uL) = det(
)
1 4 u
= u2 + 4u + 1 = 0=
s
s
= 2 + 3 and u2 = 2 3= Find the eigenvector
The two eigenvalues are u1 s
associated with u1 = 2 + 3= Solve the algebraic system

(D u1 L)u = 0
s

x1
0
0 (2 + 3)
1
s
=
=
0
x2
1
4 (2 + 3)
This is equivalent to two algebraic equations, which are multiples of each other,
s
s
(2 + 3)x1 + x2 = 0 and x1 + (2 3)x2 = 0=
This
s gives one degree of freedom to choose, say, x1 = 1 and then x2 = 2 +
3= This agrees with one homogeneous solution obtained using the elimination
s
method. In order to obtain the other homogeneous solution, let u2 = 2 3
and repeat the above calculations.
solving

4@50 u
1@50
det(D uL) = det(
)
4@50
4@50 u
= (4@50 u)(4@50 u) (1@50)(4@50) = 0=
The two eigenvalues are u1 = 1@25 and u2 = 3@25= Find the eigenvector
associated with u1 = 1@25= Solve the algebraic system

(D u1 L)u = 0

4@50 (1@25)
1@50
x1
0
=
=
4@50
4@50 (1@25)
0
x2
The two algebraic equations are

(2@50)x1 + (1@50)x2 = 0 and (4@50)x1 + (2@50)x2 = 0=
This gives one degree of freedom to choose, say, x1 = 1 and so x2 = 2= In order
to nd the other homogeneous solution, let u2 = 3@25 and repeat the above
calculations.
solving

2u
3
det(D uL) = det(
)
1
2 u
= (2 u)(2 u) (3)1 = 0=
293
The two eigenvalues are u1 = 1 and u2 = 1= Find the eigenvector associated
with u2 = 1= Solve the algebraic system

(D u1 L)u = 0

2 (1)
3
x1
0
=
=
1
2 (1)
0
x2
The two algebraic equations are

3x1 3x2 = 0 and x1 x2 = 0=
This gives one degree of freedom to choose, say, x1 = 1 and then x2 = 1= In
order to nd the other homogeneous solution, let u1 = 1 and repeat the above
calculations.
solving
5
6
2 u
1
0
8)
1
2 u
1
det(D uL) = det(7
0
1
2 u
= (2 u)((2 u)(2 u) 1) 1(1(2 u) 0)
= (2 u)[(2 u)(2 u) 2] = 0=
s
s
The three eigenvalues are u1 = 2, u2 = 2 + 2 and u3 = 2 2= Find the
eigenvector associated with u1 = 2= Solve the algebraic system
(D u1 L)u = 0
65
6
5 6
2 (2)
1
0
x1
0
7
8 7 x2 8 = 7 0 8 =
1
2 (2)
1
0
1
2 (2)
0
x3
5
The three algebraic equations are

x2 = 0, x1 + x3 = 0 and x2 = 0=
This gives one degree of freedom to choose, say, x1 = 1 and so x3 = 1= The
other two eigenvectors can be found by similar calculations.
7.2.3
Application to Three-tank Mixing
This section continues the discussion of the three-tank mixing application in

Subsection 3.7.4 where the steady state model was considered. Here we will
study the time dependent model for the amounts {(w), |(w) and }(w) in the
three 24 (L) tanks. The ow rates are as indicated in Figure 3.7.1, and the
system of dierential equations is
5
60 5
65
6 5 6
{
5@12 2@12
0
{
1
7 | 8 = 7 5@12 6@12 1@12 8 7 | 8 + 7 2 8 =
(7.2.1)
}
0
4@12 4@12
}
0
294
The solution of det(D uL) = 0 done by-hand calculations is a little messy and,
so, the MATLAB commands det() and solve() are used.
AA syms r
AA poly = det(A - r*eye(3))
% compute the determinant
-5/144 - 5/12*r - 5/4*r^2 - r^3
AA r = solve(poly)
% solve for the roots of this polynomial
AA r1 = eval(r(1))
% converts the symbolic expression to numbers
-.1264
AA r2 = eval(r(2))
-.7643
AA r3 = eval(r(3)
-.3593
From here one can solve the algebraic system (D uL)u = 0 to nd the eigenvectors, but an easier and more robust approach is to use another MATLAB
command eig(), which computes all the eigenvalues and their associated eigenvectors.
7.2.4
Eigenvalues and Eigenvectors Using MATLAB
The MATLAB command [U R] = eig(A) computes the eigenvectors and eigenvalues of the matrix A. All three symbols are square matrices. The columns
of X , indicated by X (:> m), are the eigenvectors. The matrix R is a diagonal
matrix whose m wk diagonal components are the eigenvalues associated with the
m wk column of X . One homogeneous solution is X (:> m)hU(m>m)w , and the general
homogeneous solution is a linear combination of X (:> m)hU(m>m)w = The eigenvectors
and eigenvalues for Example 7.2.3 are easily computed by [U R] = eig(A). The
columns of X have been scaled to have length equal to one, and appear dierent
than those done by-hand calculations. However, any eigenvector can be scaled
to obtain another eigenvector associated with the same eigenvalue.
AA A = [2 -3;1 -2]
AA [U R] = eig(A)
U=
.9487
.7071
.3162
.7071
R=
1
0
0
-1
AA newu1 = U(:,1)/U(1,1)
1.0000
0.3333
AA newu2 = U(:,2)/U(1,2)
1.0000
1.0000
% scale the rst eigenvector
% scale the second eigenvector
295
Return to the three-tank mixing problem in (7.2.1), and use [U R] = eig(A)

to nd the eigenvectors and eigenvalues.
AA A = [5/12 2/12 0;
AA [U R] = eig(A)
U=
.3546
.7396
-.5721
R=
-.7643
0.0000
0.0000
5/12 -6/12 1/12; 0 4/12 -4/12]
-.2201
-.0758
.9725
0.0000
-.3593
0.0000
.2898
.5047
.8132
0.0000
0.0000
-.1264
The solution to the three-tank mixing problem (7.2.1) is the general homogeneous solution plus the steady state solution, which was calculated in Subsection
3.7.4. The steady state solution is the algebraic solution of (7.2.1) when the
time derivatives on the left side are all zero. The time dependent solution is
5
6
5
6
5
6
=3546
=2201
{(w)
7 |(w) 8 = f1 7 =7396 8 h=7643w + f2 7 =0758 8 h=3593w +
=5721
=9725
}(w)
5
6
5
6
=2898
7=2
f3 7 =5047 8 h=1264w + 7 12=0 8 =
=8132
12=0
Since the eigenvalues are all negative, the time dependent solution converges to
the steady state solution as time increases.
7.2.5
1.
Exercises

s
s
(a).
Verify u = u2 = 2 3 and u = [1 (2 3)]W are the other
eigenvalue and eigenvector.
(b).
Solve the algebraic system (D u2 L)u = 0 to nd the above eigenvector.
2.
(a).
Verify u = u2 = 3@25 and u = [1 2]W are the other eigenvalue
and eigenvector.
(b).
3.
(a).
Verify u = u1 = 1 and u = [1 1@3]W are the other eigenvalue and
eigenvector.
(b).
296
4.

s
s
(a).
Verify u = u2 = 2 + 2 and u = [1
2 1]W are the second
eigenvalue and eigenvector.
(b).
5.
Use determinants to nd both eigenvalues and eigenvectors for

4@25 1@25
D=
=
4@25 4@25
6.

3 1
D=
=
1 3
7.

4 1
D=
=
2 1
8.
Use determinants to nd all three eigenvalues and eigenvectors for

5
6
3 1
0
D = 7 1 3 1 8 =
0
1 3
9.
Use the MATLAB command [U R] = eig(A) to conrm any calculations
in exercises 5-8.
7.3
Solution of x0 = Dx + f (w)
Assume the q q matrix D has q distinct real eigenvalues such as in Examples

7.2.1-7.2.4 of the previous section. Consider the solution of the linear system
of dierential equations with unknown functions x = [{1 (w) {2 (w) {q (w)]W
x0 = Dx + f (w)=
(7.3.1)
In this case, one can show, see Subsection 7.3.3, there are q linearly independent
homogeneous solutions formed by the eigenvectors um and eigenvalues um
u1 hu1 w > u2 hu2 w > > uq huq w =
Any homogeneous solution xk (w) satises x0k = Dxk and is a linear combination
of these
xk (w) = f1 u1 hu1 w + f2 u2 hu2 w + + fq uq huq w =
A particular solution is any xs (w) that satises x0s = Dxs + f (w), and they
can be found for simple cases by the method of undetermined coe!cients, see
7.3. SOLUTION OF X0 = DX + F(W )
297
Subsection 7.3.2, or by the variation of parameters method, see Subsection 7.3.3.

Since
(xk (w) + xs (w))0
= x0k + x0s
= Dxk + Dxs + f (w)
= D(xk + Dxs ) + f (w)>
the general solution is

x(w) = xk (w) + xs (w)=
The assumption that the matrix has distinct real eigenvalues does limit our
discussion (see exercise 1 for other cases), and the interested reader should
consult [1] or [7] for more details about repeated or complex eigenvalues. When
the homogeneous and particular solutions have been found, the initial value
problem, which also requires x(w) to satisfy a given initial vector x(0) = x0 >
can be solved. A number of examples will be given, and an application to time
dependent heat conduction will be presented.
7.3.1
D Has an Inverse and f a Constant Vector
The application to the three-tank mixing problem presented in the previous

section has a dierential equation for the amount in each tank. This gives a
linear system where q = 3, f (w) = f is a constant vector, and the eigenvalues of
the matrix are distinct, real and negative. The matrix D has an inverse, and
the steady state solution is dened by the solution of the algebraic system
0 = Dx + f so that x = D1 f =
The steady state solution is a constant vector, and it is also an example of a
particular solution. The general solution is a linear combination of the homogeneous solutions plus the steady state solution
x(w) = f1 u1 hu1 w + f2 u2 hu2 w + f3 u3 hu3 w + (D1 f )=
Because the eigenvalues u1 > u2 and u3 are negative, the general solution must
converge to the steady state solution.
Example 7.3.1. Find the general homogeneous, steady state and general
solution for

{1
{2
x0
0
= Dx + f

4 1
{1
6
=
+
=
10 1
12
{2
The homogeneous solutions are found by rst nding both eigenvalues and their
corresponding eigenvectors. Solve
det(D uL) = 0

4u
1
det(
) = 0
10
1u
(4 u)(1 u) 10 = 0
298
to obtain u1 = 6 and u2 = 1= Let u = 6 and nd the corresponding eigenvector

by solving the algebraic systems (D uL)u = 0:

0
46
1
x1
=
or
0
10
16
x2
2x1 + x2
10x1 + (5)x2
= 0 and
= 0=
So, for u = u1 = 6 the corresponding eigenvector is u1 = [1 2]W = In a similar

calculation for u = u2 = 1 we obtain the eigenvector u2 = [1 5]W = The
general homogeneous solution is
xk (w) = f1 u1 hu1 w + f2 u2 hu2 w

1
1
= f1
h6w + f2
hw =
2
5
The steady state solution is the solution of 0 = Dx + f
xs
= D1 f

1
4 1
6
=
10 1
12

1@(6)
1@(6)
6
1
=
=
=
10@(6) 4@(6)
12
2
The general solution is

x(w) = f1 u1 hu1 w + f2 u2 hu2 w + xs

1
1
1
6w
w
= f1
=
h + f2
h +
2
2
5
Since the rst eigenvalue u1 = 6 is positive, the general solution may not converge to the steady state solution!
7.3.2
Particular Solution by Undetermined Coe!cients
Consider particular solutions when f (w) is a simple vector function of time such
as gw + h or ghfw or g cos(fw) + h sin(fw) where g and h are given constant
vectors. Reasonable speculations are that any particular solution should have
the forms xs (w) = aw + b or ahfw or b cos(fw) + c sin(fw)> respectively. The a
and b constant vectors are to be found so that x0s = Dxs + f (w)= This is the
vector version of undetermined coe!cients that was used in Sections 6.1 and
6.4. Here we will equate vector coe!cients of the like functions of time and
solve the resulting algebraic systems.
299
Example 7.3.2. Find the particular solution of

{1
{2
x0
0
= Dx + gw + h

4 1
{1
2
6
=
+
w+
=
10 1
6
12
{2

xs (w) = aw + b
and choose a and b so that
(aw + b)0 = D(aw + b) + gw + h
a1 + 0 = (Da)w + Db + gw + h
0w + a = (Da + g)w + Db + h=
Match the vector coe!cients of w and 1 to get
0 = Da + g and a = Db + h=
The solution of the rst algebraic system is
a = D1 g

1@(6)
1@(6)
2
2@3
=
=
=
10@(6) 4@(6)
6
2@3
Use a in the second equation to solve for b
b = D1 (a h)

1@(6)
1@(6)
2@3
6
7@9
=
(

)=
=
10@(6) 4@(6)
2@3
12
32@9
Thus, a particular solution is
xs (w) = aw + b

2@3
7@9
=
w+
=
2@3
32@9
Use the homogeneous solution from Example 7.3.1 to nd the general solution
x(w) = f1 u1 hu1 w + f2 u2 hu2 w + aw + b

2@3
7@9
1
1
6w
w
= f1
w+
=
h + f2
h +
2@3
32@9
2
5

{1
{2
x0
0
= Dx + gh2w

4 1
{1
4
=
+
h2w =
10 1
2
{2
300

xs (w) = ah2w >
and choose a so that
(ah2w )0 = D(ah2w ) + gh2w
ah2w 2 = (Da)h2w + gh2w =
Matching the vector coe!cients of h2w we have
2a = Da + g
2La = Da + g
(D 2L)a = g=
Since 2 is not an eigenvalue of D, the matrix D 2L has an inverse and we may
solve for a
a = (D 2L)1 g

1
42
1
4
=
10
12
2

1@(12) 1@(12)
4
1@2
=
=
=
10@(12) 2@(12)
2
3
Hence, a particular solution is

xs (w) = ah2w =
1@2
3
h2w >
and the general solution is

x(w) = f1 u1 hu1 w + f2 u2 hu2 w + ah2w

1@2
1
1
6w
w
= f1
h2w =
h + f2
h +
3
2
5

{1
{2
x0
0
= Dx + g cos(3w) + h sin(3w)

1
3
4 1
{1
+
cos(3w) +
sin(3w)=
=
2
4
10 1
{2

xs = a cos(3w) + b sin(3w)>
301
and choose the vector coe!cients so that the dierential equation holds
(a cos(3w) + b sin(3w))0
= D(a cos(3w) + b sin(3w)) +

g cos(3w) + h sin(3w)
a sin(3w)(3) + b cos(3w)(3) = (Da) cos(3w) + (Db) sin(3w) +
g cos(3w) + h sin(3w)
= (Da + g) cos(3w) + (Db + h) sin(3w)=
Match the vector coe!cients of the sine and cosine function to obtain
a(3) = Db + h and b3 = Da + g=
The rst equation can be solved for a = (Db + h)@(3) and put into the second
equation
b3 = D((Db + h)@(3)) + g
b3L = D2 b@(3) + Dh@(3) + g
(3L D2 @(3))b = Dh@(3) + g=
Solve this for b and then nd a = (Db + h)@(3)= Another way to solve this
system is to write it as a block 2 2 algebraic system

D 3L
a
g
=
=
3L
D
b
h
This is easily solved using MATLAB

4@10
8@30
a=
and b =
=
2@10
22@30
The general solution is
x(w) = f1 u1 hu1 w + f2 u2 hu2 w + a cos(3w) + b sin(3w)

1
1
= f1
h6w + f2
hw +
2
5

4@10
8@30
cos(3w) +
sin(3w)=
2@10
22@30
The method of undetermined coe!cients does not always work or is too complicated to implement. An alternative is the variation of parameters method,
which can also be generalized to systems of dierential equations. This method
gives a closed formula for the particular solution, but it may involve some
di!cult integrations. A third way to solve systems is to use numerical approximations such as implemented in the MATLAB command ode45().
302
7.3.3
Particular Solution by Variation of Parameters
The vector version of variation of parameters is a generalization of those discussed in Sections 6.1 and 6.5. Again, assume there are q distinct real eigenvalues of the matrix D so that the homogeneous solution of (7.3.1) is a linear
combination of
xk (w) = f1 u1 hu1 w + f2 u2 hu2 w + + fq uq huq w =
Let [(w) be an q q matrix with column m equal to xm (w) um hum w
[(w) x1 (w) x2 (w) xq (w) =
(7.3.2)
Then the homogeneous solution can be written as the matrix [(w) times the
W
coe!cient vector c = f1 f2 fq
xk (w) = [(w)c=
(7.3.3)
As in Section 6.5, the variation of parameters method assumes the particular

solution can be expressed as
xs (w) = [(w)v(w)=
W
Here the column vector v(w) = y1 (w) y2 (w) yq (w)
is to be found so
that xs (w) = [(w)v(w) satises x0s = Dxs + f (w)=
In order to nd ym (w), expand the matrix-vector product by columns
xs (w) = [(w)v(w)
= y1 (w)x1 (w) + y2 (w)x2 (w) + + yq (w)xq (w)=
The derivative with respect to time may be computed by columns using the
product rule
x0s (w) = (y1 (w)x1 (w) + y2 (w)x2 (w) + + yq (w)xq (w))0
= (y1 (w)x1 (w))0 + (y2 (w)x2 (w))0 + + (yq (w)xq (w))0
= y1 x01 + y10 x1 + y2 x02 + y20 x2 + + yq x0q + yq0 xq =
Because each xm (w) um hum w is a homogeneous solution, x0m = Dxm , the above
can be rewritten using the column version of matrix-vector products
x0s (w) = y1 Dx1 + y10 x1 + y2 Dx2 + y20 x2 + + yq Dxq + yq0 xq
= (y1 Dx1 + y2 Dx2 + + yq Dxq ) + (y10 x1 + y20 x2 + + yq0 xq )
= (D[)v + [v0 =
Choose v so that xs = [v satises x0s = Dxs + f (w), which means
(D[)v + [v0
= D([v) + f (w)
= (D[)v + f (w)=
303
If [ = [(w) has an inverse, then v = v(w) should be chosen so that

[v0 = f (w) or v0 = [ 1 f (w)=
The column vector [ 1 f (w) can be integrated component by component to
obtain the desired vector v = v(w). A variation of parameters formula for a
particular solution of (7.3.1) is
Z w
xs = [(w)
[ 1 f =
(7.3.4)
The general solution is the sum of the homogeneous solution in (7.3.3) and the
particular solution in (7.3.4)
x(w) = xk (w) + xs (w)
= [(w)c + [(w)
[ 1 f >
(7.3.5)
which is a valid formula provided [(w) has an inverse and the components of
[ 1 f can be integrated.
A matrix [(w) in (7.3.2) formed by the columns of q linearly independent
homogeneous solutions is called a fundamental matrix of the system (7.3.1).
The fundamental matrix and its properties for dierent classes of eigenvalues
are more completely discussed in Chapter 9 in [7]. Here we will assume the
eigenvalues of D are distinct and real and then, with some theoretical eort,
one can show the eigenvectors are linearly independent. By using Theorem
3.7.3 and the properties of determinants
det([(w)) = det([u1 hu1 w u2 hu2 w uq huq w ])
= hu1 w hu2 w hu2 w det([u1 u2 uq ])
= hu1 w hu2 w hu2 w det([(0)) 6= 0=
Thus, [(w) has an inverse.
Example 7.3.5. Consider Example 7.3.2 and nd a particular solution by the
variation of parameters formula in (7.3.4). The matrix [(w) and its inverse are
6w

h
5hw hw
hw
1
[(w) =
(w)
=
and
[
@(7h5w )=
2h6w 5hw
2h6w
h6w
The matrix-vector product is

6w
(5@7)h6w (1@7)h6w
2w + 6
h (16w@7 + 6)
[ 1 f =
=
=
hw (2w@7 + 0)
6w + 12
(2@7)hw
(1@7)hw
Use integration by parts to compute the integrals in the vector
6w
Z w
h (8w@21 67@63)
1
[ f=
=
hw (2w@7 + 2@7)
304
The particular solution is

Z
xs (w) = [(w)
1
f=
2w@3 7@9
2w@3 32@9
>
which agrees with the particular solution in Example 7.3.2 that was found by
undetermined coe!cients.
The by-hand calculations in this and similar examples using the variation of
parameters formula (7.3.4) can be complicated, and one should consider using
a symbolic computing tool such as Maple or the symbolic toolbox in MATLAB.
The following MATLAB code will execute the formula in (7.3.4) for Example
7.3.5.
MATLAB code varpar.m
% This is a symbolic calculation to nd
% a particular solution using variation
% of parameters for a 2D system of DEs.
syms t
f = [2*t + 6;6*t + 12];
x1 = [exp(6*t); 2*exp(6*t)];
x2 = [exp(-t); -5*exp(-t)];
X = [x1 x2];
invX = inv(X)
vprime = invX*f
v = int(vprime,t)
xp = X*v
newxp = simple(xp)
Everyone should do at least one variation of parameter by-hand calculation,
and then conrm the calculation using a symbolic computation tool.
7.3.4
Initial Value Problem x0 = Dx + f(w) and x(0) = x0
Assume one has found q linearly independent homogeneous solutions and a

particular solution to (7.3.1). Form the fundamental matrix [(w) by using the
homogeneous solutions, and write the general solution as
x(w) = [(w)c + xs (w)
= f1 x1 (w) + f2 x2 (w) + + fq xq (w) + xs (w)=
The initial value problem is to nd x(w) that satises
x0 = Dx + f (w)and x(0) = x0 =
The coe!cients must be chosen so that the initial condition holds
x(0) = [(0)c + xs (0) = x0 =
305
This requires the solution of the algebraic system

[(0)c = x0 xs (0)=
(7.3.6)
When the columns in the fundamental matrix are linearly independent for all
0 w W , which is the case for q distinct real eigenvalues, then [(w) will have
an inverse and one can solve for the coe!cients in c=
Outline for x0 = Dx+f (w) and x(0) = x0 with Distinct Real Eigenvalues.
1.
Find the real eigenvalues um and eigenvectors um =
2.
Find the fundamental matrix
[(w) = x1 (w) x2 (w) xq (w) where xm (w) um hum w =

3.
4.
5.
Find xs (w) by undetermined coe!cients or variation of parameters.

Find the general solution x(w) = [(w)c + xs (w)=
Find c by solving [(0)c = x0 xs (0)=
Examples 7.3.1, 7.3.2 and 7.3.5 (continued). Find the solution of the
initial value problem

0

4 1
{1
2
6
{1
=
+
w+
and
10 1
6
12
{2
{2

{1 (0)
2
=
=
{2 (0)
1
Step 1. Both eigenvalues and eigenvectors were found in Example 7.3.1 where
determinants were used. One could also do this by using the MATLAB command
eig().
Step 2. The fundamental matrix [(w) is a 22 matrix formed by the eigenvalues
and eigenvectors
x1 (w) x2 (w)
[(w) =
u1 hu1 w u2 hu2 w
=
6w
1h
1hw
=
=
2h6w 5hw
Step 3. A particular solution was found in Example 7.3.2 using undetermined
coe!cients, and it was also found in Example 7.3.5 using variation of parameters

2w@3 7@9
=
xs (w) =
2w@3 32@9
Step 4. The general solution is the homogeneous solution plus the particular
solution
x(w) = [(w)c + xs (w)
6w

1h
f1
2w@3 7@9
1hw
=
+
2w@3 32@9
2h6w 5hw
f2
6w

w
1h
1h
2w@3 7@9
= f1
+ f2
+
=
2w@3 32@9
2h6w
5hw
306
Step 5. Find f1 and f2 by solving

f1
f2
1 1
2 5
f1
f2
[(0)c + xs (0) = x0

0 7@9
2
+
=
=
0 32@9
1
1
2
0 7@9
(

)
=
1
0 32@9

5@7 1@7
25@9
166@63
=
=
=
2@7 1@7
41@9
9@63

1 1
2 5
The solution to the initial value problem is

6w
{1 (w)
1hw
2w@3 7@9
1h
+ 9@63
+
=
= 166@63
2w@3 32@9
{2 (w)
2h6w
5hw
7.3.5
Application to Time Dependent Heat Conduction
Heat conduction models have been discussed in Sections 3.2, 3.3 and 7.1 (Example 7.1.4). The last version considered a thin wire with three interior cells
and attempted to nd the approximate temperature in the cells as a function of
time. As a simplication we assumed no internal heat source and zero temperature at the left and right ends of the wire. The model had the form u0 = Du
where D is 3 3 and u is a column vector with three temperatures.
The present model will be for four unknown temperatures and have nonzero
internal heat source such as from electrical resistance. The unknowns xl (w)
are approximations of the temperature in the wire at position {l = l{ where
{ = O@(q+1) and q = 4= By following the time dependent model in Subsection
7.1.4, the system of dierential equation is
5
60
x1
9 x2 :
:
f 9
7 x3 8
x4
65
x1
2 1 0
0
:
9
9
1
2
1
0
x
:9 2
= (N@({)2 ) 9
7 0 1 2 1 8 7 x3
0
0 1 2
x4
6
5
2
i + (N@({) )x0
:
9
i
:=
9
8
7
i
i + (N@({)2 )x5
5
6
:
:+
8
The left and right temperatures are x0 and x5 , and the initial temperatures for
the interior cells will be given.
Let the thermal properties be = 1, f = 1 and N = 0=001 with O = 10 so
that { = 2= Assume the left, right and initial temperature are always zero.
Model the increasing heat source in the wire by i = 0=1 0=1h0=1w = Then the
307
initial value problem is

5
u0
60
x1
9 x2 :
9
:
7 x3 8
x4
= Du + g + hh0=1w and u(0) = 0

65
5
2 1 0
0
9 1 2 1 0 : 9
:9
= =00025 9
7 0 1 2 1 8 7
0
0 1 2
5
6 5
6
0=1
0=1
9 0=1 : 9 0=1 : 0=1w
9
: 9
:
.
7 0=1 8 + 7 0=1 8 h
0=1
0=1
6
x1
x2 :
:+
x3 8
x4
The eigenvalues and eigenvectors can be found by the MATLAB command [U

R] = eig(A), which generates the four eigenvectors in the four columns of X
and the four eigenvalues in the diagonal positions of U. This means the homogeneous solutions, for the distinct real eigenvalue case, are X (:> m)hU(m>m)w . The
fundamental matrix is
[(w) = X (:> 1)hU(1>1)w X (:> 2)hU(2>2)w X (:> 3)hU(3>3)w X (:> 4)hU(4>4)w

so that [(0) = X= The general homogeneous solution is
[(w)c = f1 X (:> 1)hU(1>1)w + f2 X (:> 2)hU(2>2)w + f3 X (:> 3)hU(3>3)w + f4 X (:> 4)hU(4>4)w =
In order to nd a particular solution, use the method of undetermined coe!cients by assuming the particular solution has the form a + bh0=1w = Choose
the vector coe!cients so that the system of dierential equations holds
(a + bh0=1w )0 = D(a + bh0=1w ) + g + hh0=1w
0 + bh0=1w (0=1) = Da + Dbh0=1w + g + hh0=1w
= (Da + g) + (Db + h)h0=1w =
Match the like terms to obtain two vector equations
0 = Da + g and b(0=1) = Db + h=
The rst equation is easily solved by a = D1 g= The second equation can
be written as (=1)Lb = Db + h or as (D + (0=1)L)b = h= If 0=1 is not an
eigenvector, then (D(0=1)L) has an inverse and the second equation is solved
by b = (D + (0=1)L)1 h=
The initial conditions are satised by solving [(0)c = u0 (a + bh0=1(0) )
= 0 (a + b)= Since [(0) = X> the solution is c = X 1 (a + b)= Once these
computations have been done, the solution to the initial value problem is
u = [(w)c + a + bh0=1w
= f1 X (:> 1)hU(1>1)w + f2 X (:> 2)hU(2>2)w + f3 X (:> 3)hU(3>3)w + f4 X (:> 4)hU(4>4)w
+a + bh0=1w =
308
Figure 7.3.1: Heat Diusion in Thin Wire

The calculations have been implemented in the MATLAB code le heat4.m,
and the output is given in the Figure 7.3.1. Since the eigenvalues (=0009045,
=0006545, =0003455 and =0000955) are all negative, the homogeneous solution decays to zero. The particular solution decays to a and, so, the solution to
the initial problem converges to a = [800 1200 1200 800]W = This is suggested
by the right side of the graph in Figure 7.3.1.
7.3.6
1.
Exercises
Find the eigenvalues

of the following matrices:

2 1
(a).
D=
has repeated eigenvalues,
1@4 3
2 1
(b).
D=
has complex eigenvalues and
1
2

1 2
(c).
D=
has a zero eigenvalue and no inverse.
1@2 1
2.
Use undetermined coe!cients

to nd the particular solution of
2
3
x0 = Dx + f where D =
with
3 2
1
2
(a).
f=
+
w and
2

3
1
(b).
f=
h2w =
1

3.
309
Consider the system of dierential equations in exercise 2.

(a).
Find the eigenvalues and eigenvectors.
(b).
Find the fundamental matrix and its inverse.
(c).
Use variation of parameters to nd a particular solution in exercise
2a.
(d).
Use variation of parameters to nd a particular solution in exercise
2b.
4.
Consider the system of dierential equations in exercise 2 with initial
condition
3
x(0) =
=
5
(a).
Solve the initial value problem for f in exercise 2a.
(a).
Solve the initial value problem for f in exercise 2b.
0
5.
Solve
value
the initial
problem
x = Dx + f where

1 1
7
1
1
D=
>f=
+
w and x(0) =
=
2 4
2
3
0
0
6.
Solve
the initial
value problem
x =Dx + f where
3 1
4
1
1
D=
>f=
+
h3w and x(0) =
=
1 3
2
3
1
7.
Solve
value
x0 =
Dx +f where
the initial
problem

4 1
4
1
1
D=
>f=
cos(2w) +
sin(2w) and x(0) =
=
2 1
2
3
1
0
8.
Solve
5 the initial value
6 problem
5 x
6 = Dx + f where
5 6
3 1
0
4
1
7
8
7
8
7
1
3
1
5
D=
>f=
w and x(0) = 2 8 =
0
1 3
4
3
9.
In exercise 3 conrm your by-hand calculations using a symbolic computation tool.
10.
Use MATLABs command dsolve() to conrm any by-hand calculations
in exercises 4-8.
11.
Consider the application to heat conduction in a wire as implemented in
the MATLAB code heat4.m. Modify this code so that there are q = 5 unknown
temperatures. The matrix now will be 5 5, there will be ve dierential
equations and the column vectors must have ve components.
(a).
Compare the graphical output with the q = 4 output in Figure
7.3.1.
(b).
Experiment with the length of the time interval by adjusting the
increment size in the time loop from w = (n 1) 200 to w = (n 1) 500.
Observe the convergence of the approximate temperatures as time increases.
Chapter 8
Image Processing in Space

Domain
Chapters eight and nine use matrices to represent images. Basic matrix operations can be used to alter the image contrast, light-dark and blurred-sharpened
qualities of the image. In this chapter the matrix operations will be on the
original matrix, which is often called the space domain. An interesting variety
of images will be enhanced. A very nice general reference to image processing is
[2], where the indicated www site links to many interesting images. The pollen,
aerial, moon and micro chip images were used in [2], and the Mars Rover image
was taken from NASA [11].
8.1
Matrices and Images
Most images have some initial distortion or equipment noise associated with
them. Because one can some times determine the nature of the distortion, it
may be possible to eliminate or reduce the undesirable attributes in the image.
As indicated in Subsection 2.5.6, digital images can be associated with matrices
of integers. The general plan is to convert the image le to a matrix, manipulate
the matrix and then convert the manipulated matrix into an enhanced image.
8.1.1
Enhanced Images
The following pairs of images have been enhanced by simple operations on the
image matrix, which will be discussed in this and the next chapter. Figures
8.1.1 and 8.1.2 are images of pollen where the rst gure lacks contrast and is
dark. The pollen image in Figure 8.1.1 is provided courtesy of Roger Heady,
Australian National University. Figure 8.1.2 was created by using a piecewise
linear transformation and sharpening.
311
312
CHAPTER 8. IMAGE PROCESSING IN SPACE DOMAIN
Figure 8.1.1: Pollen Image
Figure 8.1.2: Enhanced Pollen Image
8.1. MATRICES AND IMAGES
313
Images in Figures 8.1.3 and 8.1.4 are an aerail photo, which initially is too
bright in some areas. The aerail image in Figure 8.1.3 is provided courtesy of
NASA. The enhanced version was obtained by a power array transformation
and gives a clearer image of the water channel.
Figure 8.1.3: Aerial Photo
Figure 8.1.4: Enhanced Aerial Photo
314
The Mars Rover image in Figure 8.1.6 has been enhanced to see the darker
regions in Figure 8.1.5. The Mars image in Figure 8.1.5 is provided courtesy of
NASA.
Figure 8.1.5: Mars Rover Photo
Figure 8.1.6: Enhanced Mars Rover Photo
315
The second image of the moon in Figure 8.1.8 has sharper craters than
the original moon image in Figure 8.1.7. The moon image in Figure 8.1.7 is
provided courtesy of NASA.
Figure 8.1.7: Moon
Figure 8.1.8: Sharper Moon Image
316
8.1.2
Grayscale Images
Grayscale images are associated with p q matrices whose components are

integers. For 8-bit images the integers range from 0 to 255 = 28 1, and for
16-bit images they range from 0 to 65535 = 216 1. The black image pixel
is associated with 0, and the white image pixel is associated with 255 (8-bit)
or 65535 (16-bit). In MATLAB one can view the image in several ways.
First, just inspect the matrix components. Second, use the MATLAB command
mesh() to generate a surface of the image where the indices of the matrix are
on the xy-plane and the intensity of the image is on the z-axis. Third, one can
map the matrix into a standard image le such as a *.jpg le. This can be done
by the MATLAB command imwrite(). The inverse of the imwrite() is imread(),
which generates a 8-bit integer matrix from an image le.
Example 8.1.1. We will create alphabetical letters from 50 40 matrices.
The letter "C" can be created by dening a 50 40 matrix to initially be zero
and then nonzero for some components to form the letter. This will produce a
letter with black background and with lighter regions to form the letter. The
following MATLAB function denes the letter "C" where the lighter region has
an input value equal to j.
MATLAB code letterc.m
function letc = letterc(g)
letc = zeros(50,40);
letc(10:40,4:8) = g;
letc(10:14,8:36) = g;
letc(36:40,8:36) = g;
Figure 8.1.9 is the mesh plot of the matrix C, which was generated by the
MATLAB function letterc(200). The MATLAB command imwrite(newC, C.jpg)
maps the matrix newC (C with 8-bit integers) into the image le C.jpg, which
is viewed in Figure 8.1.10. Figure 8.1.11 is the "negative" of this letter.
AA
AA
AA
AA
C = letterc(200);
newC =uint8(C);
newC(5:20,1:12)
ans =
000000000000
000000000000
000000000000
000000000000
000000000000
0 0 0 200 200 200 200 200
0 0 0 200 200 200 200 200
0 0 0 200 200 200 200 200
0 0 0 200 200 200 200 200
0 0 0 200 200 200 200 200
0 0 0 200 200 200 200 200
200 200
200 200
200 200
200 200
200 200
0000
200
200
200
200
200
200
200
200
200
200
317
Figure 8.1.9: Plot of the Matrix C
0 0 0 200 200 200 200 200 0 0 0 0

0 0 0 200 200 200 200 200 0 0 0 0
0 0 0 200 200 200 200 200 0 0 0 0
0 0 0 200 200 200 200 200 0 0 0 0
0 0 0 200 200 200 200 200 0 0 0 0
AA mesh(C)
AA imwrite(newC, C.jpg)
Figure 8.1.10:
Image of Letter C
AA negC = 255 - letterc(200);

AA newnegC = uint8(negC)
AA imwrite(newnegC,mac8111.jpg);
318
Figure 8.1.11:
Negative Image
Example 8.1.2. One can augment the matrices of letters to form words or
other objects. In this example four letters are created by functions similar
to the letter function letterc(g) to form the matrix qfvx and then the image
NCSU. The mesh plot of the matrix is given in Figure 8.1.12, the image NCSU
with black background is in Figure 8.1.13 and the "negative" of the image
NCSU with white background is in Figure 8.1.14. The "negative" is created
by replacing the components in the matrix ncsu by 255 qfvx(l> m), that is,
qhjqfvx(l> m) = 255 qfvx(l> m)= The following MATLAB code imagncsu.m was
used to generate these gures.
Figure 8.1.12:
Matrix NCSU
MATLAB code imagncsu.m

mesh(ncsu);
newncsu = 20*ncsu;
newncsu = newncsu(50:-1:1,:);
negncsu = 255*ones(size(newncsu))-newncsu;
319
negncsu1 = uint8(negncsu);
imwrite(negncsu1, negncsu.jpg);
Figure 8.1.13:
Figure 8.1.14:
8.1.3
Image of NCSU
Negative Image of NCSU
Color Images
Color images are decomposed into red, green and blue. Three p q matrices,
or a 3D array whose dimensions are p> q and 3> are used to represent the color
image. In MATLAB this is given by duud|(:> :> 1) for the red matrix, duud|(:> :> 2)
for the green matrix and duud|(:> :> 3) for the blue matrix. The MATLAB command imwrite() maps to the image array into the array.jpg le that is associated
with color image. The following MATLAB code le color_imagncsu.m can be
used to generate color versions of the image NCSU.
MATLAB code color_imagncsu.m
red = 10; % 220
green = 10; % 100
blue = 250; % 10
newncsu = ncsu(50:-1:1,:);
redncsu = red*newncsu;
greenncsu = green*newncsu;
bluencsu = blue*newncsu;
colorncsu(:,:,1) = redncsu;
colorncsu(:,:,2) = greenncsu;
colorncsu(:,:,3) = bluencsu;
colorncsu1 = uint8(colorncsu);
imwrite(colorncsu1, colorncsu.jpg);
320
Figure 8.1.15:
8.1.4
Center Grain in Pollen
Cropped Images
In order to cut or crop unwanted parts of an image, one needs to rst nd the
image matrix from the image le. This can be done by using the MATLAB
command imread(). Once the image matrix is found, the appropriate part of
the image matrix can be selected. Then map this back into a smaller image
le. For example, if one wants to examine the pollen grain in the center of the
image in Figure 8.1.1, use the following MATLAB commands.
AA pollenmat = imread(pollen.jpg)
AA size(pollenmat)
ans =
500 500
AA imwrite(pollenmat(200:400,100:400),mac8115.jpg)
Figure 8.1.15 contains this cropped image. In order to distinguish the dark
and white areas, there needs to be more separation between the dark and light
components in the image matrix. This is known as "contrast" in the image
matrix and will be more carefully discussed in the next section.
8.1.5
1.
Exercises
Consider the letter function letterc(g).

(a).
Experiment with dierent input values in the letter function letterc(g).
(b).
Modify the shape of the letter "C" generated by letterc(g).
(c).
Modify the background shade of the letter.
2.
Consider the image of NCSU.
(a).
Dene additional letter functions for the letters in your rst name.
(b).
Modify the MATLAB code imagncsu.m to generate your rst name
image.
(c).
Modify the background shade for your rst name image.
3.
Experiment with dierent colors in the image NCSU as generated in
color_imagncsu.m.
8.2. CONTRAST AND HISTOGRAMS
321
4.
Modify color_imagncsu.m to generate a color image of your rst name.
5.
Crop the aerial photo given in aerial.jpg so that the landing strip is the
main part of a smaller image le.
6.
Crop the Mars photo given in mars.jpg so that the rock formation in the
center right is the central part of a smaller image le.
8.2
Contrast and Histograms
Histograms of an image matrix will be introduced to quantify brightness and

contrast of an image. The shift, piecewise linear and power matrix transformations of the image matrix will be used to adjust the brightness or contrast of
an image. These will be applied to enhance the images of pollen, aerial photo
and Mars.
8.2.1
Histogram of an Image
Images often are said to be light or dark or to lack contrast. In an attempt

to be more precise about this, we use the matrix representation of an image.
Each pixels image intensity is recorded by an entry of its matrix. If the entry
is small (large), then the pixel is dark (light). If there are a large number of
dark (light) pixels, then one may say the image is dark (light). If most of the
entries have similar values, then the image lacks contrast.
Denition 8.2.1. Let n = x(l> m) be an integer from 0 to 255 and represent
a shade of a pixel at the location (l> m) in an p q image matrix, x. Dene a
histogram of the 8-bit image x to be the function from integers 0 : 1 : 255 to
the integers 0 : 1 : pq given by
k(n) = number of pixels whose shade equals n, that is, x(l> m) = n=
If the image is dark (light), the graph of the histogram will be shifted to
the left (right). If the image lacks contrast, the graph of the histogram will be
clustered. In order to nd the histogram of an image matrix, one must total the
number of components that have a particular shade. This can be done by using
three nested loops where the two outside loops range over the pixels (indices of
the image matrix) and the inner most loop ranges over all possible shades. The
rst example illustrates this for the pollen image in Figure 8.1.1.
Example 8.2.1. Find the histogram for the pollen image in pollen.jpg. First,
we nd the 8-bit image matrix sroohqpdw by using the MATLAB command
imread(), and second, the three nested loops are used to construct the histogram function. The following MATLAB code pollenhist.m does this and then
generates the graph of the histogram in Figure 8.2.1.
MATLAB code pollenhist.m
% This code creates a histogram for pollen.jpg and
% plots its graph.
322
Figure 8.2.1: Histogram of Pollen Image
clear;
% Creates an image matrix for pollen.jpg.
pollenmat = imread(pollen.jpg);
u = double(pollenmat);
[nx ny] = size(u)
nshades = 256;
hist = zeros(nshades,1);
for i = 1:nx
% creates the histogram, hist
for j = 1:ny
for k = 0:nshades-1
if u(i,j) == k
hist(k+1) = hist(k+1)+1;
end
end
end
end
plot(hist);
The histogram in Figure 8.2.1 indicates low contrast and a darker image
because the pixels are clumped together and have lower shades. In subsequent
subsections we will transform the image matrix so that the histogram is more
evenly distributed.
8.2.2
323
Shift Transformation
The shift transform either increases or decreases all of the pixels (components
of the image matrix) by the same amount. The histogram in Figure 8.2.1 for
the pollen image indicates that most of the components have shades between
90 and 150, and so a shift to the right equal to 100 would tend to brighten the
image.
Denition 8.2.2. Let x be an 8-bit image matrix and let vkli w be a positive
or negative integer. The shift transformation of the image matrix is a mapping
to another 8-bit image matrix W x whose components are
;
0>
if x(l> m) + vkli w ? 0
?
x(l> m) + vkli w>
if 0 x(l> m) + vkli w 255 =
(W x)(l> m)
=
if x(l> m) + vkli w A 255
255>
The truncation below 0 or above 255 can seriously distort the associated
image. The MATLAB command uint8() does this truncation as illustrated in
the following trivial calculations:
AA u = [-100.3 45.0 257; 23.7 150.0 301.1]
-100.3000 045.0000 257.0000
023.7000 150.0000 301.1000
AA shift_u = u + 100
-00.3000 145.0000 357.0000
123.7000 250.0000 401.1000
AA new_u = uint8(shift_u)
000 145 255
124 250 255
In terms of the MATLAB command uint8(), the shift transformation can be
written as
W x = uint8(x + vkli w) or in component form
(W x)(l> m) = uint8(x(l> m) + vkli w)=
Example 8.2.2. This example illustrates the shift transformation applied to
the image matrix of pollen.jpg. The following MATLAB code pollenlight.m is
a shift to a brighter image than in Figure 8.1.1, generates a new histogram in
Figure 8.2.2 and a new image le in Figure 8.2.3.
MATLAB code pollenlight.m
% This code uses the shift transformation to either
% brighten or darken all of the image.
clear;
[nx ny] = size(u)
324
Figure 8.2.2: Histogram of Lighter Pollen Image

nshades = 256;
shift = 100; % Transform by shifting to light.
% shift = -50; % Transform by shifting to dark.
ulight = u + ones(nx,ny)*shift;
for i=1:nx
for j=1:ny
for k=0:nshades-1
if ulight(i,j)==k
hist(k+1)=hist(k+1)+1;
end
end
end
end
plot(hist);
pollen_light = uint8(ulight);
imwrite(pollen_light, pollenlight.jpg);
8.2.3
Piecewise Linear Transformation
In order to spread the shades (increase the contrast) in the pollen image in
Figure 8.1.1 so that its histogram in Figure 8.2.1 is less clumped in the center,
we can use a linear map applied to the pixels in the center region. If the slope
of the linear map is larger than one, then the shades in the original image
325
Figure 8.2.3: Lighter Pollen Image

will be spread over a larger range of shades. In order to have the transformed
image matrix have components between 0 and 255, we apply a piecewise linear
transformation to each component of the image matrix. An illustration of this
is in Figure 8.2.4, where one can choose dierent values for the points (u1 > v1 )
and (u2 > v2 )= The MATLAB function le for a piecewise linear function is in
spread.m.
MATLAB function spread.m
function uspread = spread(r1,s1,r2,s2,u)
if (u? r1)
uspread = ((s1-0)/(r1-0))*u;
end
if ((uA=r1) & (u?= r2))
uspread = ((s2 - s1)/(r2 - r1))*(u - r1)+ s1;
end
if (uAr2)
uspread = ((255 - s2)/(255 - r2))*(u - r2) + s2;
end
Since the values of the piecewise linear function may not be 8-bit integers, they
must be rounded or truncated to be integers between 0 and 255.
Denition 8.2.3. The piecewise linear transformation of an image matrix x is
a 8-bit image matrix Wsz x whose (l> m) components are given by the composition
326
Figure 8.2.4: Piecewise Linear Function

of the piecewise linear function W (v) and the 8-bit valued MATLAB function
uint8()
(Wsz x)(l> m) uint8(W (x(l> m))=
A simple example of this is given by the following MATLAB calculations:
AA u = [60 80 150; 135 150 250]
AA newu = [spread(80,10,140,245,60) spread(80,10,140,245,80)
spread(80,10,140,245,150);
spread(80,10,140,245,135) spread(80,10,140,245,150)
spread(80,10,140,245,250)]
newu =
007.5000 010.0000 245.8696
225.4167 245.8696 254.5652
AA TpwU = uint8(newu)
TpwU =
008 010 246
225 246 255
Example 8.2.3. A more interesting illustration of this transformation is given
in the MATLAB code pollenspread.m where the pollen image is transformed to
have much better contrast. This code uses the function le spread.m to do a
piecewise linear transform of the image, creates a histogram in Figure 8.2.5 and
generates the enhanced image in Figure 8.2.6.
327
MATLAB code pollenspread.m

% This le enhances the image pollen.jpg by using the
% piecewise linear transformation as given in the function
% le spread().
clear;
[nx ny] = size(u)
nshades = 256;
r1 = 80; s1 = 10; % Transformation by piecewise linear function.
r2 = 140; s2 = 245;
for i = 1:nx
for j = 1:ny
uspread(i,j) = spread(r1,s1,r2,s2,u(i,j));
end
end
pollen_spread = uint8(uspread);
for i = 1:nx
for j = 1:ny
for k = 0:nshades-1
if pollen_spread(i,j) == k
hist(k+1) = hist(k+1) + 1;
end
end
end
end
plot(hist);
imwrite(pollen_spread, pollenspread.jpg);
8.2.4
Power Transformation
On the unit interval {2 is concave upwards and also has values on the unit
interval. The square root function {1@2 is concave downwards and has values
on the unit interval when { is in the unit interval. By replacing the { by a
normalized image component x(l> m)@255, one can adjust the relative brightness
and darkness of the image. In order to ensure the image is in the 8-bit integer
range, one must use a scaled power 255(x(l> m)@255)s and the uint8() function.
Denition 8.2.4. The power transformation of an image matrix x is a 8bit image matrix Ws x whose (l> m) components are given by the composition of
the power function {s with positive s and the 8-bit valued MATLAB function
uint8()
(Ws x)(l> m) uint8(255(x(l> m)@255)s )=
A trivial illustration is done using the array operations in MATLAB:
328
Figure 8.2.5: Histogram for Enhanced Pollen Image
Figure 8.2.6: Higher Contrast Pollen Image
329
AA u = [ 16 64 144; 50 225 40]

u=
16 064 144
50 225 040
AA newu = (u/255).^.5
newu =
0.2505 0.5010 0.7515
0.4428 0.9393 0.3961
AA TpU = uint8(255*newu)
TpU =
064 128 192
113 240 101
Example 8.2.4. The following MATLAB code aerialpower.m with s = 2 was
used to enhance the aerial image in Figures 8.1.3 and 8.1.4.
MATLAB code aerialpower.m
% This code uses the power transformation to enhance
% the photo in aerial.jpg.
clear;
aerialmat = imread(aerial.jpg);
% aerial photo
u = double(aerialmat);
[nx ny] = size(u)
c = 1.;
power = 2;
newu =255*c*(u/255).^power;
aeria11 = uint8(newu);
imwrite(aeria11, aerial1.jpg);
The MATLAB code aerialpower.m can be easily modied to use the power
transformation on other images. Figures 8.2.7 and 8.2.8 were generated by
the MATLAB code marspower.m, which transforms the Mars Rover image
mars3.jpg by the power method with s = 1@2 and s = 2> respectively= The
image in Figure 8.2.7 is much brighter because the square root function is concave down and on the unit interval it is larger than the square function.
8.2.5
Exercises
1.
Consider the MATLAB code le pollenhist.m. Alter it to nd the histogram of the Mars Rover image in mars3.jpg.
2.
Consider the MATLAB code le pollenhist.m. Alter it to nd the histogram of the aerial image in aerail.jpg.
3.
Apply the shift transformation with vkli w = 40 and

275 240=7 210

x=
=
195=2
57
37=1
330
Figure 8.2.7: Mars Rover Image Using Power 1/2
Figure 8.2.8: Mars Rover Image Using Power 2
8.3. BLURRING AND SHARPENING
331
4.
Consider the MATLAB code le pollenlight.m. Experiment with dierent values of vkli w = 100> 50> 150 and 200= Note the histograms and the
corresponding images.
5.
Apply the piecewise linear transformation with (u1 > v1 ) = (20> 90), (u2 > v2 )
= (180> 200) and

275 240=7 210

x=
=
195=2
57
37=1
6.
Consider the MATLAB code le pollenspread.m. Experiment with dierent values of (u1 > v1 ) and (u2 > v2 ) in the function spread.m. Note the histograms
and the corresponding images.
7.
Consider the MATLAB code le pollenspread.m. Alter this to apply the
piecewise linear transformation to the image le aerial.jpg. Experiment with
dierent values of (u1 > v1 ) and (u2 > v2 ) in the function spread.m. Note the histograms and the corresponding images.
8.
Consider the MATLAB code le pollenspread.m. Alter this to apply the
piecewise linear transformation to the image le mar3.jpg. Experiment with
dierent values of (u1 > v1 ) and (u2 > v2 ) in the function spread.m. Note the histograms and the corresponding images.
9.
Apply the power transformation with s = 0=5, 1=5 and

275 240=7 210

x=
=
195=2
57
37=1
10.
Consider the MATLAB code le aerialpower.m and experiment with
additional values of s = 0=3> 0=5> 0=7 and 0=9= Note the histograms and the
corresponding images.
8.3
Blurring and Sharpening
The distortion of an image may be a result of several factors such as deection

of light rays, defective equipment
and electronic noise. One model is related to heat diusion where the heat
energy spreads from high values to lower values. This is analogous to high
intensity light spreading to lower intensities, which results in a sharp image
becoming less sharp or blurred. By reversing the "diusion" process, that is,
use backward or negative time steps, one can deblur or sharpen images. Several
examples will be given and include sharpening the pollen and moon images.
8.3.1
Images in One Dimension
Blurring of a 1D image can be modeled by the explicit nite dierence model of

heat where the 1D image is viewed as the initial temperature within a long thin
wire. A simple explicit time discrete model for the approximate temperature
332
xnl x(l{> nw) was formulated in Section 3.2

= xnl + (xnl1 2xnl + xnl+1 ) with
xn+1
l
N w
=
f {2
The reverse of heat diusion is to use negative time steps and to start with the
temperature distribution in the future. This suggests that deblurring an image
may be modeled by starting with a blurred image as a future "temperature"
and apply the explicit nite dierence model with negative time steps. In the
blurring of the image, one must be careful to choose the time step small enough
so that 1 2 is positive. The backward iterations eventually start to oscillate
and give badly distorted images. The rst example illustrates this for a simple
1D image.
Example 8.3.1. The image is given by a simple piecewise constant function,
which could be viewed as proles of two rectangular buildings. Blurring could
be caused by moisture in the air, but here it will be simulated by the forward
heat diusion model of blurring. Once this is done, negative time steps will be
used in an attempt to deblur the distorted image. The MATLAB code blur1d.m
is an implementation of this.
MATLAB code blur1d.m
clear;
clf;
% This code illustrates possible blurring and deblurring in 1D.
% It is motivated by heat diusion in 1D.
% Blurring is modeled by diusion of heat in positive time steps.
% Deblurring is done by diusion of "heat" in negative time steps.
% In order to see the changes, push a key.
u = zeros(101,1);
newu = u;
u(30:50) = 10;
u(70:73) = 15;
dt = .00001;
dx = 1/100;
%
% Forward in "time" or "blurring"
%
subplot(1,3,1)
plot(u)
axis([0 101 0 16]);
title(Image)
gure(1)
pause
subplot(1,3,2)
for k = 1:10
for i = 2:100
333
newu(i) = u(i) + (dt/(dx*dx))*(u(i-1) -2*u(i) + u(i+1));

end
plot(newu);
axis([0 101 0 16]);
title(Blurred Image)
k
pause;
u = newu;
end
%
% Backward in "time" or "deblurring"
%
subplot(1,3,3)
plot(u)
for k = 1:10
for i = 2:100
newu(i) = u(i) - (dt/(dx*dx))*(u(i-1) -2*u(i) + u(i+1));
end
plot(newu);
axis([0 101 0 16]);
title(Deblurred Image)
k
pause;
u = newu;
end
The graphical output is given in Figure 8.3.1. The left graph is the unblurred
image and the center graph is the blurred image after n = 10 "time" steps
with w = 0=00001 and { = 0=01= These values have been chosen so that
the blurring and deblurring process gives reasonable results. The graph in the
right side of Figure 8.3.1 is the deblurred image after n = 10 steps. Note
the additional "peaks" in the deblurred image. A more accurate deblurred
image is found at n = 8 steps, and the reader should run the code to see this.
So, the deblurring process can be too aggressive and insert nonexistent image
components.
8.3.2
Images in Two Dimensions
Blurring of a 2D image can also be modeled by the explicit nite dierence

model of heat diusion where the 2D image is viewed as the initial temperature
within a thin plate. Heat is allowed to diuse in both the { and | directions
so that a simple explicit time discrete model for the approximate temperature
334
Figure 8.3.1: Deblurred 1D Image

xnlm x(l{> m|> nw) is
xn+1
lm
with {
= xnlm + { (xnl1>m 2xnlm + xnl+1>m )

+| (xnl>m1 2xnlm + xnl>m+1 )
N w
N w

and |
=
f {2
f | 2
The reverse of heat diusion is to use negative time steps and to start with the
temperature distribution in the future. As in the 1D case deblurring an image
may be modeled by starting with a blurred image as a future "temperature"
and apply the explicit nite dierence model with negative time steps. In the
blurring of the image, one must choose the time step small enough so that 1
2{ 2| is positive. Like the 1D example backward iterations eventually start
to oscillate and give badly distorted images. The second example illustrates the
blurring and deblurring of the image NCSU.
Example 8.3.2. Consider the image NCSU that was created in Subsection
8.1.2. The following MATLAB code, blur2d_ncsu.m, illustrates deblurring in
2D using negative time steps. The reader will nd it informative to experiment
with the time step sizes and the length of the outer time loops. The mesh()
plots are also interesting to observe, but are not printed out in this text. Figures
8.3.2, 8.3.3 and 8.3.4 are original, blurred and deblurred image les, respectively.
The deblurring is still a little fuzzy relative to the original image.
335
MATLAB code blur2d_ncsu.m

clear;
clf;
% This code illustrates possible blurring and deblurring in 2D.
% It is motivated by heat diusion in 2D.
% Blurring is modeled by diusion of heat in positive time steps.
% Deblurring is done by diusion of "heat" in negative time steps.
u = ncsu;
newu = u;
[nx ny] = size(ncsu)
lightncsu = 20*u;
lightncsu = lightncsu(nx:-1:1,:);
lightncsu1 = uint8(lightncsu);
imwrite(lightncsu1, mac832.jpg);
gure(1)
subplot(1,3,1)
mesh(u)
axis([0 ny 0 nx 0 15]);
title(Image NCSU)
dt = .00006;
dx = 1/(nx-1);
dy = 3.2/(ny-1);
subplot(1,3,2)
%
% Forward in "time" or "blurring"
%
for k = 1:20
for j = 2:ny-1
for i = 2:nx-1
newu(i,j) = u(i,j) + ...
(dt/(dx*dx))*(u(i-1,j) -2*u(i,j) + u(i+1,j))+...
(dt/(dy*dy))*(u(i,j-1) -2*u(i,j) + u(i,j+1));
end
end
mesh(newu);
axis([0 ny 0 nx 0 15]);
title(Blurred NCSU)
k
pause;
u = newu;
end
blurncsu = 20*u;
blurncsu = blurncsu(nx:-1:1,:);
blurncsu1 = uint8(blurncsu);
336
Figure 8.3.2: Original NCSU
Figure 8.3.3: Blurred NCSU

imwrite(blurncsu1, mac833.jpg);
subplot(1,3,3)
%
% Backward in "time" or "deblurring"
%
for k = 1:16
for j = 2:ny-1
for i = 2:nx-1
newu(i,j) = u(i,j) - ...
(dt/(dx*dx))*(u(i-1,j) -2*u(i,j) + u(i+1,j))-...
(dt/(dy*dy))*(u(i,j-1) -2*u(i,j) + u(i,j+1));
end
end
mesh(newu);
axis([0 ny 0 nx 0 15]);
title(Deblurred NCSU)
k
pause;
u = newu;
end
deblurncsu = 20*u;
deblurncsu = deblurncsu(nx:-1:1,:);
deblurncsu1 = uint8(deblurncsu);
imwrite(deblurncsu1, mac834.jpg);
8.3.3
Sharpen the Pollen Image
The original pollen image pollen.jpg in Figure 8.1.1 was enhanced by increasing
the contrast via a piecewise linear transformation to the image in Figure 8.2.6.
The image in Figure 8.2.6 will be deblurred or sharpened by using negative time
337
Figure 8.3.4: Deblurred NCSU

steps and the 2D heat diusion method as implemented in the Example 8.3.2.
The MATLAB code pollensharpen.m does both the contrast enhancement in
Figure 8.3.5 and subsequent sharpening in Figure 8.3.6. The reader will nd it
interesting to compare the last image with the original image in Figure 8.1.1
and to experiment with dierent values of lqwhqvlw| = 1=05 and frhi i = 0=05
and the length of the k-loop.
MATLAB code pollensharpen.m
clear;
% This code is a composition of the piecewise linear
% transformation and the negative time step diusion
% operation. The pollen image will have an increased contrast,
% be brighter and be sharper.
[nx ny] = size(u)
nshades = 256;
%
% Transformation by piecewise linear function.
%
r1 = 80; s1 = 10;
r2 = 140; s2 = 245;
for i = 1:nx
for j = 1:ny
if (u(i,j)? r1)
uspread(i,j) = ((s1-0)/(r1-0))*u(i,j)
end
if ((u(i,j)A=r1) & (u(i,j)?= r2))
uspread(i,j) = ((s2 - s1)/(r2 - r1))*(u(i,j) - r1)+ s1;
end
if (u(i,j)Ar2)
uspread(i,j) = ((255 - s2)/(255 - r2))*(u(i,j) - r2) + s2;
end
end
end
pollenspread = uint8(uspread);
imwrite(pollenspread, mac835.jpg);
%
338
Figure 8.3.5: Increased Contrast Pollen
% Brighten and sharpen the image.

%
u = double(pollenspread);
newu = u;
coe = .06;
intensity = 1.05;
for k = 1:5
for j = 2:ny-1
for i = 2:nx-1
newu(i,j) = intensity*u(i,j) ...
- coe*(u(i-1,j) -2*u(i,j) + u(i+1,j))...
- coe*(u(i,j-1) -2*u(i,j) + u(i,j+1));
end
end
u = newu;
end
pollen_sharpen = uint8(u);
imwrite(pollen_sharpen, mac836.jpg);
339
Figure 8.3.6: Brighter and Sharper Pollen
8.3.4
Sharpen the Moon Image
The MATLAB code moonsharpen.m is a variation of the pollensharpen.m code,

and also it is used to both brighten and sharpen the moon image moon.jpg. The
original moon image is brightened by 20% by using lqwhqvlw| = 1=2, and then
one negative time step is used to sharpen the image with frhi i = 1. Again, the
reader will nd it interesting to experiment with dierent values of lqwhqvlw|,
frhi i and the length of the k-loop. The original moon image is in Figure 8.3.7,
and the brightened and sharpened moon image is in Figure 8.3.8.
MATLAB code moonsharpen.m
clear;
% The original moon image is brighten and sharpened. The
% brightening is controlled by intensity, and the sharpening
% is controlled by coe and the k-loop.
moonmat = imread(moon.jpg);
u = double(moonmat);
[nx ny] = size(u)
nshades = 256;
%
% Sharpen the image.
%
coe = 1;
340
Figure 8.3.7: Original Moon Image

intensity = 1.2;
for k = 1:1
for j = 2:ny-1
for i = 2:nx-1
newu(i,j) = intensity*u(i,j) - ...
coe*(u(i-1,j) -2*u(i,j) + u(i+1,j))-...
coe*(u(i,j-1) -2*u(i,j) + u(i,j+1));
end
end
u = newu;
end
moon_sharpen = uint8(u);
imwrite(moon_sharpen, mac838.jpg);
8.3.5
Exercises
1.
Experiment with blur1d.m and vary the parameters for the time step and
the k-loop.
(a).
Vary gw = 0=00005> 0=00010 and 0=00020=
(b).
Vary n = 1 : 5> 1 : 10 and 1 : 20=
2.
Experiment with blur2d_ncsu.m and vary the parameters for the time
step and the k-loop.
341
Figure 8.3.8: Brightened and Sharpened

(a).
Vary gw = 0=0003> 0=0006 and 0=00012=
(b).
Vary n = 1 : 10> 1 : 20 and 1 : 30=
(c).
Can you do a better job of deblurring the image in Figure 8.3.3?
3.
Experiment with pollensharpen.m and vary the parameters lqwhqvlw|>
frhi i and the k-loop.
(a).
Vary lqwhqvlw| = 1=00> 1=04 and 1=08=
(b).
Vary frhi i = 0=02> 0=04 and 0=06=
(c).
Vary n = 1 : 5> 1 : 10 and 1 : 20=
4.
Experiment with moonsharpen.m and vary the parameters lqwhqvlw|>
frhi i and the k-loop.
(a).
Vary lqwhqvlw| = 1=1> 1=2 and 1=4=
(b).
Vary frhi i = 0=5> 1=0 and 1=5=
(c).
Vary n = 1 : 5> 1 : 10 and 1 : 20=
5.
Consider the sharpened image for pollen in Figure 8.3.6. Use a variation of lqwhqvlw| in moonsharpen.m to modify pollensharpen.m so that the
sharpened image will be brighter than in Figure 8.3.6.
Chapter 9
Image Processing in
Frequency Domain
The discrete Fourier transform is introduced to lter unwanted attributes of an
image or signal such as from electronic noise or from atmospheric distortion.
The discrete Fourier transform is represented by the Fourier matrix whose components are complex numbers. By using properties of complex numbers and
matrices a number of important properties of the discrete Fourier and fast
Fourier transform are derived. These are then used to lter in the frequency
domain a number of images.
9.1
Laplace and Fourier Transforms
Laplace and Fourier transformations can be used to solve dierential equations

and to reduce periodic noise in signals and images. Basically, they convert the
derivative operations into multiplication, dierential equations into algebraic
equations, and periodic functions are associated with their frequencies. This
latter property is very useful in image enhancement where one wants to discard
or lter unwanted periodic components. For example, in the Section 9.2 the
discrete version of the Fourier transform will be used to lter a "fuzzy" sine
function.
9.1.1
Laplace Transform
The Laplace transform is traditionally introduced because it is useful in solving

initial value problems for ordinary dierential equations. The Laplace transformation maps a function i (w) to another function of a parameter v= This transformation converts derivatives into multiplication by the parameter v, which
allows one to solve initial value problems by solving algebraic systems in terms
of the parameter v=
343
344
CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
Denition 9.1.1. The Laplace transform of a function i (w) is a function of a

parameter v A 0
Z 4
O(i )
hvw i (w)gw=
0
Because the improper integral is a linear operation, the Laplace transform

is also linear, that is,
O(fi ) = fO(i ) and O(i + j) = O(i ) + O(j)=
The Laplace transform of an exponential function i (w) = hdw with v A d is easy
to compute as follows
Z 4
O(hdw ) =
hvw hdw gw
0
=
=
=
=
hvw hdw gw
lim
Q $4
0
Q
h(vd)w gw
lim
Q $4
lim [
Q $4
1
1
h(vd)Q
]
(v d)
(v d)
1
=
vd
A special case is d = 0 so that O(1) = 1@v= There are a number of other rules
for computing Laplace transforms, but here we list only four important rules.
Basic Rules for Laplace Transform. Assume all functions are such that
the improper integrals in the denition exist.
1.
2.
3.
4.
O(fi ) = fO(i ),
O(i + j) = O(i ) + O(j),
O( gi
gw ) = vO(i ) i (0) and
O(i j) = O(i )O(j) where i j is called the convolution of i and j
Z
(i j)(w)
i ( )j(w )g =
The third rule is important because it converts the derivative operation into
multiplication by the parameter v= The third property is established by using
the integration by parts formula
Z
Z
ygx = xy xgy
with gx =
gi
gw gw
and y = hvw = Assume i (w) and the parameter v are such that
9.1. LAPLACE AND FOURIER TRANSFORMS
345
limQ$4 i (Q )hvQ = 0 and, thus,

O(
gi
) =
gw
=
=
4
0
hvw
Z
lim
Q $4
gi
gw
gw
hvw
gi
gw
gw
vQ
lim [i (Q )h
Q $4
Z
v0
i (0)h
(v)hvw i (w)gw]
= 0 i (0) + vO(i )=
The proof of the convolution formula is more complicated, but it will be used
in the next subsection.
9.1.2
Dierential Equations and Laplace Transform
The solution process using Laplace transforms to solve initial value problems
can be outlined in the following three steps: use the Laplace transform to
convert the initial value problem (often associated with the time domain)
into an algebraic problem (often associated with frequency domain), solve
the algebraic problem, and convert back to the time domain for the solution
of the initial value problem.
Consider the initial value problem
gx
= dx + i (w) with x(0) given.
gw
Compute the Laplace transform of both sides of the dierential equation and
use the rst three basic rules
gx
) = O(dx + i (w))
gw
vO(x) x(0) = dO(x) + O(i )=
O(
Solve for the Laplace transform O(x)

(v d)O(x) = O(i ) + x(0)
1
1
O(x) = O(i )
+ x(0)
vd
vd
= O(i )O(hdw ) + x(0)O(hdw )=
Use the convolution property with j(w) = hdw to obtain
O(x) = O(i j) + x(0)O(hdw )
= O(i j + x(0)hdw )=
346
Evidently,
x(w) = (i j)(w) + x(0)hdw
Z w
=
i ( )j(w )g + x(0)hdw
0
Z w
=
i ( )hd(w ) g + x(0)hdw
0
Z w
= hdw
i ( )hd g + x(0)hdw =
0
This agrees with the solution in Section 6.1 where the homogeneous solution is
hdw and the particular solution is from the variation of parameters formula.
9.1.3
Fourier Transform
The Fourier transform can be viewed as a variation of the Laplace transform

where the parameter v is replaced by an imaginary parameter l$ and the domain
of i (w) is expanded to the real line. The Fourier transform has similar properties,
which we will only state.
Denition 9.1.2. The Fourier transform of a function i (w) is a function of a
parameter $ A 0
Z 4
I (i )
hl$w i (w)gw=
4
Four Basic Properties of Fourier Transform. Assume all functions are

such that the improper integrals in the denition exist.
1.
2.
3.
4.
I (fi ) = fI (i ),
I (i + j) = I (i ) + I (j),
I ( gi
gw ) = lzI (i )>
I (i j) = I (i )I (j) where i j is called the convolution of i and j
Z 4
(i j)(w)
i ( )j(w )g =
4
The solution of a dierential equation can be found using the Fourier transformation. For example, nd a solution of
g2 x
+ fx = i (w)=
gw2
As in the use of the Laplace transform, compute the Fourier transform of both
sides of the dierential equation and use the above rules
I (
g2 x
+ fx) = I (i (w))
gw2
gx
) + fI (x) = I (i (w))
gw
((lz)2 I (x) + fI (x) = I (i (w))=
(lzI (
347
Solve for I (x)

I (x) = I (i )
$2
1
=
+f
Let j = j(w) be such that I (j) = $21+f and use the fourth rule with the
convolution to conclude
x(w) = (i j)(w)=
Some functions and their transforms can become complicated, but they can be
computed by using tables or symbolic computation tools.
9.1.4
Transforms Using MATLAB
The symbolic toolbox in MATLAB has commands to compute the Laplace and
Fourier transforms and their inverses. Some simple illustrations are given below.
AA laplace(t)
1/s^2
AA laplace(t^2)
2/s^3
AA ilaplace(1/(s*(s+1)))
1-exp(-t)
AA ilaplace(2/(s-3)^2)
2*t*exp(3*t)
% computes the Laplace transform
% the inverse Laplace transform
The Fourier transforms of many functions give Dirac delta functionals, which
can be viewed as a limit of impulse functions.
Denitions 9.1.3. The unit step function, K(w w0 ), or Heaviside function is
dened as
0> for w ? w0
=
K(w w0 )
1> for w w0
The impulse function L (w w0 ) can be written as
1
(K(w (w0 )) K(w (w0 + )))
2
1
for w0 w w0 +
2 >
=
=
0>
w ? w0 or w0 + ? w
L (w w0 )
The Dirac delta functional is given by the limit

(w w0 ) = lim L (w w0 )=
%$0
Another perspective on the limit L (w w0 ) is how it operates as an integral of

the product with a continuous function i (w)
Z
lim
$0
Z
i (w)L (w w0 )gw = lim
$0
w0 +
i (w)
w0
1
gw = i (w0 )=
2
348
The limit of L (w w0 ) can be thought of as an "ideal" impulse.

AA fourier(sin(t))
% computes the Fourier transform
-i*pi*Dirac(w-1) + i*pi*Dirac(w+1)
% gives delta functionals
AA fourier(cos(t))
pi*Dirac(w-1) + pi*Dirac(w+1)
AA ifourier(w,t)
% the inverse Fourier transform
-i*Dirac(1,t)
AA ifourier(w+3*w^2,t)
-i*Dirac(1,t) - 3*Dirac(2,t).
The Fourier transform identies the frequency associated with the sin(2i w)
and cos(2i w) according to the location of the delta functional
AA syms f
AA fourier(sin(2*pi*f*t))
-i*pi*Dirac(w-2*pi*f) + i*pi*Dirac(w+2*pi*f)
AA fourier(cos(2*pi*f*t))
pi*Dirac(w-2*pi*f) + pi*Dirac(w+2*pi*f)
AA fourier(8*cos(2*pi*100*t) + 4*sin(2*pi*40*t))
8*pi*Dirac(w-200*pi) + 8*pi*Dirac(w+200*pi)
-4*i*pi*Dirac(w-80*pi) + 4*i*pi*Dirac(w+80*pi).
The discrete version of the Fourier transform is similar to this.
9.1.5
Discrete Fourier Transform (DFT)
The discrete Fourier transform is derived from the Fourier transform of i (w) by
assuming i (w) is periodic on the unit interval. Let $ = 2n where n are integers
from 0 to q 1. In this case the improper integral may be approximated as
follows by letting [0 4) be approximated by [0 q]
Z 4
Z q
Z 1
I (i ) =
hlzw i (w)gw 2
hlzw i (w)gw = 2q
hlzw i (w)gw=
0
4
Approximate the integral over the unit interval by a Riemann sum

I (i ) 2q
q1
X
hl2n(m@q) i (m@q)(1@q)
m=0
= 2
q1
X
(hl2@q )nm i (m@q)=
m=0
Denition 9.1.4. Let the complex number } hl2@q and let im be complex
numbers where m are integers from 0 to q 1. The discrete Fourier transform,
DFT, is a mapping from Cq into Cq whose nwk component is
q1
X
m=0
} nm im =
349
One can view this as a matrix-vector product

GI W (f ) = Ff
where the kj-components of the qq matrix F are } nm = The matrix F is called the
Fourier matrix. Another useful perspective is that the nwk component equals
the evaluation of a polynomial at a { = } n
[Ff ]n
= sf (} n ) =
q1
X
im (} n )m where
m=0
sf ({)
q1
X
im {m =
m=0
l2@3
= cos(2@3) l sin(2@3) =
Example
s 9.1.1. Let q = 3 so that } = h
1@2l 3@2= The Fourier matrix is 33 with complex numbers as components
5 0
6 5
6
} }0 }0
1 1 1
F = 7 }0 }1 }2 8 = 7 1 }1 }2 8 =
}0 }2 }4
1 }2 }1
If f = [1 7 2]W so that sf ({) 1 + 7{ + 2{2 and GI W (f ) is

5
65 6 5
6
1 1 1
1
1+7+2
Ff = 7 1 } 1 } 2 8 7 7 8 = 7 1 + 7} + 2} 2 8
2
1 }2 }1
1 + 7} 2 + 2}
5
6
5
6
sf (} 0 )
10
= 7 sf (} 1 ) 8 = 7 3=50 4=33l 8 =
sf (} 2 )
3=50 + 4=33l
Example 9.1.2. Let q = 4 so that } = hl2@4 = cos(2@4)l sin(2@4) = l=
The Fourier matrix is 4 4 with complex numbers as components and here the
subscript denotes q = 4
6 5
6
5 0
1 1 1 1
} }0 }0 }0
9 }0 }1 }2 }3 : 9 1 }1 }2 }3 :
: 9
:
F4 = 9
7 }0 }2 }4 }6 8 = 7 1 }2 1 }2 8 =
}0 }3 }6 }9
1 }3 }2 }1
The DFT of f = [i0 i1 i2 i3 ] is
5
1 1
9 1 }1
F4 f = 9
7 1 }2
1 }3
1
}2
1
}2
65
i0
1
9 i1
}3 :
:9
} 2 8 7 i2
}1
i3
6
:
:=
8
350
Regroup the four components of f into

permutation matrix S and note
5
1
9 1
F4 f = (F4 S )(S f ) = 9
7 1
1

F2 GF2
=
F2 GF2
1 1
1 }2
1
}2
1
}2
fhyhq
frgg
1
}
}2
}3
65
i0
1
9 i2
}3 :
:9
} 2 8 7 i1
}
i3
6
:
:
8
where
1 1
with }b = hl2@4
1 }b

1 0
i0
i1
=
, fhyhq =
and frgg =
=
0 }
i2
i3

F2
even and odd subscripts by using a
Thus, the 4{4 matrix product reduces to two 2{2 matrix-vector products. A
similar reduction occurs when q = 8 = 23 and in general for q = 2g or g =
orj2 (q)= This generates a very fast implementation of the Fourier transform
called the fast Fourier transform. It requires (q@2)orj2 (q) operations, which is
much less than the 2q2 operations for the general matrix-vector product.
In applications q is typically very large and an e!cient method to do these
computations is required. In MATLAB the command t() is an implementation
of the DFT called the fast Fourier transform. The computation in Example
9.1.1 is easily done
AA t([1 7 2])
%n=3
10.0
-3.5000 - 4.3301i
-3.5000 + 4.3301i.
We will make frequent use of this command.
9.1.6
1.
Exercises
Use the Laplace transform to solve the initial value problem

gx
= f(80 x) with x(0) = 210=
gw
2.
Use Laplace transform rule three twice to nd the Laplace transform of

gx
g2 x
with x(0) = 2 and
(0) = 3=
gw2
gw
3.
Let q = 5 and consider the DFT.

(a).
Find the Fourier matrix F.
(b).
Use the Fourier matrix and by-hand computations nd the DFT of
f = [2 8 9 1 3]W =
(c).
Use the MATLAB command t() to conrm the above calculation.
9.2. PROPERTIES OF DFT

4.
351
Let q = 8 and consider the DFT.

(a).
Find the Fourier matrix F.
(b).
Use the Fourier matrix and by-hand computations nd the DFT of
f = [2 8 9 1 3 2 1 3]W =
(c).
Use the MATLAB command t() to conrm the above calculation.
9.2
Properties of DFT
The Fourier transform of a function i (w) may be discretized, and many of its
properties have analogues to the discrete case. The discrete Fourier transform,
DFT, will be dened and three important properties will be discussed. These
include the DFT of the discrete sine and cosine functions, inverse DFT and
DFT of convolutions. Like the Fourier transform of functions, the DFT will
be able to identify frequencies of periodic functions. This allows one to "lter"
unwanted parts of signals or images.
9.2.1
Denition of DFT
The discrete Fourier transform is derived from the Fourier transform. The
discrete Fourier transform, DFT, is a mapping from Cq into Cq whose nwk
component is
q1
X
} nm im =
m=0
This is a matrix-vector product

GI W (f ) = Ff
where the kj-components of the q q Fourier matrix F are } nmP
= Or, the nwk
m
component of GI W (f ) is the evaluation of a polynomial sf ({) q1
m=0 im { at
a { = }n
q1
X
[Ff ]n = sf (} n ) =
im (} n )m .
m=0
Example 9.2.1. Let q = 5 so that } = hl2@5 = cos(2@5) l sin(2@5) =

0=3090 l0=9511= The Fourier matrix is 5 5 with complex numbers as components
6
5 0
6 5
} }0 }0 }0 }0
1 1 1 1 1
9 }0 }1 }2 }3 }4 : 9 1 }1 }2 }3 }4 :
:
9 0
: 9
2
4
6
8 :
2
4
9
} }3 :
F =9
:=
9 } 0 } 3 } 6 } 9 }12 : = 9 1 } 3 }
7 } } }
}
} 8 7 1 }
} }4 }2 8
} 0 } 4 } 8 } 12 } 16
1 }4 }3 }2 }
352
If f = [1 7 2 2 3]W , then sf ({) 1 + 7{ + 2{2 2{3 + 3{4 and GI W (f ) is

5
Ff
9
9
= 9
9
7
5
9
9
= 9
9
7
1
1
1
1
1
1
}1
}2
}3
}4
sf (} 0 )
sf (} 1 )
sf (} 2 )
sf (} 3 )
sf (} 5 )
1 1
}2 }3
}4 }
} }4
}3 }2
6 5
: 9
: 9
:=9
: 9
8 7
1
}4
}3
}2
}
65
:9
:9
:9
:9
87
1
7
2
2
3
6
:
:
:
:
8
11=0000
4=0902 + 6=1554l
7=0902 1=4531l
7=0902 + 1=4531l
4=0902 6=1554l
6
:
:
:=
:
8
MATLAB easily does this computation

AA t([1 7 2 -2 3])
ans =
11.0000
4.0902 + 6.1554i
-7.0902 - 1.4531i
-7.0902 + 1.4531i
4.0902 - 6.1554i
9.2.2
DFT of Sine and Cosine
Let t = 0 : 1@q : 11@q be the vector starting at 0 and increasing in increments

of 1@q to 11@q. In order to compute the DFT of the sine and cosine functions,
rst compute the DFT of the exponential vector hl2nt and then use Eulers
formula. We shall make use of three properties of } hl2@q :
} q = 1> } n = } qn and 1 + } + } 2 + + } q1 = 0=
The rst two properties follow from Eulers formula. The third property follows
from
}(1 + } + } 2 + + } q1 ) = } + } 2 + + } q1 + } q
= } + } 2 + + } q1 + 1=
Move the terms on the right side to the left side and use the distribution property to get
(} 1)(1 + } + } 2 + + } q1 ) = 0=
Since } 1 is not zero, the second factor must be zero. Example 9.2.2 illustrates
the discrete Fourier transform of hl2nt for q = 3 and t = [0@3 1@3 2@3]W .
353
Example 9.2.2. If q = 3> then GI W (hl2i t ) is a 3 1 vector given by the

Fourier matrix with } = hl2@3
6 5 l2i 0@3 6
5
h
1 1 1
Fhl2i t = 7 1 } 1 } 2 8 7 hl2i 1@3 8
1 }2 }1
hl2i 2@3
65
6
5
1
1 1 1
= 7 1 } 1 } 2 8 7 } i 8 =
1 }2 }1
} 2i
If i = 0, then
5
Fhl2i t
65 6 5
6 5 6
1
1
1+1+1
3
}2 8 7 1 8 = 7 1 + } + }2 8 = 7 0 8 =
1
}1
1 + }2 + }
0
1 1
= 7 1 }1
1 }2
If i = 1, then
Fhl2i t
65
6 5
6 5 6
1
1
1 + }2 + }
0
} 2 8 7 } 1 8 = 7 1 + 1 + 1 8 = 7 3 8 =
0
1 + } + }2
}1
} 2
65
6 5
6 5 6
1
1
1 + } + }2
0
} 2 8 7 } 2 8 = 7 1 + } 2 + } 8 = 7 0 8 =
1+1+1
3
}1
} 1
1 1
= 7 1 }1
1 }2
If i = 2, then
Fhl2i t
1 1
= 7 1 }1
1 }2
DFT of Sine and Cosine Functions. Let [Ff ]n denote the nwk component
of the vector Ff . Then for f = hl2i t
l2i t
[Fh
]n
[Fhl2i t ]n
0>
q>
for n 6= i
for n = i
0>
q>
for n 6= q i
=
for n = q i
and
Use Eulers formula to obtain for f = cos(2i t) and sin(2i t)
[F cos(2i t)]n
=
;
?
[F sin(2i t)]n
0>
q@2>
for n 6= q i or i
for n = q i or i
0>
(q@2)l>
=
(+q@2)l>
and
for n 6= q i or i
for n = i
for n = q i
The MATLAB code ttrig.m illustrates these rules, which are analogous to the
rules for the Fourier transform of the sine and cosine functions giving Dirac
delta functionals.
354
Figure 9.2.1: DFT of Sine and Cosine
MATLAB code ttrig.m

% This code uses t() to compute the discrete
% Fourier transform of the sine and cosine.
% The real and imaginary parts of the transformed
% vectors are graphed.
t = 0:.001:1;
freq = 1:1:1001;
tsin = t(4*sin(2*pi*40*t));
tcos = t(8*cos(2*pi*100*t));
subplot(2,1,1);
plot(freq,real(tsin),freq,real(tcos));
subplot(2,1,2);
plot(freq,imag(tsin),freq,imag(tcos));
Figure 9.2.1 has the two graphs of the transformed functions. The reader should
note the locations and magnitudes of the transformed vectors, which conrms
the above important rules.
9.2.3
Inverse DFT
Let the complex number } hl2@q and let jm be complex numbers where
m are integers from 0 to q 1. The inverse discrete Fourier transform is a
355
mapping from Cq into Cq whose nwk component is

q1
1 X nm
}
jm =
q m=0
Inverse Fourier Matrix Property. Let F be the Fourier matrix. Then F

has an inverse matrix whose kj-components are q1 } nm =
Example 9.2.3. Use the Fourier matrix and show
matrix
5
6 5
1 1 1
1
1
1
FF1 = 7 1 } 1 } 2 8 7 1 } 1
3
1 }2 }1
1 } 2
5
1 + 1 + 1 1 + }2 + }
17
1 + } + }2 1 + 1 + 1
=
3
1 + }2 + } 1 + } + }2
5
6
3 0 0
17
0 3 0 8 = L=
=
3
0 0 3
FF1 is the 3 3 identity

1
} 2 8
} 1
6
1 + } + }2
1 + }2 + } 8
1+1+1
The inverse Fourier transform can be e!ciently computed by using the

same technique as in the fast Fourier transform. The following illustrates the
MATLAB command it() for computing the inverse Fourier transform:
AA f = [1 7 2]
AA g = t(f)
g = 10.0000
-3.5000 - 4.3301i
AA it(g)
1.0000
7.0000
2.0000.
9.2.4
-3.5000 + 4.3301i
DFT and Convolution
The objective is to dene a discrete convolution so that an analogue of the

convolution property can be established. Use the polynomial representation of
the DFT for two vectors a and b with q components
[Fa]n = sa (} n )
q1
X
m=0
dm (} n )m and [Fb]n = sb (} n )
q1
X
em (} n )m =
m=0
Note, the product of these two polynomials is a polynomial of degree 2q 2

with 2q 1 coe!cients. This suggests the convolution of a and b should be a
vector with 2q 1 components chosen to be the components of the product of
the polynomials. For example, for q = 3 and two vectors a = [d0 d1 d2 ]W and
356
b = [e0 e1 e2 ]W
sa ({)sb ({) = (d0 + d1 { + d2 {2 )(e0 + e1 { + e2 {2 )
= (d0 e0 ) + (d0 e1 + d1 e0 ){ + (d0 e2 + d1 e1 + d2 e0 ){2
+(d1 e2 + d2 e1 ){3 + (d2 e2 ){4 =
Dene the convolution of a and b to be the vector of coe!cients in the product
polynomial
d0 e0 d0 e1 + d1 e0 d0 e2 + d1 e1 + d2 e0 d1 e2 + d2 e1 d2 e2 =
Denition 9.2.1. Let a and b have q components and let sa ({) and sb ({)
be polynomials associated with these vectors. The convolution of two vectors,
frqy(a> b)> is a 2q 1 vector such that
sfrqy(a>b) ({) = sa ({)sb ({)=
The analogue of the convolution property for the discrete Fourier transforms
requires the use of padded discrete Fourier transforms. This is a technical
di!culty caused by the fact that the dimension of vectors increases during the
convolution of two vectors. Let a be an q vector. A padded vector of a is a
2q 1 vector, A, with a in the rst q components and zeros in the last q 1
components. For example, let q = 3 and a = [d0 d1 d2 ]W so that
A = [d0 d1 d2 0 0]W =
Let } = hl2@5
5
1 1
9 1 }
9
2
FA = 9
9 1 }3
7 1 }
1 }4
and compute the DFT of the padded vector A

6 5 6
65
6
5
5
d0
1
1 1 1
1
9
: 9 :
9 } :
9
}2 }3 }4 :
: 9 d1 : 9 1 :
:
9
9
9 d2 : = 9 1 : d0 + 9 } 2 : d1 + 9
}4 } }3 :
:
9
:
9
:
:
9
9
7 }3 8
7
} }4 }2 8 7 0 8 7 1 8
1
0
}3 }2 }
}4
The component-wise or array product of FA and FB is

5 6
6
5
5
1
1
1
9 1 :
9 } :
9 }2
9 :
9 2 :
9 4
:
:
9
9
FA= FB = (9
9 1 : d0 + 9 } 3 : d1 + 9 }
7 1 8
7 } 8
7 }
1
}3
}4
6
5
5
5 6
1
1
1
9 } :
9 }2
9 1 :
9 2 :
9 4
9 :
:
:
9
9
(9
9 1 : e0 + 9 } 3 : e1 + 9 }
7 } 8
7 }
7 1 8
}3
1
}4
6
:
:
: d2 )=
:
8
6
:
:
: e2 )
:
8
1
}2
}4
}
}3
6
:
:
: d2 =
:
8
1
9 1
9
= 9
9 1
7 1
1
5
9
9
+9
9
7
5
9
9
+9
9
7
357
:
9
:
9
: d0 e0 + 9
:
9
8
7
1
}2
}4
}
}3
1
}3
}
}4
}2
1
}
}2
}3
}4
6
:
:
: (d0 e1 + d1 e0 )
:
8
:
:
: (d0 e2 + d1 e1 + d2 e0 )
:
8
6
:
9
:
9
: (d1 e2 + d2 e1 ) + 9
:
9
8
7
1
}4
}3
}2
}
6
:
:
: d2 e2
:
8
= F frqy(a> b)=
This is true for general vectors and gives the desired analogue of the convolution
of Fourier transform for functions.
Convolution Property of DFT. Let A and B be the padded vectors of
two q vectors a and b. The 2q 1 dimensional DFT of the convolution of a
and b is the component-wise or array product of the DFT of A and B
F frqy(a> b) = FA= FB=
Example 9.2.4. The reader should conrm by-hand calculations the following
MATLAB computations, which verify the convolution property. Let q = 3 and
a = [2 3 6] and b = [8 6 3].
AA a = [2 3 6]
AA b = [8 6 3]
AA conv(a,b)
16
36
72
45
18
AA A = [a 0 0]
AA B = [b 0 0]
AA tA = t(A)
tA =
11.0000
-1.9271 - 6.3799i
1.4271 - 3.9430i
AA tB = t(B)
tB =
17.0000
7.4271 - 7.4697i
4.0729 + 0.6735i
1.4271 + 3.9430i
-1.9271 + 6.3799i
4.0729 - 0.6735i
7.4271 + 7.4697i
358

AA lhs = t(conv(a,b))
lhs =
1.0e+002 *
1.8700
-0.6197 - 0.3299i
0.0847 - 0.1510i
AA rhs = tA.*tB
rhs =
1.0e+002 *
1.8700
-0.6197 - 0.3299i
0.0847 - 0.1510i
9.2.5
0.0847 + 0.1510i
-0.6197 + 0.3299i
0.0847 + 0.1510i
-0.6197 + 0.3299i.
Application to Signal or Image Filtering
The convolution property can be restated as

frqy(a> b) = F1 (FA= FB)=
If B is a signal or image, then its DFT will reect the frequencies of the periodic parts of the signal or image. By masking or ltering out the unwanted
frequencies one can obtain a new signal or image by applying the inverse Fourier
transformation. A lter is a (2q 1) 1 matrix, a (2q 1) 1 column vector
for a 1D signal or image, whose components vary from 0 to 1 and with the same
dimension as the DFT of the padded vector. If the component is 1, then the
frequency is allowed to pass; if the component is 0, then the frequency is tossed
out.
Denition 9.2.2. Let I lowhu represent a (2q1)1 matrix whose components
are in the unit interval. The ltered signal or image is
Q hzB = F1 (I lowhu= FB)=
As an illustration consider following MATLAB code tsine.m, which lters
a low frequency sine function with an unwanted high frequency noise. Figure
9.2.2 has the noisy sine curve with the time domain in the top graph and its
shifted DFT with the frequency domain in the bottom graph. The unwanted
frequencies are 200 and 210 and are associated with the small spikes away
from the center of the shifted frequency domain. The lter matrix is zero on
the left and right parts of the shifted frequency domain and one in the center
region corresponding to the low frequencies. The top graph in Figure 9.2.3
is the ltered DFT of the noisy sine function, which has removed the higher
frequencies, and the lower graph is the ltered image in the time domain.
MATLAB code tsine.m
% This code is a low-pass lter for a noisy sine curve.
% It uses the DFT to transform the image to the frequency domain.
% The lter matrix is applied to eliminate the unwanted high
% frequencies. Then the convolution property and inverse DFT
359
Figure 9.2.2: Noisy Sine Function

% are used to transform the ltered image back to the time domain.
clear;
t = 0:.001:1; % Dene 10 sin(2 pi t) with high frequency variations.
x = 10*sin(2*pi*t) + 2*sin(2*pi*200*t) + 1*cos(2*pi*210*t);
%
tx = t(x,2002);
% Padded t.
tx = tshift(tx);
%
gure(1)
subplot(2,1,1);
plot(t,x);
subplot(2,1,2);
plot(abs(tx));
%
freqlter = zeros(1,2002);
% Low pass lter.
freqlter(1,700:1300) = 1;
newtx = freqlter.*tx;
newx = itshift(newtx);
%
newx = it(newx,2002);
% Padded inverse t.
%
gure(2)
subplot(2,1,1);
plot(abs(newtx));
360
Figure 9.2.3: Filtered Sine Image

subplot(2,1,2);
plot(t,real(newx(1,1:1001)));
9.2.6
Exercises
Let q = 3> t = [0 1@3 2@3]W and i = 0> 1 and 2.

(a).
By-hand calculations nd the DFT of cos(2i t).
(b).
By-hand calculations nd the DFT of sin(2i t).
(c).
Use the MATLAB command t() to conrm the above calculations.
2.
Consider the MATLAB code ttrig.m.
(a).
Use a variation of this code to nd the DFT of 3 cos(230t)=
(b).
Use it to nd the DFT of 3 cos(2100t)+4 sin(2120t)+20 cos(2t)=
3.
Let q = 5 and consider the inverse DFT.
(a).
Find the inverse Fourier matrix F1 =
(b).
Verify it is the inverse of F, that is, show FF1 = L=
4.
Consider Example 9.2.4 of the convolution property.
(a).
By-hand calculations verify the MATLAB computations.
(b).
Let q = 4 and use MATLAB to verify the convolution property for
a = [1 3 4 1]W and b = [8 9 4 1]W =
5.
Consider the MATLAB code tsine.m. Adjust the lter matrix, freqlter,
so that the following is ltered
1.
10 sin(2t) + 2 sin(250t) + 1 cos(260t)=
9.3. DFT IN RQ RQ
361
6.
Consider the MATLAB code tsine.m. Use a variation of it to lter the
noisy cosine function
20 cos(2t) + 3 cos(2100t) + 4 sin(2120t)=
DFT in Rq Rq
9.3
In the previous section the DFT of a single vector was dened to be another
vector with nwk component
q1
X
(hl2@q )nm im = [Ff ]n = sf (} n ) where } hl2@q =
m=0
Three important properties were the DFT of sine and cosine functions, the
inverse DFT and the convolution identity. These were used to illustrate a lowpass lter for a noisy 1D image or signal. The objective of this section is to
extend the DFT to 2D images, which are represented by matrices and not a
single vector. This will allow us to lter unwanted attributes of 2D images.
9.3.1
Denition of DFT in 2D
Let an image be represented by an q{ q| matrix i whose components are

im{ >m| where 0 m{ ? q{ and 0 m| ? q| = The 2D version of DFT and its
inverse are analogous to DFT of a single vector.
Denitions 9.3.1. The 2D DFT of the matrix i is another matrix of the same
dimension with n{ > n| -component equal to
X X
q| 1 q{ 1
(hl2@q| )n| m| (hl2@q{ )n{ m{ im{ >m| = si (}{n{ > }|n| ) where
m| =0 m{ =0
X X
q| 1 q{ 1
si ({> |)
im{ >m| {m{ | m| , }{ hl2@q{ and }| hl2@q| =
m| =0 m{ =0
n
Let GI W (i ) = j = si (}{n{ > }| | ). The inverse 2D DFT can be shown to have

m{ > m| -component equal to
im{ >m| =
q| 1 q{ 1
1 X X
}{n| m| }|n{ m{ jn{ >n| =
q{ q|
n| =0 n{ =0
Both the 2D DFT and its inverse can easily be computed by using the
MATLAB commands t2() and it2(). For example, for q{ = 2 and q| = 3 :
362

AA a = [1 2 5; 6 7 11]
a=
1
2
5
6
7
11
AA ta = t2(a)
ta =
32.0000
-5.5000 + 6.0622i
-5.5000 - 6.0622i
-16.0000
0.5000 - 0.8660i
0.5000 + 0.8660i
AA it2(ta)
=
1
2
5
6
7
11.
9.3.2
2D DFT of Sine and Cosine
The following MATLAB code in ttrig2da.m illustrates a more general rule for
2D DFT of sine and cosine functions of two variables
MATLAB code in ttrig2da.m
clear;
nx = 4;
ny = 3;
fy = 2;
fx = 1;
for i = 1:nx
for j=1:ny
u(i,j) = sin(2*pi*(i-1)/nx*fx+2*pi*(j-1)/ny*fy);
end
end
u
tu = t2(u)
AA ttrig2da
u=
0.0000
1.0000
0.0000
-1.0000
tu =
-0.0000
-0.8660
-0.5000
0.8660
0.5000
0.0000 - 0.0000i
0.0000
0.0000 + 0.0000i
0.8660
-0.5000
-0.8660
0.5000
0.0000 + 0.0000i
0.0000 - 0.0000i
0.0000 + 0.0000i
-0.0000 - 6.0000i
0.0000 - 0.0000i
-0.0000 + 0.0000i
-0.0000 + 6.0000i
0.0000 - 0.0000i
9.3. DFT IN RQ RQ
363
As in the 1D DFT of the sine and cosine functions, the 2D DFT transforms
will identify the frequencies as spikes in the matrix of the Fourier transforms.
Let { = 0 : 1@q{ : 1 1@q{ and | = 0 : 1@q| : 1 1@q| .
The n{ > n| -component of DFT(cos(2{i{ + 2|i| )) is
;
?
q{ q|
>
q 2q
{
>
0>
for (n{ > n| ) = (i{ > i| )

for (n{ > n| ) = (q{ i{ > q| i| )
for other cases.
The n{ > n| -component of DFT(sin(2{i{ + 2|i| )) is

; q{ q|
? 2 l>
q{ q|
l>
= 2
0>
for (n{ > n| ) = (i{ > i| )

for (n{ > n| ) = (q{ i{ > q| i| )
for other cases.
The following MATLAB code ttrig2d.m also illustrates these rules. Figure
9.3.1 is the graph of the 2D DFT of the "padded" image of the indicated sine
and cosine functions of two variables. The frequency domain has been shifted
so that the center frequencies are the smaller frequencies. The "spikes" are
located near a circle of radius 400 from the center. Since the "spikes" are very
large, the vertical axis in the mesh plot is the orj(1 + dev(i i wx)).
MATLAB code ttrig2d.m
% Computes the 2D DFT using t2().
% Notice the locations of the "spikes" in
% the mesh plot, which indicates the frequencies.
nx = 500;
ny = 500;
for i = 1:nx
for j=1:ny
u(i,j) = 1 + ...
2^0*(1+sin(2*pi*((i-1)/nx)*200))+...
2^2.*(1+sin(2*pi*((j-1)/ny)*200))+...
2^4.*(1+cos(2*pi*((i-1)/nx+(j-1)/ny)*141))+...
2^6.*(1+sin(2*pi*((i-1)/nx-(j-1)/ny)*141));
end
end
tu = t2(u,2*nx-1,2*ny-1);
tu = tshift(tu);
mesh(log(1+abs(tu)));
364
Figure 9.3.1: 2D DFT of Sine and Cosine
9.3.3
Convolution for Matrices
The convolution identity also generalizes to the 2D DFT. Let d be an q{ q|

matrix with components d(m{ > m| ) where the indices start at zero= The polynomial of two variables { and | associated with this matrix is dened to be
X X
q| 1 q{ 1
sd ({> |)
d(m{ > m| ){m{ | m| =
m| =0 m{ =0
The convolution of two q{ q| matrices, frqy2(d> e), is a (2q{ 1) (2q| 1)

matrix such that
sfrqy2(d>e) ({> |) = sd ({> |) se ({> |)=
For example, consider q{ = 2 and q| = 3 where we have again used MATLAB
AA a = [1 2 5; 6
1
2
6
7
AA b = [2 5 9; 1
2
5
1
2
AA conv2(a,b)
2
9
13
48
6
19
7 11]
5
11
2 8]
9
8
29
128
73
43
144
78
45
139
88.
9.3. DFT IN RQ RQ
365
sd ({> |) = 1 + 2| + 5| 2 + 6{ + 7{| + 11{| 2

se ({> |) = 2 + 5| + 9| 2 + 1{ + 2{| + 8{| 2
The coe!cient in sfrqy2(d>e) ({> |) of {2 | 4 is
88 = 11 8 = d(1> 2) e(1> 2).
The coe!cient in sfrqy2(d>e) ({> |) of {2 | 3 is
78 = 7 8 + 11 2 = d(1> 1) e(1> 2) + d(1> 2) e(1> 1)=
The convolution identity in 2D also requires the matrices to be padded by
zeros in both the rows and columns. The justication of this identity is similar
to that given for the 1D DFT. Here we have used the MATLAB notation, i i w2()>
for the fast Fourier transform in 2D.
Convolution Property for 2D DFT. Let d and e be q{ q| image matrices.
i i w2(frqy2(d> e)) = i i w2(d> 2 q{ 1> 2 q| 1)= i i w2(e> 2 q{ 1> 2 q| 1)=
The following MATLAB computation veries this for the above 23 matrices
with 2 q{ 1 = 3 and 2 q| 1 = 5:
AA t2(conv2(a,b))
1.0e+002 *
Columns 1 through 3
8.6400
-2.7193 + 2.0698i
-1.0757 + 0.8192i
-2.4000 - 1.8013i
1.0034 + 0.1752i
0.2677 - 0.1158i
-2.4000 + 1.8013i
0.5290 - 0.9709i
0.2249 - 0.4680i
Columns 4 and 5
-1.0757 - 0.8192i
-2.7193 - 2.0698i
0.2249 + 0.4680i
0.5290 + 0.9709i
0.2677 + 0.1158i
1.0034 - 0.1752i
AA norm(t2(conv2(a,b)) - t2(a,3,5).*t2(b,3,5))
4.8969e-014.
9.3.4
Application to Image Filtering
The convolution identity can be restated by taking the inverse DFT of both
sides
frqy2(d> e) = li i w2(i i w2(d> 2 q{ 1> 2 q| 1)= i i w2(e> 2 q{ 1> 2 q| 1))=
If e is an image, then the DFT of e will reect the frequencies of the periodic
parts of the image. By masking or ltering out the unwanted frequencies one
can obtain a new image by applying the inverse DFT. A lter is a matrix with
the same dimension as the DFT of the padded image. The components of the
lter usually vary from 0 to 1. If the component is 1, then the frequency is
allowed to pass; if the component is 0, then the frequency is deleted.
366
Denition 9.3.2. Let e be an q{ q| image matrix. Let I lowhu represent a

(2 q{ 1) (2 q| 1) matrix whose components are in the unit interval. The
ltered image is Q hze
Q hze = li i w2(I lowhu= i i w2(e> 2 q{ 1> 2 q| 1))=
The following MATLAB code tsine2d.m illustrates this for a low frequency sine
wave with higher frequency sine noise.
MATLAB code tsine2d.m
% This code creates a noisy 2D sine wave.
% The 2D DFT is used to lter the unwanted
% higher frequencies.
clear;
nx = 200;
ny = 600;
for i = 1:nx
% Dene the big wave with periodic noise.
for j = 1:ny
u(i,j) = 0+100*(1+sin(2*pi*((j-1)/ny)*5)) + ...
15.*(1+sin(2*pi*((i-1)/nx)*80))+...
15.*(1+sin(2*pi*((j-1)/ny)*80));
end
end
sine = uint8(u);
imwrite(sine, sine.jpg);
tu = t2(u,2*nx-1,2*ny-1);
% Padded t.
tu = tshift(tu);
gure(1);
mesh(u);
gure(2)
mesh(log(1+(abs(tu))));
lter = ones(2*nx-1,2*ny-1);
d0 = 150;
% Use ideal low-pass lter.
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
if dist A d0
lter(i,j) = 0;
end
end
end
gure(3)
mesh(lter);
l_sine = lter.*tu;
gure(4)
mesh(log(1+abs(l_sine)));
9.3. DFT IN RQ RQ
367
Figure 9.3.2: Noisy 2D Sine Wave

l_sine = itshift(l_sine);
l_sine = it2(l_sine);
l_sine = real(l_sine(1:nx,1:ny));
l_sine = uint8(l_sine);
imwrite(l_sine, sine_l.jpg);
Figure 9.3.2 is the noisy sine wave with unwanted higher frequencies. Figure
9.3.3 is the mesh plot of the real matrix associated with the noisy image. Figure
9.3.4 is the log plot of the magnitude of the shifted DFT for the noisy sine wave;
note the spikes indicating the higher frequencies away from the center. Figure
9.3.5 is the low-pass lter used to eliminate the spikes in the previous gure.
Figure 9.3.6 is the ltered DFT, and Figure 9.3.7 is the ltered image of the
noisy sine wave.
9.3.5
Exercises
Let d = [1 3 5 6; 2 4 7 19] and e = [2 3 4 7; 1 5 2 10]=

(a).
Find the t2 of d and e and
(b).
Verify it2(t2(d)) = d.
2.
Use a variation of ttrig2da.m to nd
(a).
t2(sin(2({ + 3|)) and
(b).
t2(cos(2({ + 3|))=
3.
Consider a variation of ttrig2d.m to nd the DFT of
1.
x = 1 + cos(2{200) + 4 cos(2|200) + 16 sin(2{200) + 64 cos(2{200)=

4.
Consider the vectors in exercise 1.

(a).
Find frqy2(d> e) and
(b).
Verify the convolution property for the 2D DFT.
5.
Use a variation of tsine2d.m to nd the 2D DFT of
x = 100(1 + cos(2|10)) + 30(1 + cos(2{120) + sin(2|120))=
6.
Use a variation of tsine2d.m to nd the 2D DFT of

x = 100(1 + cos(2|10)) + 30(1 + cos(2{120 + 2|120))=
368
Figure 9.3.3: Mesh Plot of Noisy Sine Wave
Figure 9.3.4: DFT of Noisy Sine Wave
9.3. DFT IN RQ RQ
369
Figure 9.3.5: Low-pass Filter
Figure 9.3.6: Filtered DFT of Sine Wave
370
Figure 9.3.7: Filtered Sine Wave
9.4
Frequency Filters in Rq Rq
The low-pass, high-pass and band-reject lters will be studied. These have various shapes including ideal (step functions), Butterworth (polynomial functions)
and Gaussian (exponential functions). Applications to the NCSU image, a light
and noisy aerial image and sharpening of micro chip image will be presented.
9.4.1
Classications of Filters
A lter is a matrix with the same dimension as the 2D DFT of the padded
image. The components of the lter usually vary from 0 to 1 and let I lowhu
represent such a matrix. Then the ltered image is given by Q hze
Q hze = li i w2(I lowhu= i i w2(e> 2 q{ 1> 2 q| 1))=
The MATLAB code tsine2d.m in the previous section illustrated an ideal lowpass lter for a low frequency sine wave with higher frequency noise. A high-pass
lter allows higher frequencies in the image to remain, and it can be viewed as a
ones matrix minus a low-pass lter matrix. The band-reject lter allows higher
and lower frequencies to pass and inhibits some middle range frequencies. Ideal
lters have components that are either zeros and ones. The following are three
examples of band-reject lters wherep
the approximate frequencies to be rejected
are near g0 and we use glvw(l> m) (l q} )2 + (m q| )2 =
Ideal Band-reject Filter.
0> if g0 z@2 glvw(l> m) g0 + z@2

I lowhu(l> m) =
1> otherwise.
Butterworth Band-reject Filter.
I lowhu(l> m) =
1
=
1 + (glvw(l> m)z@(glvw(l> m)2 g20 )2q
Gaussian Band-reject Filter.

I lowhu(l> m) = 1 h
1
2
2
2
2 ((glvw(l>m) g0 )@glvw(l>m)z)
9.4. FREQUENCY FILTERS IN RQ RQ
371
The parameters z and g0 control the width and location of the band. In
the Butterworth band-reject lter the exponent q controls the steepest of the
boundaries in the band of frequencies to be rejected. There are similar versions
of high and low-pass lters.
9.4.2
Ideal Low-pass for NCSU Image
The MATLAB code lter_ncsuid.m generates a noisy NCSU image, see Figure
9.4.1, and lters it via an ideal low-pass lter as indicated in Figure 9.4.2. The
ltered image is in Figure 9.4.3 and has blurred edges on the letters.
MATLAB code lter_ncsuid.m
% This code uses an ideal low-pass lter to
% eliminate unwanted frequencies from a noisy
% NCSU image.
clear;
ncsu = [bign(5) bigc(7) bigs(9) bigu(11)];
newncsu = 20*ncsu;
[nx ny] = size(newncsu);
nx
ny
newncsu = newncsu(nx:-1:1,:);
u = newncsu;
for i = 1:nx
% This is NCSU with periodic noise.
for j = 1:ny
u(i,j) = u(i,j) + ...
15.*(1+sin(2*pi*((i-1)/nx)*80))+...
15.*(1+sin(2*pi*((j-1)/ny)*80));
end
end
sinencsu = uint8(u);
imwrite(sinencsu, sinencsu.jpg);
tu = t2(u,2*nx-1,2*ny-1);
tu = tshift(tu);
subplot(2,2,1);
mesh(u);
subplot(2,2,2);
% Use ideal low-pass lter.
d0 = 150;
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
372
Figure 9.4.1: Noisy NCSU Image

if dist A d0
lter(i,j) = 0;
end
end
end
subplot(2,2,3);
mesh(lter);
l_ncsu = lter.*tu;
subplot(2,2,4);
mesh(log(1+abs(l_ncsu)));
l_ncsu = itshift(l_ncsu);
l_ncsu = it2(l_ncsu,2*nx-1,2*ny-1);
l_ncsu = real(l_ncsu(1:nx,1:ny));
l_ncsu = uint8(l_ncsu);
imwrite(l_ncsu, sinencsuid_l.jpg);
9.4.3
Butterworth Band-reject for NCSU Image
The ideal low-pass ltered image in Figure 9.4.3 is not entirely satisfactory.
Another approach is to use a band-reject lter where one may need to experiment with the width, z, the "slope", q, and the location, g0 , of the band. The
MATLAB code lter_ncsubu.m uses the Butterworth band-reject lter and may
be applied to either the noisy big sine wave or the noisy NCSU. This code generates a noisy NCSU image, see Figure 9.4.1, and lters it via the Butterworth
band-reject lter as indicated in Figure 9.4.4. The ltered image is in Figure
9.4.5, and one should compare it with the ideal low-pass ltered image in Figure
9.4.3. The code is similar to the above, but the lter denition is dierent as
indicated below:
% Use Butterworth band-reject lter.
d0 = 160;
n = 2;
w = 20;
Figure 9.4.2: Low-pass Filtering of NCSU
Figure 9.4.3: Ideal Low-pass NCSU
373
374
Figure 9.4.4: Band-reject Filtering of NCSU
Figure 9.4.5: Band-reject Filtered NCSU
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
if dist ~= d0
lter(i,j)= 1/(1 + (dist*w/(dist^2 - d0^2))^(2*n));
else
lter(i,j) = 0;
end
end
end
9.4.4
375
Power/Butterworth Band-reject for Aerial Image
Another application of the band-reject lter is to the noisy aerial photograph.

This image suers from too much exposure and from banded sine and cosine
noise. The light exposure is modied by use of the power transformation with
the power equal to two, and then the Butterworth band-reject lter is used
to reduce the noise. The MATLAB code lter_aerial.m starts with a light
and noisy aerial image, see Figure 9.4.6. The image is enhanced by the power
method and the Butterworth band-reject lter as is indicated in Figure 9.4.7.
The ltered image is in Figure 9.4.8. The lter portion of the code is as follows:
c = 1.;
% Use the power transformation to darken.
gamma = 2;
f_fp =255*c*(u/255).^gamma;
u = f_fp;
tu = t2(u,2*nx-1,2*ny-1);
tu = tshift(tu);
subplot(2,2,2)
d0 = 400;
% Use Butterworth band-reject lter.
n = 2;
w = 20;
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
if dist ~= d0
lter(i,j)= 1/(1 + (dist*w/(dist^2 - d0^2))^(2*n));
else
lter(i,j) = 0;
end
end
end
9.4.5
High-pass/Sharpened Micro Chip Image
This illustrates a high-pass lter to give emphasis to the higher frequencies in

the image of a damaged micro chip. The micro chip image in Figure 9.4.9 is
provided courtesy of Brockhouse Institute for Materials Research, McMaster
University. The high-pass image is then added to the original image so as to
obtain a sharper image. The reader may nd it interesting to experiment with
width and frequency threshold of the Butterworth or the Gaussian high-pass
lters. The MATLAB code lter_micro.m uses a high-pass Gaussian lter to
sharpen an image of a micro chip, see Figure 9.4.9. The high frequencies are
isolated by a Gaussian lter, see Figure 9.4.10, and then these frequencies are
376
Figure 9.4.6: Light and Noisy Aerial Image
Figure 9.4.7: Filtering Aerial Image
377
Figure 9.4.8: Filtered Aerial Image

added to the DFT of the padded image. The sharpened image is in Figure
9.4.11. The lter portion of the code is indicated below:
% Use Butterworth or Gaussian high-pass lter.
d0 = 100;
n = 4;
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
% Use Butterworth high-pass lter.
% lter(i,j)= 1/(1 + (dist/d0)^(2*n));
% lter(i,j)= 1.0 - lter(i,j);
% Use Gaussian high-pass lter.
lter(i,j) = exp(-dist^2/(2*d0^2));
lter(i,j) = 1.0 - lter(i,j);
end
end
..
.
% Update image with high frequencies.
l_micro = tu + lter.*tu;
378
Figure 9.4.9: Micro Chip Image
Figure 9.4.10:
Sharpening of Micro Chip Image
Figure 9.4.11:
9.4.6
379
Sharpened Micro Chip Image
Exercises
1.
Experiment with the parameter g0 in the ideal low-pass lter in lter_ncsuid.m.
2.
Modify the lter in lter_ncsuid.m to become an ideal band-reject lter. Experiment with the location and width of the band of frequencies to be
rejected.
3.
Experiment with the parameters q> z and g0 for the Butterworth bandreject lter in lter_ncsubu.m.
4.
Modify the lter in lter_ncsubu.m to become a Gaussian band-reject
lter and experiment with its parameters.
5.
Experiment with the parameters q> z and g0 for the Butterworth bandreject lter in lter_aerail.m.
6.
Modify the lter in lter_aerial.m to become a Gaussian band-reject
lter and experiment with its parameters.
7.
Experiment with the parameters in the high-pass Gaussian lter used
lter_micro.m. Also, modify the sharpening step to include other linear combinations:
% Update image with high frequencies.
l_micro = a*tu + b*lter.*tu;
8.
Repeat exercise 7 using the high-pass Butterworth lter.
Appendix A
Solutions to Odd Exercises

Chapter One
Section 1.1
1.
3.
5.
7.
9.
11.
13.
(a). 0.333
(b).
0.111; huuru = 0=000111
(a).
2z + 3} = 25 + 3l
(b).
}z = 17s+ 19l; }@z = (11 23l)@13
} 2 = 1@2 + ( 3@2)l and 1 + } + } 2 = 0
} + 0 = (3 + 4l) (7 + 2l) = (3 7) + (4 2)l = 4 + 2l
1@58 + (17@58)l
9@25 + (12@25)l
Prove |} + z|2 (|}| + |z|)2 : Use the dot product as in Section 1.4.
Chapter One
Section 1.2
1.
3.
(a).
(b).
(a).
x = 2{ | + 7 and y = { + 2| + 1
} = 3@5 + (9@5)l
x=
5.
7.
9.
11.
{2 + | 2 + | 2
3{
and y = 2
{2 + (| 1)2
{ + (| 1)2
(b).
} = 3@2 + l
s
1 + l = 1=0987 + 0=4551l
(a).
} = 1
s (1=0987 + 0=4551l)
(1 + l)10 = ( 2)10 hl(2@8)10 = 32l
3 + 4l = 5hl where = arctan(4@3)
Solutions of } 3 = 3 + 4l are
51@3 hl@3 (hl2@3 )0 = 1=6289 + 0=5202l
51@3 hl@3 (hl2@3 )1 = 1=2650 + 1=1506l and
51@3 hl@3 (hl2@3 )2 = 0=3640 1=6708l
381
382
13.
APPENDIX A. SOLUTIONS TO ODD EXERCISES

Prove z = z1 where z hl2@q :
z
z1
=
=
=
= hl2@q
= cos(2@q) + l sin(2@q)
= cos(2@q) l sin(2@q)
1
cos(2@q) + l sin(2@q)
cos(2@q) l sin(2@q)
1
cos(2@q) + l sin(2@q) cos(2@q) l sin(2@q)
cos(2@q) l sin(2@q)
1
Chapter One
Section 1.3
1.
3.
5.
7.
9.
11.
13.

$

$

2

$
d = 2 l + 5 m and a =
5
$

$

d e = [3 2]
[{1 {2 ] = [1 3] [2 5] = [3 2]
$

$
$
f = [f1 f2 ] and
Hint:
d = [d1 d2 ]> e = [e1 e2 ] and
compute
both
sides.
s
$
k
d k = 1 + 49> arctan(7@1)
s
s 81=87 degrees and
$

$
d @ k
d k = [1@ s50 s7 50]
s
13 = 29 + 10 2 29 10 cos(). Then cos() = 13@ 290 and
40=24 degrees.
$

k [353=55 473=55]
Chapter One
Section 1.4
1.
3.
5.
7.
9.
11.
13.
The angle between the vectors is 30 degrees, which is also given by

s
s
1 3 + 3(1) = 2(2) cos()=
s s
1(3) + 2(1) = 5 10 cos() giving 98=13 degrees.
[3 1] [{ 1] = 3{ 1 = 0 gives { = 1@3=
$

$
(a).
Hint:
d = [d1 d2 ]> e = [e1 e2 ] and compute both sides.
(b).
Both sides are equal to 11.
(c).
Proofrequires equality
for all possible vectors.
s
1 3
2
2
duhd = (det(
)) = 7 so that the duhd is 7=

2 1

2 1
12 3
det(
) = 3 and det(
) = 54=
1 1
6 3
$

$
zrun = i g = 3(30) + 1(6) = 96=
383
Chapter One
Section 1.5
1.
3.
5.
(a).
| = (1@3){ 3
(b).
{ = 3w and | = w 3
(a).
[{ |] = [2 7] + w[1 1]
(b).
{ = 2 + w and | = 7 w
[0 1] + (3@5)[1 2] = [3@5 11@5]
Chapter Two
Section 2.1
3.
5.
7.
9.
11.
13.
5 6
3
$

$
$

$

d = 3 l + m + 2 n or a = 7 1 8
2
$

$
$

(a).
2
d + e = [7 5 5] (b).
{ = [5 1 3]
s
s
$

$

$

$

(b).
d e = 3
(a).
k d k = 21 and e = 11
s s
1(1) + 2(3) 1(1) = 6 11 cos() giving 10=02 degrees
$

$
(a).
Hint:
d = [d1 d2 d3 ] and e = [e1 e2 e3 ] and
compute both sides.
(b).
Both sides are equal to 26.
(c).
Proof requires equality for all possible vectors.
$

$
zrun = i g = 10(25) + 1(20) + 2(5) = 280
Chapter Two
Section 2.2
$

1.
3.
5.
7.
9.
11.
$

$
$

$
$
s

$
d e = 3 l + m 9 n and
d e = 91
s
46
$

$
Hint:
d = [d1 d2 d3 ] and e = [e1 e2 e3 ] and
compute both cross products.
$

$

u i = [0 1 3]
$ $

$
(a).
(
d e)
f = [4 3 19] [2 1 5] = 84
(b).
84
Hint: expand the determinants.
Chapter Two
Section 2.3
1.
3.
5.
7.
[1 2 1] + w[3 1 1]
(a).
[1
p [1 1 1]
s 1 2] + (7@3)
168@3 = 3(56)@3
(b).
p
56@3
(c).
s
2
(a).

5 3
w
4
=
3 3

3
(b).
w = 1@2 and = 1@2
384

(c).
p
3@2
Chapter Two
Section 2.4
1.
3.
(a).
1(2) + 2(1) + 4(2) = 12
(b).
[1 2 4]
(c).
[1 2 4] [{ 2 | 1 } 2] = 0
(a).
[1 2 3]
(b).
[1 2 3] [{ 1 | 1 } 1] = 0
{ 2| 3| = 6
5.
(a).
2 = 1 + 2u + v
3 = 1 + 3u + 0v
1 = 1 + 0u + 0v
(b).
7.
u = 2@(3) and v = 1@(3)
[1 1 1] + (17@19) [2 3 1] + (60@19) [1 0 1] =
= [113@19 70@19 96@19]
9.
(a).
p = 45=00 and f = 1046=67
(b).
p4 + f = 866=67
Chapter Two
Section 2.5
1.
3.
(a).
sW = [5 6 2 1 5] and pW = [1000 850 1700 2220 1150]
(b).
sW p = 21470
(a).
5
65 6 5
6
1 2 4
u
10
7 0 1 1 87 v 8 = 7 1 8
1 1 4
w
4
5.
(b).
u = 18@1> v = 6@1 and w = 5@1
(a).
[8 5 7]
(b).
[18@7 24@7 0] + w[8 5 7]
Chapter Three
Section 3.1
1.
(a).
(b).
[8 19 17 23 7]
5
9
9
9
9
7
3.
5.
(c).
(d).
(a).
30
s
80
(b).
(c).
[1@3 6@2 3@(1) 5@3]
(a).
(b).
6
sin(1) sin(3)
sin(0)
9 sin(6) sin(2)
sin(1) :
:
9
7 sin(3) sin(1) sin(4) 8
sin(5) sin(3) sin(2)
5
397 394 389

391 388 383
[X Y] = meshgrid(1:1:3,1:1:3)
F = 400 - 20*X.^2 - Y.^2
F({> |) = 4{| + (2{ + 2|)200@({|)
About F(8=1> 8=1)=
9.
11.
:
:
:
:
8
6
1
27
0
9 216 8
1 :
9
:
7 27 1 64 8
125 27 8
(a).
(b).
I =
7.
2
0
1
5
1
4
7
2
2
1
7
1
1 4 3

@12
2@12
3@12
4@12
i1
386.
200.
141.
115
i2
-373.
-173.
-100.
-57.
l1 = 0> l2 = 5 and l3 = 5
Chapter Three
Section 3.2
1.
3.
2 1 6 8
Dx = [35 37]W and D + F =
6 2 14 10

2 21 2 12
Both sides should equal
=
7 3 17 17
385
386

5.
23 26 37 7
1 0 9 7
33 55
37
135
7.
Both sides should equal
9.
= 32@100>
u1 = [64=00 00=00 22=40]W >
u2 = [87=04 27=64 30=46]W
u = [167=50 135=00 102=50]W
Chapter Three
Section 3.3
1.
3.
5.
7.
x = [2 4 3]W
x = [50 2 5 1]W
y = [210 115 156=67]W and x = u = [182=5 155=0 117=5]W
Use forward and backward substitution.
Chapter Three
Section
5 3.4
1.
3.
5.
6
2 1 1 7
7 4 1 1 0 8
0
3 1 2
5
1 0 0
9 0 1 0
H43 (3) = 9
7 0 0 1
0 0 7
(a).
(b).
(c).
7.
(a).
(b).
6
5
0
1 0
9 0 0
0 :
:> S = 9
0 8 23 7 0 1
1
0 0
5
1
9 0
H4 (10) = 9
7 0
0
5
6
1 7 1 1
[D d] = 7 2 0 7 2 8
0 14 1 6
5
1
7
H32 (2) H21 (1) [D d] = 7 0 14
0
0
0
1
0
0
0
1
0
0
6
0
0 :
: > and
0 8
1
6
0 0
0 0 :
:=
1 0 8
0 10
6
1 1
b
5 0 8 = [X d]
6 6
b is
The solution of the equivalent system X x = d
W
x = [5@2 5@14 1] =
6
5
1 1 0
1 6
9 1 0 1 1 4 :
:
[D d] = 9
7 0 2
7
0 4 8
2 0
0 1 8
H43 (2@9) H42 (2) H32 (2) H41 (2) H21 (1) [D d] =
6
1 1 0
1
6
:
9 0 1 1 2
1
b
: = [X d]
9
8
7 0 0
9
4
2
0 0
0 1@9 58@9
b is
W
x = [25 89 26 58] =
5
6
1 1 1
0
[D d] = 7 1 0 4 20 8
0 2 4 10
5
6
1 1
1
0
b
5 20 8 = [X d]
H32 (2) H21 (1) [D d] = 7 0 1
0 0 14 50
b is
5
(c).
9.
(a).
(b).
(c).
11.
x = i = [40@7 15@7 25@7]W =

Hint: Do the elementary matrix products.
Chapter Three
Section 3.5
1.
3.
5.
7.
9.
3@10
1@10
1@@10 3@10
(b).
H12 (1@3) H2 (3@10) H1 (1@3) H21 (1@3) [D L] = [L D1 ]
5
61 5
6
3 1 0 0
3@10
1@10 0
0
9 1 3 0 0 :
9
0 :
9
: = 9 1@@10 3@10 0
:
(c).
7 0 0 3 0 8
7
0
0
1@3 0 8
0 0 0 4
0
0
0 1@4
5
61
5
6
3 1 0
8 3 1
7 1 3 1 8 = (1@21) 7 3 9 3 8
(a).
0 1 3
1 3 8
5
65 6 5
6
8 3 1
1
17@21
(b).
(1@121) 7 3 9 3 8 7 2 8 = 7 30@21 8
1 3 8
3
31@21
Hint: Do the row
operations.
5
6
5
6
1 0 0
1 2 0
7 0 1 0 8 and D1
7 0 1 0 8
(a).
D1
1 =
2 =
4 0 1
0 0 1
5
6
1 2 0
1
7 0 1 0 8
(b).
(D1 D2 )1 = D1
2 D1 =
4 0 1
5
6
1 2 0
1
7 0 1 0 8
(D2 D1 )1 = D1
1 D2 =
4 8 1
See text form MATLAB commands.
(a).
387
D1 =
388
11.

(a).
node 1:
[i1 0] + [0 i2 ] + [i7 0] + [i3 f i3 v] = [0 0]
node 2:
[i3 f i3 v] + [0 i8 ] + [i4 f i4 v] = [0 0]
node 3:
[0 i5 ] + [i6 0] + [i4 f i4 v] = [0 0]
node 4:
[i6 0] + [0 i8 ] + [i7 0] + [0 z] = [0 0]
Chapter Three
Section 3.65
6
65
2 1
0
0
1
0
0
0
9
9 1@2
0 :
1
0
0 :
:
: 9 0 3@2 1
9
1.
(a).
7 0
2@3
1
0 8 7 0 0 4@3 1 8
0 0
0 5@4
0
0
3@4 1
1
1 W
(b).
y = [470 625 823 3 1087 2 ] and x = [870 1270 1270 870]W
5
65
6
1 0 0
2 1
1
3.
(a).
S13 D = 7 1@2 1 0 8 7 0 5@2 1@2 8 and
0 2 1
0 0
5
S13 d = [3 2 6]W
W
(b).
y = [3 1@2 5]W and
x = [1 9 1]
b22 = 4@3 1
5.
(a).
D
1 2
b 2 = [823 1 470]W
b22 x2 = d
(b).
D
3
x2 = [1270 870]W and x1 = [470 1270]W
7-11.
See circuit3.m
Chapter Three
Section 3.7
1.
3.
5.
7.
11.
det(DW ) = 1(6) (3 + 5) = 11

det(S32 D) = 6 + 3 + 2 = 11
det(D) = 8 3
det(D) = 3 8
det(D) = 77 26 = 21
det(D) = det(H43 (8@3) H41 (1) H31 (8) D)
= 1(1)(3)(7) = 21 6 5
5
6 5
6
e12
(1)2+1 P21 @(11)
5@11
7 e22 8 = 7 (1)2+2 P22 @(11) 8 = 7 3@11 8
(1)2+3 P23 @(11)
e32
2@11
0
{ = (2 + 4(|@24)) 10({@24)
| 0 = (3 + 10({@24) + 2(}@24)) (4(|@24) + 8(|@24))
(a).
} 0 = 4 + 8(|@24) (2(}@24) + 6(}@24))=
5
6 5
6
{
2
(b).
D 7 | 8 = 7 3 8
}
4
5
6 5
6
{
14=1
7 | 8 = 7 24=0 8
(c).
}
36=0
(a).
(b).
(a).
(b).
(a).
(b).
389
Chapter Four
Section 4.1
1.
3.
5.
(a).
(b).
(c).
(a).
(b).
(c).
(a).
(b).
I (p> f) = 14p2 + 12pf + 3f2 56p 22f + 59

p = 18@6 and f = 14@7
| = 38@3
I (p> f) = 204p2 + 60pf + 8f2 7976p 1634f + 85771
p = 7=4167 and f = 68=75
| = 180=0
x(0) = 154=2983 and u = 0=04419
x(60) = 10=8893
Chapter Four
Section 4.2
1.
3.
5.
6
5
6
2 7
5 12
(a). DW = 7 0 1 8 and (b). E W DW = 7 5 19 8
1 2
10 27
Hint: Compute the transpose of DD1 = L.
(a).

30 10
p
19130
=
10 4
f
7735
5 6
5
6
0
1 1
9 0 :
9 2 1 : }1
9 :
:
(b).
Dz = 9
7 3 1 8 }2 = 7 0 8 implies four equations must
0
4 1
hold:
1}1 + 1}2
2}1 + 1}2
3}1 + 1}2
4}1 + 1}2
7.
=
=
=
=
0>
0>
0 and
0=
Thus, }2 = }1 from the rst equation. The other equations

give }2 = }1 = 0=
(c).
p = 41=5 and f = 2035=5
(d).
Predicted price at time w = 8 is (41=5)8 + 2037=5 = 1705=50=
See the MATLAB code pop.m.
Chapter Four
Section 4.3
1.
(a).
(b).
(c).
OK
No. The data are not "near" a plane.
p1 = 4=0, p2 = 1=0, f = 92 and rW r = 36 AA 0=25=
390
3.

(a).
5
9
9
9
9
9
9
D=9
9
9
9
9
9
7
5.
(b).
(c).
(a).
9
:
9
:
9
:
9
:
9
:
9
:
: and d = 9
9
:
9
:
9
:
9
:
9
:
7
8
25000
24200
23900
23200
22600
22100
21500
20500
20100
6
:
:
:
:
:
:
:
:
:
:
:
:
8
A\d gives p1 = 227, p2 = 177, p3 = 94 and f = 24618

17252
5
9
9
9
9
9
D=9
9
9
9
9
7
(b).
2 3 3 1
3 4 4 1
5 4 4 1
8 5 5 1
12 6 6 1
15 8 8 1
19 10 10 1
23 11 11 1
27 13 13 1
3
5
7
9
11
12
14
15
9
8
7
5
4
3
2
1
12
13
16
18
21
23
26
24
:
9
:
9
:
9
:
9
:
9
b=9
: and d
:
9
:
9
:
9
:
9
8
7
(8 10)@2
(7 9)@2
(5 8)@2
(4 7)@2
(3 5)@2
(2 4)@2
(1 3)@2
(1 2)@2
6
:
:
:
:
:
:
:
:
:
:
8
A\d gives d = 0=2753, e = 0=1946, and f = 0=1298
Chapter Four
Section 4.4
1.
3.
5.
function Fnm = Fnm(p)

Fnm = 100 + 2*(p(1)-2)^2 + 4*(p(2)-3)^2 + 10*(p(3)-4)^2;
AA [p Fval] = fminsearch(Fnm, [10 10 10])
Use rad_fmin.m to get
x(w) = 155=9841h0=044993w and x(60) = 10=4834
Use variation of pop_us_fmin.m
Chapter Five
Section 5.1
1.
3.
5.
b = [1 1 1]
(a).
n = [2 2 2] and n
(b). 5 u = 1@26 and 5v = 1@26 5 6
2@3
1@3
2
x = u7 1 8 + v7 0 8+ 7 0 8
0
1
0
5
6
5 6 5 6
1@2
0
5
x = u7 1 8 + v7 0 8 + 7 0 8
0
1
0

7.
9.
11.
391
There is no particular solution because

1 0
3 2 1 2
3 2 1 2
=
3 1
9 6 3 5
0 0 0 1
implies 0{1 + 0{2 + 0{3 = 1!
The free 5
variable
{3 = u and
6 is 5
6
3
3
x = u 7 0 8 + 7 1@2 8 =
1
0
The free5variable
u and
6 is5{3 = 6
1
4
x = u 7 1 8 + 7 3 8 =
1
0
Chapter Five
Section 5.2
1.
(a).
(b).
(c).
3.
(a).
(b).
(c).
(d).
5.
(a).
6
4
9 0 :
:
xs = 9
7 0 8
50
6
5
6
5 6
2@3
1@3
0
9 1 :
9 0 :
9 0 :
:
9
:
9 :
xk = u 9
7 0 8 + v7 1 8 + w7 0 8
1
5 6 0 5
6 0 5
6
5
4
2@3
1@3
9 0 :
9 1 :
9 0 :
9
:
9
:
9
:
9
x=9
7 0 8 + u7 0 8 + v7 1 8 + w7
0
0

0
3 2 1 1 6
0 5
0 0 16 1
5@3
9 0 :
:
xs = 9
7 0 8
5 1
6
5
6
2@3
1@3
9 1 :
9 0 :
:
9
:
xk = u 9
7 0 8 + v7 1 8
06
06
5
5
5
6
5@3
2@3
1@3
9 0 :
9 1 :
9 0 :
:
9
:
9
:
x=9
7 0 8 + u7 0 8+ v7 1 8
1
0
5
6 0
2 1 1 3 4
7 0 3 1 8 6 8
0 0 0 0 0
5
6
0
0 :
:
0 8
1
392

5
(b).
(c).
5
7.
1 0
9 0 1
9
7 0 0
0 0
6
1 0 2@6 1@6 6@6
7 0 1 1@3 8@3 6@3 8
0 0 0
0
0
5 6
5
6
5
6
1
2@6
1@6
9 2 :
9 1@3 :
9 8@3 :
:
9
:
9
:
x=9
7 0 8 + u7 1 8 + v7 0 8
0
0
1
6
0 1 0 0 1
0 1 0 1@2 1 :
:
1 1 0 0 1 8
0 0 1 1@2 2
Chapter Five
Section 5.3
1.
3.
W
HD15H W = [25@6]> y =
6 30@29 and x = [89@29 72@29 106@29]
1@2 1@2
0 8 > z = [2=4828 3=6552]W and
Q =7 1
0
1
x = [3=0690 2=4828 3=6552]W
Chapter Six
Section 6.1
1.
3.
5.
7.
9.
Left side = (95@3)h3w 2@3

Right side = 3[(95@9)h3w 2w@3 5@9] + 2w + 1
{(0) = 95@9 5@9
(a).
{k = Fh5w
(b).
{s = (3@5)w 13@25
(c).
{ = (138@25)h5w + (3@5)w 13@25
(a).
{k = Fh2w
(b).
{s = 10h3w
(c).
{ = 5h2w + 10h3w
(a).
{k = Fh5w
(b).
{s = (25@13) cos(w) + (5@13) sin(w)
(c).
{R = (38@13)h5w + (25@13) cos(w) + (5@13) sin(w)
{s = h5w h5 (3 + 2)g = w(3@5) 13@25
Chapter Six
Section 6.2
1.
3.
Compute the second derivative and show p{002 + n{2 = 0=

{(0) = (1@2)0 + 2 = 2
{0 (w) = (1@2) cos(2w)2 + 2( sin(2w))2
{0 (0) = (1@2) cos(0)2 + 2( sin(0))2 = 1
Compute {0 and {00 and show {00 + 4{0 + 4{ = 0=
{(0) = 1 + 0
{0 (w) = h2w (2) + 5(1h2w + wh2w (2))
(a).
(b).
5.
7.
9.
(a).
(b).
(a).
(b).
(a).
(b).
{0 (0) = 2 + 5(1 + 0) = 3
{k = f1 h3w + f2 h2w
{ = (3@5)h3w + (2@5)h2w
{k = f1 h4w + f2 wh4w
{ = 2h4w + 11wh4w
Hint: Subtract the two equations.
Hint: Show } must be equal to zero.
Chapter Six
Section 6.3
1.
3.
5.
7.
{01 = hw cos(w) + hw ( sin(w))

w
{02 = hw sin(w) +
h (cos(w))
1 0
(b).
Z (0) =

(c).
det (Z (0)) = 6= 0
(a).
{k = f1 cos(3w) + f2 sin(3w)
(b).
{ = 2 cos(3w) +s
(1@3) sin(3w)
s
(a).
{k = f1 hw cos(
2w) + f2s
hw sin( 2w)
s
s
(b).
{ = 1hw cos( 2w) +s(3@ 2)hw sin( 2w)
s
s
s
s
n=1:
( 3 + 3@2)h(2+ 3)w + ( 3@2)(2 + 3)h(2 3)w
n=4:
3hs2w + 6wh2w
s
s
n = 10 :
6h2w sin( 6w) + 3h2w cos( 6w)
(a).
Chapter Six
Section 6.4
1.
3.
5.
7.
9.
11.
{ = f1 hw + f2 h4w + (3@4)w + 1@16

{ = f1 hw + f2 h4w + (1@6)h2w
{ = f1 cos(2w) + f2 sin(2w) + 1 cos(w) + (1@3) sin(w)
{ = 6hw + (13@4)h2w + (3@2)w 7@4
{ = hw + (29@48)h4w + (1@3)h2w + (3@4)w + 1@16
1
(a).
{s = 1$
2 cos($w)
1
1
(b).
limz$1 (( 1$
2 ) cos(w) + 1$ 2 cos($w)) = (1@2)w sin(w)
(c).
Start with {s = w(d cos(w) + e sin(w))=
Chapter Six
Section 6.5
3.
{s = h3w y1 + h3w y2 = (2@9)w 1@3 where

y1 = (1@6)[(2@3)h3w w + (11@9)h3w ] and
y2 = (1@6)[(2@3)h3w w + (7@9)h3w ]
Chapter Seven
Section 7.1
1.
(a).
{
|

=
0
1
1 8
{
|
393
394
3.
(c).
(a).
(b).
5.
(c).
{
|
s
1s
h(4+ 15)w +
4 + 15

s
1s
h(4 15)w
f2
4 5
s
s
s
s
f1 = (4 15)@( 15) and f2 = (4 + 15)@( 15)
Replace

100(L) by 50 (L).

{
1
1
= f1
h(2@25)w + f2
h(6@25)w
|
2
2
f1 = 7@4
andf2 = 1@4
1
1
= (1)
h4w + 3
h2w
1
2

(b).
{
|
= f1
Chapter Seven
Section 7.2
1.
3.
5.
7.

s
1s
1s
0
1
(a).
Show
= (2 3)
1 4
2 3
2 3
s
(b).

Let u = 2 3and

0
0u
1
x1
=
implies
0
1 4 u
x2
s
s
(2 + 3)x1 + 1x2 = 0 and x1 + (2 + 3)x
s 2 = 0=
Let x1 = 1 and then
compute
x
=
(2
+
2

3)=
2 3
1
1
(a).
Show
= (1)
1 2
1@3
1@3

2 (1)
3
x1
0
(b).
=
implies
1
2 (1)
0
x2
(1)x1 + (3)x2 = 0 and (1)x1 + (3)x2 = 0=
x2 = 1@3=
Let
x1 = 1 and then compute

4@25 u
1@25
det(
) = (4@25 u)2 4@252 = 0 implies
4@25
4@25 u
u = 6@25 and
2@25=
4u
1
det(
) = (4 u)(1 u) + 2 = 0 implies
2 1 u
u = 2 and 3=

Chapter Seven
Section 7.3
1.
(a).
(b).
(c).
u = 5@2 and 5@2

u = 2 + l and 2 l
u = 0 and 2=

3.
(a).
u1 = 1 and u1 =
w
h
h5w
[(w) =
w
h
h5w
w
h
[(w)y(w) =
hw
(b).
(c).

1
1
; u1 = 5 and u1 =
1
1
w
h @2
hw @2
and [(w)1 =
h5w @2 h5w @2

5w
w
h
h (w@2 + 0)
h5w (w@2 4@10)
h5w
w 2@5
2@5
2w
3w
hw h5w
h @3
h @3
(d).
[(w)y(w) =
=
0
h2w @3
hw h5w

1
185@36
7@6
1
2w
3w
h +
+
w
(37@4)
h + (28@9)
1@6
1
2
109@36
1
1
17@26
(7@2)
h2w + (67@13)
h3w +
cos(2w)+
2
1
11@13

7@26
sin(2w)
10@13

5.
7.
Chapter Eight
Section 8.1
1.
3.
(c).
Replace letc = zeros(50,40) with

shade = 100 % for example
letc = shade*ones(50,40)
Vary red, green and blue in color_imagncsu.m
Chapter Eight
Section 8.2
1.
3.
5.
9.
In the code le pollenhist.m

replace pollen.jpg by mars3.jpg.

235 201 170
xqlw8(x 40) =
155 17
0

255 245 222
qhzx =
211 115 102
Using s = 1=5

255 234 191

qhzx =
171 27 14
Chapter Eight
Section 8.3
Have fun comparing the new images.
Chapter Nine
Section 9.1
1.
3.
395
vO(x) 210 = f80(1@v) fO(x)

O(x) = 210@(v + f) + f80(1@(fv) 1@(f(v + f)))
= O(210hfw + 80 80hfw )
x(w) = 130hfw + 80
(a).
Let } = hl2@5 and
6
5
1 1 1 1 1
9 1 } }2 }3 }4 :
:
9
2
4
} }3 :
F5 = 9
:
9 1 }3 }
7 1 }
} }4 }2 8
1 }4 }3 }2 }
396

5
(b).
9
9
F5 f = 9
9
7
1 1
1 }
1 }2
1 }3
1 }4
1
}2
}4
}
}3
1
}3
}
}4
}2
1
}4
}3
}2
}
65
:9
:9
:9
:9
87
2
8
9
1
3
: 9
: 9
:=9
: 9
8 7
23=0000
2=6910 + 9=4576l
3=8090 4=6695l
3=8090 + 4=6695l
2=6910 9=4576l
6
:
:
:
:
8
Chapter Nine
Section 9.2
1.
(a).
3.
(a).
Let } = hl2@35 and i = 1=

1 1 1
F3 sin(21t) = 7 1 } } 2
1 }2 }
l2@5
Let } = h
and
5
1 1 1
9 1 }4 }3
19
9 1 }3 }
F1
5 =
59
7 1 }2 }4
1 } }2
Chapter Nine
Section 9.3
Enjoy the image lters.
65
6 5
6
sin(21(0))
0=0
8 7 sin(21(1@3)) 8 = 7 1=5l 8
sin(21(2@3))
1=5l
1
}2
}4
}
}3
1
}
}2
}3
}4
6
:
:
:
:
8
Bibliography
[1] Robert L. Borrelli, Courtney S. Coleman, Dierential Equations: a Modeling Perspective, Wiley, 2004.
[2] Gonzales and Woods, Digital Image Processing, 2nd Edition, Prentice Hall,
2002, http://www.imageprocessingplace.com.
[3] Center for Research and Scientic Computing, North Carolina State University, http://www.ncsu.edu/crsc/ilfum.htm.
[4] The MathWorks Inc., http://www.mathworks.com.
[5] Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, 2000.
[6] Cleve Moler, Numerical Computing with MATLAB, SIAM, 2002,
http://www.mathworks.com/moler.
[7] R. Kent Nagle, Edward B. Sa and Arthur David Snider, Fundamentals
of Dierential Equations and Boundary Value Problems, Addison Wesley,
2004.
[8] Gilbert Strang, Introduction to Applied Mathematics, Wellesley-Cambridge
Press, 1986.
[9] United Nations, http://www.un.org/esa/population/unpop.htm.
[10] United States Census Bureau, http://www.census.gov/main/www/
cen2000.html.
[11] United States NASA, http://www.nasa.gov/vision/universe/
solarsystem/mer_main.html.
[12] Robert E. White, Computational Mathematics: Models, Methods and
Analysis with MATLAB and MPI, Chapman and Hall/CRC, 2004.
397
Index
Applications
angles and area, 30
circuit, 244
LRC tuning, 272
modulated LRC, 277
three-loop, 155, 236
two-loop, 103, 218
data modeling, 83, 171
ow rate identication, 196
multiple markets, 192
price, 89
radioactive decay, 177, 200
real estate appraisal, 194
sales prediction, 173
United States population, 201
world population, 173, 187
uid ow, 238
heading velocity, 25
heat conduction, 286
in a wire, 111, 123
parameter identication, 204
time dependent, 306
image, 90
contrast, 324
lter in 1D, 358
lter in 2D, 365
lters, 370
sharpen, 336, 339
inventories, 86
mass-spring, 251
damped forced, 270
dampened, 260
forced, 268
mixing
one-tank, 244
three-tank, 166, 293
two-tank, 283
motion of a mass, 49
mass-spring, 251
projectile, 244
projectile in space, 72
phasors, 43
torque, 34, 57, 61
truss
ve-bar, 149
six-bar, 133, 237
two-bar, 101
visualization, 100
work
in space, 54
independent of path, 34
on a ramp, 33
array operations, 99
auxiliary equation, 253
basis vectors, 213
block Gauss elimination, 231
box product, 62
Cauchy-Schwarz inequality, 28, 53,
98
characteristic equation, 253
complex algebraic equation, 6
n-th root, 15
complex function
exponential, 14
linear, 10, 11
quadratic, 11
square root, 12
complex number, 3
exponential form, 14
eld properties, 6
polar form, 4
399
400
convolution
two functions, 344, 346
two matrices, 364
two vectors, 356
cosine identity, 28, 52
cosine law, 22
Cramers rule, 80, 88, 161, 165
cross product, 59
curve tting, 83, 89
de Moivres formula, 15
determinant, 32, 59, 161
dierential equation, 90
general solution, 245
homogeneous second order, 252
homogeneous solution, 245
particular solution, 245
dimension of a subspace, 213
Dirac delta functional, 347
discrete Fourier transform, 351
1D version, 348
2D version, 361
convolution property, 357, 365
fast Fourier, 350
inverse, 354
dot product, 28, 52, 97
INDEX
Heaviside function, 347
helix, 72
histogram of an image, 321
homogeneous solution, 225, 252
ill-conditioned problems, 119
image, 90, 311, 313315
color, 319
lter, 365
lter in 2D, 366
p-w linear transform
contrast, 324
power transform
brighten-darken, 327
sharpen, 336, 339
impulse function, 347
inconsistent algebraic system, 225
initial value problem, 304
rst order linear, 243
second order linear, 250, 267
integer, 1
inverse matrix, 138, 165
lter matrix, 358

ltered signal or image, 358
xed variable, 222
oating point error, 2
oating point number, 2
Fourier heat law, 111
Fourier matrix, 349, 351
Fourier transform, 346
free variable, 222
Lagrange identity, 61
Laplace transform, 344
least squares, 172
least squares solution, 42, 184
line in a plane
distinct points, 39
parametric equations, 38
point and direction vector, 39
point and normal vector, 39
single equation, 38
slope-intercept, 38
line in space
parametric equation, 67
vector equation, 67
linear operator, 243, 253
linearly independent, 32, 64, 185,
212
logistic function, 202
LU factorization, 151
Gauss elimination method, 130

Gauss-Jordan method, 141
general solution, 264
MATLAB codes
aerialpower.m, 329
bigc(), 371
eigenvalue, 290
eigenvector, 290
equilibrium equations, 230
Eulers formula, 14
INDEX
bign(), 371
bigs(), 371
bigu(), 371
blur1d.m, 332
blur2d-ncsu.m, 334
bridge.m, 149
circuit3.m, 157
color-imagncsu.m, 319
complex-power.m, 12
complex-prod.m, 8
cycloid.m, 43
decay.m, 178
E-eq-bge.m, 240
E-eq-null.m, 240
euler-form.m, 16
tsine.m, 358
tsine2d.m, 366
ttrig.m, 353
ttrig2d.m, 363
ttrig2da.m, 362
lter-aerial.m, 375
lter-micro.m, 375
lter-ncsubu.m, 372
lter-ncsuid.m, 371
gauss-el.m, 136
heat-fmin.m, 206
heat-sol(), 206
heat4.m, 308
helix.m, 73
imagncsu.m, 91, 318
inv-mat.m, 147
letterc(), 91, 316
lettern(), 91
letters(), 91
letteru(), 91
lines3d.m, 75
log-sol(), 203
lrc.m, 278
ls-graph.m, 175
ls-heat(), 206
ls-log.m, 203
ls-rad(), 200
market.m, 177
marspower.m, 329
moonsharpen.m, 339
401
phasor.m, 44
pollenhist.m, 321
pollenlight.m, 323
pollensharpen.m, 337
pollenspread.m, 326
pop-us-fmin.m, 203
pop-us.m, 202
pop-world-fmin.m, 207
pop.m, 188
proj3d.m, 74
rad-fmin.m, 200
rad-sol(), 200
spread.m, 325
support.m, 136
varpar.m, 304
wire-time-cur.m, 124
wire-time.m, 114
yplrc(), 278
matrix
augmented, 99, 128
diagonal, 119
elementary, 129
Fourier, 349, 351
fundamental, 303
identity, 138
image, 90
inconsistent, 130
inverse, 138
lower triangular, 119
reduced row echelon form, 228
row echelon form, 222
transpose, 63, 182
tridiagonal, 122
upper triangular, 119
Wronskian, 255
matrix-matrix products, 109
matrix-vector products, 106
multiple solutions, 89, 226
Nelder-Mead simplex method, 199
Newtons law of motion, 20, 49
norm, 51, 53, 97
normal equation, 89, 184
nullspace method, 234
particular solution, 225, 264
402
phasor, 44
pivot component, 222
precision, 2
prime number, 1
Pythagorean theorem, 21
rank of matrix, 222
rational number, 1
real number, 2
reduced row echelon form matrix,
228
resonance, 270
roots of unity, 16
row echelon form matrix, 222
row operation, 129
Runge-Kutta numerical method, 278
Schur complement, 153
thermal conductivity, 112
undetermined coe!cients, 245, 264,
298
variation of parameters, 247, 276,
302
vector
2D, 19
3D, 47
addition, 23, 50, 96
angle, 24
augmentation, 96
column, 19, 47, 96
dot product, 97
inventory, 86
norm, 24, 51, 97
orthogonal, 97
parallel, 29, 53
perpendicular, 29, 53
position, 47
price, 86
row, 19, 47, 96
scalar product, 23, 50, 96
transpose of a column, 19, 47,
96
unit, 24, 48
INDEX
unit basis, 19
vector space, 23, 51, 97
basis vectors, 213
dimension of a subspace, 213
subspace, 209
nullspace, 211
range space, 211

Emacs A 072006 Mark

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Emacs A 072006 Mark

Hochgeladen von

Copyright:

Verfügbare Formate

Elements

1 Vectors in the Plane

4 Ax = d: Least Squares Solution

Multilinear Data Fitting . . . . . . . . . . . . . . . . . . . . . . . 191

6 Linear Initial Value Problems

7 Eigenvalues and Dierential Equations

A Solutions to Odd Exercises

Complex Numbers as Arrows . . . . . . .

Linear Combination of Vectors

Box with Fixed Volume . . . . . . .

5.3.1 Bar e with Four Forces . . . . . . . . . . . . . . . . . . . . . . . 237

7.3.1 Heat Diusion in Thin Wire . . . . . . . . . . . . . . . . . . . . 308

8.1.10 Image of Letter C . . . . . . . . . .

9.2.1 DFT of Sine and Cosine . . . . . .

Computer Sales Data . . . .

chapters are an introduction to image and signal processing.

Vectors in the Plane

Floating Point and Complex Numbers

CHAPTER 1. VECTORS IN THE PLANE

Floating Point Numbers

1.1. FLOATING POINT AND COMPLEX NUMBERS

z if and only if d = f and e = g>

CHAPTER 1. VECTORS IN THE PLANE

Figure 1.1.1: Complex Numbers as Arrows

Denition 1.1.2. The polar representation of a complex number with  =

1.1. FLOATING POINT AND COMPLEX NUMBERS

Example 1.1.3. Let } = 2 + l and z = 1 + 3l= Then

CHAPTER 1. VECTORS IN THE PLANE

So, { = 1@5, | = 7@5, which is easy to verify

1.1. FLOATING POINT AND COMPLEX NUMBERS

Figure 1.1.2: Norm(} 2 ) and Angle(} 2 )

= (d + el)(f + gl) = (df eg) + (ef + dg)l

CHAPTER 1. VECTORS IN THE PLANE

Let } = |}| (cos() + l sin()) and z = |z| (cos(*) + l sin(*)) so that

= |}| (cos() + l sin()) |z| (cos(*) + l sin(*))

Additional properties are summarized in the following theorem.

Complex Operations Using MATLAB

1.1. FLOATING POINT AND COMPLEX NUMBERS

Consider the real number 1@3.

Prove the remaining properties in Theorem 1.1.1.

CHAPTER 1. VECTORS IN THE PLANE

Complex Valued Functions

Linear and Power Functions

The linear complex function is

Example 1.2.1. Consider solving i (}) = (1 + l)} + 3 l = 7l= Let } = { + l|

1.2. COMPLEX VALUED FUNCTIONS

The linear rational complex function is

where constants d> e> f and g are complex numbers.

A quadratic complex function is

CHAPTER 1. VECTORS IN THE PLANE

cos(@2) and y = |}|

= 21@2 (cos(@6) + l sin(@6))

Note the solution of the complex quadratic equation } 2 = 1 +

1.2. COMPLEX VALUED FUNCTIONS

Figure 1.2.1: A!ne, Square and Square Root of z

The exponential function of a complex variable } = { + l| at rst look is

CHAPTER 1. VECTORS IN THE PLANE

(l|)2 (l|)3 (l|)4 (l|)5

Denition 1.2.1. Eulers Formula is

The general complex exponential function is

Application to Trigonometric Identities

1.2. COMPLEX VALUED FUNCTIONS

= (cos({) + l sin({))(cos(|) + l sin(|))

7 Eigenvalues and Dierential Equations

7.3.1 Heat Diusion in Thin Wire . . . . . . . . . . . . . . . . . . . . 308

Denition 1.1.2. The polar representation of a complex number with =

Let } = |}| (cos() + l sin()) and z = |z| (cos() + l sin()) so that

= |}| (cos() + l sin()) |z| (cos() + l sin())

cos(@2) and y = |}|

= 21@2 (cos(@6) + l sin(@6))

(|g| hl )1@2 = |g|

The second solution is |g|1@2 hl@2 = |g|1@2 hl@2 hl2@2 =

(hl hl2 )1@q

= |g|1@q hl@q hl2@q =

21@6 (cos(@12 + (2@3)0) + l sin(@12 + (2@3)0))

Figure 1.3.3: f2 = e2 + d2 2de cos()

Since 1 cos() 1> the inequality, which is called the Cauchy-Schwarz

Use the cosine identity for cos() to obtain