LINEAR

ALGEBRA
WITH APPLICATIONS

Otto Bretscher
Harvard University

PRENTICE HALL, Upper Saddle River, New Jersey 07458

 CONTENTS
Library of Congress Cataloging-in-Publication Data

Bretscher. Otto.
Linear algebra with applications I Otto Bretscher.
p. cm. Preface zx
lncludes index.
ISBN: 0-13- 190729-8
1. Algebras. Linear. I. Title.
QAI84.B73 1996
512' .5-dc21
94-36942
CJP
1 Linear Equations 1
1.1 Introduction to Linear Systems
Acquisitions Editor: GEORGE LOBELL
Editorial Assistant: GALE EPPS
1.2 Matrices and Gauss-Jordan Elimination 12
Assistant Editor: AUDRA J, WALSH 1.3 On the Solutions of Linear Systems 33
Editorial Director: TIM BOZIK
Editor-in-Chief: JEROME GRANT RICCARDI .
Assistant Vice President of Production and Manufacturing: DAVID W.
Editoriai/Production Supervision: RICBARD DeLORENZO
Managing Editor: LINDA MIHATOV BEHRENS
2 Linear Transformations 52
Executive Managing Editor: KATHLEEN SCHIAPARELLI 2. 1 Introduction to Linear Transformations and Their
Manufacturing Buyer: ALAN FISCHER Inverses 52
Manufacturing Manager: TRUDY PISCIOTTI
Director of Marketing: JOHN TWEEDDALE
2.2 Linear Transformations in Geometry 65
Marketing Assistant: DIANA PENHA 2.3 The Inverse of a Linear Transformation 86
Creative Director: PAULA MAYLAHN
2.4 Matrix Products 97
ArtDirector: MAUREEN EIDE
Cover and Interior Design/Layout: MAUREEN EIDE

Cover Image: nromcrown Chapel. copyrighl by R. Greg Hurlsey; Eureka Springs, Arkansas. v 3 Subspaces of IRn and Their Dimensions 128
© 1997 by Prentice-Hall, lnc. 3.1 Image and Kerne) of a Linear Transformation 128
Sirnon & Schuster I A Viacom Company
Upper Saddle River, NJ 07458
3.2 Subspaces of !Rn; Bases and Linear Independence 146
3.3 The Dimension of a Subspace of Rn 160
All rights reserved. No part of this book may be
reproduced, in any form or by any means,
without permission in writing from the publisher.

Printed in the United States of America.

4 Orthogonality and Least Squares 177
4.1 Orthonormal Bases and Orthogonal Projections 177
10987654321
4.2 Gram-Schrnidt Process and QR Factorization 198
4.3 Orthogonal Transformations and Orthogonal Matrices 207
ISBN 0-13-190729-8 4.4 Least Squaresand Data Fitting 218

Prentice-Hall International (UK) Limited, London

Prentice-Hall of Australia Pry. Limited, Sydney
Prentice-Hall Canada Inc., Toronto
s Determinants 239
5.1 Introduction to Deterrninants 239
Prentice-Hall Hispanoamericana. S.A., Mexico
Prentice-Hall of India Private Limited, New Delhi 5.2 Properties of the Deterrninant 250
Prentice-Hall of Japan, lnc., Tokyo
5.3 Geometrical Interpretations of the Deterrninant; Cramer's
Sirnon & Schuster Asia Pte. Ltd., Singapore
Editors Prentice-Hall do Brasil. Ltda., Rio de Janeiro Ru1e 272
Vi • Contents

/ 6 Eigenvalues and Eigenvectors 291

6.1 Dynamical Systems and Eigenvectors: An Introductory
Example 291
6.2 Finding t11e Eigenvalues of a Matrix 307
6.3 Findi ng the Eigenvectors of a Matrix 321
To my parents
6.4 Complex Eigenvalues 341
6.5 Stability 357 O~to and Margrit Bretscher-Zwicky
wzth love and gratitude
1 Coordinate Systems 371
7 .l Coordinate Systems in IR" 371
7.2 Diagonalization and Similarity 387
7 .3 Symmetrie Matrices 401
7.4 Quadratic Foro1S 4 J3
r 7.5 Singular Values 425

/ 8 Linear Systems of Differential Equations 438

8.1 An Introduction to Continuous Dynamical Systems 438
8.2 The Complex Case: Euler's Forrnula 458

/ 9 Linear Spaces 4 77
9.1 An Introduction to Linear Spaces 477
9.2 Coordinates in a Linear Space 496
9.3 Inner Product Spaces 511
9.4 Linear Differential Operators 528

APPENDIX A Vectors 546

Answers to Odd-Numbered Exercises A-1

Index I-1
 Key Features
PREFACE
. d ·I o n in the text to make the di cu. -
• Linea r rransformario~IS are mtroduce . :~~ f and easier to visualize. M apping,
ion of matri x operauon mo re mea nll1 ", u ' . .
and transf rmati n be ome a the me in the tex t thereaftet . . . .
. I . are used as a un tfytn g the m ' a
• Discrere and conrinuou dynanuca ·' .relll I" . f linear aJo ebra.
m tiYati n ~ r eiQ.enve tors. and a maJor app tcatton o "' .
A po li ce offl eer o n patrol at rni dni g ht, o runs an o ld joke, notice a man
.II 1i: nd an abundan f rhoughr-provoking (and occa. tona ll y
• The rea d er Wl crawlin g abo ut on hi s hand and knee unde r a treetlam p. He wa lk over to
deli2.htful ) problem and exerci e · investigate, where upo n the man expl ain in a tired and omew hat lurred o ice
- . d d d all throuohout the text. The major that he has lost hi hou ekeys. The poli ceman offers to help, and fo r th e n xt five
• Ab tracr concepr are mtro u e ~a Uc I f "' ·ali ty befo re the stuclent
idea are arefullv d Ye loped at \·ano u le e o genet minutes he too i search in g on bis hands and knees. At last he excla irn . . " A re
i intr duced t ab tra t ve tor pace · you absolu tely certain th at thi s is where yo u droppecl the keys?"
• \ isuali-aiion alld eollleri.I·cal internretation are emphas ized exten ively
r
"Here ? A bso lute ly not. I dropped them a block down, in the middl e o f the
street. "
throu.::hout.
"T hen w hy the dev il have you got me hun ti ng around thi s lamppo tT
' Because th i i where the li ght is ."
Hornepage
It is mathematics, and no t j ust (as Bismarck c laimed) po li tics, that con i in ·' the
art of the pos ibl e." Rather than search in the clarkness for so\uti o n to pro bl e m
of press ing interest, we contri ve a realm of proble ms whose intere t Jie · above a ll
in the fac t that so lutio n can concei vably be found.
Perhaps the lm·gest patch of li ght s urro unds the techniques of matri x ari th-
me ti c and algebra, ancl in part icttlar matri x mul tiplicatio n and row redu ctio n. Here
we mig ht begin with De ca11es, since it wa he who di scovered the co nceptu al
meeting-point of geometry ancl algebra in the identifi cation of Eu clidean pace
with ]!( 3 ; the tec hnique and application pro liferated in ce hi day. To organize
and clarify th o e i the role of a modern linear algebra course.

+ Computersand Computation
A n essenti al issue th at needs to be addressed in establishing a mathe matica l
methodo logy is the ro Je of co mputation and of co mput ing techno logy. Are the
pro per subjects o f mathe mati cs algorithm and ca lcul at io n . or are they grand the-
ori e and abstractio ns which evade the need for co mputatio n? l f the fo rmer. i
it important that the studc nt leam to carry out the computations \ ith penc il and
pape r, or hould the algorithm "press the calcu lato r· x - 1 button'· be a ll ' ed to
substitute for the tnditi o nal method of fi nd ing an in er. e? lf the lauer. . ho uld
the abstraction be taught thro ug h el aborate no tati onal mechani m . o r rhro ugh
co mputati onal exa mples and g raphs?
We seek to take a con istenr approach to these question : a lgorith ms and
co mputatio ns cu·e primary, and prec isely for thi · reaso n computer are not. Again
and again we exa mine the nitty-g ritty of row recluc ti on or matri x multipli ati o n in
order ro derive ne \ in ·igh t . Most of the proo fs. w h ther of rank- null ity the re m.
the volume-chang formul a fo r determinant , r the pectra l theore m for ·ymmetri
matri ces. are in thi , way tied to hand -o n proced ure . .
ix

X • Preface
The aim is not just to know how to compute the solution to a prob lern, b~t t:o
imagine the computations. The student needs to perform enoug.h,;ow reductl~ns
by band to be equipped to follow a Jine of argument of the form . .I f w~ calcu ate
. ' and to appreciate m advance
the reduced row echelon form o f such a ma tn x · · · , . .
what the possible outcomes to a particular com.putatton are. . .
In applications the solution to a problern IS hardly ~ore. J~portant than .rec-
. · Its· range o f v aJ1'd'1ty , and appreciating how sensitive It IS to perturbatwns
oomzmo .
of the i~put. We emphasize the geometric and qualitative nature of ~e sol~ttons,
notions . of approximation, stability, and "typicaJ" matrices. The dJscu~swn of
Cramer' s rule, for instance, underscores the value of closed-form solutJO~s . ~or
visualizing a system's behavior, and understanding its dependence from Imtlal
conditions. .
The availability of computers is, however, neither to be Ignored n~r r~-
gretted. Each student and instructor will h.ave to dec~de how much p~actice IS
needed to be sufficiently farniliar with the mner workings of the a!gonthm .. As
the explicit computations are being replaced gradu~lly b~ a theorettcal overview
of how the algorithm works, the burden of calculatton will be t~en up by te~h­
nology, particularly for those wishing to carry out the more numencal and applied
exercises.
It is possible to turn your Linear algebra course into a more computer orie~ted
or enhanced course by wrapping with this text either ATLAST Computer Exerctses
for Linear Algebra (1997, edited by S. Leon, E. Herman, and ~ · Faulkenb.erry) ?r
Linear Algebra Labs with Matlab , 2nd edition ( 1996 by D . Hill and D. Zitar.e lh).
Each of these supplements goes beyond just using the computer for computatwnal
matters. Each takes the standard topics in linear algebra and finds a method
of illuminating key ideas visually with the computer. Thus both have M-files
available that can be delivered by the Internet.
+ Examples, Exerdses, Applications, and History
The exercises and examples are the heart of this book. Our objective is not just to
show our readers a "patch of light" where questions may be posed and solved, but
to convince them that there is indeed a great deal of useful, interesting material
to be found in this area, if they take the time to Iook around. Consequently, we
have included genuine applications of the ideas and methods under discussion to a
broad range of sciences: physics, chernistry, biology, econornics, and, of course,
mathematics itself. Often we have simplified them to sharpen the point, but they
use the methods and models of contemporary scientists.
With such a !arge and varied set of exercises in eacb section, instructors
should have little difficulty in designing a course that is suited to their aims and
to the needs of their students. Quite a few Straightforward computation problems
are offered, of course. Simple (and, in a few cases, not so simple) proofs and
derivations are required in some exercises. In many cases, theoretical principles
that are discussed at Jength in more abstract linear algebra courses are here found
broken up in bite-size exercises.
Preface • Xi
The examples make up a significant portion of the tex t; we have kept abstract
exposition to a minimum. It i a matter of ta ·te whethe r general theorie . . hould
gi ve 1i se to s pecific examples, or be pasted tagether from them . In a text uch a
thi s one , attempting to keep an eye on applications, the latter i. clearly pre ferable:
the examples alway. precede the theorems in thi s book.
Scattered throughout the mathemat ical exposition are quite a few name and
dates , some hi storical accounts , and anecdotes as weil. Students of mathematics
are too rare ly shown tbat the seeming ly strange and arbi trary concept they tudy
are the results of long and hard struggles. It will encourage the readers to know
that a mere two centmies ago some of the most brilliant mathematician were
wrestling with probl ems s uch as the meaning of dimension or the interpretation
of e", and to realize that the advance of time and understanding actually enables
them , wi th some effort of their own, to ee farther than those g reat mind .
+ Outline ofthe Text
Chapter 1. This chapter provides a careful introduction to the olution
of systems of linear equations by Gauss-Jordan elimination. Once the concrete
problern is solved, we restate it in terms of matrix formali sm, and di cu the
geometric properlies of the solution s.
Chapter 2. Here we raise the abstraction a notch and reinterpret matrice
as linear transformations. The reader i introduced to the modern notion of a
function , as an arbitrary a ociation between an input and an output, which Iead
into a di scussion of inverses. The traditional method for finding the inverse of
a matrix is expl ai ned: it fits in naturall y as a sort of automated algorithm for
G auss-Jordan e limination .
We define linear transformations primaril y in tem1 of matlice . ince that
is how they are used ; the abstract concept of linearity is presented as an auxil iary
notion. Rotation in IR 2 are empha ized, both as archetypal, easily-vi ualized
examples, and as preparation for future applications.
Chapter 3. We introduce subspaces. im ages and kerne! , linear indepen-
dence, and base , in the context of IR" . This allows a thorough discu ion of the
central concepts, wi thout requiring a digres ion at thi early stage into the Jippery
issue of coorclinate systems.
Chapter 4. This chapter includes some of the mo t basic application • . We
introduce orthonormal ba e and the Gram-Schmidt proce s. along with the QR
factori zation. The calculation of correlation coefficients i di cussed, and the
important technique of lea t-squares approximations i explained. in a number of
differe nt context .
Chapter 5. Our di cuss ion of determinants is algorithmic. based on the
counting o f '·patterns" (a transparent way to deal with pernmtation ) . We deri ve
the properli es of the deterrninant from careful analysis of thi procedure. and tie
it Iogether with Gauss-Jordan elimination . The goal i to prepare for the main
application of determinant. : the computation of characteri tic polynomiaL .
Chapter 6. This chapter introduces the central application of the latter half
of the text: linear dynamical systems. We begin with di crete stems. and arc
 Preface • Xiii
xii • Preface
naturally led to seek eigenvectors, which characterize the long-t~r~ behav_i~r of I have received valuable feedback from the book's revtewers:
.
the system. Qualitative behavior is emphasized, particularl_y s~ab!ltty ~ondi_t.Ion .
Camplex eigenvalues are explained, without apology, and ued mto earller dLscus- Frank B ~atro~ s, University of Pittsburgh
sions of two-dimensional rotation matrices. Tracy Bibelnieks, University of Minnesota
Cbapter 7. Having introduced the change of coordinates to an eigenbasis Jeff 0 · Farmer, University of Nortbern Colorado
in Chapter 6, we now explore the "general" theory of coordin~te s~stem~ (s till K~nrad J. Heuvers, Michigan Technological University
finnly fi xed in ~~~, however). This Ieads to a more com~rehens!ve d1scusst~n of Mic?ael ~allaber, Washington State University
diagonalization, and to the spectral theorem for symmetrtc matnces. These tdeas Dame! King, Oberlin College
are applied to quadratic forms, and conic sections are explained. Richard Kubelka, San Jose State University
Chapter 8. Here we apply the methods developed for discrete dynamical Peter C. Patton: University of Pennsylvania
systems to continuous ones, that is, to systems of first-order linear differential Jeff~ey M. Rab~n , Uni:ersity of California, San Diego
equations. Again, the cases of real and complex eigenvalues are discussed. D an~el B. _Shapiro, Ohio State University
Chapter 9. The book ends where many texts begin: with the theory of David Stemsaltz, Technische Universität Berlin
abstract vector spaces (which are here called "linear spaces," to prevent the con-
fusion that some students experience with the term "vector"). The chapter begins D L my editor ' Ge o.rge. Lobe11 ' •.or his
I alsod thank
and Rieb . encouragement and advice
with assembling the underlying principles of the many examples that have come ar e orenzo for coordmat.Ing book production. ,
before; from this lush growth the general theory then falls like ripe fruit. Once the
reader has been convinced that there is nothing new here, the book concludes with
struct~~:l d~:~~~t:,:n~~!dth~; :!~~ ~~~g~~pported by a grant from the In-
important applications tbat are fundamentally new, in particular Fourier analysis.
(with David Steinsaltz) Otto Bretscher
Department of Matbematics
Harvard University
Cambridge, MA 02138 USA
e-mail: otto@math.barvard.edu

+ Acknowledgments
I first thank my students, colleagues, and assistants at Harvard University for the
key role they have played in developing this text out of a series of rough lecture
notes. The following professors, who have taught the course with me, have made
invaluable contributions:

Persi Diaconis
Edward Frenkel
David Kazhdan
Barry Mazur
David Murnford
Shlomo Sternberg

I owe special thanks to William Calder and Robert Kaplan for their thoughtful
review of the manuscript.
I wish to thank Sylvie Bessette for the careful preparation of the manuscript
and Paul Nguyen for bis well-drawn figures.
I am gratefu~ to t~ose who _have contributed to the book in many ways:
Me_noo Cung,. SrdJan I?tvac, Robm Gottlieb, Luke Hunsberger, Bridge! Neale,
Akilesh Palantsamy, Rita Pang, Esther Silberstein, Radhika de Silva, Jonath an
Tannenhauser, Selina Tinsley, and Larry Wilson.

. - -- - ·- - - - -.
 LINEAR EQUATIONS
INTRODUCTION TO LINEAR SYSTEMS
Traditionally algebra was the art of solving equations or systems of equation
The word algebra comes from tbe Arabic al-jabr, wbich means reduction. Tbe
term was first used in a mathematicaJ sense by Mohammed al-Khowarizmi, who
lived about A.D. 800 in Baghdad. Linear algebra, tben, is tbe art of solving systems
of linear equations.
The need to solve systems of linear equations frequently arises in mathemat-
ics, statistics, physics, astronomy, engineering, computer science, and economics.
Solving systems of linear equations is not conceptually difficult. For small
systems ad hoc methods certainly suffice. Larger systems, however, require more
systematic methods. The approach generally used today was beautifully explained
2000 years ago in a Chinese text, the "Nine Chapters on the MathematicaJ Art"
(Chiu-chang Suan-shu), which contains the following example. 1

Tbe yield of one sheaf of inferior grain, two sheaves of medium grain,
and three sheaves of superior grain is 39 tou. 2 The yield of one sheaf of
inferior grain, three sheaves of medium grain, and two sheave of superior
grain is 34 tou. The yield of three sheaves of inferior grain, two sbeaves of
medium grain, and one sbeaf of superior grain is 26 tou. What i the yield
of inferior medium, and superior grain?

In this problem the unknown quantitie are the yield of one sheaf of infe-
rior, one sheaf of medium, and one sheaf of superior grain. Let us denote these

1
8. L. v.d.Waerden: Geometry and A lgebra in Ancient Civilizariom. Springer-Verl ag. Berlin , 1983.
1 A 1011
is a bronze bowl used as a food container during the middle and late Chou dynasty (c. 900-
255 B.C.).
1
 Sec. 1.1 In trod uct ion to Li nea r y te ms • 3
2• h ap. 1 Linea r Equatio n s
. Tl e probl m ca n th
1y . . 1
11 be repre e nted by Fi nall y, we e limin ate the vari able z off the di agonal.
quantiti e by x , v. and ;::,_ re pecu ve_
the followi ng system of linear equ auons. x + 5z = 49 - 5 x Ia t equation X = 2.75
r +2r + 3;:: = 39 y - z= - 5 + last equ ati on y = 4.25
+ +
:r 3:r 2;:: = 34 z= 9.25 ~ - = 9.25
3x + 2y + .: = 26
T he yields o f inferi or, mediu m, and superior grain are 2.75 , 4.25, and 9.25 tou
To olve for .r . y. and -, we need to transform this sy t m from the form
per shea f, re pecti vely.
X+ 2_,, + 3;:: = 39 into the form y = ....
X - .. .
By ubstituting these va lue , we can check that x = 2.75, y = 4.25 , z = 9.25
x + 3v + - ?. =
34 is indeed t11e so lution of the ystem.
3x + - ·V + z =
26 z = ...
· . . 1 t rm th at are off the di agonal. those 2.75 + 2 X 4.25 + 3 X 9 .25 = 39
ln othe~ words. we . need to elt nundate ~~~ t1~e coeffi cient of the vari ables along 2.75 + 3 X 4.25 + 2 X 9.25 = 34
circled ll1 the equatwn below. an ma
the diagonal equal to l . 3 X 2.75 +2 X 4.25 + 9.25 = 26
X+ @ +® =3 9
Happ ily. in li near algebra you are almost always abl e to check your olutions.
G + 3y + ® = 34 It wi ll help you if you get into the habi t of checking now.

® +CB + z = 26 .
We can accompli h the e goals step by step, one vari a_ble at a t~me. ln the
. l' fi ed y te ms of equations by addtng equau ons to one + Geometrie Interpretation
past you may I1ave nnp t . . bl f
another or subtracting them. In thi y tem we c~n elimin ate the van a e .r rom How can we interpret thi s resu lt geometrically? Each of the t11 ree equ ations
the eco nd eq uati.on by ubtracting the fi rst equauon from the second . of the system defi nes a plane in x - y- z space. The sohttion et of the ys-
- 39 ~ x + 2 y + 3z = 39 tem consists of those points (x. y , z) which lie in all three plane , that is, the
x + ?- Y + 3 Z - 7 in tersection of the three planes. Algebraically speaki ng, the solution set con-
x + 3y + 2- = 34 - 1 t equation Y- = - 5
sists of those ordered triples of numbers (x , y, 7) which ati fy all three equa-
3x + 2y + - = 26 3x + 2y + z = 26
tion simultaneously. Our computations show that the ystem ha only one so-
To el iminate the variab le x fro m the thi rd eq~ation , we subtract_the first eql~ation lu tion. (x , y, z) = (2.75 . 4.25 , 9.25 ). T hi s means that the plane defined by the
from the tbird equation three times. We m ul ttply the first equatwn by 3 to oet three equations intersect at the poi nt (x, y . -) = (2.75 . 4 .25. 9.25 ), a hown in
Figure 1.
3x + 6y + 9- = 11 7 (3 x Ist equation).

and then ·ubrract thi result from the th ird equation.

x + 2y + 3z = 39 ~ x + 2y + 3z = 39
y - z= - 5 Figure 1 Three planes in space,
V- Z = - 5 intersecfing at a point.
3x + 2 y + z = 26 - 3 x I t equation - 4y - 8z = - 9 1

Similarly, we eliminate the variab le v off the di agonal.

x + 2y + 3z = 39 -2 x 2nd equation ..r + 5z = 49
y- z= - 5 y- z= - 5
- 4 y - 8z = -9 1 + 4 x 2nd eq uation - 12z = - I II

ß efore we elim in ate the vari able z off the diagonal, we makc the coeffi cient of ;::
011 the diagonal equal to I, by di vid ing the last eq uation by - 12.

x + 5z = 49 x + 5z = 49
y- z= - 5 ~ y- ;:: = - 5
- 12z= - 11 1 -:- (- 12) z = 9.25
4• Chap. 1 Linear Equations
Sec. 1.1 Introduction to Linear Systems • 5
2x+4y+ 6z= 0 -:-2 x + 2y + 3z = 0 ---;
4x + Sy + 6z = 3 4x+ 5y+ 6z =3 - 4(I)
7x+8y+ 9z = 6 7x +8y +9 z =6 - 7 I)
x + 2y + 3z = 0 x +2y + 3z = 0 -2(II)
- 3y- 6z = 3 -:- ( -3) y+ 2z = - 1 ~
- 6y - 12z = 6 - 6 y - 12z = 6 + 6(U)
X - Z= 2
y + 2z = - L x - z= 21
0= 0 l y + 2z = - 1

After omitting the triv ial eq uatio n 0 = 0, we have onl y two equations with
three unlmowns. The solution set is the intersection of two nonparallel planes in
Figure 2a Three planes having a space, that is, a line. This sy tem ha in fi nitely many solu tions.
line in common. The two equ ations above can be written as fo ll ows:

x= z+ 21
l y = - 2z - 1 ·

We see that both x and y are determined by ~- We can freel y choose a value of
z, an arbitrary real number· then the two equ ations above give us the values of x
and y for thi s choice of z. For example:

• Choose z = l. Then x = z+2 =3 and y = - 2~ - l = -3 . The solution i

(x y '") = (3, - 3, l) .
• Choose z = 7. Then x = z + 2 = 9 and y = - 2z - 1 = -15 . The solution
Flglll'e 2b Three planes with no i (x . y, z) = (9 - 15. 7).
common intersedion.

More generally, if we choose z = t . an arbitrary real number. we get x =

While three planes in space usually intersect at a poi?t, they _may have a
t + 2 andy = - 2t - 1. Therefore the general solution is
li in common (see Figure 2a) or may not have a common tntersectwn at all, as
= (t + 2. - 2t - = (2, - l. 0) + t ( l , - 2, l ).
s;~wn in Figure 2b. Therefore, a system of three _equations with t~ree unknowns (x, y. z) I, t)

may have a unique solution, infinitely many solutmns, or no solutwns at all. Tbi eq uation represent a line in space, a shown in Figure 3.

+ A System with Infinitely Many Solutions Figure 3 The line (x, y, z) =

Next, let's consider a system of linear equ ations that has infinitely many solutions: (t + 2, - 2t - 1, t).

2x +4y + 6z = 0
4x + Sy + 6z = 3
7x + 8y + 9z = 6
We can solve this system using elimination as discussed above. For sim-
plicity, we Iabel the equations with Roman numerals.
6• Chap. 1 Linear Equations
Sec. 1.1 Introduction to Li near Systems • 7
+ A System without Solutions . . . o urself to o btain the result shown. 17. Fi nd all o lutions of the linear syste m
In the system below, perform the elimmauons y

.
x· + 2y + 3z = 0 x - z = 2·1
+~= - I
3x
x + 2y =
+ Sy =
aI
b '
4x + 5y + 6z = 3 ~ y 0 = -6
7x+ 8)+9z= 0 whe re a and b are arbi trary constants.
18. Find all o luti ons of the linear system
Whatever values we choose for x, y, an_d z, the equ1ation u· s0 = - 6 cannot be
.
satisfied. T his system is incOJlSIStellt, that is ' tt has no so u on . x + 2y + 3z = a
x + 3y + 8z = b ,
EXERCISES x + 2y+ 2-=c
. man as three linear equations with w here a, b, c are arbitrary con tants.
GOAL Set up and solve systems Wlt? as d ~eir solutions geometrically . 19.
three unknowns, and interpret the equatwns an Consider a two-commodity market. Whe n the uni t price of the product are
P1 and P2, the quantities demanded, D 1 and D 2 , and tbe quantitie supplied,
S1 and S2 , are gi ven by
.
In E xerclSes 1 to 10• find all solutions of the linear
. systems using e li mination as
discussed in this section. Then check your solutJons. D1 = 70 - 2 PI + P2, St=- 14+3 P 1
D2 = 105 + P1- P_,' s2 = - 7 + 2 P2.
x + 2Y = 1
2x + 3), = ~
I 14x + 3y = 21
2. 7x + 5y = 3
3 12x + 4 y = 31
. 3x + 6 y = 2
1. I a. W hat i. the re lations hip between the two commodi tie ? Do they compete
like Vo lvos and BMWs, or do they complement one anotber, like shirt
x + 2y+3z= 8 and tie ?
2x + 4y = 21 5 12x + 3Y = 0 I y
6. x + 3 + 3z = 10
4. 13x +6y =3 · 4x +5 y =0 x+2y+ 4 z = 9 b. Find the equilibrium price , that is, the price fo r w hich s upply equaJ
dem and for both products.
x + 2y + 3z = 1 + 2y +
x 3z = 0 20.
The US economist and N obel laureate Wassily Leonti ef (bom 1906 in
1. x + 3y + 4 z =3 8. 4x + 5 y + 6z = 0
S t. Petersburg, Russia) was interested in the follo wing question : Wbat o ut-
x + 4y + 5z = 4 7x+ 8y + i0z = 0
put ho uld each o f the indu stries in an economy produce to ati sfy the to tal
x +2y+ 3z = 1 x + 2y + 3z = 1 demand for all products? Here we consicler a very s imple example of inpu t-
9. 3x + 2 y + z =1 10. 2x+ 4y+ 7z = 2 o utput analysi , an economy witb o nJy two indu tri e , A and B . A s ume th at
7x +2) - 3z =I 3x + 7 y + 11 z = 8 tbe con umer demand for tbeir products is, re pecrively, I ,000 and 780, in
mi!Jj ons of dollars per year.
In Exercises 11 to 13, find all solutions _of t~e linear systems . Represent your
solutions grapbically, as intersections of Lmes m the x -y plane.

11 X - 2y = 21 X - 2y = 31 13. I2x -
X - 2y = 31
· I
3x + 5 y = 17 12. I2x -4y =6 4 y= 8
Con umer
In Exercises 14 to 16, find all solutions of the linear systems. Describe your
solutions in terms of intersecting planes. You need not sketch these planes.

x + 4y + z = O x+y- z = O Which outputs a and b (in rnillion of do llar per year) hould the two in -
14. 4x +i3 y + 7z =0 15. 4x - y + 5z = 0 du tries generate to satisfy the demand ? You may be tempted to ay 1.000
7x +22y +l3z =l 6x + y + 4z = 0 and 780 respecti vely; bu t thi ngs are not quite a imple as rh at. We have
X + 4y + Z =0 to take into account the inte rindu try demand a weU. Let u sa indu try
16. 4x +13 y + 7z = 0 A produces e lectri city. O f course, produ ci ng almos t any product will require
?x + 22y + 13z = 0 e lectric power. Suppo e thal indu stry B needs I O!t worth of e lecu·ic ity for
each $ 1 of outpur B produce , and that indu try A net:ds 20~ worth of B · s
8• hap. 1 Linear Equation Sec. 1.1 Introduction to Linear System • 9
F d th output a and iJ needed
product for each l of output A produces. m_ 25. Consider the linear system
to ati fy both consumer and interindu stry demand .
X+ y - Z = -2
3x - 5 y + 13 z = 18
x- 2y + 5z = k

where k is an arbitrary number.

a. For which val ue( ) of k does this system have one or infinitely many
solutions?
b. For eacb value of k you found in part a, how many solution s does the
system have?
c. Find all solutions for each value of k .
21. Find the outpul a and b needed to sati fy the consumer and interindu stJy 26. Consider the linear system
demands given below (see Exercise 20).
x+ y- z =2
X+ 2y + Z = 3
x + y + (k2 - 5)z = k

where k is an arbitrary con tant. For which choice(s) of k does this sy tem
have a unique solution ? For which cboice(s) of k does the system have
infinitely many olutions? For wbich cboice(s) of k i the system incon istent?
Consumer 27. Emile and Gertrude are brother and sister. Emile has twice as many sister
as brothers, and Gertrude has just as many brothers as sis ters. How many
ch.ildren are there in this family?

22. Consider the differential equation 28. In a grid of wires, the temperature at exterior mesh points is maintained at
constant values (in oq as shown in tbe figure below. When the grid i in
d2 x dx thermal equilibrium, the temperature T at each interior mesh point i the
- - - - X= CO (t
dt 2 di average of tbe temperatures at the four adjacent points. For example,
Thi equation could de cribe a forced damped osc_illator, a we ~i ll see in
T3 + T, + 200 + 0
Chapter 9. We are told that the differential eq uat 1on has a solut1on of the T2 = 4
form
Find the temperatures T1, T2 , and T3 wben the grid is in thermal equilibri um.
x(r) = a sin(t) + öcos(l).
Find a and b, and graph the solution .
23. Find all solutions of the system

1-~~~8~=~ 1· for
200°
.--- ---.-- ---...oo
V a. >.. = 5, b. >.. = 10, c. >.. = 15.
24. On your next trip to Switzerland, you should take the seeni e boat ri_de from
Rheinfall to Rheinau and back. The trip downstream from Rbemfa U to
Rheinau takes 20 minutes, and the return trip takes 40 minutes· the distance
._---~---~---~00
between Rheinfall and Rheinau along the river is 8 kilometers. How fast
'2 does the boat travel (relative to the water), and how fa t does the river Rhein
ftow in thi s area? You may assume both s peeds to be con tant throughout the
journey.
 -· c
Se . 1.1 ln troductio n to Linear y tcm • 11
r.
hap . 1 Linea r Equ ations X y
2
. . . " . ol nomial of the form .f(t) = 0 + hl + c t )
29. Fi nd a pol 1110 ITII UI of degi e - (a I? Y _ 1) (2 ,) and (3. 13). k · tch thc
who e graph g e. through the po1nt ( I. · ·- '
• oraph of t.his polynomlal. ,
~ . . 2 .1 ol nomial of thc fo rm .f (r ) = a + /JI + er - )
30. Fi nd a polynomwl of cleglee (< P . Y (? ) ('"' · ) where p q r
1 tl · ohtl1e pomts(l.p). _,q . .), / , '.'
who e grap 1 goe 11ouo I a ol nomi al exi. t for nII choices of p,
are arbitrary co n tanL. Doe. . uc 1 p
2 l
q. r?
m
31. Find all point (n. b. c) in pace for which th

x + 2y + 3z = a
4x+5y+6::=b Exp lain briefly how you fou nd these graph . Argue geometri call , without
7x + 8v + 9-= c solving the ystem algebraically.
b. ow olve the system algebraically. Verify th at the graph yo u sketched
in pan a are com pati ble with yo ur algebraic olution.
has at least one -olu tion. 34. "A certain person buy . heep, goat and hog , ro the number of 100, fo r
. . . I I a y to ol e when they are in trian g ultr form ,
32. Linear s tems are pm t1 u ar Y e t
l 00 crown ; the sheep cosr him a crown a-piece: the goat 1~ crown ; and
that is. all entri above or below the diagonal are zero. the hog , 3 ~ crow ns. How many had he of eachT (From Leonhard Euler:
Elements of A lgeb ra. St. Peter. burg, 1770. Translated by Rev. John Hewlett. )
= - 3 Find al/ olu tions to th i problem.
- 3x 1 + x2 = 14 35. Find a y tem of linear equ ations with three unkn own who e oluti on are
a. X 1 + 2X2 + .\"3 = 9
-.\"1 + 8.r1 - 5.r3 + X ,J = 33 X = 6 + 5t . y= 4 + 3t .
where t is an arb itrary constant.
Solve thi lower triangular y tem by Forward sub titution, fi nding first x1,
36. Bori s and Marina are hopping fo r chocolate bar . Bori obse rve , '·If 1 add
then x 2 , then x 3 , then x.~. half my money to yours, it \ ill be enough to buy two chocolate bar .·· Marin a
b. Sol e the upper triangu lar ystem 11
naively asks. "Tf 1 add half my money to yours. how man_ can we bu ?" Bori
repl ie , ·'One chocolate bar." How much money did Bori have? (Fro m Yuri
.x 1 + 2x 2
- X3 + 4.r4 = -3 Chern yak and Robert Rose: T!Ie Chicken from Minsk. Ba ic Book . New
x2+ 3x3+ 7x-1= 5 York, 1995.)
x3 + 2x4 = 2 37. Here i another method to olve a system of linea r eq ualionc: Sol e one of
X-1 = 0 the equations for one of the vari ables, and substitute the re ult in to the other
eq uations. Repeat this process unril you run out of vari able or eq uatio n .
33. Consider the linear ·ystem Co nsider the exa mple di cu ed earl y in thi ecti on:
.r +-.v+3 - =39
X+ y= I x+ 3y+ 2z =34
I 3.r + -Y + :: = 26
x+ - v = 1
2"
We can olve the fi rst equ ation for x .

where r is a nonzero constant. .\" = 39- 2y- 3:::

a. Derermi ne the x - and y-intercepts of the lines x + y = I and x + ( t /2) Y = t ' Then we . ubstitute into the other equ ati ons.
ketch these Ii nes. For whi ch values of the constant 1 clo these line
intersect? For Lhese va lues of 1, the point of intersecti on (x , y) depencl on (39 - 2y- 3::) + 3y + 2:: = 341
the choice of the co nstant 1, that is, we can co n: ider x ancl y as [uncti ons I 3(39-2y -.)::) +2y + .:= 26
of 1. Draw rou gh sketche. of these functi ons.
12 • Chap. 1 Linear Equatio ns Sec. J .2 Ma trice and Gau -Jordan Eliminat io n • 13
We can s implify. The four co lum n f Lhe matri
.1
1-4~ =8z : ~~I
/ ,1,

Now -' =z- 5, o th at -4(z - 5) - 8z = - 91 , or T he th ree rows of the matri x ~ / [ 2~

- 12z = - 111. Note rha t the first co1umn of thi s matri x correspo nd to the first variab le of Lhe
111 syste m , w h.il e the first row corre p nds Lo the First eq uation .
\ e find tbat z = - = 9.25 . Then /
lt is c ustomary to Iabe l the e ntrie of a 3 x 4 m atri x A with doubl e sub cript
12
as fo ll ows:
y = z - 5 = 4.25
a,2 Cl t3 Clt4]
and C/22 a 23 C/24 .

x = 39 - 2y - 3z = 2.75 .
C/32 C/33 C/34

The fir t s ubscript refer. to the row, and the econd to the co1umn: The e ntry aij
E plain h this method i es entially the same as the method di c u sed in i · located in the ith row a nd the jth co lumn .
thi ection: only the bookkeeping is different. Two matrice A and Bare equa l if they have the sa me ize and corresponding
entries are eq ua l: aiJ = biJ.
If the number of rows of a m atrix A eq uaJ the number of column (A i
n x n). then A is ca ll ed a square matrix, and the e ntries a 11 , a2_ , ... C/ 1111 form the
(main ) diagonal of A. A square ma trix A is called diagonal if all its entrie off
.2 ~L\TRICf.S ~rn GA"CSS-JORDAN the main diagonal are zero, that is a;j = 0 whenever i =1- j. A quare matrix A is
ELDfiNATIO_ - called upper triangular if all it entries be low the m ain diagonal are zero. that is.
a;j = 0 whenever i exceeds j. Lower triangular ma trice are defined analogou Jy.
. ~-rems of linear equations are u uall ol ed b compurer. Suppo e A matrix whose en tries are a ll zero is called a -ero matrix and is denoted by 0
·o 1 ·e e following tem on a omputer: (regardles of its ize) . Cons ider the matrices
+ 4: =-
c~ [~
[I4 [I 42] '
~l
0
I
X - )'

~]
2
r -r 5y + -= 5 . A -- B-
- 3 3
1~X !Oy - -= 1 5
0

un
Whar information about this stern would tbe compute r need to ol e it? Witb the 0
right oftware. alJ you need to enter i the pattem of coefficient of tbe ariable
and the numbe on tbe rigbt-hand ide of the equations:
D = [~ ~ ]. E= 0
2
8
5
4 2]
I 5 .
The matrices B, C , D , E are square , C is diagonal , C and D are uppe r trian g ular ,
and C and E are lower triangular.
10 - 1 I M atrice with only one column or row are of particttlar interest.

All the i.nformation about tbe y tem i conveniently stored in thi arra of num-
1
be ed matri:x. Sin e tbe matri.x abo e bas three rows and four colurnns, A ma tiix with only one column i ca!led a co lumn vector, or simply a vector .
ed · x read ""three b four' _ The entries of a vector are called its compon.enrs . The set of all co lumn ec tor
with n components is denoted by IR".
A ma trix with only one row i called a row vecror.
In th.is text the term vecror refer to co lumn vector , unles stated
otherwise. The reaso n for our preference for column ec tor will b c me
in thi sense by lhe Engüsh llllllhernati ian appare nt in the next section .
 Sec. 1.2 Matrices and Gauss-Jordan Elimination • 15
14 • Chap. 1 Linear Equations
but. working with the augmented matrix requires less writing, saves time and i
Example of vectors are easter to read. lnstead of dividing an equation by a scalar, 1 you can divid~ a row
by a scalar. lnstead of adding a multiple of an equation to another equation you
can add a multiple of a row to another row. '
As you p~rform elimination on the augmented matrix, you should always
remernher the bnear system lurking behind the matrix. To illustrate this metbod
~e perfon11 the elirnination both on the augmented matrix and on the linear syste~
a ( olumn) vector in ]]{4 , and [ I 5 5 3 7 ] a row vector with five compo- lt represents.
111
nent . Note that the 11 columns of an m x n matrix are vectors in 1R • 8 4 2x+ 8y + 4z = 2 -;-2
In previous cour e in mathematic or phy ic , you may h~ve thought about 5 1 2x + 5 y + z= 5
vector from a more aeomet1ic point of view. (See the appendtx fo r a ummary 10 - 1 4x+ 10y - z=
of ba ic fact on vec t~rs.) In thi ou r e it will often be helpful to think about a
vector numerically, as a (finite) eq uence of numbers, which we wi ll u ual ly write + ,(.

[~
in a column. 4 2 x+ 4y + 2z = 1
In our digital age, infom1ation i often tran mitted and storecl as a equence 5 1 -2(1) 2x + 5 y + z= 5 -2(D
of numbers. that i , as a vector. A sequence of 10 econd of mu ic on a CD i 10 - 1 - 4(D 4x + 10y - z= -4(D
tored a a vector with 440,000 components. A weather photograph ta.ken by a
satellite is digitized ancl transmitred to Earth a a sequence of nurober .
,(. +

[~
Consider aga.i n the sy tem 4 2 x + 4y + 2z =
-3 -3 ; ] + (-3) -3 y - 3z = 3 -;- (-3)
2x + 8y + 4z = 2 - 6y - 9z = - 3
-6 -9 -3
2x+ 5 + z =5
4x + JOy- z = l .j, ,(.

[~
Sometimes we are interested in the matrix
4 2 I] -4(ll) x + 4 y + 2z = 1 - 4(ll)
1 1 -1 y+ z= -1
-6 -9 -3 +6(ll) - 6y - 9 = -3 +6(ll)
,(. ,(.

wbicb contains the coeffi cients of the sy tem called ils coefficient matrix.
By contrast, tbe matrix [~
0
1
-2
1 -15] X

y +
2 =
z = -1
5

0 -3 -9 ...;-(-3) -3z = - 9 ...;-( -3)

8
5
4 2]
I -
,(. ,(.

~ +2(Ill)
-n
0 -2 +2(TII) X 2z = 5
10 - 1
1 l - (fll) y + z= -1 - (lll)
which di spl ays all the numerical infom1ation containecl in the system is called its 0 1 z= 3
augme11ted matrix. For the sake of clarity, we will often indkate the position of ,(.
the equal signs in the equations by a dotted line.

[~ 11]
0 0 X = 11
8 4 : 2]
5 1 : 5 0
1 0 -4
3
y =-4
z= 3
10 - 1 : I

Even when you so lve a linear system by hand rather than by computer, it
may be more efficient to perfon11 the e lirnination on the augmented matrix rathe r 1In vector and matrix algebra, the term " calar" is ynonymou s wilh (real) number.
than on the system of equations. The two approaches apply the same concept,
 Sec. 1.2 Matrices and Gauss-]ordan Elimin ation • 17
16 • Chap . 1 Li nea r Eguation s
In Step 1 we are merely writing down the equations in a di ffe rent order.
The olution is often represent d a a ector. This will certainly not affect the solutions of the sys tem.

[/~ ;]~
0 1 - I -1 x3 - xs =4
+ 2x3 + 4x4 + 2xs = 4 ~
X4-
4 2 4 2 2x t + 4x2
4 3 3 3 2x t + 4x2 + 3x3 + 3x4 + 3xs = 4
In thi s exa mpl e, the process of elimination works very _smoothl y. ::~ an 6 6 3 6 3x t + 6x2 + 6x3 + 3x4 + 6xs = 6
elimin.a te all entries off the di agonal and can make every co ffi ctent on the dta.oonal
e ual to I . The process of elimination works . w II unle _s we encou_nt_er a zero
-!- +
a~no the di agonal. The e zero repre ent miss tn g terms Jn some equauon . T he 2x t + 4x2 + 2x3 + 4x4 + 2xs = 4
[/~
4 2 4 2
follo:_,ing example illu trates how to solve uch a sy tem: 0 1 - 1 - 1 X4 - x 5 = 4
x 3 - X4 - X5 =
2x 1 + 4x 2 + 2x3 + 4x4 + 2xs =
4
4
4
6
3
6
3
3
3
6
;] XJ -
2x t + 4x2 + 3x3 + 3x4 + 3xs = 4
3x t + 6x2 + 6x3 + 3x4 + 6xs = 6

2x 1 + 4x 2 + 3x3 + 3x4 + 3xs = 4 Now we can proceed as in the previous examples.

3x 1 + 6x2 + 6x3 + 3x4 + 6xs = 6

The augmented matrix of thi ystem is Step 2 Divide the cursor row by the cursor entry to make the cursor
entry equal to 1.
00 I I - 1
2 4 2 -4 2
A= 2 4 3 3 3 Step 2 does not change the solutions of the system, because the equ ation
[
3 6 6 3 6 corresponding to the cursor row has the same soiutions before and after the
operation.
As in the previous examples, we are tryin g to bring_the matrix_into di agonal fo rm .
To keep track of our work we will place a cursor 111 the matn x, ~s. you would on 2x 1 + 4x2 + 2x3 + 4x4 + 2xs =
[/~ ~r2
4 2 4 2 4 -7-2
a computer screen. Inirially, the cursor i placed at the top posttJOn of the fir t 0 1 -1 - l X) - X4 - x 5 = 4
nonzero column of the matrix. 4 3 3 3 2xt + 4x2 + 3x3 + 3x4 + 3xs = 4
6 6 3 6 3x t + 6x2 + 6x3 + 3x4 + 6xs = 6
0 - 1 - 1
4
4
2
3
4
3
2
3
i] -!- -!-

n
Xt + 2x2 + x3 + 2x4 + x 5 = 2

~]
6 6 3 6 2 I 2 1
0 1 - 1 -1 XJ - X4 - x 5 =4
Our first goal is to make the cursor entry equal to l. We can acco mplish thi s in 4 3 3 3 2x t + 4x2 + 3x3 + 3x4 + 3xs = 4
two steps as follows: 6 6 3 6 3xt + 6x2 + 6x3 + 3x4 + 6xs = 6

Step J If the cursor entry is 0, swap the cursor row with some row Step 3 Eliminate alJ other entries in the cursor co1umn by ubtracting
below to make the cursor entry nonzero. * suitable multiples of the cursor row from the other rows.*

*We may also add a multiple of a row, of course. Think of th is as subtractin g a negati ve mult iple of
- - *To make the process unambi guous, swap the cursor row with thc jir .\'1 row below that has a nonzero a row.
ent ry in Lhe cursor column.
 ec. 1.2 Matrices an d Gau -Jo rdan Elimin ati o n • 19
18 • Chap. 1 Linea r Equatio ns
+ 2x4 + xs =

[~
.r 1 +2x 2 + X3 2 2 2 I - (I !)
2

[/~
2
0 I - l - I
- 2(1)
X3 - X4 - x5 = 4
2.r 1 + 4.r 2 + 3..r:~ + 3x.J + 3xs = 4 - 2(1)
0
0
/ I
I
- 1
- I
- I
l
2o] - n
4
4 3 3 3
6 6 3 6 iJ - 3(l) x 1 + 6x _ + 6x3 + 3x4 + 6xs = 6 - 3(1) 0 3 -3 3 0 - 3(11)
~ Step 3

[~
x 1 + 2x2 + + 2x4 + x5 = 2 2 0 3 2

n ~j
1 X3
2 2
0 - I
- I
- I
I
X3 -
X -
X4 - x5 = 4
X~+ x 5 = 0
0
0
/ l
0
- I
0
- J
2 -4
4
-2l
0
.,
-3
., 3x3 - 3x4 + 3xs = 0 0 0 0 6 - 12
0 .) .)

l ~ Step 4
Convince yourself that thi s operatio n does not cha nge the o lu tion of the

[~
te m (Ex.erci e 18). 2 0 3 2
No w we have taken care of the first colum n (the firs t variable), o we ca n
move the c ur o r to a new posiri o n.
0
0
l
0
- 1
0
- 1
/ 2
-2]
_: -7-2
F oll owing the a pproach take n in Sectio n 1.1 , we move the c ursor down 0 0 0 6 - 12
di agonally that i , down o ne row and over one colum.n. ~ Ste p 2

~j [~
2 I 2 2 0 3 2 -2(III)
0
0
I
l
- I
- I
- I
I
0
0
1
0
-I
0
-1
;d
-2]
4
-?
+( III)

0 3 -3 3 0 0 0 6 - 12 -6 111)
For o ur method to work a. before, we need a nonzero c ursor e ntry. Since ~ Step 3
not o nl y the cur or e ntry but al o all e ntries below are zero, we cannot acco mplj h
this by swapping the c ursor row with some row below, a we did in St:e p I. l t
would not help us to wap the c ursor ro w with the row above; thi would affect
the first column o f the m atrix, which we bave already fi xed. Thu we have to
E-
-
[I0
0
0
2
0
0
0
0
1
0
0
3
- 1
0
0
0
0
)' 1
0 -~J
give up o n the econd column (tbe econd vari able); we w ill m ove tbe c ursor to
the next colurrm .
W he n we try to appl y Ste p 4 to thi matrix, we ru n o ut of col umn : th e

~j
2 i 2 1 proce s of row reductio n comes to an e nd . We ay that rhe matri E i in reduced

[t 0
0
0
/ 1
I
3
- 1
-I
-3
- I
1
3
row echelon fo nn, or rref for s hort. We w rite
E = rref(A),
w he re A is the a ug me nted m atri x of the sy te m.
To sumn1arize:

A matri x is in reduced row echelon form if it sati . fi e all o f the fo llowing

Step 4 Move the cursordown di agonally, that is, down one row and condition :
over one column. lf the new cursor e ntry a nd all e ntries below are zero , a. If a row has no nzero e ntries, the n the fir t nonze ro e ntry is 1. ca lled the
move the cursor to the next column (remaining in the sa me row). Repeat thi leading I in thi s row.
step if necessary. Then return to Step 1. b. lf a column co ntains a leading 1, the n a ll othe r e ntries in tha t co lumn
are zero.
c. If a row co ntains a leadin g I. the n ach row abo e contain a Ieadio cr
He re, since the cur or entry is 1, we can proceed directly to Step 3 and I furthe r to the left . "'
eliminate all other e ntries in the cursor column.
 Sec. 1.2 Matrices and Gauss-] ordan Elimination • 21
20 • hap. 1 Linear Equ atio ns
A matrix E in reduced row chelon fo rm may contain row of zeros, as in
the example above. By condi tion c, these rows must appear a · the last rows of
the matri x.
Convince your elf tbat th e proced ure outl ined above (repeatedly perfo rm:ing
Step 1 to 4) indeed produces a matrix with these three properties.
Below, ' e circle the leading l ' in tbe reduced row eche lon form of the For exampl e, if we set s = t = 0, we get the particular solution
matrix. 2

[~
2 0 3 0 2 0
0
0
CD0 - 1
0
0
(D : -2
2 2
0
-2
0 0 0 0 0
Here is a summary of the elimination process outlined above.
Thi matrix repre ents rhe follow ing y tem:
Solving systems of linear equations
G+-X2 + 3x4 2
Wri te the augmented matlix of the system. Place a cur or in the top entry of
I G) X~ = 2
the first nonzero colurnn of thi matrix.

I G)
-2
Step 1 If the cursor entry is zero, swap tbe cursor row witb some row
below to make the cursor entry nonzero.
Again, we say tbat tbi sy tem i in red uced row echelon form. Tbe leading
variables correspond ro rhe leading l ' s in the echelon fonn of the matrix. We Step 2 Di vide tbe cur or row by the cursor entry.
also draw the staircase fo rmed by the leading variable . That is where the name Step 3 Eliminate all other entries in the cursor column, by ubtracting
ecbelon fonn come from: accordi ng to Web ter, an ecbelon is a Formati on " like suitable multiples of the cursor row from the other rows.
a sedes of steps." Step 4 Move tbe cursor down one row and over one column. If the
Now we can solve each of the equations above for the leading variable. new cursor entry and al! entries below are zero, move the cursor to the next
column (remaining in the same row). Repeat the last step if neces ary.
x 1 = 2 - 2x2 - 3x4
X3 = 2 + X4 Return to Step 1.
x 5 = -2 The process ends when we run out of rows or columns. Then the matrix
is in reduced row echelon form (rret).
We can freely cboose the nonleading vari ables, x 2 = s and x4 = t , where s
Write down the linear system corresponding to this matrix, and solve
and t are arbitrary real numbers. The leading vari ables are then deterrnined by
each equation in the system for the leading variable. You may choo e the
our choi ces for s and t ; that is, x 1 = 2 - 2s - 3t, x 3 = 2 + t, and .x:5 = -2.
nonleading variables freely ; the leading variables are then deterrnined by these
This system has infinitely many so lution , namely,
choices. lf the echelon form contains the equation 0 = l , then there are no
solutions; the system is inconsistent.
Tbe operations performed in the Steps I, 2, 3 are called elementmy row
where s and 1 are arbitrary real numbers. We can represent the solutions as vectors operations: swap two rows, divide a row by a scalar, or subtract a multiple
in ~ 5 . of a row from another row .
XI 2- 2s- 3t
X2 s Below is an inconsistent system.
X3 = 2+t (s, 1 arbitrary)
x1 - 3x2 - Sx4 = - 7
X4 t
3x, - 12x2 - 2..r3 - 27x4 = - 33
xs -2
- 2x , + 10..>:2 + 2x3 + 24x~ = 29
We often find it helpful to write this solution as - x, + 6x2 + x3 +14x4 = 17
 Sec. 1.2 Matrices and Gauss-jordan Elimination • 23
22 11 Chap. 1 Linear Equations
The augmented matrix. of the system is an asteroid or rninor planet; it is only about 1000 km in diameter. The public
was very interested in this discovery. At that time, the nurober of planets in the
-3 0 -5 solar system was till an issue debated by many philosopher and repre entatives
-7]
l j
- 1
- 12
10
6

The reduced row echelon form for this matrix is

-2 - 27
2 24
14
- 33
29
17
.
of the Church. Piazzi was able to track the orbit of Ceres for forty days, but then
it was lost from view because Piazzi feil ill. Gauss, however, at the age of 24,
succeeded in calculati ng the orbit of Ceres, even though the task seemed hopele s
on the basis of a few Observations. His computations were so accurate that the
German astronomer W. Olbers (1758-1840) located the asteroid on December
31, 1801. In the course of bis computations, Gauss had to solve systems of 17

l ~ ! H ~~J .
linear equation .. In dealing with rhi s problern Gauss also used the method of
least sq uares which he had developed araund 1794 (see Section 4.4). Since Gauss
at first refused to reveal the methods which led to tbis amazing accomplishment
0 0 0 0
ome even accused him of sorcery. Gauss later described his methods of orbit
(We leave it to you to perform the elimination.) . computation in hi book Theoria Motus Cmporum Coelestium (1809) .
Since the last row of the echelon form represents the equauon 0 = 1, the The method of solving a linear system by Gauss-Jordan elimination is called
system is inconsistent an algorithm, 1 that is, ·'a procedure for solving a mathematical problern in a finite
nurober of steps that frequently involves repetition of an operation" (Webster's
This method of solvino linear systerns is sometimes referred to as Gauss- New World Dictionary, Th.ird Edition) .
Jordan e/imination, after the"'German mathematician Carl Fiiedrich Gau_ss ( 1777- Gauss-Jordan elimination is weil suited for solving linear systems on a com-
1855; see Figure 1), perhaps the greatest mathematician of m?dern tunes, and puter, at least in principle. In practice, however, some tricky problems associated
the German engineer Wilhelm Jordan (1844-1899). Gau~s h1~self called the with roundoff errors can occur.
method eliminatio vulgaris. Recail that the Chinese were usmg thts method 2000 Numerical analysis teils us that we can reduce the proliferation of roundoff
years ago. errors by modifying Gauss-Jordan elimination with partial or complete pivoring
How Gauss developed this method is noteworthy. On January I, 18~1, the techniques. Partial pivoting requires us to modify step 1 of the algorithm as
Sicilian astronomer Giuseppe Piazzi (1746-1826) discovered a planet, ":Iuch he follows: Swap the cursor row with a row below to make the cursor entry as !arge
named "Ceres," in honor of the patron goddess of Sicily. Today, Ceres IS called as possible (in absolute value). This swap is performed even if the initial cursor
entry is nonzero, as long as there is an entry below with a !arger absolute value.
In the first ex.ample worked in the text, we would start by swapping the firs t row
fpe 1 Corl Friedrich Gouss
with the Ia t.
oppeors on the Germon 10-mork note
(in foct this is the mirrar image of o AY5393908 A5
2 3 2 1
well-known portroit of Gouss).1 UJ 26]
I 3 2 3 2 34
u 2 1 2 3 39
~
:::>
UJ
0 In modifying Gau s-Jordan elimination, an interesting quest.ion arises: If we
(!U
z transform a matrix A into a matrix B by a equence of elementary row operation ,
I
obbe UJ
N and if B is in reduced row echelon form, is it necessari ly true that B = rref(A)?
FortunateJy (and perhaps surprisingly) this is indeed the case.
In this text, we will not utilize this fact , so there is no need to pre ent

-
Do.."KI-•e..~

~~
.!
~~ "f:r·'"·'oOt'l
t....,.oi9G'I
• the somewhat technical proof. If you feel ambitious, try to work out the proof
yourself, after studying Chapter 3 (see Exercises 3.3.65 to 3.3.67).

1The word algorit hm is derived from the name of the mathematician al-Khowarizmi, who introd uced

1Reproduced by permission of the German Bundesbank. thc tenn algebra into mathematics (see page 1).
 Sec. 1.2 Matrices an d Gauss-Jo rdan Elimj natio n • 25
24 • hap. 1 Linear Equ ations
In Exercise l t~ 1 ~ fi~d all so lu tio ns o f the equ ati ons wi th paper and pencil u ing
Gauss-Jordan e ltrru natw n. Show all your work . Solve the ystem in E xerci e 8
as a sy. tem in the variable x 1, x 2, x 3 , x 4 , x 5 .

x + y- 2z =51 3x + 4 y - z = 8 1
1.
2
· 16x + 8 y - 2 z = 3
3. x + 2 y + 3z = 4
I2x + 3 y+ 4 z = 2
X+ y = l
XJ + X4 = 0
4. 2x - y = 5 5. x2 +x3 =0
XJ + x2 = 0
3x + 4y = 2
X1 + x4 = 0
Pick the mallest
such h. x 1 - ? x2 + xs =3
x1 + 2x2 2x 4 + 3x5 = 0
X3 + 3x4 + 2xs = 0
Swap the itl1 row 6. XJ - 2x 5 = 2 7.
X3 + 4x4 - X = 0
with t11e h th row.
X4 + xs = I
xs = 0
+ 2x 5 - x 6 = 2
x4
Subtract a1if ti me x2+ 2x4 + 3xs = 0 I 9. + 2x2 + X5 - X6 = 0
the i t11 ro' from S. I 4x4 + 8xs = 0 X1

the h th row,
x 1 +2x2 + 2x3 - X 5 + X6 = 2
for allh * i 4x 1 + 3x2 + 2x3 - x 4 = 4 x 1+ 2x3 + 4x4 = -8
Sx1 + 4x2 +3x3 - x 4 = 4 x2 - 3x3 - x 4 = 6
10. ' 11.
-2x l - 2x2 - X3 + 2x 4 = - 3 3x 1 + 4x2 - 6x3 + 8x4 = 0
11xl + 6x2 + 4x3 + x 4 = l l - x 2 + 3x3 + 4x 4 = - 12
2x l - 3x3 + 7xs + 7x6 = 0
- 2xl + x2 + 6x3 - 6xs - 12x6 = 0
12. x2 - 3x3 + xs + Sx6 = 0
- 2x_ + X4 + X5 + X6 = 0
2xl + x2 - 3x3 + 8xs + 7x 6 = 0

Solve the li near systems in Exerc i es 13 to 17 below. You may u e techno logy.

3x + 11 y + 19z = - 2 3x + 6y + 14z = 22
13. 7x + 23 y +39z = 10 14. 7x+ l 4y+ 30 =46 7

- 4x - 3 y - 2 z = 6 4x + Sy + 7z = 6
3x + Sy + 3z = 25
fpe 2 Aow chart for Gauss-Jordan
'
Display reduced ro w echelon form
15. 7x + 9 y+ 19- = 65
- 4x +5 y + ll z = 5
eliminatian. Ais an m x n matrix.
3x l + 6x2 + 9x3 + Sx4 + 25xs = 53
16. 7xl + 14x2 + 2 1x3 + 9x4 + 53x 5 = 105
In Fi gure 2 , we present a ftow chart for Gauss-Jo rdan elimination. - 4xl - 8x2 - 12x3 + Sx4 - l Oxs = 11
2xl + 4x2 + 3x3 + Sx4 + 6xs = 37
EXERCISES 4xl + 8x2 + 7x3 + Sx 4 + -Xs = 74
GOAL Use Gauss-Jordan elimination to solve linear systems. Do simple prob- 17. - 2x l - 4x2 + 3x3 + 4x4 - Sxs = 20
lems using paper and pencil , and use technology to solve more complicated x1 + 2x2 + 2x3 - x 4 + 2x - = 26
problems. Sx 1 - l Ox2 + 4x3 + 6x-1 + 4x = 24
 Sec. 1.2 Matrices and Gau -J ordan · limin ation • 27
26. hap. 1 Linear Equations
IO\ are in red uced r w echelon fo rm . before a nd after the reaction. For example, becau e the numbcr of oxygen
18. Oe term.ine ' hich of the mau·i es b

u
atoms m ust remai n the same,

~] i 2 0

~]
· l~ ~]
- 0 2... l 0
0 J .) 0 0 1 0 0 2a +b = 2c + 3d.
b. c. [
0 1 4 0 0 0 0- Wh ile th~r~ are many po s ible choice fo r a, b, c, and d w hich balance th e
0 0 0 re~ct 1 on,
1t 1 customary to u e the sma ll est possib le pos it ive intel!e rs. Ba lance
d. Lo 2 3 -+ ] tht reactio n. ~
19. Fi ndall 4 x J matri e · in red uced row echelon form . 30. Fi~d a pol ynomial of degree 3 (a po lyno m ial f tb e fo rm j(l = a + br +
20. We ay that two 111 x n matri c _ in r du ed ro ech Ion fom1 are of the __ a me ct· + dt 3) w hose g raph goes thro ug h the poin ts (0, I ), ( l , 0 , (- I , 0) . an d
type if they o ntai n th e ame number of leading I ' in the sa me post u ons .
• J (2 , -15). Sketch the g raph f th i c ubic .
For examp le, 31. Find the poly no m ial of degree 4 w hose gra ph goes through the p int ( L 1).
....) (2, - I ), (3. -59) , ( -l. 5), -2. - 29). Graph this polynomia l.

[~ 0
are of the ame type. How many type
and
0
of 2 x 2 matrice · 111 reduced row
32. Cubic splines. S uppose yo u are in c ha rge of the des il! n of a ro ll er coa te r
r_1de .. T hi imp le ride wi ll not make any left or rig bt t~m , tha t i , th e track
IJ.es 111 a vert ica l pla ne. The figu re below s hows the ride as viewed from the
tde. The point (ai. bi ) are given to yo u, an d you r job is to connect th e dot
echelo n fo rm are there . in a reasonably mooth way. Let ai+ l > a i .
21. How many t pe of 3 x 2 matrice in reduced row eche lon form are there
( ee Exercise 20)?
22. How many type of 2 x 3 matrice in reduced row echelon form are there
( ee Exercise 20)?
23. Suppose yo u apply G au -Jo rdan e lim.ination Lo a matri . Explai n how you
can be . ure th at the re u lting matri x is in red uced row echelon fo rm.
24. Suppose matti x A is tTan fo rmed inro matrix B by means of an e lementary
row operation. I there an eleme ntary row operation tha t tra nsforms B into
A? Expla.in.
/ (a", b.)
25. Su ppose m atr ix A is tra nsformed into matrix B by a seque nce of e leme n- I
tary row operations. 1 there a equence of e le me ntary row operation that
tran fo rms B into A? Exp lain (see Exerci e 24) .
26. C o nsider an 111 x 11 matrix A. Can yo u transform rref(A) i nto A by a seque nce
of eleme ntary row o peration (see Exercise 25)?
27. I there a eq uen ce of e lemen tary row Operati o ns that transfo rms O ne method often e mpl oyed in such de ign pro blems i the tech niqu e of ubic
sp l111~s.
We choo e fi (t). a poly norn.ia l of degree 3, ro defi ne the sha pe of
I 2 3] 1 0
0 I
the n de betwee n (ai - l · bi _ 1 ) a nd (a i, bi). for i = I, ... . n.
[7 8 9
4 5 6 in to
[ 0 0
Exp lain.
i8. Suppo e yo u subtract a mu ltiple of an eq uation in a ·y tem fro m a nothe r
equatio n in the system . Explain why the two sy tem s (before a nd afte r thi .
operati o n) have the . a me so lutio n .
29. Balancing a chemical reaction. Co nside r the c he mi ca l reacti o n
a N02 + b H20 -r c HN02 + d HN0 3,
Ob io us ly, it is re JUirecl that }; (n,) = bi a nd fi(n, _ 1 ) = b, _ 1, f r i = 1. .... 11 •
where a, b, c, and d are un known pos itive integers. T he reacti o n mu. t be
balanced , that is, rhe numbe r o f ato ms of each e le me nt m ust be rhe sa me To g uara ntee a smooth ride at the po ints (a i . bi). we ' a nt the li rs t and th
28 • hap . 1 Linear Equations Sec. 1.2 Ma trices a n d Gauss-Jo rdan Eliminatio n • 29
econd de rivatives of f; and fi+I to agree at the e poi nts. 37. Fo r so me backgro und o n thi exerc i e, see Exe rci e 1. 1.20.
.f;'(a;) = .f/+1 (a;) and Co ns ider a n eco nomy w ith three ind ustries, / 1, 12 , / 3 • W hat o utput x 1,
.f/' (a;) = fi'~ 1 (a;) , for i = l. . . . , n - I. x2, x3 ho uld they produ ce to sati fy bo th co ns umer de mand a nd inte ri nd u tr y
de mand ? The demand s put o n tbe three industri e are show n be low.
Expl ain Lhe pract:ica! ignifi ance of the e condi ti o ns. Ex plain w hy , fo r the
convenience of th ride r , it i al o requi red rhat
f((a o) = .r,;Can) = 0.
Show that sati fying all t:he e condition s amourit to so lving a yste m of linear
equati o n.. Ho w many ariable ar in thi s sysre m ? How m a ny equ atio ns?
(Note: It can be hown that thi sy te m ha a unique o lution .)
33. Find a polyno mial f( t ) of degree 3 uc h that f (l ) = 1, /(2) = 5, .f'(l ) = 2,
and .f' 2 = 9 where .f'(t ) i the de ri vati e of f (t ) . Graph thi polynomi a l.
34. T he dor product of two ve tor

.i? =
x,~2 ] and
_ YI]
Y2
Consumer
y= .
[ Xn
[ ;ll
in ffi: 11 is defined by 38. lf we consider more tha n three indu trie m an input-output model, it i
X· y= Xl)ll + X_ }'2 + · · · + XnYn· cumber ome to repre ent all the demand in a diagram a in Exercise 37.
Suppose we have the industries / 1 , 12 , . • . , 1". witb outputs x 1• x 2 . .. , x 11 •
Note that the dot product o f two vectors is a calar. We ay th at the vectors The outpul vector i
x and ji are perpendicular if ,\: · y = 0 .
Find all vectors in W.: 3 perpendicul ar to
x,]
Ul
_ X2
X =
[x·~~ . .

Draw a sketch. The corzsumer demand vector is

35. Find a U vectors in IR4 which are perpendi cul ar to the three vector

(see Exercise 34).

[lJ · [iJ· [~J where b; is the consume r de m an d o n ind ustry I;. T he de m a nd vector for
36. F ind all solu tio ns x 1, x 2 , x 3 of the equati on indu stry l j is

b= X 1VI + X2Ü2 + X3VJ, alj]

- 0 2j
w here v·J -- . '
[
Onj

where a ij is the de m a nd industry l j puts on industry I;, fo r each $ 1 of o utpul

industry lj produces. F o r exa mpl e , a 3_ = 0 .5 means that ind u try h needs
30 • hap. 1 Linear Equation s Sec. 1.2 Matrices and Gauss-j ordan Eli mina tion • 31
SO ~ wortl1 of product from industry ! 3 fo r each $ 1 worth of goods 12 pro- The position vector of the center of mass of thi s system is
duces. The coeffi cient a ; ; need not be 0: producing a product rn ay reqUJre
oood or ervice frorn the am indu try.
:. Find the fo ur demand vector for the economy in Exercise 37.
b. Wbat i tl1e meani ng in economi terrn of Xj Üj? _
where M = m 1 + m 2 + · · · + mn .
c. What i the rneaning in economi c terrn s of x 1v1 + x2v2 + · · · + x" V,, + b?
d. What is the meaning in economic terrns of the equ atio n Consider the triangular plate sketched be1ow. How mu st a total mass of
1 kg be distributed among tl1e tl1ree vertices of tl1e plate so tl1at the plate can
x 1ü1 + x2Vi + · · · + x" v~, + b = .\:? be supported at tl1e point [ ~] , that is, rcm= [ ~]? Assurne that the mas of
39. Consider tlle economy of J rael in 1958 .* The three industries considered the plate itself is negtigible.
here are
11: agricu lture,
1'2 : manufacturing,
h: energy.
Outputs and demands are rnea ured in miltion of 1 raeli pound , the curr ncy
of Israel at tl1at time. We are told tl1at

b=
13.2 ]
17.6 .
[ 1.8
ü1 =
0.293]
0.014 ,
[ 0.044 üi = [~.0.01207] ' ü3 = [~.0.2017]
16

a. Why do the fir t component of ü2 and ü3 eq ual 0?

b. Find tl1e outputs x 1• x 2 , x 3 required to arisfy demand .
40. Consider some particles in the plane witl1 po ition vectors r1 , r2, ... , r" and
mas es m 1. m 2, . . . , m" .

41. The momentum P of a system of n particles in space with masses m 1, m2 ,

. . . , m" and velocities Ü1 , Üz, . .. , Ün is defined as

Now consider two elementary particles witl1 velocities

The particles collide. After tlle collision, tl1eir respective velocities are ob-
served tobe

Assurne tl1at the momentum of the system is conserved throughout tl1e colli-
sion. What does this experiment tell you about tl1e masses of the two particles?
*W. Leontief: lnput·Output Economics, Oxford Uni versit y Press, 1966.
(See figure on page 32.)
 Sec. 1. 3 On th e Salu tions of Li near Sy tems • 33
32 • hap. 1 Linear Equations
For example, 5(47) = 11.5 means that the time from unrise to un set on
Particle I
February 16 i · I L hour and 30 rni nu te . For locations clo e to the equ ator
the function S(t) i wei l appro ximated by a trigonometric funcrio n of the form

S(t) = a +b cos (~~~) + c sin (~~~)

(the period i 365 days o r I year). Find thi s approximation for Bo mbay and
graph your o lution. Acco rding to thi s model , bow long is the Iongest day of
the year in Bombay?

·
·rY m
· the United
42. The sketch below represent. a maze of one-way st.reet _m a Cl
States. The traffic volume through certain block dunng . an h~ur has been .ON THE SOLUTIONS OF LINEAR SYSTEMS
mea ured . Suppose that the vehicles leaving t.he area dunng tlu hour were
exactl y the sa me a tho e entering it. ln the last ection we di cussed how a sy tem of linear equatio n can be solved
by Gauss-Jordan el imin ation. N ow we will inves ti gate what thi method tell us
" about the nurober of so lution s of a linear ystem. How man y ol uti on can a
JFK Street linear system poss ibl y have? How can we tell whether a system has any solutions
Dun ·ter Street at a ll?
Fir t, we ob erve that a ]jnear ystem ba no olutions if (and only if) it
reduced row echeJon fo rm contains a row of the form

[0 0 0 0
repre enting the equation 0 = 1. ln thi case we ay that the system is inconsisrent.
lf a linear y tem is consistent that is, if it does bave solutions) how many
o luti ons can it have? The number of olutions depend o n whether or not there
Mt. Auburn Street are nonleading variable . If there i at lea t one nonleading variable, then there
will be infinitely many o lu tion , since we can a sign any value to a no nleading
variable. If all variable are lead ing. on the otber hand , there will be only one
solution , since we cannot make any choices in assigning value to the variables.
Winthrop Street We have sbown: 1

Fact 1.3.1 Number of solutions of a linear system

A linear system has either
What can you say about the traffk volume at the four location s indicated by • no so/utions (it is incon istent),
a question mark? Can you figure out e~actly how_much traffic there was • exactl) one solution (if the y tem i con i tent and all ariable are
an each block? If not, describe one poss1ble scenano. For each of the four Jeading). or
locations, find the highest and the lowest possible traffic volume.
• infinitely many solutions (if the system i con i t nt and tbere are non-
43. Let S (t ) be the Jength of the tth day of the year in Bomba~, lndia (measured leacling variable ).
in hours, from sunrise to sunset). We are given the followmg values of S(t):
S(t)
47 11.5
1Starting in th is section. we will number the definitions we g ive and the fac ts we deri e. l11e nth
74 12
fact stated in Section p .q is labeled as Fact p.q .n .
273 12
 ec. 1.3 On th e o luti ns o f Lin ear Sy tems • 35
34 • Chap. 1 Li near Equation
b. The eq uation rank (A) = 111 mean that there i a leading I in each r w f
EXAMPLE 1 ... T he red uced ro w ec I1e Ion l-01·11's
•
of the au.2:mented matrice of three sy. tem are
~
the echeJon form of A. Thi im plies that the echelo n form of th aug mented
given below . H O\. many Ol utio n are ther in each case?
matrix doe not contain the row
1 2 0 0

l~
2 0 [0 0 0 0
0 0 0
a. 0 l Themfore, the y tem mu t have at lea t one o lution.
0 0 0
0 0 c. The eq uatio n rank (A) = 11 mean s that th ere is a lead ing I in each colu mn.
0 0 0 0
i.e. , all variab le are leading. T herefore. eithe r the y. te m i incon i tent r
it has a uniq ue so lutio n (by Fact 1.3. 1 .
Solution d. There are nonleadi ng variab les in th i ca e. Th erefore, either the sy. tem i
a. Jnfi nüely many lutio n (the econd variable is nonl ead ing). inconsistent or it has infin ite ly many ol ution (by Fact 1.3. 1) . ~
b. E ·actl one olution (a ll ari ·~b le are leadin g . EXAMPLE 4 ... Co nsider a linear sy tem w irb fewe r eq uatio ns than unk.nown . How man y olu-
c. o solution (the third row represenrs the eq uaüon 0 = I). tio n couJd thi ystem have?
E ·ample 1 hows that the number of leadin g 1 s i~ the echel_o n forn~ teil s
u abour the number of o lution of a linear y tem . T h1 ob er at10n motJvate Solution
the fo ll owiJ1g definition: Suppose there a.re 111 eq uation and n unknown ; we are to ld that 111 < n. Let
be the coefficient matrix of the ystem. By Examp le 3a.

Definition 1.3.2 Rank rank (A) Sm < 11. .

1
The rank of a matrix A is the number of leading 1' in nef(A) . Th us the sy tem w ill have infin ite ly many so lution or no solution at all (by
Examp le 3d). The key Observat ion i that there a.re no nl eading ariabl es in thi
~ ~
case.
EXAMPLE2 ...

rank
[ 7
I
4
2
5
8
since rref
I
4
[ 7
2
5
8 9
] [CDo Q)o -'l
36= 0
2.
0 0
Fact 1.3.3 A linear yste m wi th fewe r eq uation than un.known - ha e ithe r no o lutio ns
or infi ni te ly ma ny o luti ons.
\.

EXAMPLE 3 .... Consider a system of m linear equation wi th n un know ns. It s coefficie nt matrix To illu trate thi o b ervatio n, con ider two eq uatio ns in tlu·ee variable : Two
A has the ize m x n . Show that: planes in pace ither in ter ect in a lin e or are parall e l (see Figure I), b ut they
a. rank(A) sm and rank(A) S n. will never intersect at a po in t! Th is mean that a sy ·tem of two eq uation w ith
three unkno\ n ca nnot have a un iq ue o lution.
b. Jf rank(A) = m. then the sy tem i con istent.
c. lf rank(A) = 11 , then the sy -tem has at most one so luti on. EXAMPLE 5 ... Co n ider a linear ystem of 11 eq uation w ith 11 unknov ns. Whe n doe thi y tem
have a unique o lu tion? U e the rank of the coefficien t matri x A.
d. If rank(A) < 11 , then the system has eilher in fi ni tely many so luti ons, or no ne.

Solution Figure 1 o) Two planes intersect in o line. b) Two parallel planes.

a. By the defin ition of the reduced row eche lo n form , there is at most o ne
leadinrr 1 in each row and in each co lumn . lf there i a lead in g I in each
row, then rank (A) = m; otherwise, rank(A) < m . L ikew ise for the columns.

IThi is a preliminary , rather technica l delinition. In Chapter 3 wc wi ll gain a heller conccptual

(a) (b)
undcrstandi ng of the rank of a matrix.
36 • Chap. 1 Linear Equations
Sec. 1.3 On the Solutions of Linear Systems • 37
Solution
We ob erve fir t that rank(A) s n , by E amp.le 3a. lf rank(A = n then the system
has a unique oluti o n. by Example 3, part b and c. However, if rank (A) < n ,
then the sy tem doe not have a unique so lution, by Example 3d. ~

Fact 1.3.4 A linear y tem of n equations with n unknowns ha a unique olutio n if (and :~ [1]
onl y if) the rank of its coefficient matrix A i n. Thi means that
1 0 0 0
0 1 0 0 Figure 2
rref A) = 0 0 0

0 0 0
The unique olution of thi y te m i x = 2, y = 1. The equation

+ The Vector Fonn and the Matrix Fonn of a Linear System

We now introduce some notations that allow us to represent y tems of linear i ca lled the vector tßr.m fth~ linear ~stem. Note that the vectors [ 3 ] and [ 1 ]
are the colurnns of 1ts coefficJent matn x. I 2
equations more succinctJ y. As a imple examp.le, consider the linear sy tem 1
ow consider the genera l linear ystem

3x + y = 71· a,,x, + a 12X2 + · · · + a, x" = b1

1 X+ - Y = 4 az1x1 + a22x2 + · · · + GznXn
11

= bz
[o Section 1.1 we interpreted the olution of thi system as the intersection of two
lines in the x -y plane. Here i another interpretation. We can write the system
above as
We can write

[
3x + y ] = [7] ,
X +2y 4 O J JX J + OJ2X2 + · · · + GJ X 11 11
] [ b1]
Gz JXJ + anx_ + · · · + a2nx" b
or
[
Gm I ~-I + Gm2:~-2 + · · · + G 11111
:"Cn = b;" '
or
or u, Vz VII /;
i t i i

x, [a.,]
.
a21
[a". ]
a22
[a,"]
a zn

To solve this system means to write the vector [

of [ ~] and a scalar multiple of [ ~] .

~] as the sum of a scalar multiple

This problem and its solutiön can be

a,;,,
or, more succinctly,
+ xz
a,;,2
+···+ XII .

a,;,"
=
[1J
represented geometrically, as illustrated in Figure 2.
38 • Chap . 1 Linear Equ ations S c. 1.3 On the Sol ution of Linear Systems • 39
This is the vectorf orm of the linear ystem. In thi s context, the following definitio n Note that the product kr i defi ned only if the nu rober of columns of A
is useful: matche the number of components of x. Also, note th at the product Ax i a
vector in IR • Thi s de fi nit~o n allows u · to represent the lin ear system with vector
111

form x1ii1 + · · · + x" ü" = b as , . v ~ • _ ,

Definition 1.3.5 Linear combinations
A vector b in ~ 111 is called a linear combination of the vectors ii1 , ii2 .. . , v"
in ~~~~ if there are scalars x 1, x 2 , •• . , x" such that the matrixform of the I i near system.
b= x 1ii 1 + x2ii2 + · · · + x" ii" . Below are so me examples of matrix products.

EXAMPLE 6 ...

Solving a linear system with augmented matrix [A : b] amounts to writing

[~
0
the vector b as a linear combination of the column vectors of A . 2
We now give another defini tion that permits an even more compact repre-
sentation of a linear system . We define the linear combination
EXAMPLE 7 .... The product

0
to be the product of the m x n matrix 2

i unde fi ned, because the number of columns of the matri x doe not match the
OLJmber o f component of the vector. <111111

A = v1 vz
EXAMPLE 8 .... If

and the vector

x= [~;] . and xi any vec tor in IR 4 , find D~r.

x"
Solution

Definition 1.3.6 The product Ai l

- 0
lf the column vectors of an m x n matrix A are ii ~> v2 , . ••, ii", and x is a Dx = ~
vector in ~~~, then the product Ax is defined as [

Thu D."i = x, for all x in JR4 .

v" EXAMPLE 9 .... Re present th ystem

2x 1 - 3-~2 + Sx3 = 71
ln words, Ai is the linear combination of the columns of A with the compo- 9x l + 4~1 2- 6.\3 = 8
x
nents of as coefficients.
in matri x form A-r = b.
'
 Sec. 1.3 On the o lution of Linear Syste ms • 41
40 • Chap. 1 Linear Equation
characterization invol ves the dot product of two vector ; you may wi h to rev iew
Solution thi s concept in the appendix (Defi rliti on A4).
Begin by writing !l1e tern in vector form. Consider the product of an m x n matrix A with a vector ,r in IR".

[~ a21 J
aii a22 J a2"
= X1 . + X2 [ a12
. + · · · + Xn [ a1". J
[
Ax = b,
a,:,l a~,l a~n
where A = [~ -3
4 G11X1+ a12X2 + · · · + G 111 X 11 J
a21X1 + a22X2 + · · · + a211Xn
Here are two important algebraic rul es conceming the product Ax .
[
Gm I~ I + am2~2 + · · · + Gmn~n
Fact 1.3.7 For an m x n matrix A, two vectors x and y in IR.", and a calar k, Note tbat the first component of Ax is the dot product of the fir t row of A
a. A(x + y) =A-i+ A_v, with x etc.; the ith component of Ax i the dot product of the ith row of A with
x. Thu we have hown:
b. A(kx) = k(Ax).

We will prove the fir t equation, and leave the verification of the second as
Exerci e 45. Fact 1.3.8 x
lf i a vector in IR." and A is an m x n matrix with row vector
then

A (x + y) = V1 V2 VII
[
XI+

Xn
Yll
X2 + Y2

+ Y11 [
-- w1
W2 -J
- _
X =
[ ~I ·-~ ]
W2 ·X
.

- Wm - Wm ·X
= (x1+ Y1)Ü1 + (x2 + Y2)Ü2 + · · · + (x" + y")v"
= x1ü1+ x2~ + · · · +x"Ü" + Y1Ü1 + Y2Ü2 + · · · + y"Ü" (i.e., the ith component of A.r is the dot product of w; and ,r).
= Ax + Ay

In Definition 1.3.6 we express the proQuct Ax in terms of the columns of EXAMPLE 10..,_
the matrix A. This product can also be expressed in terms of the rows of A. This

[! 0
2
+
+
0. 1
2 -1
+
+
(-1)·2]=[III]
3 ·2

1A
(Compare with Example 6.)
box • marks the end of a proof.
42 • Chap. 1 Linea r Eq uation Sec. 1.3 On t he S lution s o f Linea r ys tem s • 43
L t us defi ne two othe r operatio n in o l in g matri ce . c
We ca n find a vector in 1R111 such th at the ys te m E.l: = c is incon si. te nt: Any
vecto r whose ~ast com po ne nt is no nzero will do . Using this vecto r c, how can we
find a vec to r b in 111 such th at the system A.l: = b is in con is tent?
Definition 1.3.9 Sums of matrices T he key idea i to work backw ard thro ug h Ga us -Jordan e limjnatio n. We
know th at there is a seq uence of e leme ntary row o peratio n, which tran fo rm A
T he sum of rwo matrices of the - ame size is defi ned entry by ntry.
into E. By invetting each of th e e Operatio ns a nd pe t·fo rming them in re er e
11 o rder, we ca n find a seq uence of e le me nt ary row Operat ions that tran fo rms E int
.
0! ]
+ [
.
h11
A (fo r exam ple, in tead of di vid ing a row by k we multipl y the ame row b k ).
If we apply the ame ro w Operation to the a ug mented matri x [ E : c], we end
. . . a,;m b,;,t
up w ith a matrix [ A : b] s uch that the , y -rem A.r
= b i inco n i. tent, a de ired
Scalar multiples of m atrices c
(reca ll that the sy tem Ex = is inco nsi te nt and eJe men tary row operatio n do
not change the e t o f o luti o ns . For example Iet
Tbe produ r of a sca lar k w ith an 111 x n matti is defined entr by entry.

Cl ! !

k :
[
Clm \

ote th at the e definitions genera lize the co rre pond ing operation fo r vec-
ror. (see the Appendix , Defi ni tion Al ).

EXAMPLE 11 ~

r~ ilJ
I
2
A=
3
EXAlVIPLE 12 ~ 4

".) [ - 21
-!- t

r~ ~l
4 4 8
We w ill co nclude thi secti on with an example that applies many of the
2 -7-(2) 2 4
concepts we have introduced.
3 3 6
EXAl\•I PLE 13 ~ Consider an 111 x n malri x A wi th rank(A) < m . Show that there is a vector h in 2 Jl
IR"' such that the syste m A.l: = b is inconsistent. -1- t

r~ ;l J2
4 - 4(/ I)
Solution
Consider the reduced row echelon form E = rref(A) . S ince the rank of the matri x "
..) - 3(1 1)
A is les th an the number of its rows, the last row of E does not conta in a leading - (II)
I and is therefore zero.
-1- t

r~ ~l il
0 0 0 + 4 ( //
E = rref(A) =
L E=
I
cJ =
I 2
0 0 0 +3 (11)
000 00 0 0 0 + (I I )
44 • Chap. 1 Li nea r Equation s
Sec. 1.3 On th e Sal ution s of Li near Sys tems • 45
The rea onin g above how that the sy tem
7. Con ider the vector v 1 ii2 , ii 3 in JR2 ketched below. How man y olurion
does the system
Ai = b

in con istent.
have? Argue geometricaJly.

EXERCISES
GOALS Use the red uced row echelon form of the augmented matrix to find
the number of so lution of a Linear sy tem. App ly the definition of the rank of
a matrix . Compute the product Ax in tenn s of the column or the rows of A.
Represent a linear sy tem in vector or in matrix form .
8. Consider tbe vectors Ü1, Ü2, Ü3, Ü4 in IR 2 ketched below. How many solutions
1. The redu ced row echelon form of the augmented matri ces of three sy tems does the system
are given below. How many oluti ons does eacb . ystem have?

a.
1 0
0 1
[ 0 0 0
; ~ ~]
: 1
b. [6 0 ) ~] 0
c. [ 0
1 0
0 1 ~]
have? Argue geometrically.

Find the rank of the matrice in Exerci es 2 to 4.

I 'J 3]
2.
[0 0 1 2
0 1 9. Wri te the system

x + 2y + 3z = I
5. a . Write the system 4x + Sy + 6- = 4
7x+8 y +9z = 9
X + 2y = 71
I3x + y = 11 .in matri.x form.

in vector form . Compute the dot products in Exercises 10 to 12 (if the products are defined).
b. Use your answer in part a to represent the system geometricall y. Solve
the system and represent the so lution geometrically.
6. Consider the vectors ii 1, ü2 , ü3 in IR 2 sketched be low (ii 1 and ii2 are para ll el).
How many solutions does the system 10. mHJ II. [ I 9 9 7] m 12. [ I 2 3 4] m
Compute the products Ax in Exercises 13 to 15 u ing paper and pencil. In each
have? Argue geometricall y. case, compute the product two ways: in terms of the column of A (Definition
1.3.6), and in tenns of the rows of A (Fact 1.3.8).

14. u; !] [-n 15. [I 2 3 4] m

 Sec. 1.3 On the Solutions of Linea r Systems • 47
46 • Chap. 1 Linear Equa tion
Compute the prod uct A."\- in E · reise 16 to 19 us in g paper and pe ncil (if the 27. True or false? Justify your answers.
Consider a system A.x = b.
product are defin d).
a. If Ax = b is inconsistent, then rref(A ) contains a row of zeros.

16. [ ~ ~ J [ -3] 17. [41 25 18. un[~] b. 1f rref(A) contains a row of zeros, then Ax = b is incon istent.
28. True or fa lse? Ju stify your answers.
Suppose the matrix E is in reduced row echelon form.
a. If we omit a row of E, then the remaining matrix is in reduced row echelon

19.
[ -~
20. a. Fi nd
-5 -nm form .
b. lf we omit a column of E, then the remaining matrix is in reduced row
echelon form.
29. True or false? Justify your answer.
Consider a system Ax = b. This system is consistent if (and only if)
rank(A) = rank[ A : b].
iJ .
- 1
30. If the rank of a 5 x 3 matrix A is 3, what is rref(A)?
31. If the rank of a 4 x 4 matrix A is 4, wbat is rref(A)?
32. True or false?
b. Find
Consider the system Ax = b, wbere A is an n x n matrix. Thi s sy tem has a
- 1 unique solution if (and only if) rank(A) = n .
4 33. Let A be the n x n matrix with all 1 's on the diaoonal and all O's outside the
diagonal. What is Ax, where .X is a vector in~~~?
21. U e technology to compute the prod uct 34. We define the vectors

in ~3.
a. For
22. Consider a linear system of three equations with three unkn owns. We are
told that the ystem ha a unique olution. What does the red uced row ech-
elon form of the coefficient matrix of this system Iook like? Explain yo ur c]
f
k
'
an wer.
23. Consider a linear system of four equations with three unknown . We are
compute Ae1, Ae2, and Ae3.
told that the system has a unique solution. What does the reduced row ech-
elon form of the coefficient matrix of thi s system Iook like? Explaio your
b. If B is an m x 3 matrix with columns v1, li2 , and ü
Be3?
ans wer.
35. In ~~~, we define
c
24. Let A be a 4 x 4 matrix , and Iet b and be two vectors in ~ . We are to1d
4

that the system Ax = b has a unique soluti on. What can you say about the 0
number of . ol ution of the sy tem Ax = c? 0
c
25. Lel A be a 4 x 4 matri x, and Iet b and be two vectors in ~4 . We are to ld
e; =
that the sy. tem Ax = b is incons istent. What can you say about the number +-- ith component
of olutions of the system Ax = c?
c
26. Let A be a 4 x 3 matrix, and Iet band be two vectors in ~4 . We are told 0
that the system Ax = b has a unique so lution . What can you say about the
number of so lutions of the system Ax = c? If A is an m x n matrix, what is Ae;?
48 • Chap. 1 Linear Equations
Sec. 1.3 On the olutions of Linear ystems • 49
36. Find a 3 x 3 mau·ix A such tJ1at
c. !f ?z
x1 and are so lutions of tbe homogeneou sy tem Ai= Ö, then .~ 1 +x2
1s a solut1on as weU. .
d. If x i a soluti on of tbe homogeneaus sy tem Ax = Ö, and k is an arbitrarv
constant, then kx is a solution as weil. -

37. Find aJI ector x such that 48. Consider a so lution x1 of the linear system AX..= b. Justify the facts tated in
part a and b:

Ai = b, where A = [~ ~ !] and b= m a. If x" is a solu~ion of the system Ax = Ö, then .X 1 + ;" is a olution of the
system Ax = b .
b. If .rz x
is another Solution of the system Ai = b, then 2 - ..t 1 i a solution
of the system Ax = Ö.
38. a. Using technology , generate a "random' 3 x 3 matrix A (the entries may c. N~w s~~pose A is a 2 x 2 matrix. A solution vector x1 of the ystem
be either single-digit integers or number between 0 and 1, depending on Ax = b JS ske~cbed below (in JR.Z). We are told that the olution of the
the technology you are using). Find rref(A . Repeat this experiment a system Ai = 0 form the Une sketch~d below. Sketch the line con isting
few time . of all solutions of the system Ai = b.
b. What does the reduced row echelon form of "most" 3 x 3 matrice Iook
like? Explain.
39. Repeat Exercise 38 for 3 x 4 matrices.
40. Repeat Exercise 38 for 4 x 3 matrices.
41. How many olutions do 'most" systems of three linear equation with three
unknowns have? Explain in terms of your work in Exercise 38.
42. How many olutions do ·'most" systems of three Unear equations with four olutions of Ai= Ö
unk:nowns have? Explain in terms of your work in Exercise 39.
43. How many solutions do "most" systems of four linear equations with three
unknowns have? Explain in terms of your work in Exercise 40.
44. Consider an m x n matrix A with more rows than columns (m > n) . Show lf you are puzzled by the generality of this problem, think about an exarnple
that there is a vector b in IR."' such that the system Ax = b is inconsistent. first:
x
45. Consider an m x n matrix A, a vector in ]Rn, and a scalar k . Show that

A(kx) = k(Ax).

46. Find the rank of the matrix 49. Consider the table below. For some linear systems A.t = b. you are given
either the rank
• -
of the coefficient matrix A, or the rank of the auomeoted
b
a b matrix [A : b]. For each system, state whether the system could have no
0 d solution, one solution, or infinitely maoy solution . There may be more than
[ 0 0 one possibility for some system . Justify your an wers.
where a, d , f are nonzero, and b, c, e are arbitrary numbers.
47. A linear system of the form
Numberor Number or Rank Rank
equations unknowns or A or [A: bJ
is called homogeneous. Justify the following facts: a. 3 4 2
a. All homogeneaus systems are consistent. b. 4 3 3
b. A homogeneaus system with fewer equations than unknowns has infinitely c. 4 3 4
many solutions. d. 3 4 3
50 • Chap . 1 Linear Eq uati n Sec. 1.3 On the So lutlo ns of Linear Systems • 51
SO. Consider a linear ystem A.~ = b, where A i. a 4 , 3 matrix . We are to lcl that
rank [ : b] = 4. H ow man o lu tio n do thi sy. tem have? 57. Express the vector [
1
~ J as the um of a vector on the li.ne y = 3x and a
51. Con ide r an 111 11 matri · A . an r x 5 matrix 8. and a ector -~ in IR". For vector o n the line y = x j 2.
w hich choi ce of m. 11 , r. 5. p is the product
y

defined.
52. Con ide r the mauice

A = [: ~] and B-
- [0 -l]0 .
I

Can you find a 2 x _ matri x C such that

A Bx) = Cx ,

X
for alJ vectors .:r in IR2 ?
53. lf A and B are two m x 11 matrices, i

(A + B )x = A_~ + B.r,
for alt x in l "?
54. Con sider two vector ü1 and ü2 in IR 3 that are not parall I. Wtli ch vector in
IR 3 are linear combinations of ü1 and Vi? Describe the set of the e vectors
geometrically. lnclude a sketch in your an wer.

55. ls the vector [ i] a linear comb;nat;on of [ i] and [ !]1

56. ls the vector

30
- I
38
56
62

a linear combination of

I 5
7 6
I 3
9 2
4 8
9 -2
2 -5
3 4 ?
5 7
2 9
 r l
S c. 2. 1 In trod uctio n to Lin ea r Tran for m ati o n and Their lnve rse • 53
your encodecl position will be

Y=[y'Y2 ]=[l31]
220
(see Figure 1).
The cocling transformation can be represented as

LINEAR
TRANSFORMATIONS or, more uccinctly, as

Iy= Ax j.
Tbe _matrix A is called the coefficient matrix of the tran formation , or simpl y its
matnx.
A transfonnation of tbe form

y =Ax .
1 INTRODUCTION TO LINEAR i called a linear transform.ation. We will discuss thi s important concept in greater
detail below.
TRANSFORMATIONSAND THEIR
~s tbe _hip reacbes a new position, tbe sailor on duty at headquarter in
INVERSES Marsedl e recetves the encoded messaae
b

lmagine yourself cruising in tbe Mediterranean as a crew member on a French

coast guard boat, looking for spies. Periodically, your boat radios its position b= u~~J.
to headquarters in Marseille. You have to expect that communications will be He must determine tbe actual position of the boat. He will have to solve the linear
imercepted. So, before you broadcast anything, you have to tran form tbe actual system
position of the boat,
Ax= b,
or, more explicitly,

x, +3x 2 = 1331
(x 1 for Eastern longitude, x 2 for Northern latitude), into an encoded position
I 2x t + Sx2 = 223 ·
Figure I

You use tbe followin g code:

I
Yt = x, + 3x2 encoded posi rion
.Y2 = 2x , + Sx2. . [Y1] = [131]
Yo 220
For example, when the actual position of your boat is 5°E, 42°N or
:~l]ual :os[iti~n] LJ~
52
[ ·' 2 42 y
54 • Chap. 2 Linear Transformations Sec. 2.1 ln troduction to Linear Transformationsand Their Inver es • 55
Here is hi solution . Is it correct?
c d.1ng, w1.thmatnx. A = [I2 3]
5
~
X =
[Xt
X2
] = [ 43 4] .r y

As the boat travels on and dozen of positions are radioed in, the sailor get Figure 2
decoding, with matri x B =[ - S 3]
2 -1
a Iinie tired of olving all those linear systems, and he thinks there mu st be a
general formula to si mplify thi s task. He wants to so lve the system

Xt + 3x2 = Yt I
.Since. the _decod~ing transformation x
= By is tbe inverse of the coding
transforfT!atwn y = Ax, we say that the matrix B is the inverse of the matrix A.
I2x t + 5x2 = Y2 We can wtite thi as B = A - 1•
when y 1 and .Y2 are arbitrary con tants, rather than particul ar numeri cal value . Not al l linear transformation
He is looking for the decoding transformation
y -+ x

that is, the inverse 1 of tbe coding transformation are invertible. Suppose some ignorant officer chooses the code

X --7 y. Yt = x1 +2x2
with matrix
Y2 = 2xt + 4x2
Tbe method of solution is nothing new. We apply elimination as we have
for a linear system witb known val ues Yt and Y2· for the French coast guard boats. When tbe sailor in Marseille has to decode a
position, for example,
Xt + 3x1 = Yt I Xt + 3x2 = Yt I --+
I
2xt + 5x1 = Y2 - 2(1) I -X2 = -2)11 + )12 -7- ( - ]) - [ 17889] '
b=

Xt + 3x1 = Yt I -3(11) Xt = - 5 y 1 +3y21 he will be chag1ined to discover that the system

I X2 = 2y t - Y2 I X2 = 2 y1- Y2
Xt + 2x2 = 89 1
The formu la for tbe decoding transformation is I2x t + 4x2 = 178
Xt = - 5y 1 + 3y2 has infinitely many olutions, namely
X2 = 2yt- Y2.
or
-5
x = By, where B = [
2
where t is an arbitrary number.
Because this system does not have a unique solution, it is impossible to
Note that the decoding transformation is linear and tbat its coefficient matrix is recover the actual position from the encoded position: tbe coding transformation
and the coding matrix A are noninvertible. This code is useles !
-5 T
B= [
2 Now Iet us di sc uss tbe important concept of linear Iransformations in oreater
detail. Since linear transformation are a special cla of functions , it ;ay be
The relationship between the two matrices A and B is sbown in Figure 2. helpful to review the concept of afunction first.
Con ider two set X and Y . A function T from X to Y is a ru le that a ociate
with each element x of X a uniq ue element y of Y. The 'et X is called the domain
of the function , and Y is its codomain. We will sometimes refer to x as rhe input
Figure 3 Domoin X ond codomoin Y of the function. and to y as it output. Figure 3 hows an examp le where domain
1
We will discuss the coocept of the inverse of a tran sfom1ation more systematicall y in Section 2.3. of o fu nclion T. X and codomain Y are finite.
56 • Chap. 2 Linear Transform ations Sec. 2. 1 lnt rod uct io n to Li near Transformatio nsa nd Their Inve rses • 57
In precalcul us and calcul us you tudi d functions whose in put and outpul Yl = a1 I XJ + a 12x2 + ··· + a 111 X 11

are calars, i. e., whose domai n an d cod omain are the real number ~ or subsets Y2 = a 2 1X 1 + a22x2 + ·· · + a2 11 X"

of R for exa mple,

1'2 - 2 Ym = a", JXJ +G 111 2X2 + ··· + G 111 11 X 11 .
.f x) = ex, g(t ) = - - .
r- 1 The output var iable y; are linear functions o f the input variab les x;. In so me
ln mul6 variable ca lculus you may have ncountered function whose input and/or branches of mathematics, a first-order function with a constant term , such as
output \ ere ector . Y = 3xl - 7.x2 + Sx3 + 8, is called linear. Not so in linear alge bra: the linear
fun cti on of n variables are those of the form y = c 1x 1 + c2 x 2 + ... + cnx,11 for
EXAMPLE 1_." some coeffi cients c 1 c2 ... , c,. .
y = Xj" + xi + x~ EXAMPLE 3_." The linea r tran sfonn ation
Tbi formul a defines a function from ~- to ~ - The input i the vector .r = y, = ? x 1 + 3x2 - 9x3 + 8x4
and the output is the calar y. Y2 = 6x1 + 2x_ - 8x3 + 7x4
Y3 = 8x1 + 4x2 7x4
4
(a function from JR to JR3 ) is represented by the 3 x 4 matrix
EXAMPLE 2 _."

r=
CO (t)]
s i~(l )
A =
[8
7
6
3
2
-98]
-8 7 .
[ 4 0 7
EXAMPLE 4_." The coefficient matrix of the identity transformation
Tbi formula defines a function from IR to IR3 , with inpu t l and Output r.
Yl = X ]
We now return to the topic of linear transformation s. Y2 = X]

Definition 2.1.1 Linear transformations Yn =

A function T from 1Ft" to !Ft111
is called a linear transformation if the re is an (a linear transformation from iR" to lR" who e output equals .its input is the n x n
m x n matrix A such that matrix

r ~ ~ ~l
T (x ) = A x ,

for all x in~".

0 0 1
This is a preliminary definition; a more conceptual characteiization of linear trans- (all entries on the main diagonal are 1, and all other entries are 0). Thi matrix i
formation will follow in Fact 2.2.1. called the identiry marrix and is denoted by ! 11 •
Make sure you understand that a linear transforma6on is a special kind of
1 0
6 ~l
fun ction. The input and the output are both vectors. If we denote the outpul vector
T (.( ) by ji, we can write I_ = [ 13 = 0 1 etc.
[ 0 0
5i = Ax. We have already seen the ide n6ty m~ttrix in other context . For example,
Let us write this equation in terms of its components. we have shown that a linear system A.i = b of n equations with n un.knowns ha
a unique oJution if and only if rref(A) = 111 (see Fact 1.3 .4).
(/111]
(/211 XJ ] [ G JJXI + GJ 2X2 + · · · + GJ
11
Xn]
X? G2 JX l + a 22X2 + · · · + a 211X11 EXAMPLE 5_." Give a geometric inte rpretation of the linear transformation

am 2 Gmn
[
;" = aml~l + am2~2 + · · · + a,.m~11 .ii = Ax. where A = [~ -6J.
58 • hap. 2 Linear Transformations Sec. 2. 1 Introduction to Linear Transformation and Their Inverse • 59
Solution
A Straightforward computat ion show that
T

and
r [~J = U ~ nm UJ
Figure 4

Tm= [; i ~J [~J = m
Solution
Fir t, we rew1ite the linear tran formation to how the component : Note that T

co lumn .
m i the fir I column of the matrix A, and T n] IS it third

Thi observation can be generali zed as follow :

Now consider the geometric rel ation hip between the input vector

- [Xl ]
r-
X2
Fact 2. 1.2 Consider a Jjnear trao formation T from Rn to "' Then the matri x of T i
0
and the corresponding output ector 0

y= [ )'~ ] = [ -::~ ] . T (e,.) . where e; =

~ ith .

We ob erve that the rwo vector .t and 5 ha e the same length , Jx~ + xi, and that 0
they are perpeudicu lar to one another (because the dot product equal s 0). From
x
the sign of the components, we know that if i. in the first quadrant, then y will
be in the second, as hown in Figure 4. To ju ti fy thi result, write
x
The outpur vecror y is obtained from by rotafing through an angle of 90°
(!f radians) in the counterclockwi e direction. Check that the rotation is indeed
x
counterclockwise when is in the econd third , or fourth quadrant. ~
EXAMPLE 6 .... Con ider the linear transformation T (x) = Ax , with

A=u in T hen
Find 0
0

Tm and r[H U; VII = V;,

whe.e for simplicity we wrire Tm forT ([ m by d fi nition of the prod uct A e; .

0
60 • Chap. 2 Linear Transformations Sec. 2.1 Introdu ct ion to Linear Tra nsforma tionsa nd Their In erse • 61
The vector el. e2 . ... . eJI in IR" are so me times referrecl to a the Sf!ll'l!Ja~·d In Exerci ·e 9 to 12, decide whetber the g iven matrix i invertible. F in d the
ve tors in II. The Standard ec tOL - I ' e2, e3 in IR 3 are often denoted by i ' j' k. in verse if it exists. In Exe rc ise 12, the constant k i arbitrary.

EXERCISES
GOAL Use the concept of a linear transformation in terms oF the formul a .v =
9. [ ~ ~] 10. [ ~ ~J
13. Prove the fo llow in g Fact :
11. u~] . 12. [ ~ ~]
kv. and interpret imple linear tTan formatio ns geometrica lly. Findthe inverse of
a linear tran fonnation fro m IR 2 to IR 2 (if it exi ts). Find the matrix of a linear a. The 2 x 2 matrix

A = [~
tran fonnation column by column .

Consider the transfmmations from IR 3 to IR defined in Exerc ises I to 3. Which

!]
of these tran Formation are linear . i in vertible if and only if ad - bc =1= 0.
= Hint: Consider the ca e a =1= 0 and a = 0 separately .
."1 = 2x2 )' I = X )'I X2 - X3
b. ff
1. Y2 = X2 + 2 2. Yl = 3x3 3. V2 = X 1X3

:V3 = 2x2 )'3 = X 1 Y3 = Xj - X2

[
Cl
c
b] . . .
d 1 uwert1ble
4. Find the matrix of the linear tran Formation
Yl =9xl + 3x2 - 3x3 then
)'2 = 2xl - 9x2 + X3 1
}'3 = 4x l - 9x2- 2x3 a b]- 1 [ d -ab ] .
[c d - ad- bc -c
y4 = Sx 1 + x2 + Sx3.
5. Con ider a linear tran Formation T From iR to IR 2 , w here
3 (The formula in part b i worth memorizing .

14. a . For wh.ich c hoices of the con tant k i the matrix [; i] invertible?

b. For which cho.ices of the consta nt k are a.ll entrie of [; i ]-I integer?
Find the matrix A ofT.
15. For which choices of the con tant a and b is the matrix
6. Con ider the tran Formation T From ffi: 2 to JR 3 given by
A= [a -b]
b a

invertible? What is the inver e in thi s case?

ls this transformation Linear? If o , find its matrix .
7. Suppose ü1, ü2 , •.. , Ün are arbitrary vect:ors in IR 111 • Consider tbe transforma- Give a geometric interpretation of the linear transformations defined by the ma-
tion from IR" to IR 111 gi ven by trice in Exercises 16 to 23. Tn each case decide whether the tra.nsformation is
invertible. Findtheinver e if it ex.ists and interpret it geometrically.

l
X2
xl
:

.x"
J= X1u1 + X2V2 + · .. +x"un.
_ _ _

Is this transformation linear? IF so, find its matri x A in te rms oF the vector
- - -
U1, U2, .. . , U11 •
16.

20.
[~ ~]
[~ ~]
17.

21.
[-1 0]
0

[-~ 6]
- 1
18.

22.
[00.5 0.50]
[6 -~J
19.

23.

8. Find the inverse of the linear Iransformation Consider the circular face on the fo llowing page. For each of the matrice A in
Yl = x 1 + ?x2 Exercise. 24 to 30, draw a sketch howing the effect of the linear tran Formation
Y2 = 3xl + 20x2. T (x) = A.r on thi Face.
62 • Chap. 2 Linear Transformation Sec. 2.1 lntroduction to Linear Transformationsand Thei r Inverses • 63
35. In the French coast guard example in this section, suppose you are a spy
watehing the boat and ·listening in on the radio messages from tbe boat. You
collect the foiJowing data:

• When the actual position is [ ;] , they radio [ ~;] .

4

• When the actual position is [

4
~ l they radio [ ~~ l
24. [ ~ -b] 25.
[~ ~] 26. 27. [ ~ -n Can you crack their code, i.e., find the coding matrix , assuming tbat the code
is linear?
36. Consider a linear transformation T from ~~~ to ~m. Using Fact 1.3.7, justify

?
28. [ b ~] 29.
0 - 1[-1 0] 30. the following equations:
T ( ü + w ) = T (Ü ) + T (w), for all vectors ü, w in IR",
31. In Chapter 1 we mentioned that the image of Gauss hown on a German bill
,. ., T (kü) = kT(Ü) , for all vectors ü in IR" and all scalars k .
.. is a mirror image of Gau s' actual likenes . What linear transform atio n T
can you apply to get the acrual picture back? 37. Consider a linear transformation T from IR2 to iR2 . Suppose ü and ÜJ are two
x
arbitrary vectors in IR 2 , and is a th.ird vector wbose endpoint is on the line
segment connecting the endpoints of ü and w. Is tbe endpoint of the vector
T(x ) necessarily on the line segment connecting the endpoints of T ( Ü) and
T (w )? Justify your answer.

32. Find an 11 x n matrix A such that Ax = 3x, for all x in ~~~ .

33. Consider tbe tran formation T from ~2 to ~ 2 , which rotates any vector .:r
througb an angle of 45° in the counterclockwise direction .

Hinr: We can write ~r = ü + k (w - ü). for some scalar k between 0 and 1.

Exercise 36 is helpful.
38. The two column vectors ü1 and ~ of a 2 x 2 matrix A are ketched below.
Consider the linear transformation T(Jr) = Ax. from IR 2 to IR 2 . Sketch the
X vector

You are told that T is a linear transformation (thi s will be shown in the next
section). Find the matrix ofT.
34. Consider the transformation T from ~2 to ~ 2 which rot.ates any vector .X
through a given angle cp in the counterclockwise direclion (compare with
Exercise 33). You are told that T is linear. Find the matrix ofT in terms of cp . 0
64 • hap. 2 Linear Tran fo rmat ion Sec. 2.2 Linear Tra nsformations in Geometry • 65
39. Tm or fal ? If T is a lin ar tran formati on fro m lR" to lR111 • then b. Whic h poi nts

T
.r l]
X2
: = XJT el ) + x _T el) + ... +x" T (e"),
[ ,\n
~] ?
are tran sfonned to [ Explain .
\ bere e1 . e2 . ... , e" are the Standard ectors jn JR" .
40. Describe all linear tran formation fTom lR (= JR 1) to JR. What do thei r graph
Iook like?
43. a. Con ;der the vector " ~ [! l Js the tran format;on T (i) ~ " ·i (the dot

41. De: c1ibe all li near tran formation from JR 2 to lR 1) . What do their product) from 3 to lR linear? lf so fi nd the matrix ofT .
graphs Iook like? b. Consider an arbitrary vector ü in 3 . I the tran fonnation T (x = u. x
42. When you repre ent a three-dimensional object graph ically in the plane (o n linear? If so, fi nd the matrix ofT (i n term of the component of ii).
paper, the blackboard, or a computer creen). you have to trans form patial c. Conversely, consi.der a li near tra nsformation T from iR 3 to iR. Show that
coordinates. there is a vector ü in 3 such that T x
= ü . .X, fo r all x
in R 3 .
3
44. The cro s prod uct of two vector in _ is defi ned by

[ ~~ ] .
X) [ :~ ] X[ ~~ ] [ :2:~ =:~~~ ] .
a3 b3
=
a1b2-a2b 1

into plane coordinates [ ~ ]. The simplest choice is a linear tran fo rmalion, Consider an arbitrary vector ü in JR 3 . Is the transformation T (x) = ü x x
fro m JR 3 to JR 3 li near? Jf so, find its matrix in tenns of the components of the
fo r ex an1ple, the one given by the rnatrix vector ii.
45. True or fa.lse? lf T is a linear tranfo rm ation fro m JR 3 to JR 3 , then

[-: 1 0]·
- 2 0 I
fo r all vectors ii and
r cü x
win JR3 .
w) = T (ü) x T (w)

See Exerci se 44 fo r the defini tion of the cros

a. Use this tran form ation to represent the uni t cube with corner points product.

[~] · [g]. [!]· [~] · [l]· [!] [~] · [l] 2.2 LINEAR TRANSFORMATIONS
IN GEOMETRY
Include the images of the x 1, x 2 , and x 3 axes in your sketch.
In the Ia t section, we defi ned a linear transformation a · a function .v = T Cr) from
lR" to 1R111 such that
y =A.~ .

fo r some m x n matrix A.
EXAMPLE 1 .... Consider a ]j near transformati on T (_r) = Ax fro m lR" to lR111 •
- -+-- +--- +--+ Y1
- I a. What i the rel ationship between T(Ü) . T(w) , and TC+ w), where ü and w
are vectors in lR"?
- I
b. What is the relationship between T (v) and T (k ü), where ü is a ector in lR"
and k is a scalar?
66 • Chap. 2 Linea r Transfo rma tions Sec. 2.2 Li near Transformati ons in Geometry • 67

= T l~; ] =
T(kv) = kT(Ü)

T (x) T (x ,e, + x 2 e2 + . . . + x" e" )

'
. ''
Xn

Figure 1 (o) lllustroting the property 'ü kü = T (x ,e,) + T (x2e2) + ... + T (x"e") (by property a)
rv
r + wl = r !vl + T!Wl. = X t T (e, ) + X2 T (e2) + ' .. + x" T (e/1 ) (by property b)
(b) lllustroting the property
r !kVJ = kT !vJ. (a) b)

Solution
a. Applying Fact 1.3.7 we find that
T (Ü + w) = A ü + w) = AÜ + Aw
In words: The transform of the sum of two vector equal the um of the
= T ( Ü) + T (w).
T (e")

l :_:~ ]
Xn
= A,t (by Definition 1.3.6) •

Here is an example illu trating Fact 2.2. 1.

transforms. EXAMPLE 2 ~ Con ider a linear Iransform ati on T from R 2 to ~ 2 . The vectors T(e 1 and T( e 2)
b. Again , apply Fact 1.3.7: are sketched in Fi gure 2. 1 Sketch the image2 of the uni t quare under thi s trau -
T (k v) = A(k - ) = =
kAÜ kT (Ü) .
formati on.

In words: The transform of a calar mulüpl e of a vector i the caJar multipl e Solution
of the transfonn . -<1111
e
The unü square consi ts of aJJ vectors of tbe form .t = x 1 1 + x 2 2 , where x 1 and e
Figure I ilJu strates these two properties in the case of the linear Iransfor- x2 are between 0 and 1. Therefore, the image of the uni t square consists of all
mati on T from 2 to ~ 2 which rotate a vector through an angle of 90° in the vectors of the form
counterclockwi se direction (compare with Example 2. 1.5). T (-'i) = T (x ,e, + X2e2) = XtT (e ,) + x2 T (e_).
In Example 1 we saw that a linear transformation sati sfies the two equation s
T ( ü + w) = T (v + T (w ) and T (k ü) = k T (Ü). Now we will show that the where x 1 and x 2 are between 0 and 1. The last step folJows from Fact 2.2.l. One
converse is true as well: Any tran sformation from R" to R"' which sati sfies thesc such vector T (x) is shown in Figure 3.
two equations is a linear tran form ation.
Figure 2 x T
~
Fact 2.2.1 A tran formation T from IR" to IR"' is linear ' if (and onl y if)
a . T ( ü + w) = T (v) + T (w), for all ü, ÜJ rin R", and
b. T (k v) = kT (v) , for all ü in R" and all sca ars k.

Proof In Example I we saw that a linear transformation sati sfi es the equations a and b.
To prove the converse, consider a Iransformation T from IR" to !R"' that satisfies
equations a and b. We must show that there is a matrix A such that T Ci ) = Ax,
for all X in !R". Let eI ' . . . ' eil be the Standard vectors introduced in Fact 2.1. 2.
1
ln ma ny texts, a linear transfonnaii on is defined as a fun ction from IR" to IR 111 wit.h these two
propen ies. T he order of presentatio n doe not rea ll y mauer; what is important is that you thin k o f a linear
Irans formation both as a fun ction T from lR" to IR"' of the form T (x) = Ax and as a fun cti on w hich has
the propen ies a and b ·tated in Fact 2.2. 1.
Do ma in Codomain
1Note that the.re are two sli ghtly di ffere nt ways to re present a linear Lran formati n fr m R 2 to IR -
geometricall y. Somet imes we wil l draw two di ffe rent planes to represen t do main and codo ma in (a here) . .
and sometimes wc will draw the in put .r und the outpul _v = T(.r ) in the same pl ane (as in Examp le 2. 1.5).
The fit" l represent, tion is les crowded. wh.i le the second one cln rifie the geome tric relation hip between
X and T (x ) (for examp le, in Exa mple 2. 1.5, ;r ~ nd TC:r) are perpendicu lar).
2The imoge o f a subset S of the do ma in consists of Lhe vectors T (x). for a ll .1' in S.
68 • Cha p. 2 Linea r Transformations Sec. 2.2 Linear Transformatio ns in Geo metry • 69
Yz l f you th.in k of the doma in IR 2 of the tran sfo rmatio n T as a rubber heet,
you can im agine that T ex pands the sheet by a fac tor of 2 in the ü2 -di:recti on and
co ntrac t it by a fac to r of ~ in the ü1-direction. ~

. . . - . . -. --,'
:
Figure 3 Figure 4
The vector T (x) in Fi gure 3 is in the shaded parallel ogram sho wn in Figure 4.
Conversely, any vector b in the shaded parallelogram can be wci tten a.
b= XI T(el) + x2 T e2) = T(x l e l + x2e2) ,
for two calars x 1 and x 2 between 0 and 1. This shows that the image of the uni t
square is rhe parallelogram defined by T(e 1) and T (e2). ~ a . See Figure 1 for the case rx = rr j2. Fo r an arbitrary rx, we illu strate on ly the
For generali zatio ns of thi example, see Exercises 35 and 36.
EXAMPLE 3 ... Consider a linear transformati on T from IR 2 to IR 2 uch th at T (Ü1)
T (v2 ) = 2 ü2 for the vectors ü 1 and iJ2 . ketched in Figure 5 . On the am e axes,
sketch T Cr , for the vector -~ given in the sam e figure. Ex pl ain your o lutio n.
Nex t, we prese nt some c lasses of linear transfo rmations from IR2 to 2
that
.._, I are of interest in geometry .
~
~ Rotations
Consider the transformati o n T fro m JR2 to IR2 that rotates a vector through an
angle a in the counterclockwise directi on, 1 as shown in F igure 8. Recall that in
Example 2. 1.5 we studied a rotation with a = rr / 2.
EXAMPLE 4 ... Let T be the coun terc lockwise rotatio n througb an angle a.
a. Draw sketches to illu trate that T is a linear transfo rmation.
b . Find the matrix of T .
Solution
property T (kv) = k T (iJ), leav ing the property T (Ü + w ) = T (v) + T (w) as
an exercise.
= 4v1 and Fi g ure 9 show that tbe vectors T (k v) and k T (v) are equal ; convince
yourself that the two vectors have tbe ame length and the same argument.
b. The matri x of the tran sfo rmation T is

Solution
Using a parallelogram , we can represenr x as a linear combi nation of iJ 1 and Ü2 :

as shown in Figure 6.
By Fact 2.2.1 ,
Figure 8 A rototion. Figure 9
T (x) = T (c 1ü1 + czv2) = c1T Cii1) +c2 T Cii2) = ~c 1 ü 1 +lc2ii2.

T he vector c 1ü1 is cut in half, and the vector c2 ü2 is doubl ed, as sho wn in Fi gure 7 . .Y = T(x) T(kii ) = kT(ii)

Figure 5 Figure 6 Figure 7

X k ii

uz

1 It is eas iest to dcti ne a rotation in terms of po hu· coordi nates. The le.ngth of T (.r) equuls the length

of .~ . and the argument (or polar angle) of T (.r) exceeds the argument of .r by a .
 70 • Chap. 2 Linear Tra nsform atio ns ec. 2.2 Lin ear Transformation in Geo metry • 71

r
)=[-sicosn aJ
T(ii 1
a r sin a

Figure 10
Figure 11 Figure 12

by Fact 2.1.2. To find T (e 1 ) and T(e2), apply ba ic tri gonometry , as show n

in Figure l 0. ote that the vector T(e 1) and T e2) both have length I. -<lllll Solution

Fact 2.2.2 Rotations

Co nsider the va lu e T(e1) = [~] and T (e2 =[ -!l th at i , the colu mn of the

matrix. Note that T (e1) and T (e2 ) are perpendi c ular vector of the ame Jength,
The matrix of a counterclockwi e rotarion through an ang le cx i a how n in F igure 11.
- incx] · We can ee that the vector T (e 1) and T (e2 ) are obtained from 1 and 2 by e e
CO Ci fi r t perfo m1ing a counterclockwi e rotation through the angle a, where tan cx =
b/ a. and then a dilation by the fac tor r = Ja2 + b 2 , the length of the ve tor
T (e J) and T (e2).
This forrnula is important in many applications, fTom phy ic to computer To veri fy th at the tran form ation act in the ame way on all ve tor
graphic · it is worth memorizi.ng. (not ju t e1 and e2), write the vector T (e 1) = [ ~ J in term o f it polar coordin-
Here are two special ca es:
ate. :
• The matrix of a counterclockwi e rotation through an angl e of I is

[a] [ CO Ci ]
[
cos ~
in~ ] = [O -1]
r
b - . r sin cx ·
'
sm -
JT
cos - 0 ' l
2 2 as illus trated in Figure 1- .
as in Example 2. 1.5. Then
• The matrix of a counterclockwise rotation through an angle of n /6 (or 30°) is
[~ - r sin cx] - r [ co cx - in cx ] ·

~l
. Jr ] [.T./3 r cos cx sm cx cos Ci

[
cos ~ -s m ~ =

~
. Jr
Slfl - CO - l The matrix [ - b ] is a scalar mul tiple of a rotatjon matrix ( ee De fi ni tion 1.3.9
6 6 - (/
2
and Fact 2.2.2).
:J The refore,
+ Rotation-Dilations
EXAMPLE 5 ..... Give a geometric interpretation of the linear transformation - [a -b]-
T (x) = b
(I
x =r [ CO.
. Ci
111 CX

T (i) = [ ~ -b]-
a
x, A ve tor X is first rotated th rough an a ng le cx in the counterc lockw ise di recti on;
the re ulting vector i then mul t.i plied by r, whi ch repre e nt.. a dil ati on, a · how n
where a and b are arbitrary constants. in Figure 13. -<lllll
72 • hap. 2 Linear Tran forma tions Sec. 2.2 Linear Transfo rmations in G ometry • 73

T(.t ) = [a - b] X_
a
= I'
[cos
.
sm a
er - sin er
cos er
J~ .I
b

:r
Figure 13
Figure 14
} ':7

' Fact 2.2.3 Rotation-dilations and

To interpret the linear transformati n

T (x) -- [ab -b]- a

X

Here is a more tangible de cription of thi s transformaüon. Consider a tack

~]
of thin sheets of cardboard. viewed from the side, a hown in Figure 15. Align a
geomerrically, write the vecror [ in polar coordinate : boarcl vertically against the edges of the sheets, as shown. Hold the bottom edge
of the board in place while pu hing the top edge to the right. The higher up a
sheet is, tbe further it gets pushed to the right, wirb its elevation uncbanged .

'
J
[ ba = [ rr c? s er
sm a
J The transformation T(.'t) = [ ~ t] x i called a shear paraJJel to thex 1-axi .
' More generally, we define:
'
'
'

Definition 2.2.4 Shears

where r = .Ja2 + b2 and tan(a) = a~. Then T is a counterclockwise rotation Let L be a line 1 in JR2 . A linear Iransformation T from JR2 to JR2 is called a
through the angle a followed by a dilation by the factor r. We call T a shear parallel to L if
rotation-dilation .
a. T(Ü) = ü, for aU vectors ü in L, and
b. T(x) - .t is in L for all vectors x in JR2 .
+ Shears
EXAMPLE 6 .... Con ider the linear tran sformaöon

-
y= ['0 1I] 2 -
X.
Figure 1S
Board ~
T

To understand this transformation geometrically, sketch the image of the unit

square.

Solution
By Example 2, the image of the unit square is the parallelogram defined by T(e 1)
and T(e1 ), as illustrated in Figure 14 (compare with Figure 4). Note that

1Convention: All lines considered in lhis text run through the origin unless stated otherwi e.
74 • hap. 2 Linear Tran forma tions ) Sec. 2.2 Linear Transformations in eome try • 75
Property a rnean- that the tran Fonnation leav . the line L un hanged , and + Projections and Reflections
prop rty b say th at the tip of any vector -~ i moved parallel to the l me L , as
illu trated in Figure 16. Conside r a 1ine L in IR 2 . For any vector ü in 2 , there is a uniqu e vect r ÜJ in L
Let us check propertie a and b for the linea r tran ·fo rmati on such that v- ÜJ is pe rpendicu lar to L , as shown in F igure 18.
Th i vector w is ca ll ed the orthogonal proj eclion of ü onto L , clenoted by
- = [ 1
T(x)
0 ~1 J.t proj~.ii (" orthogona l' means "perpend ic ular"). Intuiti vely, you canthink o r proj ~.ii
as th e shacl ow ü ca ts on L if we shin e a Ught straight down on L.
How can we generalize the idea of an orthogo na l projection to lin in II("?
dj scu _ed in Examp le 6, where L is the x1 -axi .
Let L be a line in IR". i.e., the set of al t scalar multipl es of some no nzero vector
ü, w hi ch we may choo e a a unit vector. For a given vector ii in IR 11 , i it po -
a. T [ -~ J = [ ~ fJ[-'~ J = [ x~ J. The line L re1nains unchanged. sible to find a uruque vector w in L uch that ii - ÜJ is perpendicu lar to L (i.e.,

b. T [ :,::~ J _ [::~ J = [ ~
The vector T (.t) - ; is in L.
n[;~ J_[:~ J = [ XJ l
~2 ~X2 J _ [;~ J = [ !;1
perpendicu lar to ü)? lf uch a vec tor in
exis t . it i. a calar multiple
of ü. See Figure 19. We have to c hoo e k o that ü -
that is,
w
w
= kü
i perpendi cul ar to Ii,

EXAMPLE 7 .... Con ider two (nonzero) perpendicular vectors t-:i and w in IR2 . Show that the
ü . cü - w) = ü . cv - kü) = ü .ü- k (ü . ü ) = ü . ü- k = o.
tran Fonnation

T(.r) = .x + (ii · ,r)w We have u ed the facr that ü · ü = l ( ince ü i a unit vector . Therefore. ü - kü
i perpendi cul ar to ü if (and on ly it) k = ii · ü.
is a shear parallel to the line L spanned by w(see Figure 17).

Solution
We Ieave it to you to verify that T i a Linear h·ansformation . We wi ll check
properties a and b of a shear (Definition 2 .2.4). Figure 18 Projecting onto o line.

a. If v is a vector on L, then

T (v) = ü + (ü. ü)w = ü.

because ü . ü = 0 ( ince the vector ü and v are perpe ndicu lar).
b. For any vector -~ in IR the vector
2

r (x) - ; = ; + (ü . x) iü - ; = Cü . .x) w
is in L , becau e thi vector is a scalar multipl e of ·ÜJ.

Figure 16 Figure 17
Figure 19
X
ll

0
76 • Chap. 2 Linear Transformations ec. 2.2 Linear Tran Fo rm ations in eometry • 77
We ummari ze:

Fact 2.2.5 Orthogonal Projections

• Let L be a line in IR" con isting of all calar multiple of ome unit
v ctor ü. For any vector v in IR" there is a uniqu e vector win L such
th at ti - w i perpendi ul ar to L , namely, ÜJ = (Ü · ti)ü. This vector ÜJ 0
v
called the orrhogonal projection of onto L: Figure 21
proj'-ü = (ü. v)u
• The transformation T (li) = proj L(ti) from !Rn to !R" i linear.
Definition 2.2.6 Reflections
Let L be a line in IR". Fora ector - in lR", the vector 2 proj L v)- ü i called
The verification of linearity i straightforward; u e Fact 2.2.1. We will check the refl.ection of ii in L:
property b here, and leave property a as an exer ise: refcü = 2( proj '-v)- ü = 2(ü · ii)ii- ü.
proj L(kv) = Ü · kÜ)Ü = k(Ü · Ü)Ü = k projL V) where ü i a unit vector in L.

v v in
Con ider again a line L and a vector ti in 1R 2 . The reftection ref L of
the line L i shown in Figure 20.
Figure 21 illu trates the relationship between projection and refiection:
v
If i perpendicular to L we expect th at reh- = -ü. Th.i result can be
confirmed by computation: lf ü is perpendicul ar to L , then ü . ü = 0, so that

refLü = 2(ü · ii)ii- ü = 0- ii = - - .

We leave it as an exercise for you to verify that the reflection in a Line in !Rn is a
Now solve for refL ü: li near transfom1ation from !R" to ]Rn .

v
ref'- = 2(projL Ü) - v= 2(ü · ü)i:t - ii,
EX ERCI SES
where il i a unit vector in L. GOALS Check whether a transformarion is linear. Use the marri ces of rotation
We can use the formula 2(proj'-ti) - ii to define the refi ection in a line and rotation-dilation s. Apply the definitions of shear . projections. and ref1ec-
in !R". tions.

1. Sketch the image of the unit square under the Linea r Iransformation

Figure 20 - [3 2I] -
T (x) =
1 X.

2. Find the matrix of a rotation through an angle of 60° in the counterclockwi e

direction.
3. Consider a linear tran formation T from JR2 to JR 3 . Use T (e 1) and T(e 2 ) to
describe the image of th.e unit square geom trically.
4. Interpret the following linear tran formation geometrically:

- = [ l
T (x)
1
-1]~ ~
78 • hap. 2 Linear Tran fo rmation s Sec. 2.2 Linear Tra nsformation s in Geom etry • 79
5. T he matrix 13. Suppose a li ne L in IR 2 co ntains the uni t vector
- 0.8 - 0.6 ]
[ 0.6 - 0.8
repre nts a rotatio n. Find the ang le of rotati n (i n radian s).
Find the matrix A of the linear transfo rmation T (t) = ref'-x . Give the entri e

6. Let L be th line in IR' that con i t of a ll scalruc multiple of the vector [ ~]. of A in terms of u 1 and u 2 •
14. S uppose a line L in ~" contains the uni t vector

Find the orthogonal projectio n of the ector [ : ] nto L.

- r~~J
u= . .

7. Le t L be the line in IR' which co nsists of all calruc multipl s of Ul F ind

u"
a. Find the matrix A of the li near transfo rm ati on T (x) = proj Lx . Gi ve the
e ntries of A in terms of the co mponents u 1 of ü.
the reflection o f the ecl<lr [ : ] in the line L . b. What is the sum of the d iagonal entries of the matri x you fo und in part a?
15. Suppo e a line L in lR" conta.ins the uni t vector
8. Interpret tbe following linear transfom1 ati on geo metri call y:

T Cx) = [ ~ - ~ J~
9. Interpret the following li near tran Formati on geo meLTicall y:
ü= r:; J.
un
u"

T x) = K Find the matrix A of the linear transformati on T (,r ) = reft_x. Gi ve the entri es
of A iJ1 terms of the components u ; of ü.
10. Find the matrix of the orthogonal projection onto the line L in ~ 2 be low. 16. Let T (x ) = refLx be the re flection in the Line L mR2 sho wn below .
a. Draw sketches to illustrate that T is li near.
b. Find the matrix of T m
terms of ct .

11. Refer to Exercise 10. Fi nd the matri x of the refl ectio n in the line L.
12. S uppose a line L in ~ 2 contain the unit vector 17. Consider the linear transformation

- [u1]
LI =
u2
.
T (x- ) = [ 1
1
1] -
I X.

Fi nd the m atri x A of the linear transformati on T (x ) proj '-x . Give the a. Ske tch the image of the unit square under this transformati on.
entrie of A in terms of u 1 and u 2 . b. Explain how T can be interpre ted as a projecti on fo llowed by a dil ation.
80 • Chap. 2 Linear Transformati ons Sec. 2.2 Lin ear Transformatio ns in eo metry • 81
18. fnterpret the tran fo rmatio n 29. Let T and L be tran sformations from IR" to IR". Suppose L is the inverse of
T , that is,

T (L (.K)) = .r. and L (T (x)) = x

a a reft ection fo ll owed b a dilalion.
fo r all x in IR". lf T i a linear transform ation , i L hnear a well? Hint:

Find the mau·ices of the linear tran form ation from JR3 to JR 3 give n in Exerci e x+ y = T(L (x) ) + T(L ()i) ) = T(L (x) + L ()i ),
19 to 23 . Some of these tran fonnati ons have not bee n form all y defi ned in the
because T is linear. Now apply L on both side .
text. Use common sen e. You may as ume that all rhese t.ransformation are
linear.
J
30. Find a nonzero 2 x 2 matrix A such that Ax is parallel to the vector [ ~ J,for
19. The orthogonal projection onto the .x- y plane. aU .r. in !R2 .
20. The reftection in the x- z pl ane.
21. The rotation about the 7-axi through an angle of rr /2, counterclockw i e as
viewed from the positive z-axis.
31. Find a nonzero 3 x 3 matrix A such that Ai is perpeod;cular to [ ~ J.
for all

22. The rotation about the y-axi through an angle a, counterclockwi se a viewed
,r in IR3 .
from the positive y-ax.is. 32. Consider the rotation matrix D = [ c?
sm a
a -
111
COSCi
a J and the vector ü =
23. The reftection in the plane y = .
24. Consider the linear transformation [~~~; J, where a and ß are arbitrary angles.
rcx>=[i ~J .x. a. Draw a sketch to explain why Dü = [c?s(a + ß>].
SIO(a + ß)
b. Compute Dü. Use the resuJt to derive the add.ition theorems for ine and
a. Sketch the image of the unit square under this transformation . cosine:
b. Show that T is a shear.
c. Find the inverse transformation and describe it geometrically. COS (et + ß) = . .. , sin (a + ß) = .. . .

25. Find the inverse of the matrix [ ~ ~] , where k i an arbitrary constant. 33. Consider two distinct hnes L 1 and L 2 in .IR 2 • Explain why a vector ü in IR2
Interpret your result geometrically. can be expressed uniquely a
26. Let

rcx>= [ -
-I
1
4] -
3 X where Ü1 is in L1 and Ü2 in L 2 . Draw a sketch. The tran formation T (v) = ü1
is called the projection onto L 1 along L 2 . Show aJgebraically that T i Linear.
be a Linear transformation. Is T a shear? Explain. 34. One of the five matrices below represents an orthogonal projection onto a line
z
27. Consider two linear transformations y = T(x ) and = L(Y), where T goes and another represents a reftection in a line. Identify both and briefty ju tify
frorn IR" to !Rm and L goes from IRm to IRP. Is the tran. formation z= L(T(,r) your choi.ce.
linear as weil ? (The rransforrnarion z = L(T (x )) is called the composite ofT
[I
and L.)
28. Let

A=[~ ~] and B = [P
r
q]
s
.
A = 3 ;
J 2 2]
J
2
2
1

'
' B =
t!
3
:J. c= 3
f! n
D= -~u 72] '
2 2
Find the rnatrix of the linear transformation T(x) = B(Ax) (see Exerci se 27).
Hinr: Find T{e 1) and T(e2 ) . 3 2
1
2 E= ~ =
[ -1
- 1
2 - I
;J
82 • Chap. 2 Linear Transformations Sec. 2.2 Linear Transformations in Geometry • 83
2 2
35. Let T be an in vertible linear Iransformation from lR to JR . Let P be a
parallelegram in JR 2 w ith one vertex at tbe origin . Is the image of P a
pru<lllelogram as weil ? Explain. Draw a sketch of the image.

39. Let

36. Let T be an invertible linear transformation fro m JR 2 to JR-. Let P be a

be nonzero perpendi c ular vectors. Find the matrix A of the hear
parallelogram in JR 2 . Is the image of P a parallelogram as weil ? Explain.

T(x ) = ; + Cii -x)w

(See Example 7). Gi ve the e ntries of A in terms of a and b.

40. Let P and Q be two perpendicular Iines in iR 2 . For a vector ; in JR 2 , what is
projpx + projQx? Give your answer in term of x. Draw a sketch to ju tify

tr your an wer.
41. Let P and Q be two perpendicular lines in JR 2 . For a vector ; in JR2 • what
i the relationship between refpx and refQx? Draw a ketch to ju tiEY your
ans wer.
42. Let T(,t) = projL ..r be the projection onto a line in '";,". What is the re lation hip
between T (.t) and T(T(x))? Justify your answer carefully.
37. Let T be a linear transformation from JR2 to JR2 . Three vectors 1, ~, w in
v
lR 2 and the vectors T(ii 1), T (v 2 ) are shown below. Sketch T (w). Explain. 43. Find the inver e of the rotation matrix

A = [ c? a -sina ] .
Xz 5111 Ct CO a

Interpret the Jjnear transformation defined by A - 1 geo metric all . E plain.

l) '

44. Find the in verse of the rotation-dilat ion matri

38. Let T be a linear transformation from JR 2 to JR 2 . Let ii 1, ü2 , w be three
vectors in lR 2 , as shown on the following page. We are told that T(ii 1) = ii 1
and T(ii2) = 3ii2 • On the same axes, sketch T(w).
A= [a
b
-ba ]
(assunting that A is not the zero matrix). Interpret the linear tran formation
defined by A - I geometrically. Explain.
45. Consider two linear tran formation T and L from JR2 to JR 2 . We are told that
T (v 1) = L(ii,) and T(v _) = L (Ü_) for the vectors ü1 and ü2 skerched on th
following page. Show that T (-"r) = L (,t), for all vec tors ;r in .JR2 .
84 • hap. 2 Linear Transformation
Sec. 2.2 Li nea r Transformations in Geo met ry • 85
Find the functio n f(t) defi ned in Exerci e 47, graph it (u ing techno logy ,
and find a number c between 0 and rr/2 . uch that j(c) = 0. U e your an wer
to find two perpendicul ar unit vectors v1 and ü2 such that T(ÜJ) and TC 2 ) are
perpendi cu lar. Draw a sketch.
49. Sketch the im age of the unit circle under the linea r transformation

- [50 0]2 -
T(x) = X.

50. Let T be an invertible linear transfom1ation from ' 2 to JR2 . Show that the
46. Con ider a hear r from ~- to ~ . Show that there are perpendi
2
cul ar vectors image of the unit circle is an eLEpse centered at the origin. 1 Hint: Con ider
- and w 2
in 2 . uch that T (r) = .X + (v · ~r)w , for all .r in ~ . two perpendicular unit vecLOrs v1 and ü2 such that T (ii 1) and T (~) are per-
47. Let r be a linear tran formati on from JR to ~ . Consider the function
2 2
pendicular (see Exerci e 47d). The unit circle consist of aU vectors of the
form

ü = cos(t)ü 1 + sin(t)v2 ,

from ~ to JR. Show: where r i a parameter.

a. The function j(t ) i continuous. You may take fo r granted that the func- w
51. Let w1 and 2 be two nonparallel vector in ~ 2 . Coo ider the curve C in JR2
tion in (t) and cos(r) are continuous, and also that ums and product of which consist of all ector of the form co (t 1 + in (r) w
where r i a w_,
continuou function s are continuou . parameter (draw a sketch). Show that C is an eUip e. Hint: You can interpret
b. f(rr/2) = - f( O). C as the image of the unit circle under a suitable Linear transformation; then
c. There is a number c between 0 and rr/2 uch that f(c) = 0. U e the use Exercise 50.
intermediate value theorem of calculus, whi ch teil u the following: lf a 52. Consider a linear traosfonnation T from JR2 to JR2 . Let C be an ellipse in JR 2 .
function g(t ) is continuou for a ::: t ::: b, and L i a number between g(a) Show that the image of C under T i an ellipse as weil. Hint: Use the result
and g(b), then there i at least one number c between a and b such that of Exercise 51.
g(c) = L.
d. There are two perpendicular unit vectors v1 and ~ in ~ 2 such that the
vectors T (ii 1) and T (v 2 ) are perpendicular as well. (See Fact 7.5.2 for a
generalization.)
1
An ellipse in JR- centered at the origiu may be defined as a curve that can be parametrized a ·

w
for two perpendicular vec tors 1 and w2.Suppose the length of Wt exceed- the length of w2. Then we
cull the vectors±w 1 the semimajor axes of the ellipse. and ±w1 the semiminor axes.
Convemion: All ell ipse considered in this tex t are ce ntered at the origin unless stated otherw i e.

48. Refer to Exercise 4 7. Consider the linear transformation

r (x) = [ ~ _ ~ J; .
-
86 • Chap. 2 Linea r Transformations Sec. 2.3 TheInverse of a Linear Transformation • 87
T T- i
.3 THE INVERSE OF A LINEAR y
TRANSFORMATION
Let' s first review the concept of an invertible function.
Figure 2 Afunc1ion T.
lts inverse r-l.
Definition 2.3.1 Invertib le functions
A function ow consider the case of a linear transfo nnation from lR" to _ m given by
T:X~ Y .Y = Ax ,
is called invertible if the equation T x) = y ha a unique solution x in X for where A is an m x n matrix .
each y in Y. Here X and Y are arbitrary ets. The tran Formation y = Ax is invertible if the linear system

Ax = .Y
Consider the examples in Figure L, where X and Y are finite set . ha a unique so lution ,i in lR" for alt v
in !R117 • Using tecbnique developed in
If a function T: X ~ Y is invertible. its inverse y - l: Y ~ X is defined by Section 1.3 ("On the so lution of linear ystems') we can deterrnine for which
matrice A thi s is the case. We examine the ca e m < n, m = n , and m > n
y - 1 (y = (the unique x in X suchthat T x) = ).
eparate ly.
See the example in Figure 2.
Note that Case 1: m Ax y
< n • The system = has fewer equation (m) than unknowns
y - 1(T X)= x for all x in X , and (n) . Since there must be nonJeading variable in this ca e, the sy tem ha either
T(T-l(y)) = y, for a1l y in Y. no olutions or infinitely many sol ution s, for any given y in ._"' (Fact 1.3.3).
Therefore, the tran Formation ji = .X i noninvertible.
If a fu nction T is invertible, then so i y - I , and As an example, consider the case when m = 2 and n = 3, i.e., the ca e o f a
( T - 1) - 1 = T . linear tran formation from 3 to 1?.' 2 . Figure 3 ugge ts rhat ome coJJap ing take
place a we tran form JR 3 linearly .into IR2 . In other ward , we expect that m any
If a function is given by a formula, we may be able to find its inverse by point in ~ 3 get transformed inro the ame point in R 2 . The algebraic rea o ning
olving the formula for the input variable( ). For example the inverse of the above l:_ow that thi is exactly what happen : for some y in JR2 for example
function for y = 0), the equaüon T(."i) = A.r = y wi ll ha e infinüely many o lution x in
x3 - 1 JR 3 . See Exerci e 2. 1.42 for an example.
y= - - (from lR to JR) (Surpri ingJy, there actually are invertjbJe (but nonlinear transformation
5 from JR 3 ro JR2 . More generaJly, there are nonlinear in ertible tran formation
is from II (Q J! l1l fof any tWQ positive integer /1 and m.)
x = J 5y+l
Figure 3

Figure 1 Tis invertible. Ris not invertible: the equotion R(x) = y0 hos two solutions, x1 ond X7. S is not invertible: there T(.r =A.r
is no x such that S(x) = Yo·
T R s
88 • Chap. 2 Linear Transformations Sec. 2.3 The lnver e of a Linear Transformatio n • 89
Case 2: rn = n • Here. the number of equation in the sy tem A .~ = b matches
Fact 2.3.3 An m x n matrix A i invertible if and only if
rhe m1mber of unkno n . By Fact 1.3.4, the sy t m A.r = y has a unique so lution
.\- if and only if a. A is a square matrix (i.e., m = n), and
I 0 0 0
b. rref(A ) = !". -v-. tri tnY ~J./ 1 ft..l rn".(
0 I 0 0
nef(A) = 0 0 0 = !., .

EXAMPLE l ... Is the matrix A invertible?

0 0 0

h [~ ~]
2
We concl ude that the tran formation = .v Ax
i invertible if and only if rref(A) =
5
/
11
, or, equi alentl y. if rank A) = 11. 8
Con ider the linear tran formation from !R2 to !R2 we tudied in Section 2.2 .
You may have een in the la t exercise ection that rotation , reflection , and shears
are invert.ible. An orthogonal projection onto a line i noninvertible. (Why ?) Solution
Case 3: rn > n • The tran formation )i = Ax
u 3]
i nonjnvertible, because we can

[~ ~]
2 -+ 2 -+
find a vector y in !R111 uch that the ystem Ax = y is inconsistent ( ee Example 5 -4(1 -3 -6 -;- -3)
1.3.1 3; note that rank(A)::: n < 111 here). 8 -7(1) - 6 - 12
As an exam ple, consider the ca e when m = 3 and n = 2, i.e., the case of a
linear transformation from _ 2 to JR3, a shown in Figure 4. lntuitively. we expect
that we cannot obtai n all points in JR3 by transforming JR2 linearly into lR . For
many vector .v in JR 3 the eq uation T (.\-) = .Y wi ll be inconsistent. [~ J]
2
1
-6
-2(TI)
-+
+6(ll) [~
0
1
0
-i]
We now introduce some notation and summarize the results we derived A i not invertible, mce rref(A) =I h
above.

Definition 2.3.2 Invertible matrices Fact 2.3.4 Let A be an n x n matrix.

A man·ix A is called invertible if the linear transformation y = kr i invert- a. Con ider a vector b in lR". If A is invertible, then the y tem kr = b
ible. The matrix of the in ver e tran formation 1 i denoted by A- I. If the x
has the unique olution = A- 1b. lf Ais noninvertible, then the sy tem
transformation y = Ax
is inve1tible, its inver e is y. x =A-I A.\- = b ha infinitely many olutions or none.
b. Consider the special case when b = Ö. The system A.t = Ö has .r = Ö
a a so lution. lf A is invenible, then tills i the only olution. If A .i
noninvertible, then there are infinitely many other olutions.
Figure 4

If a matrix A i invertible, how can we find the inverse matrix A- I? Con ider
the matrix

~]
1 1
A = 2 3
[ 3 8

or, equivalently, the linear Iransformation

Yt

Y
y
l] = [ 2x XJ + X2 + X 3]
+ 3x· + 2x .
1The
[ Y3 1

3x, + 8x2 + 2x
invcr e transformation is linear (see Exercise 2.2.29).
90 • Chap. 2 Linear Transformation Sec. 2.3 Th e lnver e of a Lin ear Transformation • 91
To fi nd the in v r tran for mation. we olve thi " yste m for the inputvariab le x;. Thi s process ca n be de cribed succinctly as fo llows :

X1 + X2 + X3 = .\' 1
2x 1 + 3x2 + 2x3 = - 2(1) Fact 2.3.5 Finding the inverse of a matrix
3x 1 + 8x2 + 2x > = .\'3 - (1)
To fi nd the inve rse of an n x n matrix A , form the n x (2n) matrix [A : I ,.]
and compute rref[A : I"].
x, + X:!+ X3 = Yl - (ll)
• lf rref[A : I" ] i of the form [I" : B], then A is in vert ible and A- 1
= B.
X = - 2 )1 1 +
5x2 - x 3 = - 3y , -5(II) • If rref[A : I" ] is of anoth er form (i.e. , its left half i · not I" , the n A
not in vertib le ( note that the left half of rret1A : I"] i n:ef(A)).

x, + X3 = 3y,- Y2 --7

.\'2 = - 2y , + Y2
X3 = 7 y , - 5 y2 + )'3 -;- (- 1) The inverse of a 2 x 2 matrix is particularly easy to find .

x, + X3 = 3y, - Y2 - (Hl) Fact 2.3.6 The inverse of a 2 x 2 matrix

X = -2y , + n --7

X = - 7y, + 5y2- )'3 a. The 2 x 2 matrix

A= [ ~ ~ ]
= 1Oy1 - 6Y2 + v3
= -2yl + Y2 is invertible if and only if ad - bc =!= 0.
X3 = - 7 )'I + 5 Y2 - )13 b. If

Wehave fo und the inverse tran formation ; its matrix i

in ve11ib le. then

B = A- 1 =[ ~~
-7
-~
5 - 1
~]· a
[c
b]-l
d
I [
- ad- bc - c
d -ba ].
We can write the computations performed above in ma!rix form .

0] Campare with Exercise 2 .1.13 .

u n [~
1 I 1 0 --7 1 I 1 0 -(!!)
3 2 0 1 - 2(1) 1 0 -2 I 0 --7

8 2 0 0 - 3(1) 5 -1 -3 0 1 -5(U) EXERCI S ES

GOALS Apply the concept of an in vertibl e f unctio n. Derermine whether a

j]
matrix (or a linear transform ati on) is invertible, and find the in ver e if it ex i t .

u n u
0 1 3 - 1 --7 0 1 3 - 1 - (lll)
l 0 -2 1 I 0 - 2 1 --7

0 - I 7 - 5 -:-(- 1) 0 -7 5 Oe ide whether the matrice s in Exercise 1 to 15 are in ertib le. lf they are, find
the inverse. Do the computation with paper and pe ncil. Show all your work.

[~
0 0
l 0
0
lO
- 2
-7
-6
I
5
j] ]. u~ ] 2. [ ~ :] 3. [ ~ 7]
92 • Chap. 2 Linear Transformations
Sec. 2.3 Th e Inverse of a Linear Transformation • 93

s. u3 n 6. [~0 0~ ~1 ] ~] 29. For which choices of the constant k is the matrix be low inve rtibl e?

9. [; :]
10.
[ 1
I
I 2
3
l
[i 1
2
4
k
I ]
k2
30. For wh ic h c ho ices of the constants b and c is the matri x below in vertib le?

12. [ i -i ~ ~] [ -~ - b - c
~ 0~]

runJ
31. For which choices ofthe co nstants a, b. a nd c i the matrix bel ow invert ib le?

~ ~]
2 3
4 7
1 0 a b]c
14.
15. [ 7
11
14
25
25
50
[ -a
-b
0
- c 0

Decide w hether the linear tran formation in Exercise 16 to 20 are invertib le.
32. Find aJI matrice [ : ~] uch that ad- bc = I a nd A- 1 = A.
Find the inverse transformation if it ex.ists. Do the computations with p aper and
penciJ . Show all our ork. 33. Cons ider the matri ce of the form A = [ : _! l wbere a a nd bare arbitrary
co nstant . For which choices of a a nd b i A- I = A?
16
y 1 = 3x 1 + Sx2
17 1 = x1 + 2x2
· +
. ·_ = Sx 1 8x2 · )12 = 4x l + 8x2 34. Consider the di agonal matri.x

18.
.V t =
Y2 .r3
=
X2
19. Y2 =
)' I =
+ 2x2
X1
x,
+ X2 +
+ 3x3
X3
A =
a
0
0 0]
b 0 .
[
• 3 = x, Y3 = x 1 + 4x2 + 9x3 0 0 c
Y l = x , + 3x2 + 3x3 a. For which c hoice of a, b, and c is A invertible? If it i in vertible, w hat
20. y_ = x 1 +4x2 + 8x3 ( is A- 1?
Y3 = 2x , + + 12x3 7x2 b. For which choice of the diagonal element i a diagonaJ matrix of arb i-
trary size) invertible?
WJ1icb of the fo!Jowi.ng fu nctions f from lR to lR (Exerci es 21 to 24) are in vert- 35. Con si der the upper triangular 3 x 3 matrix
~
ible?

21. f(x) = x 2
22. f(x) = 2x A =
[ 0
{/
0
b
d
0 f
c]
e .

Which of the foUowing (nonlinear) transformarion from JR 2 to 2 (Exerci es 25 a. For which choice of a. b. c. d . e, and f is A in ertible?
to 27) are invertible? Find the inverse if it exists. b. More generally, when i an upper triangular matrix (of arbitrary s ize)
invertible?
c. If an upper triangular matrix is invertib le , is it inve r e an upper triangular
25. [Y
Y2I] =[xt]
x2 • Y2
x2
26
xf + x2
[YI ]=[
27.
Y2
J [Y1] = [x1+x2
x 1· x2
J· matrix as weil ?
28. Find the inverse of the Lineartransformation T : JR -+ JR below.
4 4 d. When is a lower triangular matrix invertibJe?
36. To deterrn ine whether a square matrix A is invertible, it i not alway - nec-

r r ~ J + r ~ J+ r ~ J
essary to bring it into reduced row echelon form . Ju tify the following rule:
X?
J
;~ =X 1
XI
- 16
22~ J+ -3
13
-2
8
-2
3 To determi ne whether a square matrix A i invertible, reduce it to triangular
T
r X2 X3 X4
• form (upper or lower), by elementary row operations. A is in e rtible if (a nd
on ly if) a ll entries on the diagonal of this triang ul ar form ar nonzero.
94 • hap. 2 Linea r Transfo rmations Sec. 2.3 The In verse of a Linear Transformation • 95
37. Jf A i, an in ertible matri and c i a nonzero calar, i th matri x cA invert-
the mul tiplicatio ns and di visions are counted . As an example, we exami ne
ible? [f so. w hat is the relationship between A- I and (cA )- 1? in vertin g a 2 x 2 matrix by e li mination.
~ ~I ]
38. Find A- I fo r A = [
39. Consider a quare matrix w hich differ from the idenlity matrix at ju t
entry, off the diagonal. For exam ple,
ne
[~
b
d
.(.
1
0 n -:-a, requires 2 operati ons: b ja and ! ja

A~ -~[
0 0]
I
0
0
J
. u b'
d
.(.
e
0 ~]
(where b' = b ja, and e = ! ja)
-c (I ), requires 2 operatio ns: cb' and ce

[~ ~]
In brreneral is a matrix M o f thi form invertible? If so, what is the M - ? b' e
40. Show that if a squarematrixA ha two equal column , then A i not invertible. d' g -:-d', requires 2 o pe ration
41. Which of the following linear Iransformatio ns T: IR 3 ~ IR 3 are invertib le? .J,
Find the in verse of th ose that are invertible.
[~ ~]
b' e -b'(! /) , require 2 operation
a. R efl ection in a plane . g'
b. Projection onto a plane.
c. Dilation by 5 (i.e. , T (v) =S v, fo r all vectors ü). ,).
d. Rotation about an axis.
42. A square matrix i called a permuta!ion matrix if it contai n rhe entry I [6 0
1
e'
g' {]
exactl y once in each row and in each column , with all otl1er e ntrie bei ng 0 .
The w hole process requires 8 operations. Note that we do not count operatio n
Examples are /" and
witll a predictable res ult such as I a , Oa, aja , Oja.
0 0 l] a. How many operations are required to in vert a 3 x 3 matrix by eliminati on?

[ 1 0
0 I
0
0
. b. How many operati ons are required to in vert an n x n matrix by elimin ation?
c. If it takes a slow hand-held calculator 1 second to invert a 3 x 3 matrix. how
Are permutati on matrices in ertible? 1f o, i tlle inver e a permutation matrix long wiU it take the same calcul ator to in vert a 12 x 12 matrix? A ume
as weil ? that the m atrices are in verted by Gau s-Jordan elimination , and th at the
duratio n of the co mputati on depends o nly on the number of mul tipli cati o n
43. Con ider two in vertible 11x 11 matrices A and 8. the linear tran Form ation
and divisions involved.
y = A( Bx) invertible? If o, w hat is the inver e? Hint: Solve the equation
y = A( B x) fust fo r Bx, and tllen for x. 46. Consider tlle linear sy te m
44. Consider tbe n x n matrix Mn which contai n all in teger l , 2, 3 . . . n 2 a
its e ntries, w ritten in eque nce, col urrm by co lu mn ; for exam ple.
w here A is an invertible matrix. We can olve this y tem in two d iffere nt

~ ~ ~ :! !!]
ways:

M, [ • B y findin g tlle reduced row echelon form of the aug mented matri x
[A : b].
a. Derermine the rank of M4. • By computing A- 1
, and using tlle formul a .r = A- 1
b.
b. Deterrrune the rank of M", fo r an arbi trary n :::: 2.
c. For whi ch integers n is M 11 in vertibl e? In generaJ , which approach reqwre fewer Operations? See E ercise 45.
45. To gauge the complexity o f a computational task, mathem ari c ians and com- 47. Give an example of a noninvertible function f from IR ro IR and a number b
puter scienti ts count the numbe r of e le mentary Operatio ns (addiri ons, s ub- such that the equation
tracti o ns, multiplication , and di vision ) required. S ince additions and s ub-
f (x ) = b
tractions require very little work compared to multipli cati on. and divi sio ns
(think about performing these operations o n e ight-di git numbers), often only ha a unique solution.
96 • hap. 2 Linear Tra nsformations
111
48. Give an exam ple of an m x n matrix A and a vector b in IR
li near ystem
A .~ = b
has a unique solution. Why doe n' t thi example contradict Fact 2.3.4a?
49. lnpur- ourpur analysis. (This exerci e builds on Exerci es 1. 1.20, 1.2.3 7,
1. 2.38. and 1.2.39.) Con ider th indu tries J 1. h , . . .. l n in an econo my.
Suppo. e the consumer demand vector is b, th outpur vect.or .i _an.d the
dem and vector of the jth indu try is Üj (the ith co mponent aiJ of vj I S the
demand indu try l j puts on industry J; , per unit of output of l j . As we. have
een in Exercise 1.2.38, the output ."K j u t meets tb e aggregat:e demand 1f
X
'--y---'
agg regate demancl outpul
Tbis equ ation can be wri t:ten more succinctly a
-
Vn
or Ai+ b = x. The matrix A is ca lled the rechnology matrix of thi economy;
its coeffi cient.s aij describe tbe interindustry demand , whi ch depends on the
technology u ed in the producti on proce s. The equ ation
Ax +b = x
describes a linear system ; we can wri te it in the custo mary fo rm :
x - Ax = b
r"; - Ax = b
u " - A)x = "b
x
lf we want to know the output required to sati sfy a given consumer demand
b (this was our objecti ve in the previous exercises), we can solve this linear
system , preferably via the augmented matrix.
In economics, however, we often ask other questi ons: l f b changes,
how will .X change in response? If the consumer demand on one indu stry
increases by 1 unit, and the consumer demand on the other industries remain s
unchanged, how will ; change? 1 If we ask question s like these, we think of
x
the output as a f un.ction of the consumer demand b.
1
The re levance of quesli ons like these became parl icul arl y c lear durin g WWII . when lhc demand on
certa in indu lries s uddenl y c hanged dramalicall y. When U.S. Pres ide nt F. 0 . Roosevell as ked for 50,000
Sec. 2.4 Matrix Prod ucts • 97
such th at the _ lf the matrix (/" - A) is invertibl e,2 we can expres .X a a f-unction of
b (in fact, as a linear transformati on):
-
X = Un - A)
_,-b.
a. Consider the example of the economy of Israel in 1958 (discussed in
Exercise 1.2.39) . Find the technology matrix A , the m atrix (!11 - A), and
its in verse, Un - A) - 1•
x b. In the example discussed in part a, suppose tbe consumer demand on
agriculture (Industry 1) is 1 unit (1 million pounds , and the demand on
the other two industries are zero. Wh at outpur ; is required in th is case?
How does your answer relate to the matrix (!11 - A) - 1?
c. Expl ai n, in terms of eco nornics, why the diagonal elements of the rn atri x
(!" - A) - 1 you fo und in part a must be at least 1.
d . If the con umer demand on man ufacturing increa es by l (from whatever
it was), and the consumer dernand on the two otber industries remain the
same, bow will the output bave to cbange? How does your an wer relate
to the matrix U~~ -Ar ' ?
e. Using your answers in parts a to d as a guide, explai n in general (not just
fo r this example) what the columns and tbe entrie of tbe matrix (/11 - A) - 1
tel! you, in tenns of econornics. Those wbo have studied multi vari able
calcu lus may wish to consider the partial derivatives
OX;
abj
50. This exercise refers to Exercise 49a. Consider the entry k = a 11 = 0.293 of
the technology matrix A. Verify that tbe entry in tbe first row and the fir t
column of (!11 - A) - 1 is tbe value of tbe geometrical eries l + k + e + ... .
Interpret thi s observati on in tenn of economic .
MATRIX PRODUCTS
Recall the composite of two functions: The composite of the funcöons y = sin (x)
and z = cos(y) .is z = cos(sin (x)), as illustrated in Figure l.
Sirnilarly, we can compose two linear Iransformation .
To understand tbis concept, let's return to the coding example di cussed in
Section 2. 1. Recall that the pos ition ~r = [ ~~ ] of your boat is encoded and th at
airplanes 10 be bu ilt , il was easy e nough to pred icl thal lhe coun1ry would have 10 produce rnore a lum inurn .
Unexpcclcd ly. lhe demand fo r copper drn malically increased {why?). A copper hon age then occurrcd ,
which was solved by borrowin g il ver from Fort Knox. People realized thal in pm-oulpul ana lys is ca n be
effec1i ve in mode ling and predicting c ha ins of increased demand like lh i . After WWf!, tJ.lis 1echniq ue
rapidly gained acceptance, and was soon used 10 mode l the econo mie o f more 1han 50 counlrie .
2 T his will always be the case for n ' ·producli ve" econo my. See Exercise 2.4.75.
98 • hap. 2 Linear Transform at ions
Sec. 2.4 Yfatrix Products • 99
o that

Z1 = 6(x 1 + 2x2) + 7(3x l + 5x2 ) = (6 · 1 + 7 · 3)xJ + 6 · 2 + 7 · 5)xz

= 27xl + 47x2 ,
Figure I
X------------ o. (_in (.r)) Z2 = 8(x 1 + 2x2) + 9(3xl + 5x2) = (8 · l + 9 · 3)x l + (8 · 2 + 9 · 5)x2
= 35xl + 6lx2 .

~~]
This shows that the compo ite is indeed Jjnea r, with matrix
you radio the encoded po iti on .Y = [ to M arseill e. The coding trao for m-
ation i 6 · 1+7·3 6·2+7· 5]-[27 47]

"=A.r. with A = [ ; ;J. [ 8·1+9 · 3 8·2+9 · 5 - 35

b. We can use Fact 1.3.7 to how that the transformation T(i)

61 .

B (Ax)
linear:
In Section 2.1 we left out one detai I: your po ition i rad ioed on to Pari s, a you
wou ld expect in a centrally govemed country like France. Before broadca ting to
Pari . the po iti on y is agai n encoded , using the linear transformarion T(v + ÜJ = ß (A(ü + w)) = B (AÜ + Aw) = B Av) + B(AÜJ ) = T(v ) + T (ÜJ
T(kü) = ß (A(kü)) = B k(A Ü) = k ß Av)) = kT(v )
: = s.v. with B = [ ~ ~] Once we know that T i linear we ca n find it matrix by computing the
this time. and the ailor in Mar eille radio the encoded position to Pari s ( ee z e
vector T (e1) = B (Ae 1) and T 2 ) = ß (Ae2 ); the matrix of T i then
Figure 2). e
[ T 1) T (e2 ], by Fact 2.1.2.

We can think of the mes age = received in Pari s as a function of the actual
po it ion i. of the boat: T (e1) = B (Ae 1) = B(fi r t co lumn of A = [ ~ ~] [~ J = [;~]

DJ= [ :n.
z= B (A.t) ,

z
the compo ite of the two tran fo rmation ji = Ai and = B y. ls this transforma- T(e2) =B Aez) = B( econd column of A) = [ ~ ~]
z
tion = T (i) linear. and , if o, what i its matrix ? We will how two approaches
to these important que tions, one u ing brute force (a), and one using ome
We find that the matrix of the linear tran formation T (-"r ) = B (Ai) i
theory (b).

a . We write the component of the two transformations and substitute.

- ~ = 6y l + 7 y2 Yl = x 1 + 2x2
and
Z2 = 8y1 + 9y2 Y2 = 3x l + 5x2

Figure 2 T hi s re ult agrees with the result in a, of our e.

The matrix of the linear transformation T(i) = B tA.r) is call d the product
of the matrices B and A, written a BA . Thi mean t.hat
Marsei lle: ji

~,whoreA n T(r ) = B(A:r) = ( BA )i

_ = [ ;
ßoat: x / y = A-r
for all vectors x in IR 2 (see Figure 3).
100 • hap. 2 Linear Transformations Sec. 2.4 Ma trix Prod uct • 101

== B(Ax) = (BA)x ,
Mar ·eille: _\-
ji- -in IR"} ll * q
where BA = [ 27 47 ]
35 61
Y IO IA.
rr>q

/
/__,. = Ax. whereA =
[I?]S
3

Figure 3 Boal:i
Figure 6 .r in IR''

Now let's Iook at the product of !arger matrices. Let B be an m x n matrix

and A an 11 x p matrix. These matrice represent linear transformation a hown
in Figure 4. Definition 2.4.1 Matrix multiplication
Again, tbe composite tran formation z = B Ax) is linear (the justification
given above applies in this more general case as weil) . The matrix of the Unear a. Let B be an m x n matrix and A a q x p matrix. The product BA i
z
transformatioo = B(Ax ) i called the product of tbe matrices B and A, written defined if (and only if) n = q .
as BA . Note tbat BA i an m x p matrix (as it represents a linear tran formaöon b. If B i an m x n matrix and A an n x p matrix then tbe product BA i
from P to IR 111 ) . As in the case of ~ 2 , tbe equation defined as the matrix of the linear transformation T(x ) = B(Ax) . Thi
means that T (x) = B(Ax) = ( BA )~'r, for all x
in JRP . The product BA
z= B (Ax) = ( BA )-: is an m x p matrix.

x
holds for all vectors in ~", by definition of the product BA ( ee Figure 5).
In the definition of the matrix product B A, the number of column of B
tches tbe number of rows of A. What happeos if the e two number are Although this definition of matrix muJtiplication does not 0oive us concrete
different? Suppose B is an m x n matrix and A is a q x p matrix , with n # q. 0

~nstruc~ons
0

for computing the product of two numerically given matrices, such

z
In thi case, the tran fonnations = B ji and y = Ax cannot be composed, mstructJOns can be derived ea ily from the definition.
since the codomaio of y = A.r
is different from the dom ai n of = B _- ( ee z As in Definition 2.4. 1 Iet B be an m x n matrix and A an n x p matrix.
Figure 6). To put it more plainl y: The output of y = Ax is not an acceptab.le Let 's tbink about the column of the matrix BA .
input for the transformation z
= B 5. In this case, tbe matrix product BA i.
undefined. (ith column of BA ) = (BA )e;
= B(Ae;)
= B (ith column of A)
Figure 4 Figure S
If we denote the columns of A by ü1, Ü2 • . . . , Üp, we can write

ji in IR" z = B(Ax) = (BA)x ji in IR"

u" BÜp

x in IR'' x in IR"
102 • Chap. 2 Linear Transformations Sec. 2.4 Matrix Product • 103
Let ü,, Ü2, .. . , Üp be Lhe co1umns of A. T hen, by Fact 2.4.2,
Fact 2.4.2 The matrix product, column by column
Let B be an 111 x 11 matrix and A an 11 x p matrix with co1umns ü, u2, . ..
Üp · Then the product. B A is

BA = B Vi V2

The ijth entry of the product BA is the ith com ponent of the vector B -1, which
is the dot produ ct of the ith row of B and ü1 , by Fact 1. 3.8.

To find BA , we can multiply B with the co lumns of A, and combine Fact 2.4.4 The matrix product, entry by entry
the resulting vector .
Let B be an m x n matri x and A an n x p matrix. The ijth e ntry of BA i
the dot product of the ith row of B and the j th co1umn of A:
b,, bl 2 bi n
Thi is exactly how we computed the product .b22
a,,J
b21 b zn a 12 Gij
a 21 a22 a11 a 2p
BA =
b;J b ;z b in l""
a:, , an2 {lllj anp
on page 99, using approach b.
bml bm1 bmn
For practice Iet u multip1y the same matrices in the reverse order. The first

column of AB is U;][~ l the econd i U;] [~ l is the m x p matrix whose ijth entry is
II

b;JGJj + b; zG2j + ···+ b; 11 0 11 j = L b;kakj.

AB = [ 3
1 2
5
J[68 79 J = [22
58
25
66
J. k= l

•1 0
Campare the two previous disp1ays to see that AB =I BA : matrix mu1tipti- EXAMPLE 1 .... __./ /"-... y ../
cation is noncommutative . This should come as no surpri se, because the mat1ix \"rl /f 6 7][1 2 1{2 [6·1 + 7 ·3 6. 2 + 7 . s J=Z[27 47 J
product represents a composite of transformations. Even for functions of one vari- \Ls 9 3 s Jr s . 1 + 9 . 3 8 .2+9 .5 ) 35 61
able, the order in which we compose matters. Refer to the first examp1e in this
We have done these computation before (where ?).
sect ion and note that the function s cos(sin(x)) and sin(cos(x)) are different.

+ Matrix Algebra
Here we discuss ome ru1e of matrix a1gebra.
Fact 2.4.3 Matrix multiplication is noncommutative: AB =!= BA , in general. However
at times it does happen that AB = BA ; then we say that the matrices A and • Consider an invertib1e n n matrix A. By definition of the in er e. A
B commute. mu1tip1ied with its inverse repre ents the identity transformarion.

Fact 2.4.5 For an inve1tible n x n matrix A,

lt is useful to have a formu la for the ijth entry of the product BA of an AA - 1 =In and A - 1 A = I 11 •
m x n matrix B and an n x p matrix A .
104 • Chap . 2 Li n ea r Transformations Sec. 2.4 Ma trix Products • 105
• Compo in g a lin ea r tran formatio n w ith the identity transformation , on e ither the equation by B - 1
from the left:
side. leave tJ1e transformation unchanged.
B-
1
y = B- 1
B Ax = l"A x =Ai
Now we multipl y by A - t from the left:
Fact 2.4.6 For an m x n matrix A,
A - 1B - 1
y= A - 1 Ax = i
A l " = ImA = A.
Thi s computation shows that the linear transformati o n

• lf A is an 111 x n matrix, B an I! x p m atrix, and C a p x q m atrix, w hat is y =BA i

the relalion hip b tween (A B )C and A(BC)? in invertible, and that its inverse i
One ay tothink abo ut this prob le m (perhap not the most e legant) is
to write C in tenns of its col umns: C = [ ü, Ü2 Üq ] . Tben
x=A- 1 B- 1
y.
(A B )C = AB)[ ü, U2 vq ] = f (AB)Üt (A B )~ (AB)Üq ]
Fact 2.4.8 lf A and B are inverti bl e n x n matrices, then B A is in verti bJe as weil, and
and
BA r ' = A - 1 B - 1
•
A(BC = A [ Bii t B ü_ A(Bvq) ].
Pay artenti o n to the order of the matri ces.
Since ( AB - ; = A BÜ;) by definition of the matrix product, we find that
(A B )C = A BC).

[~J s - 1 by B A
1
To verify th.i resu lt, we can multipl y A - (i n eitber order),
Paris:
and check tb at the res ult i /11 :
Fact 2.4.7 Matrix multiplication is associative
(A B )C = A(BC)

We can write impl y ABC for the product (AB ) C = A ( BC).

BA A-l ß- 1 [ ]
Mar eil le: ;.~
Everything worked out!

~
A mo re conceptu al justification of thi s property is that matrix multi- To understand the order of the factors in the formula ( B A) - 1 = A - 1 B- 1
pbcation i associative because compo ition of functions i associative. The better, th.ink about our French coa t g uard story again.
two linear transformation
Boat: [~J Torecover the actua.l position; from the doubly encoded po itioo ::, you firs t
z
apply the decoding transformation y = B-I and then the decoclin g tran fo rmation
T (x) = ((A B )C)x and L(i ) = (A( BC)) ,r
Figure 8 .X = A- 1 y. Theinverse of:: = Bkf. is therefore; = A- t s - 1 ~, a illu trated in
are identical becau. e, by definition of matrix multipli cation , Figure 8.
Tbe following result is often useful in find.ing inverse :
T (.r) = ((A B )C)i = (A B )(Cx) = A (B(Ct))

~~
IRq- Rp ---+- " - ~"'
and
Fact 2.4.9 Let A and B be two n x n matrices such that
L (x) = (A ( BC)) x = A ( (BC)x ) = A ( B (Cx)) .
""--c ~ BA = I" .
(AB)C~ The domain and codomains of the linear transformation defined by the Theh
Figure 7 matrices A, B , C, BC, AB, A ( BC) , and (AB)C are shown in Figure 7 . a. A and B are both invertib le,
• If A and B are invertible n x n matrices, i BA invertible as weil? lf so, b. A- 1 =Band B - 1 = A, and
what is its inverse?
c. AB = I 11 •
To find the inverse of the linear transformation
y= BA x,
It follows from the definüion of an invertible function that if A 8 = In and
we solve the equation for x, in two steps. First, we mu ltiply both sides of BA = 111 , then A and B are inverses, tbat is, A = B -1 and B = A - t. Fact 2.4 .9
106 • hap. ~ Linea r Transformations Sec. 2.4 Matrix Product • 107
make the point th ar the eq uation BA = !" alone guaranr es ~ha~ A and ~ at:e + Partitioned Matrices
inver es. Exerci I- and 79 will help you to under. tand the s1g nd1cance of" tht.
laim bert er. lt _i : o~1etim~ u sef~l to break a !arge matrix down into small er ub matrice by
s lt ~ tn g 1t up wtth h on zo~tal o r vertical lines th at go all the way through th e matrix.
Proof To demoo strate th ar A is in verübl e it uffices to how that the linear sy ·te m =Ö A.r Fot exa mpl e, we can thmk of the 4 x 4 matrix
has on ly the olurion .K = Ö(by Fact 2.3.:.4b)._ lf we n1ll!tipl y tl1e equatio n A ..\- = Öby

l~ ~ ~ ~l
B from the left, \ e find that BA.\-= BO = 0, o .\- = 0. ince BA = !" . There fo re,
A is invertible. If we multipl y the equ ati o n B A= !" by A - 1 fr m the ri g ht, we A=
find th at B = A - 1• T he matrix B. being the in verse of A. i it lf in vertible. and 9 8 7 6
B - 1 = (A - 1) - 1 = A (rhi s fo ll ow from the definition of an invertib le fu nction; see 5 4 3 2
page 6) . Finall y, AB= AA - 1 = !" . • a a 2 x 2 matri x who e "entries · are fou r 2 x 2 matrice :

You can u e Fact 2.4.9 to chec k o ur work w hen you compu te the in ver. e
of a matrix. For example. to check th at

B _ l [ d i the in verse of A = [ ~ !]. A = l: :! :l ~~: ~~~ l

5 4 3 2
=[
ad- bc -c
it suffices to verify that B A = h with A11 = U~ J etc.

BA= - -
1-[ d
ad- bc -c
b] = I [
d ad - bc
ad - bc
ac - ac
bd - bd ]
ad - bc
= 1,
-
T he ubmatrices in such a partition need not be of equa l size; for exarn ple,
we could have

Compare with Fact 2.3.6.

ß =
[
41 2
5 36 ] = [ 4l 52 63 J = [ ~II
Here is another example illustraring Fact 2.4.9. 7 8 9 7 8 9 21

EX.J\MPLE 2 ..... Suppose A. B , and C are three n x n matrice. and ABC= I". Show that B A useful property of partitioned matrices is the following :
invertible, and expre s- 1
in term of A and C .

Solution Fact 2.4.12 Multiplying partitioned matrices

Wrile ABC = (A B )C =
!". Wehave C(AB) = !," by Fact 2.4.9c. Since matJ.i x Partirioned matrice can be multiplied as thou 011h the ubmat.rices were calar ,
multiplication i as ociati e, we can write (CA ) B = !" . Applying Fact 2.4.9 again , i. e., u ing the formul a in Fact 2 .4.4.
we conclude that B i invettible, ancl B- 1 =CA. ~

Fact 2.4.10 Distributive property for matrices

lf A, B are 111 x n matrices, ancl C, D are n x p matrices, the n
A(C + D ) = AC+AD , and
(A + B )C = AC+ BC .

is the partitioned mat.rix whose ij th 'entry" is the matrix

You will be asked to veri fy thi property in Exe rci e 63. II

A ;t B lj + A ;2 B2j + · · · + A;"B j = 11 L Aii:Bkj ·

k= l
Fact 2.4.11 If A is an m x n matri x, B a n n x p matrix, ancl k a scalar, then
·provided that all the product A ;k Bkj are defined .
(kA)B = A(kB) = k(AB).
Verifying thi · fact i left as an exercis . On the ne t page i a nume ri cal
You will be asked to verify this property in Exercise 64. example.
108 • hap . 2 Linear Transfor mat ion s Sec. 2.4 Mat ri x Produc • 109
E:XAMPLE 3 .... We have to solve thi s . ystem for the ubmatrice Bij · By equ atio n l , A 11 mu t
be in.vertibl e, and B 11 = Ali . By equati on 3, B21 = 0 ( multi ply by A-II1 from

[~ ~ I -:JI~
l ; ~ ] 7 8 9
t he nght). Equaüon 4 now . im pli fies to B22 A 22 = 1111 • Th refo re, A22 mu t be

1
1
Ii
invertible, and Bn = A22 . Lastly, equation 2 become A A 12 + B1 2A22 = 0, or
B 12A n -- - A-11 A 12, 01· B 12=- A -11 1A12A - I .
22

~ [[~ m~ ;J+[-:J[7 8] [~ m~J + [ - :J [9] J a. A is in vertib le if (and o nl y if) both A 11 a ncl A 22 are inve1tib le (no condition
is imposed on A 12).

= [ -~ ~~ I ~~ J. b. lf A is in vertible its in verse i

Compute this product without u ing a partition and see whether you find
the same re ult. ~
In thi s simple example, using a partition i somewhat poinrles ; Example 3 Yerify this result for the example below.
merely illu trate Fact 2.4.12. Example 4 below how a mo re sen ibl e applicati on EXAMPLE 5 ....
of the concept of partjti.oned matrice .

-; l
I 2 3 - I 2 -1 2 I
EXAMPLE 4 .... Let A be a partitioned matri x 2 4 s 6 - ] -3 -3
0 0 1 0 0 0 0 1 0
0 0 0 1 0 0 0 0 I
0 0 0 0 0 0 0 0
where A 11 is an n x n matrix, A22 is an m x m matri.x, and A 12 i an n x m matri x.

a . For which choices of A 11, A 12, and An is A invertible? E X E RJ,C{(

b. If Ai invertible, what is A- 1 (in terms of A 11> A 12 , A_2)? GOALS Compute matrix product column by column and e nrry by entry. Inter-
pret matrix multiplicatio n in term of the underly ing Jjnear transformation . e
Solution the rules of matrix al gebra. Multiply pa.rtitioned matrice .

We are looking for an (11 + m) x (n + m) matri.x B . uch that

lf pos ible. compute the matrix pr duct in Exerci e l to 13. u ing pape r and
BA=/" +"'= [/"
O /~,]. penc il.

Let u partition B in the same way a. A: 1.

[~ nu ~J 2. [ -~ -~J [ 7 n 3. [I4 2s 3][1
6
2]
3 4
B = [B 11 B1 2 ]
B21 B22
where 8 11 is n x n, B22 is m x m , etc. The fact that B is the inverse of A means
4.
[~ -n [i 6] 5.
u!] [~ ~] 6. [(/ b] [ -b]
c d
d
-· (I

[~ ~n [! !]
that 0 2
7. 1 2 8. [~ ~ J[~ ~ J
- I

or, using Fact 2.4.12,

BIIAII = / II
9. u;J[-~ -4J 10. [I 0 - II un 11. [J 2
31m
B1rA 12 +

B21A12 +
B12An
B21AII
B22 A22
= 0
= 0
=Im
12. m[J 2 31 13. [ 0 0 11
[ a
d
g
b
e
h
{][!]
110 • Chap. 2 Linear Transformations Sec. 2.4 Matrix Product • 111
7·?1
14. For rhe matri es ~ ]; ,: [ ~ ~ ] ·
A -- [I
I B= [I 2 3 ],
-I] 0 .
I
29. Find a nonzero 2 x 2 matrix B suchthat [;

30. If A is a noninvertible n x n matrix , can you alway

matrix B such that AB = 0?
find a nonzero n x n

31. For the matrix

D ~ [ \ l E ~ [5] ,
find a matrix A such that
B=

]
determinewhichofthe25matri x products AA. A B . AC , ... , E D , E Eare
defined , and compute those which are defined .
15. Compute the matrix product How many so lutions A doe this problern have?

-2
5
-5 ][8~
11
-1]
2 .
- I
32. Can you fi.nd a 3 x 2 matrix A and a 2 x 3 matrix B ucb that the product
AB is h? Hint: Con ider any 3 x 2 matrix A and any 2 x 3 matrix B. There
is a nonzero ,r in ~ 3 such that Bx = Ö. (Why?) Now think about A Bx .
33. Can you find a 3 x 2 matrix A and a 2 x 3 matrix B ucb that the product
Explain why the re ult does not contradi t Fact 2.4.9. AB is invertible? (The hint in Exercise 32 still works.)
34. Consider two n x n m atrice A and B such tbat the prod uct AB is invertible.
For two invertible n x n matrices A and B, detennine which of the formulas tated Show that the matrices A and B are botb invertible. Hint: A B(A B ) - I = !"
in Exerci ' 16 to 25 are nece sarily uue . and (AB) - 1AB= ! " . Use Fact 2.4.9.

16. (/11 - + A) =I"- k . (J"-rl)

A)(l" (I;, 1A).: 35. Consider an m x rz matrix B and an n x m matrix A such that

,. 17. (A + 8 ) 2 = A 2 + 2AB + 8 2 .
18. k is invertible, and (A 2 ) - 1 = (A- 1) 2 .
a. Find all soltitions of tbe linear ystem
~ 19. A +Bis invertible, and A + B )- 1 = A- 1
+ s- 1
20. (A- ß) ( A + ß) = A 2 - 82 . A,r = Ö.
21. A ßß - IA - 1 = 1" .
b. Show that the linear system
22. ABA - 1 = B.
1
23. (ABA - 1) 3 = AB 3 A -
24. Un + A)(l + A- ) = 21" + A +A-I
1

is consistent, for alJ vector b in lR"'.

11

25. A - 18 is invertible, and (A- 1 8) - 1 = ß - 1 A .

U e the given partition to compute the product in Exerci es 26 and 27 . Check
your work by computing the same products without using a partition . Show all
your work.
c. What can you say about rank(A)? What about rank( B)?
d. Explain why m ::=: rt.
36. Find all 2 x 2 matrice X uch that AX = B , where

~- [t ~ ~ n[~ i:n ~ r ~ H:~ 1 Z?. [ 1 1

37. Find all 2 x 2 matrice X which commute with A = [

38. Find all 2 x 2 matrices X which commute with A = [

6 ~] .
6 ; ].
28. Find a nonzero 2 x 2 matrix A such that A 2
= [ ~ ~]. 39. Find tbe 2 x 2 matrice X w hich commute w ith all 2 2 matrice .

~ff;EJ ~~)
112 • Chap. 2 Linear Transformations Sec. 2.4 Matrix Product • 113
40. Consid r tvvo _ 2 rnatri e . A and B. We are told that
A[ ~] = U] and A [;] = Ul These two equation · can be combined to

8 - 1 =[~ ;].m1d (AB)- = [~ ~ J.

1
form the matrix equation A [ ~ ; ] = [ i ~].
Find A . 45. Using the last exercise as a guide, justify the following statement:
41. Consider the matrix Let Ü1, Üz, ... , Ün be some vector in R" such that the matrix
- ina ] .
CO a.
We know that the linear transformation T(x) = D(l .~ i a counterclockwi e
rotation through an angle a . .
a. For two angles, a and ß consider the product D(l Dß and DßDC: . Arg~mg
geometrically de crib the linear transforrnation _v = D(l D px and y =
DpD(l>r. Al:e the two transformations the same?
b. Now compute the products D(l Dß and DßD(l. Do the re ults make sen e
in terms of your answer in part a?
Recall the trigonometric identitie is invertible. Let Ui1, w w in !R"' . Then there
2 , .. . , 11 be arbitrary vector

in (a ± ß) = sin a cos ß ± cos a si n ß i a unique linear Iransformation T: lR" -+ !Rm such that T ( v; ) = w;, for all
cos(a ± ß = co a cos ß -=f sin a sin ß . i = I , .. . , n. Find the matrix A of th.is rransformation in term of S and

42. Consider the lines P and Q in JR2 sketcbed below . Con ider the linear tran -
formation T (x) = ref Q ( ref p (.t)), that is, we first reflect x in P llild then
reflect tbe result in Q.
B = WJ w_ W 11

.\ 46. Find the matrix of the linear transformation T: JR2 -+ JR3 with

a. For tbe vector x given in the figure, sketch T(x). What angle do the
vectors x and T (x) enclose? What is the relationship between the lengths
of x and T(x)?
b. Use your answer in part a to describe the transformation T geometrically (compare with Exercise 45).
as a reflection, rotation, shear, or projection. 47. Find the matrix A of the linear transformation T: JR2 -+ R 2 with
c. Find tbe matrix of T.
43. Find a 2 x 2 matrix A #- fz such tbat A 3 = h
44. Find all linear Iransformations T: JR2 -+ JR2 such that T [; J= [ i] and

T [;] = [ ~]. Hint: We are looking for the 2 x 2 matrices A such that (compare with Exercise 45).
114 • Chap. 2 Linear Transformations Sec. 2.4 \tlatrix Product • 115
48. Con ider the regu lar tetrahedron sketched below, who e cent er is at the origin . 49. Find the matrices of the tran s formation T a nd L defined in Exerci e 48.
50. Consider the matrix

E ~ [ -~ 0I 0
0
]
0 1
- 3-
- [-'] - I
I
and an arbitrary 3 x 3 matri x

[ b
(/

A = : e
h

a . Compute EA . Cornment on the relation hip between A and E A. in tenns

of the technique of elimü1ation we learned in Section 1.2.
b. Con ider the matrix

~]
0
Let T : ~ 3 -7 ~ 3 be the rotation about the axi throu gh the points 0 and P2 !
which tran form P 1 inro P3 . Find the image of the four corners of the 4
0
tetrahedron under thi tran formation .
T and an arbitrary 3 x 3 matrix A. Compute EA. Comment on the relation-
Po -7 hip between A and E A .
P, -7 p3 c. Can you think of a 3 x 3 matrix E such that EA i obtained from A by
p2 -7 wapping the Ja t two row for any 3 x 3 matri x A)?
p3 -7 d. The matrice of the forms introduced in parts a. b, and c are ca lled ele-
menrary: An n x n matrix E i e lementary if it can be obtained from / 11
Let L: ~3 -7 ~ 3 be the reflection in the plane through the poims 0, Po, by performing one of the three elementary row operatio ns on / 11 • Oe cribe
and P3 . Find the images of lhe four corners of the telrahedran under lhi the format of the three type of e lementary matrice .
transformation. 51. Are elementary matrices invertible? If o, i the inver e of an e le mentary
L matrix elementary as weil? Explain the ignificance of your an wer in term
Po -7
of elementary row operation .
P, -7
52. a . Justify the fo ll owing: If A i an m x n matrix , then there are elementary
p2 --7 111 x m matrice E 1 E1, .. . , Ep uch that
p3 --7
Describe the transformations in parts a to c geometrically.
a. T- 1 b. Find uch e lementary matrices E 1, E 2 , ..., Ep for
b. L - 1
c. T 2 = T o T (the composite ofT with it elf)
d. Find the images of the fo ur corners under the transformations T o L and
A= [0I 23 ].
L o T. Are the two transformations the same? 53. a. Justify the fo llowing: If A i an m 11 matrix , then th re i an in ve rtible
T oL LoT 111 x m matrix S uch that
Po --7 Po -7
P, --7 P, --7 rref(A) = SA .
p2 -7 p2 --7
p3 -7 p3 --7
b. F ind such an invertib le matrix S for
e. Find the images of the fo ur corners under the transformatior} L o T o L.
Describe this Iransformation geometricall y.
116 • Chap. 2 Linear Transformati ons Sec. 2.4 Matrix Prod uct • 117
54. a. Justify the following: A ny invertible matrix is a product of e leme ntary 59. Knowing an LU -factoriza tion of a matri x A make it much ea ier t ol e a
matrice . linear sy tem
b. Write A = [ ~ ~] a a product of elementary matrice . Ax = 'b .
55. Write all po sible form of eiern ntary 2 x2 matrices E. Jn each case, describe Cons ider the LU -factori zation
the transformation y = Ex geo meu·ically.

l
-~
56. Consider an invertibl n xmatrix A and an n x n mau'ix 8 . A certa in
11
equence of elementary row operations tran forms A into 1".
a. What do you get when you apply the ame row Operations in the ame
A=
1
- I
-5
2

4
6
- I
6
6
20
;~] Jl-1
43
-~ ! 8
HJ l~~
-5 J 0
2
I
0
0
order to the matrix AB ?
b. What do you get when you apply the same row Operations to 1"? Suppo e we have to o lve tbe ystem Ax = LU ,r = b, where
57. Is the product of two lower triangular matrices a lower triangular matrix a
weil? Explain your an wer.
58. Con ider the rnatrix

a. Set ji = Ux, and solve the ystem L ji = b, by forward ubstitution (fi nding
first y ,, then Y2 etc .) Do thi using paper and pencil. Show all your work.
b. Solve the system !j ."K = ji, by back substitution , to find the olution of x
a. Find lower triangular elementary matrices E ,, E2, .... Em s uch that the the y tem Ax = b. Do thi usjng paper and pencil. Show all your work.
product

is an upper triangular matrix U. Hinr: Modify Gau -Jordan eljmination a

follows : In Step 3, elimjnate only the entries in the cur or co.lumn below
tbe Jeading l ' . Thi can be accomplished by mean of lower triangular
/'~
x- -- ---------.... E
elementary matrice . Al o, you can skip Step 2 (divi ion by tbe c ur or A
entry).
b. Find lower triangular elementary matrices M 1, M2 , .•• , Mm and an upper
triangular matrix U such tbat 60. Show that the matrix A = [ ~ b] cannot be written in the form A = LU ,
where L i lower triangular and U i upper triangular.
61. ln this exerci e we wil l examine which invertible n x n matrice A admit
c. Find a lower triangular matrix L and an upper triangular matrix U such an LU -factorization A = LU , a discu ed in Exercise 58. The following
that definition will be u eful: Form = I , ... , n the principal submarrix A (m) of A
is obtained by omitting all row and columns of A past the mth. For example,
A =LU. the matrix

~]
Such a representation of an invertible matrix is called an LU -factori zation. 2
The method outlined in thi s exercise to find an LU -factorization can A = [; 5
be streamJined somewhat, but we bave seen the major ideas. An LU - 8
factorization (as introduced here) does not always exür (see Exercise 60) .
ha the pr.incipal submatrices

n
d . Find a lower triangular matrix L with 1 's on the diagonal, an upper tri -
angular matrix U with I 's on the diagonal, and a diagonal matrix D such
that A = LDV. Such a representation of an invertib le matrix is called an
L DU -facrorization.
A<'> = (1], A<2l _ [ 1
- 4 ;J. A'" = A = [;
2
5
8
118 • Chap. 2 Linear Transformation
Sec. 2.4 Matrix Products • 119
We , ill show that an invertibl 11 x 11 matri x A adm.its an LU -factorization
A = LU if (and only if) all it · principal submatrice are invertible. where B and R are n x 11 matrice R is invertible), k i a calar and ii i a
row vector with n components. Compute s- 1 AS.
a. Let A = LU b an LU -factorization of an n x 11 matrix A. Use partitioned
matrice to show that A (m) = L (m l u <ml for m = .I , ... , n. 69. Consider the partitioned matri x
b. U e part a to how that. if an invertible n n matri~ A l~ as an LU-
factorization then all its principal ubmatrices A (m ) are mvert1ble.
c. Consider an 11 x 11 matrix A whose principal submatrices are alt invertible.
Show that A admit an LU -factorizat.ion. Him: By induction you can where A 11 is an invertible matrix. Derermine the rank of A in te rm of the
a ume that A<n- ll ha an LU-factorization A <n- l ) = L' U'. Use partitioned ranks ofthe matrices A 11 , A 12, A 13 , and A23 .
matrices to find an LU -factorization for A. Alternatively, you can explain 70. Consider the partitioned matri x
tlü result in terms ofGauss-Jordan elirn ination (if al.l p rincipal ubmatrices
are invertible, then no row swap are required).
62. a. Show that if an invertible n x 11 matrix A adrn.it an LU -factorization , then
it adm.its an LDU-facrorization (see Exercise 58d).
where ii is a vector in IR", and w is a row vector with n compone nt . For
b. Show that if an invertible 11 xn matrix A adrn.its an LDU -factorization, then
this factorization is un.ique. Hint: Suppose that A = LtDtU t = L2D2U2 .
which choices of ii and w i A invertible? In these ca e , what is A- I?
71. Find al t invertib.le n x 11 matrices A such that A 2 = A.
Then U2U1- 1 = D2 1L:i 1L 1D 1 is diagonal (why ?) . Conclude that U2 = Ut.
63. Prove the disTributive laws for matrices: 72. Find an n x 11 matri x A whose entries are identical such that A2 = A .
73. Consider two 11 x n matrices A and B wbose entrie are positive or zero.
A(C + D) = AC+ AD Suppo e that all entrie of A are less than or equal to s, and all colurnn ums
of B are les than or equal to r (tbe jth column sum of a matrix i the sum
and
of the entrie in its jth col umn ). Show that aU entrie of the matri x AB are
A + B )C = AC + BC. le th an or equal to sr .
74. (This exercise builds on Exercise 73 .) Consider an n x n matrix A who e
64. Consider an m x n matrix A, an 11 x p matrix B, and a scalar k. Show that
e ntrie are positive or zero. Suppose tbat all column um of A are !es than
(kA)B = A(kB) = k AB) 1. Let r be the largest column sum of A.
a. Show that the entrie of A" are less than or equal to r 11 , fo r alt positi e
65. Consider a partitioned matrix integers n.
b. Show that
A = [ A~I A~2 ] lim A" =0
n->
where A 11 and A22 are square matrice . For which choices of A 11 and A22 is
A invertible? In these ca es, what is A- I? (meaning that all entrie of A 11 approach zero).
c. Show that the infinüe series
66. Consider a partitioned matrix
+ A + A 2 + .. . + AII + ...
A=[Atl
A21 A22
0] /II

converges (entry by entry) .

d. Compute the product
where A 11 and A 22 are square matrices. For which choices of A t, , A21, and
A22 is A invertible? In these cases, what is A _ , ?
(!" - A) (!" + A + A 2 + ... + A II

67. Consider two matrices A and B whose product AB is defined. Describe the
ith row of the product AB in terms of the rows of A and the matrix B. Simplify the result. Then Iet n go to infinity, and tim show that
68. Consider the partitioned matrices 1
Un - A) - = In+ A + A 2 + · · · + A" + · · ·.
A= [ ~ ~] and S= [ b ~ ]. 75. (Thi s exercise builds on Exercise
c i e 2.3.49 .)
73 and 74 above, a weil a Exer-
120 • Chap. 2 Linea r Transformation Sec. 2.4 Matrix Products • 121
a. onsider tbe in du tri 1 1 • • • •• J" in an economy . We say that industry
c. Find the matrix for the composite Iransformation which li ght undergoe as
J. i produclive if th e jth co lumn sum of th, technology matri x A is Ies
it fir t passe through the sung lasses and then the eye.
tl~a n 1. What does thi mean in renn of economjcs? d. As yo u put on the unglas es, the sign al you recei ve (inte n ity, long- and
b. We say that an economy i produclive if all of its indu tri es are productive. short-wave s ignal s) undergoes a transformation . Find the matrix M of th i
Exercise 74 show that if A is the technology rnatrix of a produclive transformation.
econom then tl1e matrix !" - A is invertible. What doe this result
tell you about the ability of a prod uctive economy to sati sfy any kind of
co nsumer demand '.
c. Interpret rhe formula J(- i;
(!" - A) - I = !" + A + A 2 + ... + A + ....
II
M

derived in Exerci e 74d in tenn of econom.ic .

76. The color of light can be repre ented in a vector

[~]
Li ght passcs through glasses and
thc n through eyes.

77. A vii Jage i divided into three mutu ally exclu ive group called clans. Each
where R = amount of red , G = amount of green, and B = amount of blue. per. on in the village belang to a c lan, and thl identification i permane nt.
The human eye and the brain transform the incoming signal ioto the signa1 There arerigid rules conceming marriage: A per on from one clan can marry
only a person from one other clan . These rules are encoded in tbe matrix A
below. The fact that the 2-3 entry i I indicate that marriage between a man
·from c lan lll and a woman from clan Il i allowed. The clan of a chlJd i
detennined by the mother' clan, a indicated by the matrix B . According to
tbi scheme ibling belang to the ame clan.
wbere
Husband ' clan Mother· clan
R+G + B I Ilill I Ilffi
intensity I = - ---"-
3 --

[~ I 0 ~
B~u T
Wife 0 Child'
long-wave ignal L= R- G
R+G
A= o(D li clan 0 1 ll clan
shon-wave igoal S =B - - - 0 0 m I 0 JlJ
2
The identification of a person with clan I can be repre ented by the vector
a. Find the matrix P representing the transfonnation from

b. Consider a pair of yellow sunglasses for water sports which cuts out all
blue light and passes all red and green light. Find the matrix A whlch
represents the Iransformation incoming light undergoes as it passes through
the sung lasses.
and likewi e for the two other clans. Matrix A tran form the husband' clan
into the wife '. clan (if x represents the husband ' clan, then Ax represent
the wife's clan).
a. Are the matrices A and B invertibl ? Findtheinver e if they exi t. What
do your answers mean , in practical term ?
b. What i the meaning of 8 2 , in term of the rule of the community.
122 • Chap. 2 Linear Tran sformation
Sec. 2.4 Ma tri x Product • 123
c. V hat i, the rn e·~n i ng of AB and BA, in tenn of rhe rul e of the c mmu -
nity? Are AB and BA the ":une?
d. Bueya i" a _ ung ' oman ' ho ha ' many ma le firs r c u in , bolh on her
to find AB for A = [ ~ ~ J and B = [ ~ ~ J. Fir t c mpute
mother· and on her father' side . The kin hip betw en Bueya and each
h 1 = (a + d)(p + s)
of her ma le cousi n ca n be represented by one of the fou r diag rarns below:
h2 = (c+d)p
lt 3 = a(q - s)
lt 4 = d(r-p)
hs = a + b).
h6=(c - a)(p+q)
h 1 = (b- d) (r + s).
Then

!
B"'Y' ~ 0 A mo lo ""' ""''"
79. Let N be the er of allposit ive integers, I. 2, 3, .... We define two funcli n
.f and g from N to N:
f(x) = 2x , fo r all x in ·

( ) x /2 if .r is eve n
g X = { X+ 1)/2 jf X iS Odd
Find formula for the co mpo ite function g(f(x)) and f(g(x ). I one of
/ / / them Lhe identity tran fom1ation from · to N? Are the fun ction f and g
invertible?

S2 0 I
0 S2 I
00 l
80. Ceometrica / optics. Consider a thin bi-convex lens with two pheri cal face .

~ 0 ~ 0 ~ 0
In each of the four cases, find the matri x which g1ve yo u the cou in 's
cl an in terms of Bueya'. clan. This is a good model for th e lens of the human eye and for the len e u ed
e. Accorcling to the rules of the vi ll age, cou ld Bueya marry a firs t cou. in? in many optica l in trument , uch a readi ng gla es, amera . micro cope ,
(We do not know Bueya·s clan.) and tele copes. The line through the centers of the sp here defining the t-; o
78. As backgrou nd to this exercise, . ee Exerci . e 2.3.45. faces i called the optical axis of the Jen . .
a. If you use Fact 2.4.4, how many mu ltiplications of numbers are necessa ry
to multipl y two 2 x 2 matri ce ?
b. [f you use Fact 2.4.4, how man y multiplication · are needed to multiply an ,'

m x n and an n x p matrix?
In 1969, lhe German mathematician Volker Strassen surpri ed the mathemat-
ical commun ity by showing that two 2 x 2 matrice can be muitipliecl with
enter of sphere ,'
onl y seven multiplications of numbers. Here is hi s trick: Suppose you have """ Cemcr of 'flhcre
dcfin ing the righ1 fa e defimng thc left facc
124 • Chap. 2 Linear Transformation
In thi exerci e. we Jearn how we can track the path of a ray of light as it
pa se through the lens, provided that the fo ll owing conditions are satis fi ed :
• The ray hes in a plane with the opticaJ axis.
• The ang le tJ1e ray makes ' ith th optica l axi s ma ll.
Sec. 2.4 Matrix Products • 125
We wanl to know how the o utgoing ray depe nds on the incomin g ray , that i ,
we are interested in the tran Formati o n
We wil l see that T can be approx.irn ated by a linear Irans form ati on provi ded
lhat m is s mall , a we assumed . To tudy thjs transformatio n, we di ide the
palh of the ray into three seg rnents, as s hown below .

To keep track of the ray, we introduce two reference planes perpendicular to

n IIT
the optical axis, to lhe left and to tbe right of the Jen .

We have introduced two auxi li ary reference plane , directly to the le ft and to
t he right of the lens. Our tran Formation [
m
x ]
n
~ [ Y] can now be represented
a the compo ite of three simpler Iransformation :

From the definition of the slope of a line we get the relation u= x + Lm

and y = w + Rn .

Left refercnce Ri ght rcference

plane plane

II
We can characterize the incomi ng ray by its . lope m and its intercept x with
the left reference plane. Likewise, we characterize the outgoing ray by slope
n and i ntercept y .

[ mu ] = [ x + Lm
m
] = [ I L] [ x ] ;
01 m
[ y ] = [ I R] [ w]
n 01 n

[~J [6 ~tl [:.J -. [ ~J [6 nL:J

126 • Chap. 2 Linear Transformation Sec. 2.4 Matrix Products • 12 7
It would Iead u too far into phy ic ro derive a fonnula for the transformati n
you want y to be independent of x (y must depend on the slope m alone) .
Explain why l jk is called thefocallength of the len .
b. What value of k enables you to read thi s text from a di tance of L
0.3 meters? Consider the figure below (whi ch i not to scale).
here. * Under the a umption we have made, lhe tran Formation 1• weil
approximated by

for some po itive con tant k (thi formula implies that w = v).

The transformation [ x ]
111
~ [ 1lY] is represented by the marrix product
c. The telescope. An astronornical telescope consists of two lense with the
[l R][ I 0][1 L]-[1 - Rk L+R-kLR ]
0 1 -k 1 0 I - -k I - kL .
same optical ax.is.

a. Focusing parallel rays. Consider the Jens in the human eye, wi th the
retina a the right reference plane. In an adu lt, the di stance R i about
t.,:'
-'
0.025 meters (about l inch). Tbe ciuary muscles allow you to vary the r ; ; - - - --- :
shape of tbe Jensand thu tbe Jen constant k , within a certain range. What
value of k enables you to fo us paralle l incoming ray , as shown in t:he
figure ? This alue of k will all ow you to . ee a distant object clearly. (The
cu tomary unit of measurement for k is 1 di opter = 1 m1 ter .
D

Left reference Right reference

plane plane

y
Find the matrix of the tran Formation [ x ] ~ [ Y],
in term of 1• 2 , k k
m n
and D . For given values of k 1 and k_, how do you choo e D o that
parallel incorning rays are converted into parallel outgoing ray ? What is
the relation hip betweeo D and the focal lenglh of the two len e , I jk 1
and l/k2?
81. Tru e orfalse? l f A i an 111 x 11 matrix and B i an 11 x p matrix , then
rref(A B ) = rref(A) rref(B) .
Hint: In terrns of the transformation

*See, for exampl c, Paul Samberg and Shlorno Sternberg: A Course in Matlr ematics .for Student.~ c~(
Physics 1. Cambridge University Press. I 991.
 Sec. 3.1 Image and Kernet of a Linear Transformation • 129
f
X: the domain of f Y: Lhe codomai n off

image /)

SUBSPACES OF
~n AND THEIR Figure 1

DIMENSIONS EXAMPLE 2 .... The image of the exponential function

f: ~ ~ ~ g i ven by y = ex
consi ts of a ll positive number .

EXAMPLE 3 .... More generall y, the image of a fu nction

1 IMAGE AND KERNEL OF A
LINEAR TRANSFORMATION f: ~ lll
consi st. of all numbers b uch that the line y = b intersects the grapb of f
You may be farniliar w ith the notion of the image of a function (po ibl y more than once), as illustrated in Fi g ure 2. <1111

f:X ~ Y, EXAMPLE 4 .... The im age of tbe fun ction

f: ~ ~ ~- c~~(:~ ]
where X and Y are arbitrary e t .
given by f( t ) = [

the unit circle ( ee Figure 3).

Definition 3.1.1 Image
The function f i · calJed a parmnetri-ation of the urrit citcle. More generaJi y
Consider a function f from X to Y. Then a parametri zation of a c urve C in ~ 2 i a fu nction g from lll to R 2 wbo e image
is C. ,....
image Cf) = {values the fun ction f takes in Y)
= (f x):x in X} EXAMPLE 5 .... U the function f: X~ Y is invertible, th en image(f) = Y . For eacb y in Y tbe re
= (y in Y: y = f(x) , for some x in X}. 1 one (and o nl y one) x in X such that y = f(x), namely, x = f - 1 (y). ~

Note that image(f) is a subset of the~aj Y of f.

Figure 2 b is in the image of f, )'

\ since b = f( xtl = f! x2J =

tC\.Y')' f(xJl = f( x4 ); c is not in the
EXAMPLE 1 .... Consider the function in Figure I, whose domain and codomain are finite set . ~ image of f because there is
no x such that c = f(x).

c ___J I
' '
______ _ _ J I _ _ ____ __ J I __ ____ _____ _ _____ J I _ _ _
1
Some authors use the Lerm " ran ge" for what we call the i mage, w hi le othcrs use the term " range"
' ' '
for what we call the codomain. Make sure Lo check which detinition is uscd when you cncounter thc tcrm L-~----~·------~
· ------------~
·-.x
·•range" in a text.
128
130 • hap . 3 Subspaces of ~~~ and Their Dimen ions Sec. 3.1 Im age an d Kemel of a Linear Tra nsfo rmation • 131

---- _1\r) = [ cos(r)

111 (1)
J

Figure 5 im(7)

Figure 3
Since the vectors [ ~] and [ ~] are paral le l, the image of T tbe line of al t

EXAMPLE 6 ~ Con ider the linear transfonnation T from ~3 to ~ 3 which projects a vector or- scalar multiples of the vector [ ; J,as illu tr ated in Fi gure 5.
e -e
thogonally into the 1 2 plane, a illu trated in Figure 4.
e -e
The image of T i the 1 2 plane in ~ 3 . <111111
EXAMPLE 8 ~ Describe the image of the linear tra nsformati on T: R2 -+ JR3 given by the matrix
EXA..iVIPLE 7 ~ Describe the im age of the Jj near n·an formation T : R 2 -+ ~ 2 given by the matri x

A = u~l Solution
Solution
The image T con i ts of alJ vector of the form
The imageofT consist of ali value ofT , i.e., all vectors of the form

T [XI]= A [.r1]
X2 .\ z
= [ 21 63][XL
X2
]= Xl [ J] +
2
Xz [3]
6
T [ ;; l ~ [: ~ J[;; l ~
that is all li near combinations of the vec tors
X, mm + ,,

UJ and Ul
Th is is the plane '· pan ned" by the two vecto rs. in an intui tive, geomerric en e
Figure 4
(see Figure 6). <111111

The ob ervatio ns made in Exam ple 7 an d 8 motivate the following defi ni-
ü ti on.

Defin ition 3.1.2 Span

Co n ider the vec tors ·ü1 • ü_ . .. . ü" in R 111 • T he s t of a ll linear combin ation
of the vectors ü1, ü2 , • • • , ü" is called their span .
span (Ü 1.... . Ü") = {c1Ü1+···+c"ü": ;ar bitra.ry ca lar }
T(üJ
132 • hap. 3 Su bspaces of R" and Their Dirnen ions ec. 3. 1 Im age a nd Kerne! of a Linear Transformatio n • 133
• lf ii is in im(T ) and k i an arbitrary ca lar, then k- is in the image as we il
(we say the image is closed under cal ar multiplication). Again, Iet' verify

[il
thi s: The vector ii can be written as ü = T (w) for ome w in " . T hen
kü = kT (w) = T (kw), o th at kii is in the image.

Let' s summarize:

0
Fact 3.1.4 Properties of the image
The image of a linear transformation T (from IR" to !Rm has the folJowing
Figure 6 properties:
a. The zero vector Ö in JRm i contai ned in im(T).
b. The image is closed under addition: lf ii 1 and ii 2 are both in im(T), then
Fact 3.1.3 For a linear transfonnation
so is üi + Ü2.
T: IR" --+ lR111 given by T (x) = Ax c. The image i closed under calar multiplication: If a vector ii is in im (T)
tbe imageofT is the span of the colu mn of A.I We denote the image ofT and k is an arbitrary scalar, then k v i in the image as weil.
by im (T) or im (A).

It follow s from prope11ies b and c that the image is closed under linear
To justify this fact, we write the transformation T 111 vector form a · In
combinations : If some vectors iii , ~ •. .. , Üp are in the image, and c" c2, ... , cp
Examples 7 and 8: are arbitrary scalars, then c 1 iii + c2 ii 2 + · · · + cP iiP i in the image as weil.
If the codomain of the transformation is IR 3 (that is, m = 3), we can interpret
thi property geometrically, as shown in Figures 7 and 8.
X? ]
XI
T (x ) =Ax= vi Vn :- = XI VI + X2Ü2 + · · · + Xn Vn
[ v
Figure 7 lf is in the imoge, then L
.X n so ore oll vectors on the line L in JR3
sponned by v. k ii

This shows that the image of T consi ts of all linear combinations of the vectors
ii;, that is, the span of iii , ii 2, . . . , iiw
1
Figure 8 lf v1 ond v2 ore in the
/1i Consider a linear transformation T from IR" to lR111 gi ven by T (x) = Ax, for
imoge, then so ore oll vectors in the
. ome m x n matrix A. The imageofT has some remarkable properties: plane Ein IR 3 sponned by VI ond v2• - - - - --- - -,- E
• The zero vector Ö in !Rm is contai ned in im(T), since Ö= AÖ = T(Ö). ''
'

• lf ii i and ii 2 are in im(T), then so is iii + ii2 (we say the image is closed
under addüion). Let's veri fy thi s: Since ÜI and ii 2 are in im(T), there are ''
v
vectors WI and w2 in lR." ·uch that iii = T (w i) and 2 = T(iil 2 ). Then
v
ii i + ii2 = T(wi ) + T(w2) = T(w i + w2), so that iii + 2 is in the image as
weil.

The imageofT is also called the column ~pace of A.

1
134 • Ch ap. 3 Subspaces of ~~~ and Their Di mensions / Sec. 3.1 Image and Kern et o f a Lin ea r Transfo rmatio n • 135
EXAMPLE 9 .... Co n ider an n x n matrix A. S how that im(A 2 ) i contained in im (A) , i.e., each
vector in im(A 2 ) i al o in im(A).

Solu tion T

Consider a vecto r w
in im(A 2 ) . B y definitjon oF the image, we ca n write ÜJ =
A 2 ü = AA ü, for ome vector ü in IR" . We h ave to how th at ÜJ can be re pre e nted
as w = A ü, For ome ü in IR". Let ü = A ü. Tben

w= AA Ü = Aii .

Figure 9

+ The Kernel of a Linear Transfo nnation

When you tudy functi ons y = f x of o ne variable, you are offen interested EXAMPLE 10 .... Consider again the o rthogonal proj ecti on o nto the e -e
1 2 plane, a linear tran For-
in the eros oF this function , i.e . the solu tions of tbe equ ation f(x) = 0. For mation T fro m IR 3 to IR 3 , a show n in F igure I 0.
example, tbe function ) = in (x ) has in fi nitely man y zero name ly, a ll in- T he kernet of T con ist of all vector who e orthogonal projectio n onto
teger mul tiples of n . The zeros of a linear transformation are of intere t as the 1 e -e2 e3
pl ane is Ö. The e are the vectors o n the -axi (the calar multi pl e
weil. of eJ). ~

Definition 3.1.5 Kemel EXAMPLE 11 .... Fi nd the kerne t of the linear tran Formati on T fro m IR to IR 2 given by
111 1
Consider a linear tran formation T (."i) = Ai from IR" to IR • The kemel of T
i tbe set of all zeros of the transformation, i.e. , the olution of the equati o n - [I
T (x ) =
1 2l J1 J-
X.
T (i) = Ö (see Figure 9).
In otber words: The kerne t of T is the soluti on et of the linear syste m
Solution
Ai = 0. We have to olve the equati on

We denote tbe kerne! of T by ker(T) or ker (A) .

T(i) = [ ~ 1
3
J- 0,-
X =

Figure I 0
For a linear transformation

T: lR" --+ IR"' , jj

• im (T ) is a subset of the codomain IRm of T , and

• ker(T) is a subset of the domain IR" ofT .

1 T(Ü)
The kemel of T is also called thc null space of A .
136 • Chap. 3 Subspaces of R" and Their Dimensions
Sec. 3.1 Image and Kernel of a Linear Transformation • 13 7
i.e., we ha e to ol e a linear sy tem . Since we ha e stud ied thi prob le rn carefully
The kerne! of T consists of the solutions of the system
in Section 1.2, we only sketch the solution here.
x1 - 6x3 + 6xs = 0
.[I
rref 1 1] _ [ I
1 2 3 - 0
0
I
- 1]
2
x2 + 2x3 - 2x 5 = 0
x 4 + 2xs = 0

l[ l
The so lutions are the vectors

1
X2
x -2s
6s - +6t21
X= X3 = S ,

[ X4 -21
X5 I

where s and 1 are arbitrary constants.

In the solution above. t i an arbitrary constant. The kerne! i the line panned by

the vector [ -~] m~ 3

Con ider a linear transformation T(x) = Ai from IR" to IR"', where n exceeds
~ ker(T ) ~ I 6s- 6t
-2s + 21
s
-2t
' · t ru-bittary scalru-s I
m (a in Example 11). There will be nonleading vmiable for the equation T (i) = We can write
A .~ = Ö, that is, this sy. tem ha infinitely many so lutions. Therefore, the kernel of
T con ist of infinitely many vectors. Thi s agree with our intuition : we expect 6s - 61 6 -6
some coll apsing to take place a we transform IR" into IR 111 • -2s + 2t -2 2
s = 1 +t 0
-21 0 -2
EXAMPLE 12 .... Find the kerne! of the linear Iransformation T : IR5 ~ IR4 given by the following 0
matrix:
This shows that ker T ) is spanned by the vector
5 4 3

[
-~
6
6 6 6
-2
7 8 10
1 and
6 6 7
0 -2
0 I

Solution
We have to solve the equation T (x) = Ai= Ö.

rref(AJ = [ ~
0 0
! -6
2
0
0
Consider a linear transformatlon T fro m IR" to IR"' given by T (.r) = A,r, for
some m x 11 matrix A .
138 • Chap . 3 Subspace of IR" a nd Th eir Dime nsions
Sec. 3.] Im age and Kerne t of a Lin ear Transfo rmat io n • 139
Defin ition o f
T he kerne ! of T ha the follow ing propertie. : in ver1ibi lity
Fact 3. 1.7 (2.3. 1 and 2.3.2)
(vi) ~ ,... (i) ~ ,... (ii )

Fact 3.1.6 Properties of the kernel

a . The zero vecto r 0 in lR" is contain d in k r(T).
Fact 2.3.3
b. The kerne ! i clo-ed unde r ums.
c. T he kern e! is c losed unde r scalar multipl .

(iii )

C o mpare the e pro perti of the kerne t w ith the corre po nd ing propertie of
the image Ii ted in F act 3. 1.4.
The veiifi cation of Fac t 3. 1.6 is Straig htforward and left as Exercise 49. Defi ni tion ofrank ( 1.3.2)
Co n ider a quare matri x A of ize n x 11 . we j u t ob erved , the kern
of A co ntain s tbe zero vecto r in iR.". Tt is pos ib le rhat the kern e t co n i ts of the
zero ecror_alone . fo r exam pl e. if A = ~· · More ge nerall y , if i in _ertibl . the n (iv
ke r(A) = {0) , since the y tem Ai = 0 ha o nl y the o lutio n .x = 0 . However,
if A i not invertible, the n the _Y tem A.t = 0 ha in fi nite ly m any o lution (by
Fact 2.3.4b), o that ker A -+ {0}. Examples 3b a nd 13
of Section 1.3 '

Fact 3.1.7 C o n ider a square matri x A. The n

( v)
ker(A) = {0} Figure II
if (and o nly if) A i in vertibl e .
EX ERCI SES

We co nclude thi s . ecti o n with a summary that re lates ma ny co ncep t we GOALS U e the concept of the image and the kerne! of a linear tran sformati o n
have introduced thus far. L a te r, we w ill ex pand thi s sum mary . (or a matrix). Express the image and the kerne! of any matiix as the pan o f some
ector . Use kemel and image to de termine whether a matri x is in vertible.

Summary 3.1.8 For each m atri x A in Exerc ises I to 13, fipd vector that pan the kerne ! of A.
Le t A be an 11 x 11 m atii x . The fo ll ow in g tate me nts are equi val e nt (i.e ., they
Use paper and pencil.
are either all true or al l fa l e):
i. A i in vertibl e.
ii. The linear syste m Ax = b ha a unique so lutio n ,r, fo r all b in lR" .
l. A=U ~] 2. A = [~ ~] 3. A = [~ ~]

A~
n ~ :]
iii. rref(A) = !" . 1
4. A = [ 1 2 3] 5. [ : 2 6. A [:
iv. ra nk (A) = n .
3
v. im (A) = IR" .
vi. ke r(A) = {0} .

1
Note that b is in Lhe image of A if and only if the · yste m Ax = b is consis te nt (by defin it io n of the
imagc) .
Figure 11 briefly reca lls the justificatio n for these equiv ale nce :
140 • Chap. 3 Subspaces of IR" and Thei r Dirn nsion
Sec. 3.1 Image and Kerne! of a Linear Tranformation • 141

7. A ~ [ l 2
3
2 n 8. A =[~
I
2 ~] 9. A = : [' ~] 27. Give an exampl e of a noninvertible fun cti o n

f: lR ~ IR

['
0 2

~ [~ n
2 3 w ith im (.f) = IR.
h~ -!l
1 - 3
1 2 11.
lo.A 4 -6 28. Give an exampl e of a parametrizatio n of the ellipse
0 0 3
-1
I 2 0 0 3 0
? l
r+ - = 1
- I -I I 4
0 0 1 0 2 0
[ - 1I I 0 -2 in IR2
= ee Example 4).

il
12. 13. A 0 0 0 I I. 0 (
A = ~ -1 -2 0
0 0 0 0 0 0 29. Gi ve an example of a function wbo e image i the unit sphere
-2 - 1 3
0 0 0 0 0 0
x2 + / + z2 = I
Foreach matrix A in Exerc ises 14 to 16, find vector that pan the im age of A. in 1R 3 .
Give a few vecror as po ible. U e paper and penc il.
30. Give an example of a matri x A such that im (A) is panned by the vector [ ~] .

15. A = [ ~ 1
2
I
3 4
I]
HJ vector m~'.
31. Gi ve an example of a matrix A

in
uch th at im (A) i the pl ane w ith norma l

32. Give an example of a linear transformation whose image i tbe line panned
For each matrix A in Exercises 17 to 22, describe the image of the tran form ation by
T (x) = A.r geometricall y a a Jjne, pl ane, etc. in 1R2 or JR 3 ).

17. A = u ~] 18. A = u 1~ ] in JR 3 .
~[:
1
19. A = [ -~ -4 -6
2 3
_:] 20. A
]
1
1 l] 33. Give an example of a linear transformation w hose kerne! is the plane
x + 2 y + 3z = 0 in JR 3 .

21. A ~ [! i] 7
9
6
22. h [~
1
4
5 ~]
34. Give an example of a linear transforn1ation whose kerne! i the line panned
by

Describe the images and kernels of the transformations in Exercises 23 to 25

geometricall y.
in JR 3 .
23. Reftection in the line y = x/3 in JR2 . 35. Consider a nonzero vector ü in JR 3 . Arguing geometrically, de cribe the image
24. Orthogonal projection onto the plane x + 2y + 3z = 0 in !R 3 . and the kerne! of the linear transformation
25. Rotation through an angle of rr /4 in the counterclockwise direction (in 1R2 ). T: JR 3 ~ lR g iven by T (i) = ü . .r.
26. What is the image of a function
36. Consider a nonzero vector ü in JR3 . Arguing geometrica lly, de cribe the image
f: JR ~ JR givenby f(t) = t 3 + at 2 +br +c, . and the kerne! of the linear tran formation
where a , b, c are arbitrary scalars? T: JR 3 ""7 JR 3 g iven by T (.r ) = ü x .r.
142 • Cha p. 3 Subspaces of IR" and Th eir Dimensions Sec. 3.1 Image and Kerne! of a Li n ear Transformation • 143
37. For the matrix For whi ch vectors ji is thi s system cons i tent? The answer allow you to
expre s im (A) as the kerne! of a 2 x 4 matrix B .
43. Using your work in Exerci se 42 as a guide, explain how yo u can write tbe
image of any rnat1i x A as the kerne! of some matrix B.
44. Consider a matrix A, and Iet B = rref A .
de cribe th im ages and kern I of the m atrice A, A
2
,
3
and A g o metrica ll y. a . Is ker(A) nece sarily equal to ker(B )? Explain.
b. Is im(A) necessarily equal to im (B)? Expiain.
38. Con ider a quare matrix A.
a. What i the relation hip bet• een ker(A and ker(A- ? More gene rally 45. Consider an m x n matrix A with rank(A) = r < n. Explain how you can
4
what can o u say about ker(A), ker(A 2 ), ker(A 3 ), ker(A ) , .. .. write ker(A) as the span of n. - r vectors.
2 3
b. What can you say about im(A , im (A ), im(A ) •.. . ? ..-
~ · Con ider a 3 x 4 matrix A in red uced row echelon fo rm . What can yo u say
39. Con ider an m x 17 m atrix A and an 11 x p matrix B. about the image of A? Describe all case in tem1 of rank(A), and draw a
a . What i the relationship between ker AB) and ker(B)? Are they always sketch for each.
equal? I o ne of them aJways contaioed in the other?
b. W hat is the relarion hip b rween im (A and im (A B)? 47. Let T be the projection along a line L 1 onto a line L 2 (see Exerci e 2.2.33 ).
Describe geometrically the image and the kerne) ofT .
40. Con ider an 111 x 17 matrix A and an 11 x p matri x 8. lf ker(A = im (B , w hat
can you ay about the product AB? 48. Consider a 2 x 2 matrix A with A2 = A .
. . [ 0.36 0.48 ]. a. If w is in the image of A, what is the relationship between w and A w?
41. Con 1der the mamx A = 0.4 _ b. What can you say about A if rank (A) = 2? What if rank(A) = 0?
8 0 64
a. Descri be ker(A) and im(A ) geometri call y. c. If rank(A) = 1, show tbat the linear transformation T (x) = Ax is the
b. Find A2 . If v is in the image of A w hat can you say about Av? projection onto im(A) aJong ker(A ) (see Exercise 2.2.33) .
c. Based on your answers in parts a and b, describe the transformation T ,'t) =
49. Verify that the kerne! of a linear transformation is closed under addition and
Ax geomerricaJly.
scalar multiplication (Fact 3 .1.6).
42. Express the image of the matrix
SO. Consider a square matrix A with ker(A 2 ) = ker(A 3 ). Is ker(A 3 ) = ker(A 4 )?
Justify your answer.

~ l1 ~]
1
,.,
2 .)
51. Consider an m _x n matrix A and an n x p matrix B uch that ker(A) = {0}
A 3 5 and ker(B) = [0}. Find ker(A B).
4 7
52. Consider a p x n matrix A and a q x n matrix B. and form the partitioned
a the kerne! of some matrix B. Hint: The image of A co nsists of all vectors matrix
y in !R4 such that the system Ax = ji is consistent. W1ite thi s system more
explic itly:

x 1 + x2 + X3 + 6x4 = Yl What is the relationship between ker(A), ker(B) , and ker(C) ?

x, + 2x2 + 3x3 + 4x4 = Y2 53. In Exercises 53 and 54, we will work with the binary digit (or bit ) 0 and
x, + 3x2 + 5x3 + 2x4 = Y3
J, instead of the real numbers R Addition and multiplication in thi y tem
x, +4x2 + 7x3 Y4
are defined as usual , except for the rule 1 + 1 = 0. We denote thi number
y tem with lF2 , or simply IF. The et of aJJ vectors with 17 component in IF
Now reduce rows:
is denoted by lF"; note that IF" consists of 2" vectors. (Why?) In information
x1 - x3 + 8x4 = 4 y3 - 3y4 technology, a vector in IF 8 is called a byte (i.e. a byte is a string of 8 binary
x2 + 2x3 - 2x4 = - Y3 + Y4 digits).
The basic ideas of linear algebra introduced so far (for the real number )
0 = Y1 - 3)13 + 2 Y4
apply to lF without modifications.
0 Y2 - 2y3 + Y4
144 • Chap. 3 Subspaces of ~" and Their Dimens ions Sec. 3. 1 Image an d Kerne! of a Linear Transfo rm at io n • 145
A Hamming matrix with 111 rows i a matri x which ·ontain all non zero will work with blocks of o nl y fo ur binary digits, i.e., with vector in Ir". For
vector in F111 a it co lu mns (in any ord er). Note that there are 2111 - I example:
column . Here is an ex ample:
. .. j t O I J j l O O t j i O I O j l O l l j l OOO I .. .

ll
0 0 0 I 3 row Suppo e these vectors in JF4 have to be transmitred from one computer to
l 0 0 23 - I = 7 colurru1 another, say from a satellite to ground control in KoUI·ou, French Gui ana (the
0
Stati on of the Eu ropean Space Agency) . A vec tor ii. in JF4 is first tran fo rmed
in to a vector v = Mu in JF1 where M is the matrix you found in Exerci e 53 .
a. Ex press the kerne! of H a the pan of fo ur v ctors in IF 7 of the form v
The last fo ur entries of are just the entrie of u;
the first three entrie of
v are included to detect errors. The vector v is now transmitted to Kourou .
We assume that at most one error will occur during tran rril ssion, that i , tbe
* * * * v
vector ÜJ received in Kourou will be either (if no error ha occurred), or
* * * * ÜJ = +v e. , if tbere is an error in the itb component of the vector.
*1 *0 *
0 U4 =
*
0
a . Let H be the Hamroing matrix introdu ced in Exercise 53. How can the
U1 = 2 = U3 =
computer in Kourou use H w to detem1ine whetber there was an error in
0 I 0 0 the transmission? If there is no error, what is H w? If there is an error,
0 0 1 0 how can the computer determine in which component the error was made?
0 0 0 b. Suppose the vector

b. Form tbe 7 x 4 matrix 1

0
1
W = 0
1
0
0

is received in Kourou . Derermine whether an error wa made m the

transmission and, if so, correct it (i.e., find v and il).

Explain wby im (M) = ker(H ). If x is an arbitrary vector in JF4 , what is

H (M x)? Kourou

54. (See Exercise 53 for some background.) When in formation is transmitted,

there may be some errors in the communication. We present a method of
adding extra information to messages so that most eiTors that occur during JF'3
transmission can be detected and corrected. Such methods are refen·ed to as SateUite H
error-correcting coäes (contrast these to codes whose purpose is to conceal detect
infom1arion). The pictures of man ' s first landing on the moon (in 1969) were error

televised just as they had been received and were not very clear, since they M tra nsmiss ion
JF'1 ~ JF7 JF1 IF'~
contained many errors induced during transmi ssion . On later missions, much
11 encode ii possible error ÜJ correct error ii decode ü
clearer error-corrected pictures were obtained.
In computers, information is stored and processed in the form of string
of binary digits, 0 and l. Thi s st.ream of binary digits i · often broken up
into "blocks" of eight binary digits (bytes). For the sake of simplicity, we
146 • Chap. 3 Subspaces of IR" and Their Dim ensio n
Sec. 3.2 ubspaccs o f 11
; Ba es a nd Linear lnd epe nd nce • 147
2 SUBSPACES OF IR''; BASES y
AND LINEAR INDEPENDENCE
In the last ction we have een that both the image and the kerne[ of a linear Irans- w
fo rm ation contain the zero vector (of th e codoma.in and the domain. re -pectively) ,
are clo ed under addition. a nd are c lo ed unde r calar multip lication . S ubset of
IR" that ha ve the e three properties are called ubspace of IR".

Definition 3.2.1 Subspaces of IR"

ub e t W of IR" i called a subspa e of IR" if it ha the fo ll owing propertie :

a. W contains the zero vector in IR" .

b. W i closed unde r adclition: If w1 ancl w2 are both in W , then o i
w, + w2.
Figure 1
c. W i closed unde r scalar multiph cation: If wi in W and k i an arbitrary X

calar, the n kw is in W.
y

Fact 3.1.4 a nd 3.1.6 tel ! us the following:

w
Fact 3.2.2 If T : !R" --7 IR"' i a linear transformation, then
• ker(T ) is a subspace of IR11 , and
• im (T) is a subspace of IR"'.

ii in W

The term space m ay eem surpn smg in this context. We have seen in
examples that an image or a kemel can well be a line or a plane, w hi c h is no t
something we ordina rily call a "space." In modern mathematics the term "space"
i a pplied to structu res Lhat are not even re mote ly imil ar to the space o f o ur
experience (i.e., the space in which we li ve, w hi ch i nai vely ide ntified w.ith JR 3 ). .r

The term space is used to describe IR" for any value of n (not just n = 3). A.ny
subset of the space IR" with the properties li sted above i ca ll ed a s ubspace of IR",
even if it i. not ·'three-dimensional."
- ii= (- l )ii
Figure 2
EXAMPLE 1 ..... Js W = { [ ~] 2 2
in JR : x ::: 0 , y ::: 0} a subspace of JR ?

The answer is no: W contains the zcro vector. and it i c losed unde r addition.
Solution but it i not c losed under muJtipli cation wi th negative sca la r ( ee Fi oure 2). ~

Note tha t W consists of a lt vect.ors in the first quadrant, including the positive
axes, as shown in Figure I.
148 • Chap. 3 Subspaces of R" and Their Dimensions Sec. 3.2 Su bspa ces of IR"; Bases and Linear lndependence • 149
y

Figure 3 0

Figure 5
y

as a linear combination of ii 1 and i:i2 • Therefore, ü is contained in W (since W i

ii
closed under linear combination ). Thi hows that W = 2 , a claimed. <11111

Similarly, the on ly subspaces of JR( 3 are JR( 3 itself the plane through tbe
origin , the line tbrough the origin, and the set {0} (Exerci e 5). ote the hierarchy
of subspace , arranged according to their dimen ions (the concept of dimension
will be made precise in the next section).

ÜJ
11: Dimension 3
Subspaces of JR 2 Subspaces of JR 3

]R3
Dimension 2 JR2 planes through Ö
Dirnenion I lines through 0 lines th!ough Ö
Dimension 0 {0} (0}
f1911re 4

Solution We have seen that both kerne! and image of a linear transformation T are
subspaces (of the domain and rhe codomain of T , respectively). Conversely,
Note that W consists of all vectors in the first or third quadrant (including the can we express any ubspace V of IR(" as the kernel or the image of a Linear
axes). See Figure 3. Again, the answer is no: While W contains 6 and is closed transformation (or, equivalently, of a matrix)?
under scalar multiplication, it is not closed under addition (see Figure 4). <11111 Let us consider so me examples. A plane E in JR(3 is usually described either
by giving a linear equation such as
EXAMPLE 3 .... Show that the only subspaces of ~2 are IR2 itself, tbe set {0}, and any of the lines x1 + 2x2 + 3x3 = 0,
through the origin.
or by giving E parametrically, as the span of two vectors : for example,
Solution
Suppose W is a subspace of IR2 which is neither the set {Ö} nor a line through the
origin. Wehave to show tha~ W = JR( 2 . Pick a nonzero vector iJ 1 in W (we can find
such a vector, since W =I= {0}). The ubspace W contains the Line L spanned by
UJ [-H
In other words, E is described either a
and

v
Üt , .but W does not equal L. Therefore, we can .find a vector 2 in W which is not
v
in L (see Figure 5). Using a parallelogram, any vector in JR2 can be expressed ker [ 1 2 3]
150 • Chap. 3 Subspace of !R.11 and Their Dimensions Sec. 3.2 Sub paces of lR 11 ; Bases and Linear Independence • 151
or

im [ ·~ _; ] .
- 1 1
Similarly, a line L in rn! 3 may be described either parametrically, as the spa n of
the vecror

or by the two linear equations ~~

/J
XI - X2 -
2xz +
X) = 0 I
= 0 .
IXI - X3 Figure 6

Therefore. _ F~gure
6 _:;how~ tha~ we need on ly li 1 and ü2 to span the image of A. Since
v3 = + u2, the vectors Ü3 and ü4 are redundant: tbey are li near
~u1 . and u4 ~ v 1
- 1
-2
- 1l . J co mbmatJon of u1 and ü2 .
im (A) = pan (li 1, Ü2 , Ü3, v4 ) = span(li 1, ü2 )
A subspace of rn!" i u ually eilher presented as the olution set of a homo- The image of A can be spanned by two vectors, but clearly not by fewer than
geneous linear system (i. e., a a kerne!), or given parametrically as the span of ~- ~
ome vectors (i.e., as an image). In Exercise 38 you will see that any subspace of _ _ We oft~ n ~vish to expre s a ubspace V of IR" as the span of some vector
IR" can be represented as the image of a matrix. VI' V2 .. . ' Vm 10 V. It is reasonable to require that none of the 1 be a linear v·
Sometimes a subspace that has been defined as a kemel must be given a an combination of the other ; otherw ise we might as we!J omit it. We fi rst introduce
image, or vice versa. The translation from kerne\ to image is traightforward: we some u eful terms:
can represent the solution set of a linear system parametrically by Gau s-Jordan
elimination (see Chapter I) . A way to write the image of a matrix as a kerne\ i
Definition 3.2.3 Linear independence; basis
di scu ed in Exerci e 42 and 43 of Section 3.1.
Con icler a sequence Ü1, . . . , Ü111 of vectors in a subspace V of JR".
The vectors Ü1, . . . , Üm are called linearly independent if none of them i
+ Basesand Linear Independence a ljnear combination of the others. 1 Otherwise the vectors are called linearly

hu~ ~ n
EXAMPLE 4 ..... Consider the matrix dependent .
We . ay that the vector ü1 ••• , Ü111 form a basis of V if tbey span V
and are linearly independent. 2

The vector u" u2 , ü3 , Ü4 in Example 4 span

Find vectors li 1, ü2 , . .. , Ü111 in IR3 tbat span the image of A. What is tbe smallest
number of vectors needed to span the image of A? V = im (A),
but they are linearly dependent because Ü-t = ü1 + ü2 • Therefore, they do not fom1
Solution v
a basis of V. The vectors 1• ü2 on the other band, do pan V and are Iinearly
We know from Fact 3.1.3 that the image of A is spanned by the columns of A:
1Thi s det'i nition makes -ense i f' 111 is <ll least two. We say th at a ·in gle vector, v 1 , is linear!
indcpendcnt if ii 1 'I Ö.
2B conve ntion. Lhe empty et 0 is a basis of the space {0} .
152 • ha t . 3 Subspaces of Ii(" and The ir Dimens ions
Sec. 3.2 Subspaces of JR"; Bases and Linear lndependence • 153
independen t (neither of them i a mul tiple of the other). T h refore, the ve~ tor
u1 - , are a ba i of \1 = im(A). T he vector u1. - 2 do not form a ba. 1 of IR 3. Solution
h;w -v r. While they ar linearly independent, they do not span IR 3 (they span To fi nd the relations among these vectors, we have to solve the vector equation

p
only a plane).
T he following defi nition w ill he lp us to ga in a better conceptual understand - I

~m
1

Definition 3 .2.4
ing of linear independen e.

Linear relations
Cl
2
3
4
5
+c, [
10
i] +c,
11
+ C4
4
9
16
25
Con ider the vector u1 u2•. . .• Um in IR" . An eq uatio n of the form
o r the matrix eq uati on
C1 - 1 + C2 -2 + · · · Cm -m = 0
u;.
I]~~ r~1~ [0
is call ed a (lin ear) relation among tbe vector There i al ways the trivial 6 2

~
relation, with c 1 = c2 = .. · = cm = 0. Nontrivi.al relation may or rnay not 7 3 4 Cl 0
ex.i t among the ectors u;.
For example, am o ng the vector introdu ed in Examp le 4 we have the non-
[i 10
8
9
5
7
11 25 C4 0
A
trivial relation
In other words, we have to fi nd the kernet o f A. To do . o, we compute
rref(A ). Usi ng technology, we find that
because u4 = u, + u2.
1 0 0 0
Fact 3.2.5 The vectors ü1, ••• , um in lR" are linearly dependenr if (and only if) there are 0 l 0 0
no nt1·ivial relation amo ng the m. rref(A) = 0 0 I 0
0 0 0 l
0 0 0 0
Proof • If one of the Ü; is a linear combination of the others, then
This shows that the kerne! of A is (Ö}, because there is a leading I in each column
of rref(A). There is o nly the tri vial re lati on among the four vectors, and they are
and we can fi nd a nontrivial relation by subtracting v; from both sides of the therefore linearly independe nt. <011111

equation above:
Since any relation
C[V I +·· · +Ci- I U;- 1 + (- l ) u; + Ci+ l Vi + l + ··· + CmVm = 0
• Conversely, if the re is a nontri vial relation Cl UI + C2 Vz + ···+ Cm Um = Ö
Cl ul + ... + C;V; + . .. + Cm Um = 0 , with C; =/:- 0 ,
can alternati vely be wrinen as
then we can solve fo r Ü; and thus expre s Ü; as a linear combination of the
other vector . •
EXAMPLE 5 ..,._ Determine whether the fo llowing vectors are li nearly independe nt.
1 6 2 1
C2
cl
.
J -
= 0,
2 7 3 4 [
3 8 5
C;n
9
4 9 7 16
5 10 11 25
we can generalize the observation made in Example 5.
154 • Chap. 3 Subspace of IR 11 and Their Dirnen ions Sec. 3.2 Sub paces of IR"; Base and Linear lnclepcnd nce • 155
~-----------------------r

Fact 3.2.6 The vect r - , ..... -", in !R" ar linear! ind pendent if (a nd on ly il)

ker v", = {0) E= im(AJ

or. eqUI alentl y. if there is a leadin g I in ea h column of

Figure 7

rref
be expre. ed uniquel y a a linear combinat ion of the v,, namel y. a

0 = Oü, + Oü2 + · · · + Oü",.

Butthis mea n th at there i only the tri ial relation among the v;: they are linear!
indepenclent (Fact 3.2.5). •
We co nclude thi ectio n with an important alternative characterization of a
basi . As an example, consider the plane E = im (A) in ~ 3 introduced in Exam-
If the vectors v1, v2 , . .. , v", are a basis of the subspace V of lR", then any ple 4 ( ee Figure 7).
vector - in V can be expres ·ed a a linear combination of the ba is vecto r ( ince The vectors v1, ü2 v3 , v4 do not form a ba i of E. sin ce any vector in E
they pan V) . Thi s repre entation i in fact u11.ique. can be expre ed in more than one way as a linear combination of the Ü;. For
example, we can write

Fact 3.2.7 v
Con ider the vecto r iJ 1, 2 , ... , v",
in a sub. pace V of lR".
T he vector Ü; are a basi of V if (and only if) any vector ü in V an but also
be expre. ed uniquely as a linear combi nation of the vector- V;.
v-1 = O:V , + Oü2 + Oü3 + I. ü.j .
Because any vector in E can be ex pre ed uniquely a a li near combination of ü1
Proof Suppo e the vector v; fom1 a basis of V , and consider a vector ü in V. Si nce. and iJ2 alone, the vectors ii ,, v2 fo rm a ba is of E.
the ba i vector span V, the vector v can be written as a I inear comb ination of
tbe v;. We have to demonstrate rhat thi s repre en tation is unique. To do ·o, we EX ERCI S ES
con ider two repre. entations of ü, name ly
GOALS Check whether a subset of !R" is a ubspace. App ly the .concept of
ü = c,ü, + c2v2 + · · · + CmÜ", = d, ü, + d2Ü2 + · · · + dmv",. linear independence (i n terms of Definiti o n 3.2.3. Fact 3.2.5, and Fact 3.2.6).
By ubtraction, we find Apply th oncept of a ba is, both in t rms of Defi nition 3.2.3 and in terrn of
Fact 3.2.7.
(c 1 - )v, + (c2 -
d1 dz)Ü2 + · · · + (cm - d",)Ü," = 0,

which is a relation among the Ü; . Since the Ü; are linearl y in dependent, we have Which of the sets W in Exerci es 1- 3 are sub pace of JR 3 ?
c; - d; = 0, or c; = d;, fo r all i: the two repre entations of iJ are ide ntical, as
claimed .
Conversely, uppose that each vector in V can be expres ecl uni q uely as a
li near com binati on of the vector v; . Clearly, the v; pan V . T he zero vector can
156 • Chap. 3 Subspaces of ~~~ and Their Dimensions
Sec. 3.2 Su~spaces of R11 ; Basesand Lin ear Independ ence • 15 7
ln E xerci se 20 and 21 , decide whe th er the vector are linearly indepe ndent. Y u
may use technology .

3. W = I[4;:;~ : ~~ ] :
7x + 8y + 9z
x , y, z arbi~ary constants}
O-m - ln l:!J
4. Consider the vectors ü1, ü2 , • . . , Üm in iR" . I span(i:it , . . . , tim) neces arily a
sub pace of iR"? Justify your answer. 7 2
I 3 5

[i m
1
5. Give a geometrical description of all subspaces of JR3 . Ju tjfy your answer. .21. 5 3
6. Con ider two sub paces \1 and W of iR". 4 9
a . I the intersection \1 n W necessarily a ubspace of iR"? 2 7
b. Is the union \1 U W necessarily a sub pace of IR"? -or which choices of the constants a, b, c d , e, f are the following vector
linearly independent? Justify your answer.
7. Consider a nonempty subset W of iR" that is closed under addition and under
scalar multiplication . Is W necessarily a subspace of lR"? Explain.
~ Find a nontrivial relation among the following vectors:

[!].
9. Consider the vectors i:i 1, ü2 , in IR", with ii 111 = 0. Are these vectors
... , V111
linearly jndependent? How can you teil?
@ Consider a subspace \1 of IR". We define the orthogonal complement V .1.. of
\1 as the set of those vectorsw in IR" which are perpendic ular to all vector
in \1, that is,w· u
ü = 0, for all in \1 . Show that V ..L is a subspace of Jl{".

In Exercises 10 to 19 use paper and pencil to decide whether the given vectors
are linearly independent.
24. Con ider the lin~ L spanned by[;·] in IR 3 . F ind a basis of L ..L (see Exer-
cise 23). 3
25. Consider the subspace L of IR 5 spanned by the vector below. F ind a basi of
L ..l. (see Exercise 23).

26. For whi ch choices of the constan ts a , b, . .. , m are the vectors below linearly
independent?

{/ e f k
b 1 g /'11.
c 0 h 1
d 0 0
I 0 j 0
0 0 1 0
158 • Chapo 3 Subspaces of IR11 and Their Dimensi o ns Sec. 302 Subspaces of R 11 ; Basesand Linear Independence • 159
Find a ba i of the imag o f each matrix in Ex rci es 27 to 33° lf you are puzzled, think fir t abou t the peci al case whe n V i a plane
in JR 3 What is m in thi s ca e?
0

un n
I c. Show that any subspace V of lR" can be represented as the image of a
27. 28.
[i 2
3
29.
[! 2
5 ~] matrixo
39. Cons ider some linearl y independent vectors ü1, ii2 , 000, Ü111 in lR" and a ector
ü in lR" that is not contained in the pan of ü" ü2 , 000 Ümo Are the vector

30.
[~
I
0
0 !] 2
31. [ 1
3
5
6-
7
8
J 32.
[~ i J
ü, , Ü2, 000, Ü111 , Ü necessari ly lin early independent? Justify your answer.
40. Con sider a n m x n matrix A a nd an 11 x p m atrix B

of the product AB linearly independeot a weil ?

We are told that the
0

co lumn s of A and the column s of B are linearly independento Are the colurnn

[~
1 2 0 3
q 41. Con ider a n m x n rnatrix A and a n n x m matrix B (with 11 =!= m) uch that
33.
0
0
0
0
Jo
I 4
0 0
0 '0 !]
34. Con ider the 5 x 4 rnatrix A below:
AB= Im (we say tbat Ais a left inverse of B)o Are the columns of B linearly
independent? What abou t the columns of A?
42. Co nsider sorne perpendicular unit vectors ü1, ii2, 000, Ü111 in lR11 0 Show that
the e vectors are necessari ly linearly indepe nde nt. Hint: What happen whe n
you fo rm the dot product of Ü; and both ides of the equation below?

A = u, 3 U-1

43. Con ider three linearly independent vectors ü1, ~. ü3 in lR11 Are the vector
0

ü, ü, + ü2 , ü1 + ü2 + ü3 linearly independent as well? How can you tell?

44. Consider the li.nearJy indepe nde nt vectors Ü1, ü2 , 000, Üm in lR 11 and Iet A be
an invertible m x m matrixo Are the column o f the foiJowing matrix linearly

We are told that the vector [ jJis in the kerne! of A. Write V, as a linear
independent?

combination of - , , Ü2, Ü3o

35. Consider the linearly dependent vectors ü1, ü2 , 000, Ü111 in lR" where ü1 =!= 00
Show that one of the vector Ü; (for i = 2, 000, 11) is a linea r combi nalion of U111 A
the previous vectors ü1, ü2 , 000 Ü; _ 1 o

36. Con si der a linear transfom1ation T from lR" to JRP a nd some linearly dependent
vector ü1, ~. 000, ü", in lR"o Are the vectors T(Ü J), T (~), 000, T ( Ü111 ) linearl y
dependent? How can you tel!?
37. Consider a linear transfom1ation T from lR" to JRP and some linea rly indepen- 45. Are the co lumns of an invertible matrix Jinearl y independent?
dent vectors ü,, Ü2, 000, Ü111 in lR"o Are the vectors T( v 1) , T(ü2 ) , 000, T(Ü 111 )
46. Find a basis of the ke rne! of the matrix belowo
necessarily linearl y inde pe ndent? How can you tell?
38. a. Show that we can find at mo t n linearly independent vectors in lR"
~]
0

l 2 0 3
b. Let V be a subspace of lR" Let m be the largest number of linearly
0 0
0

independent vectors we can find in V (note that m :s 11 , by part a)o C hoose
so me linearly independent vectors ü,, ü2 , 000, Ü111 in V 0 Show that the
vectors ü,, Ü2, 000, Ü,11 spa n V, and a re therefore a bas is of V 0 This
exercise shows that a ny subspace of ~" ba a basi so
[ 4
Ju stify your a nswer carefully , ioeo. ex pl ain how you know that the vector you
found a re linearly independent and pan the kerne!.
160 • h ap. 3 Sub paces of IR 11 and Th eir Dirn nsi n ,
Se . 3.3 The Dime nsion of a Subspace o f IR" • 161
4
47. Con ider three linear!_ independent ve tor ü1• ü2, ü3 in lR . Find

48. Expre the plane E in JR 3 with equation1 .q + 4x 2 + Sx3 = 0 a the kernet of

a matri · A and as the image of a matrix B.

49. Express the llne L in JR 3

spanned by the vecto r [
A and as the kerne! of a rn atri x B.
i]
1
as the image of a matJi.x
Figure 1 The vectors v v2 form o
1,
bosis of E.

50. Consider two ub paces V and W of lR". Let V + W be the et of all vector
in lR" of the fom1 ü + tÜ, where ü i in V and ÜJ in W . I V + W nece aril y w w w
Consider some vectors ü1, ü2 , • •. , vP and 1 , 2 , ••. , 9 in a subspace V
a subspace of lR"? of lRn ·. If tbe vectors Ü; are linearly independent, and the vectors Wj span V,
lf V and W are two di tinct Jine in lR-', what is V + W? Draw a ketch. then p ~ q.
51. Consider tv~o sub pace V and W of IR" who e intersection con i t only of
the vector 0.
a. Con ider linearl y independent vector u1, ü2 , •• • , Üp in V and w1, w2 . . . ,
For example, Iet V be a plane in JR3 . Our geometric intuition tells us that
wq in W. Explain why the vectors ü1, ü2 , .. . , Üp, w1, w2 , ... , Ü!q are
we can choose at most two linearly independent vectors in V, so that p ~ 2, and
linearly independent. we need atleast two vectors to span V , so that 2 ~ q : Therefore, the inequality
b. Con ider a basis ü,, Ü2, . . . , ÜP of V and a basis ÜJ 1, w2 , .. . , Ülq of W .
p ~ q does indeed hold in thi s case.
Explain why Ü1, u2 , •• • , ü", w1, w2 , ... , wq i a ba i of V + W see
Exercise 50). Proof Tbis proof is rather technical and does not reaUy explain the result conceptually.
52. True or false? If V is a ubspace of IR", then there is an n x 11 matrix A such In Chapter 7, when we study coordinate systems, we will gain a more conceptual
tbat understanding of tlü matter.
Since the Wj span V, we can express each v;
as a linear combination of the
V = im(A). vectors wj :
(Hint: See Exercise 38.)

We write each of the e equations in matrix form:

THE DIMENSION OF A SUBSPACE OF ~n

ra~l ]
r
Consider a plane E in IR 3 . Using our geometri c intllltiOn, we observe that all
bases of E consist of exactly two vectors (any two nonparallel vectors in E will Wq
a~t ]
: -
_- VJ Wq = Üp
do; see Figure I). One vector is not enough to span E , and three or more vectors
{/ lq Opq
are linearly dependent. More generally, all bases of a subspace V of IR" consist
of the same number of vectors. To prove thi s impo1tant fact, we need an auxiliary
result.
162 • Chap. 3 Sub pace of IR" and Their Dim en io ns Sec. 3.3 The Dim ension of a Sub pa e of J?." • 163
We combine all the e equ ation int o o ne matri x equatio n: Thi s algebraic definition of dimen sion re pre. ents a maj r step in the de-
velopment of linear al gebra: it a llow u. to co nceive of pace. w ith m re than
three dimen s.ion s. Thi s idea is of'ten poorly understood in popu lar culture, where
·ome mystici sm . uJTounds higher-dimen ·ional space . the German mathe-
matici an Hermann Weyl ( 1885-1955) said: " We are by no mean ob li ged to . eek
illumination fro m the mystic doctrines of spiriti t to obta in a clearer i ion of
multidime nsion a l geometry" Raum, Zeit, Materie, 19 18) .
The first mathematician who th ought about dimen ion fro m <U1 algebraic
point of view may have been the Frenchman Jean Le Rond d ' Ate mben ( 17 17-
1783). In an article on dim ension in the Encyclopedie he write :

The way of cons idering quantities having more than rhree dime n ion
i jusr a right as the other, becau e Ietter can alway be viewed
The kerne! of A is comained in th e kernel of N (if A.\- = 0, then M = Ar a repre enting number , whether rational or not. 1 sa id above
N.'i = Ö. But the kemel of i {Ö}, ince the Ü; are linearl y independent that it was not po ibl e to conceive more than three dimen ion .
(Fact 3.2.6). Tberefore, the kerne] of A is {Ö} a weil. This impli e th at the q x p A thoughtful gent leman [un homme d'esprit] w ith hom I am
matrix A has at least as rn an y row as colu mns, o that p .::: q (by Fact 1.3.3). • acq uainted believe th at nevetthele one could view duration as a
foUith dimen sio n . . . . Thi idea may be challenged , but it ha ,
Now we are ready ro pro e rhe fo ll owing: it seem to me, ome meri t, were it only that o f being new [cette
idee peut etre conteslee, m.ais elle a, ce me semble, quelque merite,
quand ce ne serait que celui de Ia nouveaute] . (Encyclopec/ie,
Fact 3.3.2 All bases of a ub pace V of iil&" co n ist of the same nurnber of vector . vo l. 4, 1754.

Proof Consider two bases ü1, • • • , Üp a:nd ÜJ 1, ••• , wq of V. Since the Ü; are linearly
independent and the Üij pan V, we bave p.::: q. by Fact 3.3.1. Likewi e, . ince
the Üij are linearly independent and the Ü; spa n V, we have q .::: p. Therefore,
p=q.
Consider a line L and a plane E in JR 3 . A ba is of L co nsi t of ju t one
vector (any nonzero vector in L wi ll do) , while all ba es of E con ·ist of two
vector . A basis of JR 3 consists of three vectors (t he vectors e1. e2, e3 are o ne
po sible choice). In each case, the number of vector. in a ba is COJTespond to
what we intuiti vely ense is the dimension of the sub pace.
Definition 3.3.3 Dimension
Consider a sub pace V of lR". The number of vectors in a basis of V is caUed
the dimension of V denoted by dim(V). 1
1
For lhi s defi nition to make sen e. we have to be sure that any subspace of IR'." has. a basis . This
verificat ion is left a Exercise 3.2.38.
The homme d'esprit wa no doubt d ' Alembert himself. D ' Alembert wi hed
to proteer him e lf against being attacked fo r what appeared as a ris ky idea.
The idea of dimen ion wa later tudied more y tematically b the German
mathematician Hermann Günther Gra smann (1809-1877), who introduced the
• concept of a subspace of IR". Gras mann' methods were only slo wly adapted,
partly becau e of hi ob eure writing . Gra mann ex pre ed hi idea in the book
Die lineare Ausdehnung lehre. ein neuer Zweig der Mathematik (The The01y of
Linear Extension, a New Branch of Mathematics) , in 1844. Simi lar work \ a
done at about the ame time by the Swi mathematician Lud wig S hlä fli ( 18 14-
1895).
Today , dimen ion is a tandard and central tool in mathemaric . . a v eil a
in phy ics and stati stic . The idea ha been applied to certain n nlinear ub-
set of IR", called manifold , th us genera lizing t.he idea of curve and • urface
in IR 3 .
Let u rerurn to the more mundane: w hat i the d imen io n of IR" it e lf?
Clearly. IR" ought to have dimension /!. Thi i- indeed the ca. e: thc vectO I'" el
e e
2 , ... , 11 form a basi of IR" ca ll ed it ' sranda rd basis.
A plane E in IR 3 is rwo-dimen ional. Above, we mentioned th at we ca nnot
find mo re than two linearly inde pende nl ectors in E . and that we need at I a t
two vector to pan E. lf two ector in E are linea rl y indepen de nt, they form
a basi of E. Likewi e . if tvvo vec tors pan E, 1hey fo rm a ba i of E. These
ob erva tion s ca n be generali zed a f llow :
164 • Chap. 3 Subspaces of JR:" and Their Dimensions
Sec. 3.3 The Dirnen io n of a Su bspace of JR" • 165
Fact 3.3.4 Consid r a subspace V of IR" with dim (V) = d. Thi corre po nds to the ys tem
a. We an find at most d linearly independent vector in V.
x 1 + 2.x2 + 3x4 = 0 I
b. We need at least d vector to span V . I X3 + 3x4 + 5.xs = 0
c. Jf d vectors in V are linearly independent then they form a basis of V . with general solution
d. If d vector pan V , then they form a ba i of V .
Vi V2 V3

~:: : l~ tl ~ i
The point of parts c and d i the following : by Definition 3.2.3, some
vectors form a basi of a ubspace V if they are linearly independent and span
V. However, when we are deaüng with "the right number' of vectors (namel y, s [ +t [ +' [
the dimension of V), ir suffice to check onl y one of the rwo prope.rties ; the other
will follow .

Proof We demonstrate parts a and c, and Ieave b and d a Exerci e 58 and 59.
Part a: Consider ome linearly independent vectors Ü1• ü2 , • • • ÜP in V and
choo e a basis iü~, iü 2 , ..• , iüd of V . Since the ÜJ; pan V we have p s d , as
claimed (by Fact 3.3.1).
Part c: Consider some linearly independent vector ü1 , .•• , üd in V . We
have to how that the v; pan V. Pick a ü in V . Then the ector ü1, ••• , üd v
will be linearly dependent. by part a. Therefore, there is a nontrivial relation
We know that c =!= 0. Why?) We cao solve the relation for ü and thu s express
ü as a linear combination of the -;. We have hown that an y vector ü in V i a
linear combination of the Ü;, that is, the Ü; span V. •
From Section 3. 1 we know that the kerne! and image of a linear trans-
forrnatio n are subspace of the domain and the codomain of the transformation,
respectively. We will oow examine how we can find base of kerne) and image,
and thus determine their d.imensions.
The three vector ü, Ü2, Ü3 pan ker(A), by co ostruction. To verify that
they are al o linearly independent, con ider the component corresponding to the
nonleading variables: the secood, fourth , and fifth. As you exami ne the econd
components, I , 0, and 0, respectively, you realize that ü1 can not be a linear
combination of Ü2 and ü3 . Similarly, we can ee that ü2 isn't a linear combination
of ü, and ü3· likewise for Ü3 . Wehave howo that the vector ü 1, ü2, ü3 form a
basis of the kerne] of A. The nurober of vector in thi basis i.e., the di men ion
of ker(A)) i the nurober of nonJeading variable .
dim (ker(A)) (number of non leading variable )
(number of column of A ) - number of Jead ing vari able
(number of column of A ) - rank (A) = 5 - 2 = 3
The method we u ed in Example 1 applie to all matrices .
Fact 3.3.5 Con ider an m x n matrix A . Then,
dim(ker(A)) = n- rank(A) .

+ Einding a Basis of the Kernel

EXAMPLE 1 ..... Find a basis of t~ ~etof the fo ll owing matrix , and determine the dimension + Einding a Basis of the Image
of the kernel.
Again , we study an exam ple, m aking sure that our procedure applie to the general
A = [I2 24 0 39 0]5 case.

EXAMPLE 2 .... Fi nd a ba i

~ ~ ~ ~
Solution I 2
5 1 2
Wehave to solve the linear system Ai = Ö. As in Chapter l , fi rst we fi nd rref(A) . T IR IR' with matrix A [
0 1

A = [ 21 2 0
4
3
9
0]
5 - 2(/) -+ rref(A) = [ 01 2
0
0 33 0]5 . and determ ine the dimens io n of the image.
- 1 - 1
 166 • Chap . 3 Subspaces of I!(" and Their Dimensio ns
Sec. 3.3 The D imens io n of a Sub pace of ~~~ • 16 7
Solution _Co n. ider the co lu m ns ii 1, ii2 , ü5 of A corre po nd ing to the column of E

We know rhat the columns of A pan the image of A Fact 3. 1.3) , bu t they are
co n ta 1 n 1 ~g ~e ~ead in g l" s. Sin ce ÜJ 1,2, ww
5 are linearl y independent, so are the
vectors u, , u2, us . T he vectors ii 1, ü2, ü5 span the im age of A . . in ce anv vector in
linearly dependent in thi s exampl e. (Why?) To construct a bas is of im (A), we the i.mage of A can be expressed as '
could find a relatio n among the co lumn of A, exp ress o ne of the co lumns as a
linear combination of the other , and then omü this vector as redun dant. After c, ii , + c2iJ2 + c3Ü3 + c4Ü4 + csils = c 1ü, + c2ii2 + c (ii 1 - 2ii2) + c4(2ii 1 - 3ü2) + c5ü5.
repeati ng this p ro edure a few time , we would have some linearly inde pendent
i.e., it ca n be written a a linear combinati on of ü1, ü2 , ü5 a lone .
vector that still pan im (A), i. e. we would have a basi of im (A) . We pro pose
We have show n that ii 1, ü2 , ü5 is a basis of im (A), and dim (im (A)) = 3. <011111
a method fo r fi nding a basis of im (A) whi ch u es the ideas just di scussed , but
preseot them in om ewhat streamlined form. The fo ll owing de finiti o n w ill be help ful in stating these re ults more gen-
We fi rst find the reduced row echelon fo rm of A. era lly:
- ws
u, vs U/3 Ul4

il- ~ ~ U !l
U3 'U4 IJ II tll"2
V2
0 I 2

u
0 I 2 Definition 3.3.6 A column of a matri x A is call<ui..a_pivot column if the correspo ndjn o column
- 2 -3
A=
0 l 2 I
0
of rref(A ) contains a leading 1. ~ o
1 0 1 E rref(A) 0 0
1 - l - 1 0 0 0

As shown above, we denote the i th columns of A and E by Ü; and ÜJ;, respectively.

Fact 3.3.7 The pi vot ·c olumn. of a matrix A fo rm a bas is of im (A) .
We have ro express ome of the Ü; a linear combinations of the other vectors Üj to
identify redundant vectors. The con·e pondin g problem for the ÜJ; is easily so lved
by inspecti on: B ecause the number of pivot columns i the rank of A by definiti o n, we
have the fo ll owing re ult:

ln general, we can express any column o f rref(A) th at does not contain a

leading 1 a a linear combination of column that do co ntain a leading 1. Fact 3.3.8 For any matrix A,
lt may surprise you that the . ame relationship hold among the correspo nding
rank (A) = dim(im(A)).
columns of the matri x A. Verify the following for yourself :

Tru s interpretation of the rank a a dimen io n i co nceptua ll y more sati fac-

Let us explain why correspondin g rel ations ho ld amo ng the columns of A and tory than the rather technical Defi niti o n 1.3.2.
-; E = rre f(A) . Consider a relatio n c 1ii 1 + c 2iJ 2 + · · · + c 5ii 5 = Ö among the columns Co nsider an m x n matrix A . Facts 3.3.5 and 3.3.8 teil us that
of A. Thi s relation can altemati vely be written as dim (ker(A)) = n - rank(A). and

ln~ö
dim (im (A)) rank(A).
Adding the e two equati o n , we obtain the follo wing :

A
Fact 3 .3.9 Tf A is an m x n matrix , then
j, Since A and E have the same kerne ls (the reduced row eche lon form wa de fined dim (ker (A) + dim (im (A)) = 11 .
~\ so that the systems Ai = Ö and Ex = Ö have the sa me soluti o n ), it follo w that

L For matri ces of a g iven size the !arger th k rne l. the . malle r the irnage ,
and vice versa. We ca n inte rpret the fo rm ul a

n - dim (ker(A)) = dim (im (A))

I
168 • hap. 3 Subspaces of IR'' and Their Dimen ions Sec. 3.3 The Dimension of a Sub pace of IR" • 169
The linear system

L = im(A)
v" [~; l =b
c"

=
ker (A) L... : the plane has a uniq ue solution if (and only if) the n x n matrix
perpendicular to L

Figure 2

geometri call y a fo ll ow : Con ider a lLnear tran Formati on

T (i) = A.l- from " to IR"' .

is invertible.
Note that n is the dimension of the do rnain IR" of the tran fo rmation T . The
We have shown tbe following re ult:
qu antiry dim (ker(A)) counts the number of dim en ions th at co ll apse as we perform
tbe transformation, and dim (im (A ) ouots the number of dimen ion th at ur ive
after the transformatioo. Fact 3.3.10 The vectors ü ~, Ü2, . . . , Ü11 in IR" form a basis of lR" if (and only if) the matrix
A an example cons ider the orthogonal projection ont o a line L in IR 3 ( ee
Figure 2).
Here, the dimen ion 11 of the domain i 3, two dimen ions coli apse (the
kerne! i a plane) and we are left 'i ith the I-dimensional image L.
u"
n - dim(ker(A)) = dim(im(A))

3 -2 = I
I
is io vertible.

+ Bases of ~ 11
EXAMPLE 3 ... Are the fo llowing vector a basis of IR4 ?
We know that any basis of IR" con ·ists of n vectors, since we have the standard
e
basis e1, ••• , 11 • Conversely, how can we teil whether 11 given vector u1, ü2 , ••• ,
Ün in IR" fo rm a ba is?
The Ü; fo rm a basis of IR" if every vector b in IR" can be written uniquely
as a linear combin atio n of the v; (Fact 3.2.7):
Solution

c,l We have to check whether the matrix

[::
[~ i i ~]
VII
170 • h ap. 3 Subspaces of ~~~ and Th eir Dimen ions
Sec. 3.3 Thc Dim ens io n of a Subspace of R11 • 171
is invertible . U ing te hno logy. we fin d that

rref [~ ~ l I] 8 9
1 7 = 14"
4.
un 5. [ 5 -4 3] 6. [;
3
2
l n
5 3 2 3 2
[~
The ec tor -; fo rm a basi of IR 4
.

Fact ..).3 .4 part c and d), applied to \1 = IR" and Fact 3.3. 10 provide us
7.
l 2
0 0
0
~] 8.
li
6
2
4
9
4
9
6
i]
IO.l~
wid1 new rules for o ur ummary. 0 2

Summary 3.3.11 Con ide r an n x n matrix

9. [ l - I - I l ]
- 1
1
4
-3
-6
3
-i]
In Exerci e I I to 20, fi nd a ba i of the image of the oiven matrix and th u
determi ne the dimensio n of the image. U e paper and pen~i l. '
A= II

;J u l] u n
3
11.
[~ 12. 13. 2

Then, the fo ll owing tatement are eq uivalent:

i. A i invertibl e.
ii. The linear sy tem Ax = b has a unique o lu tio n ;:r, fo r allb in IR" .
14.
un 15. [ 5 -4 3] 16.
[1~
3
2
~]
li ~] l~ ~~]
2 8 1 1
iii. LTef(A) = /".
2 6 1 2
iv. rank(A) = n . 17. 18.
2 4 1 9
V. im (A) = IR" . 2 2 1
vi. ker(A) = [0}.

~ ~]
0 2 2 3 2
vii. The Ü; are a ba is of IR".
viii. The Ü; span IR" .
ix. The Ü; are linearly independe nt.
19. [
-3
4 -6
- 1 3
-;] 20. [ i 6
2
4
9 6
4
9

For eachmatri x in Ex rci es 2 1 to 23, find a basi o f the kerne! and a ba i o f the
EXERCISES
image. and thus determine their dimensions. U e Fact 3. 3.9 to c heck your \ o rk.
GOALS Use the concept of dimen ion. Apply the ba ic res ults a bout linearl y

[-: j]
independent vectors in a d-dime nsional space (Fact 3.3.4). Find a ba is o f the
kernel and the im age of a linea r transfo rm ati o n.
21.
- I
I
-1
0 l 2l]
- 2 22
[lI 22 2 1 2
1
- I -2 0 3 . 2 4 3 3
In Exercise 1 to I 0, find a ba i of the ke rn e! of the o jven matrix , and thu s 2 -2 - 1 3 4 0 0 - I
detem1ine . the dimension of the kerne!. Use paper and pe1~cil. I
l
23.
172 • Ch ap. 3 Subspaces of IR" and Their Dim ensions Sec. 3.3 The Dimcnsi n of a Subspacc of JR" • 173
24. Find a basis of the ubspace of IR! 5 spanned by the vector below: 31. Le t ·V be the ub pace of IR4 defi ned by the eq uation

1 3 3 8 0
2 6 2 4 4
3 9 4 9 5 Find a linear transfo rmation T from l!i 3 to IR 4 such that ker(T) = {0) and
2 6 1 1 5 im (T) = V. Oe cribe T by it matrix A .
1 3 2 5 32. Fi nd a ba is of the ubspace of 4
which con i t of all vector perpendicular
to both
25. Pick a basis of JR!5 from the fo llowing vectors (if pos ible) :

[~
3
6
9
6
3
2
4
1
'
8
4
9
I
0
4
5
5
[-!] and m
3 2 5 33. A sub ·pace V of IR" i calJed a hype1plane if V i defi ned by the homogeneou
linear eq uation
4 2 4 I
3 3 I 2
2 5 7 9
where at least one of the coefficient c; is no nzero. \Vhat i the dimen ion of
1 7 0 1
a hyperplane in "? Ju tify your an wer carefully. What i a hyperplane in
4 5 8
JR2 ? What is it in IR! 3 ?
26. Consider the matrix 34. Co nsider a subspace V in IR!" whi ch is defined by m homogeneou li near
eq uations:
[~ ~ ].
1 2 0
A = 0 0 I a, , x, + a 12X2 + · · · + G J X
11 11 = 0
Find a matrix B such that ker(A) = im (B). G2 1X 1 + G22X2 + · · · + G2nXn = 0

27. Determine wbetber the vectors below form a basis of ffi!4 .

28. For which choi ce(s) of the constant k do the vectors below fo nn a basis
of ffi! 4 ?
What i the relation hip between the dim ensio n of V and the q uantity n - m?
State your an swer a an inequa lity. Explain carefull y.
35. Consider a nonzero vector ü in IR." . What is the dimension of the space of all
vector in IR!" whi ch are perpendicul ar to ü?
36. Can you find a 3 x 3 matrix A such that im (A) = ker (A)? Expl ain.
37. Give an example of a 4 x 5 matri x A with dim(ker(A)) = 3.

[~] · [~] · [~]

38. a. Consider a linear Iran sform atio n T from IR 5 to JR3 . What are the po ible
values of dim(ker(T ))? Expl ain.
[!] · b. Co n ider a linear Iransformation T fro m IR 4 to IR 7 . What are the po - ib le
value of di m(im (T))? Explai n.
29. Find a basis of the subspace of IR! 3 defined by the equ atio n 39. We are told that a certai n 5 x 5 matrix A can be written as
2x, + 3x2 +x3 = 0.
A = BC
4
30. Find a basis of the subspace of IR! defined by the equation
where B i a 5 x 4 matrix and C i 4 x 5. E pl ai n how you know tb at 1,

2x , -x2 +2x3 + 4x4 = 0. not in vertible.

17 4 • hap. 3 Subspace f JR" and Their Dimensio n Sec. 3.3 The Dimension of a u bspace of ;Rn • 17 5
40. on ider two ubspa e V a nd W of IR", w he re V i co ntained in W. Exp lain ~n Exerci es 4~ to 5_2, we wi ll study the row space of a matrix. The ro w pace of
w hy dim(V ) ::: dim (W). (Thi s statement eem intuiti ve ly rathe r obv io u . an ~ n ~ n matnx A ts defined a · the span of the row vectors of A, i.e., the set of
Still, we can no t rely o n o ur inruition when dea lin g w ith IR".) thetr hnear combi nations. For example, the row space of the matrix
41. Consider two ubspace V a nd W of IR", wh re V is co nta:ined in W. In 2 3
E ercise 40 we Iearned Lh at dim (V) :=: dim (W). Show that if dim (V) = I 1
dim(W), then V = W. 2 2
42. Co ns id er a ubspace V of IR" with dim(V) = n . Ex pl ai n w hy V= IR".
the et of all row vectors of the form
43. Cons ider two ub pace V and W of IR" , w ith nW
= (Ö} . What is the
re lat:ion h.ip between dim(V ) dim (W) and dim (V + W)? (For the de fin itio n a[l 2 3 4]+b[l 1] +c [ 2 2 2 3].
of \1 + W, see E ·e rc i 3.-.50; Exerci e 3.2.5 1 i he lpful.)
49. Find a basis of the row sp ace of the matrix be low.
44. Two ubspace V a nd W of IR" are call ed complemenrs if a ny vector in x
IR" can be expre. sed u nique ly as ; = ü + w,
where ü is in V and Ü!_ is in 0 1 0
W. Show that V and W a.re complements if (and on ly if) V n W = [0] and E= 0 0 l 2 0OJ
3
dim (V + dim (W) = n. [ 0 0 0 0 l
45. Consider some linearly independent vector ü,, Ü2•. .. , ü" in a subs pace V 0 0 0 0 0
of IR", and some vector 1, w w_, ..., wq
wh ich pan V. Show that there i
50. ~onside~ an m x n matrix E in reduced row echelon fo rm . Using your work
a bas is of V which con i t of oll the Ü; a nd some of the Wj. Hint: Find a
111 Exerc1se 4_9 as a guide, exp lain how you can find a bas i of the row pace
ba is of the image of the matrix
of E. What 1s the relationship between the dimen ion of the row space and
the rank of E?
51. Con sider an arbitrary m x n matrix A.
a. What is the relationship between the row spaces of A a nd E = rref(A)?
A= Wq Hint: Examine how the row space is affected by e lementary row opera-
tion s.
b. What is the relationship between the dimension of the row space of A and
the rank of A ·.
52. Find a basis of the row space of the matrix below.
46. Use Exercise 45 to construct a basi of IR4 which co n i t of the vector
1 l 1

and some of the vectors

[il [il
in IR4.
e,' e2, e3, e4
A=
[
2 2 2
1 2 3
1 3 5
53. Cons ider an n x n matrix A . Show that there are scala.r c0 , c 1, ••• , c" (not
all zero) suchthat the matrix co ln + c 1 A + c2 A2 + · · · + cnA" is noninve rtible.
47. Con ider two ubspace V and W of !R". Show that
v
Hint: Pick an arbitrary nonzero vector in lR". The n the n + 1 vector- ü
dim (V) + dim (W) = dim (V n W) + dim(V + W) A ü, A 2 v, .. . , A" ii will be linearly dependent. (Much more i true: There ar~
scalar co, c,, . .. , c" uch that co l" + c 1A + c2A 2 + · · · + 11 A" = 0 . You are
(for the defi nitio n of V + W, see Exerci. e 3.2.50). Hin/: Pi ck a basis ü" not asked to demoostrate this Fac t here.)
ü2 , .•• , Üm of V n W . Using Exercise 45, co n truc t bases ü 1, Ü2, ... ,
54. Con ider the matrix
iim. ü,, ii2, ... , v" of V and ü 1, ü2 , ... , ii 111 • w 1, w ivq
2 , ... , of W .
v w iv
Show that ü 1 , 1~ 2 , . . . , Üm , ii , , ii2, . .. , 17 , 1 , 2, . .. w11 is a basis of V+ W.
(Demonstratin g linear independe nce is a bit tricky .)
l
A = [2 -2] 1 .
48. Use Exercise 47 to answer the following question : If V and W are subspaces
Find calars co, c 1, c2 (not all zero) uch that the mau·ix c0h + c 1A + c1 A 1 i
of IR 10 , with dim V) = 6 a nd dim(W) = 7 , w hat are the poss ible dime n io ns
noninvert:ible (see Exerci e 53 .
of V n W?
17 6 • Ch ap. 3 Subspaces of R11 and Their Dirnen io ns l
55. Con ider an m n m atri x A. Show tbat the rank of A is m if (and ? nl y if) A
has an invertibl 111 x m ubmatri · i.e .. a matri x obtained by de le t1ng n - 1n

col umn of
111
56. An 11 x 11 matrix A i called nilporenr if A = 0 fo r some positive integer
m . Examp le are trian gular matri ces who e entri o n th di agona l are all
0. Con ider a nilpotent n n man·i x A, and choo e the rn a ll~st nu mber m
' UCh that A"' = 0. Pick a ve tor - in IR" uch that Am- li) =!= 0. Show that.
the vector v, Av, A 2 ü, ... , A"' _ 1 _ are linearly inde_pendent. Hinr : Co nsider
ORTHOGONALITY
v
a re latio n co - + c 1 A v + c2 A2 - + · · · + c", _ 1Am - I = 0. Mul tiply both ides of
the equation with Am-I to show that co = 0. ext, how that c1 = 0, etc. AND LEAST
57. Co nsider a nilpotent n x n matrix A ( ee Exerc i e 56). e the re ult demon-
strated in Exerci e 56 to how that A" = 0.
58. Explain why yo u need at lea t d vecto rs to span a space of d imension d
SQUARES
(Fact 3.3 .4b).
59. Prove Fact 3.3.4d: If d ve tor span a d-dimensional space, they fo rm a ba is
of the space. ORTHONORMAL BASES
3
60. If a 3 x 3 matrix A repre e nt the orthogo nal proje tio n onto a plane in !R ,
AND ORTHOGONAL PROJECTIONS
what is rank (A )?
61. Consider a 4 x 2 matrix A and a 2 x 5 matrix B. Not all bases are created equal. Wben working in a plane E in 11(3 , for example,
a. What are the po sible dime nsio ns of the kerne! of A B ? it is particularly convenient to use a basis ü1, ü2 consisting of two perpendicular
b. What are the pos ible dimensions of the image of A B ? unit vectors (see Figure 1).
62. Tru e or f alse? There is a 7 x 6 matrix A with rank (A ) = 4 and a 5 x 7 m atrix In thi s section we will develop some vocabulary to deal witb uch bases.
B with rank B ) = 4 such that BA is the zero matti x. Then we will use a basis consisting of perpendicular unit vectors to study the
63. Co nsider an m x n matri x A and an n x p matrix B. orthogonal projection onto a subspace. In the next section we will construct such
a. Wbat can you say abo ut the re lationship between rank (A) and rank(A B)? a basis of a sub pace of IR" .
b. What can you say abo ut the relationship between rank( B) and rank(A B)? First, we review some basis concepts.
64. Consider two m x n matrices A and B . Wbat can you say abo ut the relationship
between tbe quantitites rank(A) , rank(B), and rank(A + B )? Definition 4.1.1 Orthogonality, length
65. Consider a 4 x 5 matrix A in reduced row echelon form. Suppose A has the
same kerne! as the matri x
a. Two vectors v and win IR" are called perpendicular or orthogonal 1
if
v·w=O.
B=
0 1 0
0 0 1 3 0
0 0 0 0 1 .
2 0] b. The length (or norm) of a vector v in II(" is llilll = ~-
c. A vector ü in II(" is called a unit vector if its lengrh is I, that i , IJ ÜII = 1,
[ 0 0 0 0 0 or ii · ü = 1.
Show that A = B. Hint : Think about relations among the columns of A .
66. Consider two m x n matrice A and B in reduced row ecbelon form. Show lf vi a nonzero vector in II(" , then
that if A and B have the same kernet , then A = B. U e Exercise 65 as a
u- = - 1 v-
guide. IJÜIJ
67. Suppose a matri x A in reduced row eche lon form can be obtained from a is a unit vector (Exercise 25b).
matrix M by a sequence of elementary row Operations. Show that A =
rref(M) .

1The two terms are synonymous: "perpendicu lar'" comes from Latin and .. orthogo nal'" fro m G ree k.
177
178 • hap. 4 Orthogonality and Lea t qua res Sec. 4. 1 Ortho normal Bases and Orthogona l Projections • 1 79
The fo iJ ow i.ng properties of orthonormal vectors are often u eful :

Fact 4.1.3 a. Orthonormal vectors are linearly independent.

b. Orthonormal vectors ü1, • •• , Ü11 in IR" form a basi of lR11 •

L...-----~ l' t

Proof a. Con sider a relation

Figure 1

among the orthonormal. vectors ü1, ü2 , ..• , Ü111 in IR.11 • Let us form the dot
Definition 4.1 .2 Orthonormal vectors product of each side of thi s eq uation with ü;:
The ectors ü1, ü2 ... , -~~~ in IR11 are called orth onom w / if they are all unit
ector and orthogonal ro o ne another:
ifi = J, Becau e the dot product i di tri butive (see Fact AS in the Appendix),
if i =/= j.
c , (ü, · Ü;) + c2(Ü2 · Ü;) + · · · + C;(Ü; · Ü;) + · · · + C 111 (Üm · Ü;) = 0.

EXAMPLE l ... The ectors e e1 • . .. , e

1• 11 in ~" are orthononnal. We know that Ü; · Ü; = 1, and al1 otber dot products are zero. Therefore,
c; = 0. Since this bolds fo r aU i = 1, . .. , m, ü follows that the vector Ü;

EXAMPLE 2 ... For any scalar a the vector [ co.

IO Ct'
, a] [-sina] CO Ct'
.
are orthononnal ( ee F1 gure 2 .
....
are linearly independent.
b. Thi fo ll ows from part a and Summary 3.3.11 (any n linearly independent
vector in ][{II form a ba is of lR"). •

EXAM PLE 3 ... The ectors

l J
l l/2J
l l/2J Definition 4.1.4 Orthogonal complement
- l /2
1/ 2 l/2 - - 1/2
u, = ~~ . - 1/ 2 , V3 = 1/ 2 Consider a subspace V of ][{". The orthogonal complement V .l of V is the
- 1/ 2 - l /2 set of those vectors ,r in IR" which are ot1hogona1 to all vectors in V:
4
in IR are orthonormal (verify this) . Can you find a vector ü4 in 1R4 such that all v .L = {x in ][{17 : ü · ,t = 0, for all v in V}.
v v
Lhe vectors ü1, 2 , 3 , Ü4 are orth onormal (Exercise 16)? _...

Figure 2 v
If we have a basis Ü1, 2 • . . . , Ü111 of V , then v.1 i the et of those vector
x in IR" whi h are orthogonal to all the Ü; (Exercise 22):
-sin a J
[ cos (l V .l = {- .
x tn 1[])11
JN>. :
-
u; · x- = 0 , for
. 1. = I , ... , m J.

cos a ] For example, a vector .~ in IR 3 is orthogonal to all vectors in a plane E if (and

[ stn a
only it) i.t is orthogonal to two vectors ü1, ü2 that form a ba ·i of E (see Fig-
ure 3).
In Figure 4, we sketch the orthogonal complements of a line L and o f a
plane E in JR 3 . Note tbat both L.l and E.l are subspace of IR 3 .
180 • Chap. 4 Orthogonality and Least Squares Sec. 4.1 Ort honormal Bases and Orthogonal Proje tions • 181
X
x-w X

Figure S V

This vector ÜJ is cal led the orthogonal projection of; onto V, denoted by proj 11 ; .
Figure 3 Jn Fact 2.2.5 we stated the fo nn ula
x (v1·x)v 1.
proj 11 =
Now co nsider a subspace V of ii{" with arb itrary dimension m . Suppo e we
have an orthonormal bas is Ü1 ~, .. . , Ü111 of V . Con ider a vector _-y: in ". 1 it
w
still pos ible to fi nd a vector ÜJ in V such tb at ; - i in v .L ? If uch a ex i t w
we can wri te

- 0
- 0
for some sca.lars c; (since ÜJ is in V). It is required that

E
X- ÜJ = X- CJ Ü1 - CzUz - · ·· - C111 Vm
L""
Figure 4 be perpendicul ar to V, that is, perpendicul ar to ali the vectors v;:
V;. (x - w) = Ü;. (x - CJ VJ - ... - C;V; - ... - CmVm)

Fact 4.1.5 If V i a subspace of iR", theu it orthogonal complement v .L is a sub pace = v; ·-r -CJ(U; · Ü1)- · · · - C;(Ü; · Ü;)- · · · - C111 (Ü; · Üm)

of lR" as weil. = Ü; ·X - C; = 0 •
Thi s equation holds if (and only if) c; = v; .x.
Proof We will verify that v .L is closed under scalar multiplication and Jeave the verifi- We have sbown that there is a unique wm V uch that ,r - w is in V .L ,
namel y,
cation of the rwo other properties a Exercise 23. Consider a vector win v .L and
v
a scalar k. We have to show that kw is orthogonal to all vectors in V. Pi ck an w= (vl . x)vl + ... + (Ü", ..r)Üm .
arbitrary vector v in V. Then (kw) · v = k(w · v) = 0, as claimed. •
Let us summarize.

+ Orthogonal Projections
Fact 4.1.6 Orthogonal projection
As mentioned at the beginning of thi s section, it is often con venient to work with
an orthononnal basis of a subspace V of iR" (an orthonormal basis is a bas is of v v
Consider a subspace V of lR" with orthonormal basis 1, 2 , .•. 111 • For any v
V th at consists of orthonormal vectors). In the next secti on, we will see how vector -r in IR" there is a unique vector w in V such that _-y: - w is in v .L.
such an orthonormal bas is of a subspace can be fo und. Assuming we have an This vector w is called the orthogonal projecrion of; onto V, denoted
orthon ormal basis of a subspace V of lR", we will now examine how we can find by proj 11 .r. We bave the fo rmula
the orthogonal projecti on of a vector in IR" onto V. proj vx = Cv1 · ,r) üJ + · ··+(um · x)vm.
Before we deal with the general case, Jet us brieft y revi ew the case of a one-
dimensional subspace V of !R". In thi s case, an orthonormal bas is of V consists of The transformation T (x) = proj 11 ,r from lR" to lR" is linear.
v
a single unit vector, 1 • In Secti on 2.2, we fo und that for any vector ; in lR" there
is a unique vector ÜJ in V such that x- ÜJ is perpendicul ar to V (see Figure 5). We Jeave the verifi ca üon of rhe last assertion as Exerci e 24.
182 • Chap. 4 Orthogooality aod Least Squares Sec. 4.1 Orthono rmal Ba es and Orthogonal Pro ject ions • 183
The n

· - c- -)- c- - -
proJ vX = VJ·X v1 + Vz·x )vz = 6-v1+ 2v- 2 = [ ~] [ -~]
~ +_i [~] = ~ .

To check this ans wer, verify that x- proj vx i perpend icular to both ü1 and ii2 .
,'
....
,'
What happens wheo we apply Fact 4.1.6 to the su bspace V = lR" of lR" with
orthonormaJ ba is ü~. ü2 , . . . iin? Clearly, proj vx x
= ,r , for all in lR" . Therefore,

Figure 6
[:l [~ ] for all x in IR" .
x = Cii1 · x)ü1 + ... + Ciin · ,t)ün ,

Fact 4.1.7 Consider an orthonormal basis ü1, . . . , Ü11 of IR" . Then

Note that proj vi i tbe sum of aJ I vectors (Ü; · ,t) Ü; repre e nti ng the orthog- ; = (Ü J . x)iiJ + ... + (Ü/1. x )Ün
x
onal projections of onto tbe Lines spam1ed by the vectors Ü;. This i perhaps a
for aJ I x in IR".
good way to memorize the formula for proj 1 ,x.
For example, projectiog a vector in 1~3 onhogonally oot.o the x 1-xz plane
amounts to the same as projecting it onto the x 1-axis, then onto the xz -axis, and Thi means that if you proj ect .r onto all the lines panned by the basi
then adding the resultant vector (see Figure 6). vector Ü; and add the resullant vectors, you get the vector x
back. Figure 7
illustrates this in the case n = 2.
EXAMPLE 4 ~ Con s ider the subspace V= im (A) of ~4 • where Wh at is tl1e practical ignificance of Fact 4.1.7? Whenever we have a basi
x
ü1, ... , Ü11 of IR", any vector in ffi. 11 can be expressed uniquely as a Linear
1 1] 1 - 1 combination of the Ü; , by Fact 3.2.7 :
A = [i -~ .
Find proj vx, for Figure 7

Solution
The two columns of A form a basis of V. Since they happen to be orthogonal
we can construct an 01thonormal basis of V merely by dividing these two vectors
by their Iength (2 for botb vectors):

-
l/2] - -l/2
1/2
l/2]
VJ = l/2
[1/2 ' [ - ~~; . Vz =
0
184 • Chap. 4 Orth ogonality and Least quares Sec. 4.1 Orthonormal Bases and Orthogo nal Pro jections • 185
To fi nd the calars C; . we need to so lve a lin ear ys te m, w hi ch may invo lve a fa ir
amount o.f compu tati on. However. if the ba is Ü1• .. • , ü" i o rtho no rmal, we can
find the ·; much more eas ily: translated 5·
C; = V;· X

~ [ ~] a
Figure 9 X

EXAMPLE 5 .. By u ·ing paper and pencil , expres the vectm X a linear combination

of
Fact 4.1.8 Theorem of Pythagoras
-
Ui = 3l [ I '
; ] ih- =~[-~]
3
2
'
Con ider two vectors x and y in ffi: 11
• The eq uatio n

11x + .YII 2 = llxll 2 + II.YII 2

holds if (and onl y if) x and y are orthogonal (see Fig ure 9).

Solution
Since v1• Ü] . v3 i an orthononnal basi of JR 3 , we have .Proof The veri fica tion is straightforward :

11 -r + .Y II 2 =c.x + )i) · c.x + )i) = .x + 2c.x .

.'i · )i) + .Y . .Y = llxll
2
+ 2 (.'i . )i) + 11511 2
= llx ll2 + II.YII2 if (and only if) .X· y = 0 •

Now we can generali ze Example 6.

+ From Pythagoras to Cauchy
EXAMPLE 6 ~ Co nsider a line L in JR3 and a vector x in JR 3 . W hat can you say about the
relation hip between the length of the vectors x and projJ ? Fact 4.1.9 Consider a subspace V of IR" and a vector x in lR 11
• Then,

ll proj v.'ill :::: ll.r ll .

The Statement is an equ al ity if (and o n.ly if) .~ is in V .
Solution
Appl ying the theorem of Pythagoras to the shaded right tri angle in F ig ure 8, we
x
find that Jl proj ur ll :::: ll xll . The statement is an equality if (and on ly if) i in ~ Proof We can wri te .X = proj v.X + (x- proj v.r) and appl y the theorem of Pythagoras
(see Fi gure 10).

Does this inequality hold in higher-dimensio nal ca es? We have to examine 11-rll 2 = II proj 1, .rll 2 + 11-r - proj 11 .r ll 2
whether the theorem of Pythagoras holds in IR".
lt fo llows that II proj 11 ,t ll :::: llxll , a claimed. •
Figure 8 Figure 10

x - proj .r
1, (lranslated)

L
186 • Chap. 4 Orthogonality and Least Squa re Sec. 4.1 Ort honormal Bases and Orthogonal Project ions • 18 7
y

\1

Figure 12

Figure II This formula allows us to find the angle between two nonzero vector x and
y in ~3 :

For example, Iet V be a one-di men ional sub pace of ~~~ panned by a X· y x ·y
(nonzero) vector )i. We introduce tbe unit vector
cos (X = -=-- :- or a = arcco
lli ii ii.Y II 11-tii iJy iJ .
- 1 - In lR", where we have no intui tive notion of an angle between two vecto r , we
u = IJ )i ll y ca n use this form ul a to define the angle:
in \1 (see Figure 11 ).
We k:now that
Definition 4.1.11 Angle between two vectors
proj v.r = Cü . x )ü = ~ CY. x)y ,
IJ y ll- Consider two nonzero vec tors xa nd y in !R". The angle a between the e
for any .r in ~~~ . Fact 4.1. 9 teils us that vectors i defined a
X· )l
ll.ill:::: II proj vxll = 1[ ; 112 cY · x)jil\ = ll; ll 2 15i · .rl ll5i ii- a = arccos lli iiii.YII .
11
To justify rhe last step, note that llkiill = lk l ll vll , fo r all vector v in !R" and all
scalars k (Exercise 25a). We conclude that We have to make sure that
lx · .Y I _ X·y
m s llxii- arccos _ _
llxiiii.YII
Multiplying this equation by II .YIL we find the follow.i ng useful inequality:
is defined, i.e., that
X·)l
Fact 4.1.10 Cauchy-Schwarz inequality 1
IJxl l ll)ll
lf x and ji are arbitrary vectors in !R", then
is between -1 and 1 or, equivalently, that
Ii · .Y I s lli ii ii.Y II-
This statement is an equality if (and only if) x and ji are para lleL !11,:11·11~11/- 11ir11· 1 ~11 s L
This follows from the Cauchy-Schwarz inequality, l.r · _YI s ll.i ii ii.Y II (Fact 4 . 1. 10):
Consider two nonzero vectors x and ji in ~ 3 . You may know an expression divide both ides by ll,riiii.YII-
for the dot product x·
ji in terms of the angle a between the two vectors (see
Figure 12): EXAMPLE 7 .... Find the angle between the vectors
x · ji = llxiiii.Y II cosa.

1
Named after the French mathcmatician Augustin-Lou is Cauchy (1789- 1857) and the German math -
emat ician Herman n Amandus Schwarz ( 1843- 192 1).
and
i = [iJ
188 • Chap. 4 Orthogonality and Least Squares Sec. 4.1 Orthonormal Bases and Orthogonal Projections • 189
Solution Ca ncer rate
(de viation
X·V [ X from mean)
a = arccos ll.ill ll._vll = arccos "]":2 = 3 United Statcs
•
Here is an application to statistics o:f some concepts introduced in this section.
10

•
Consider the meat consumption (in grams per day per person) and incidence of Great Britain Meat consurnption
colon cancer (per 100,000 women per year) in various industrialized countries: (de viat ion from mean)

- 100 e 100
Country Meat consomption Cancer rate I rael

Japan 26 7.5 e
Finland
Finland 101 9.8 e -10
Israel 124 \6.4 Japa n
Great Britain 205 23.3
United States 284 34

Mean 148 18.2

Figure 13

Can we detect a positive or negative correlation 1 between meat consumption

A positive correlation is indicated when most of the data points (in our ca e,
and cancer rate? Does a country witb high meat consumption have high cancer
all of them) are 1ocated in the fir t and third quadrant.
rates, and vice versa? By "high" we mean "above average," of course. A quick
To process these data numerically, it is convenient to represent d1e de-
Iook at the data showssuch a positive correlation: in Great Britain and the United
viation for both characterist:ics (meat con umption and cancer rate) as vectors
States, both meat consumption and cancer rate are above average. In the three
in JR5 :
other countries they are below average. This positive correlation becomes more
apparent when we Iist the data above as deviations from the mean (above o.r below - 122 -10.7
the average). 47 8.4
X= - 24 y= - 1.8
57 5.1
Meat consumption Cancer rate 136 15 .8
Country (deviation from mean) (deviation from mean)
We wiJl call these two vectors the deviation vectors of the two characterisücs.
Japan -122 - 10.7 In the case of a positive correlation. most of the cotTesponding entries x,-, y;
Fin land - 47 - 8.4 of the deviation vectors have the same sign (both positive or both negative). In
Israel -24 - 1.8 our example, thi s is the case for al l entries. This means that the product x; y,- will
Great Britain 57 5.1 be positive most of the time; hence, the sum of all these products will be positive .......
United States 136 15.8 But thjs sum is simply the dot product of the two devi ation vectors.
Still using the term "correlation' in a colloquial sense, we conclude the
following:
Perhaps even more informative is a scatter plot of the deviation data (Fig-
ure 13).
Consider two characteristics of a population, with deviation vectors ,( and )i.
There is a positive correlation between the two characteristics if (and only if)
_\: · .Y > o.
1
We are using the term "correlation" in a colloquial, qualitative sense. Our goal is to quantify this
tenn.
190 • Chap. 4 Orthogonality and Least Sq uares Sec. 4.1 Orthonormal Bases and Orthogonal Pro jections • 191
Acute angle Righ1 angle Obw se angle
r= I r= - 1

X
X
.r
Figure 15 (aI y= mx, for positive
m. (b) y= mx, for negative m. (a) (b)

.Y y
y y r=l Y;= 111.X; r=- 1
w ~ ~
Figure 14 (o) Positive correlation: x. y> 0. (b ) No correlotion: x· y= 0. (c) Negative correlation: x· y < 0.

A positive correlation between tbe characteristics means that the angle cx

between the deviation vectors i le tban 90° (see Figure 14).
We can u e the cosine of the aDgle a between .:r and :li as a quantitative
n1easure for tbe correlation between the two characteristics.
Figure 16

Definition 4.1.12 Correlation coefficient

The correlation coefficient r is always between -1 and 1; the cases when
The correlation coefficient r between two characteristics of a population is r = 1 (representing a perfect positive correlation) and r = -I (perfect negative
the cosine of the angle a between the deviation vectors .X and y for the two correlation) are of pruticular interest (see Figure 15). In both cases, the data points
characteristi cs: (x; , y1) will be on the straight line y = mx (see Figure 16).
X·y
r = cos(a) = - --
Jixii ii.YII
EXERCI S ES
GOALS Apply the basic concepts of geometry in IR" : length angle..s orthog-
onality. Use the idea of an orthogonal projection onto a subspace. Find thi s
In the case of meat consumption and cancer, we find that projection if an orthonormal basis of the subspace is given.

4 182.9
,. ~ -19- 8-.5-3-. -2 1-.5- 3-9 ~ 0 ·9782 ·
Find the length of each of the vectors ü in Exerc ises 1-3.
The angle between the two deviation vectors is arccos(r) ~ 0.2 1 (radians)
~ 120.

Note that the length of the deviati on vectors is inelevant for the corre-
lation : if we had measured tbe cancer rate per I ,000,000 women (instead of
100,000), the vector y would be 10 times Jonger, but the correlati on would be the
same.
 c=z:ss

192 • hap . .f Orthogon ali tya nd Least qua re · Sec. 4. 1 Orthono rmal .Bases and Orth ogonal Proj ection • 19 3
Find the anale a between each of the pairs of vectors ü and v in Exercises 4-6. of the weight. The strin g of the pulley line ha the ame ten ion everywhere ;
"' hence, the fo rces F2 and F3 have the same magnitu cle a F1. A su me that the

4. I; = [ : lv n;
= [ 1
magnitude of each force is 10 pounds. Find the angle et so tbat the magni tude
of the force exerted on the leg is 16 pound . . Round your answer to the nearest

l.
degree. (Adapted fro m E. Batsehelet lntroductiort to Ma thernatics fo r Life

~ ~i ~ m
Scientists, Spri nger, 1979).

6. ü [

v
Foreach pair of vector 1;, li sted in Exerci es 7-9, detemline w he the r the angle
et v
bet:ween ii and is acute. obtuse, or rigbt.

7. ü = [ _; ] v= [ ~] ;

9.ü~L1J , ~[il
10. For which choice( ) of the constant k are the vectors

perpendicular? 14. Leonardo da Vinci and the resolution offorces. Leonardo (1452- 1519) a ked
11. Con ider the vectors hirnself how the weight of a body upported by two strings of different lengtb,
is apportioned between the two string .

in IR". 8 E A

a . For n = 2, 3, 4 find the angle between ü and v.

For n = 2 and 3, repre ent Ionger Lring shoner string
the vectors graphically.
b. Find the Iimit of this angle as n approaches infinity.
12. Give an algebraic proof for the triangle inequality weig ht

llii + wll :::: llii ll + llwll-

Three force are acting at tbe point D : the ten ions F 1 and F~ in l.he Lring
Draw a sketch . Hint: Expand llii + wll 2 = (ii + w). (ii + w). Then use the and the weight W. Leonardo believed that
Cauchy-Schwarz inequality.
13. Leg traction : The accompanying figure shows how a leg may be stretched IIFdl EA
by a pulley line for therapeutic purposes. We denote by F1 the vertical force IIF2II EB
 194 • Chap. 4 Orthogonality and Least Squares
Sec. 4.1 Orthonorma l Base and rthogonal Projection • 195
B E A
a. Check that ,r = ( I . !, ! , i· i6 , ...)
is in e2 , and find llxll . Reca llth formul a
for the geo metri c seri es: I + a + a 1 + a 3 + · · · = 1/( I - a , if - I < a < 1.
b. Find the ang le between ( I , 0 , 0 , . . .) and ( I , ~ .!· k· .·..
c. Givc an cx.ampl e of a seq uencc (x 1 , .x2 , . . . that c nvergcs t (th at is
X 11 = 0) but does 1101 belang LO f2.
D
lim 11
d. Let L be the subspace ofe 2 spanned by ( 1. 2I , 4I , 8I , . . . . p·1nd tI1e 11 hogona I
w projecti on of ( I , 0, 0, ... ) o nto L. .
The Hilbert pace e2 was in it ial ly used mo tly in phy ic : Werner !"1e1 en-
Wa he righr? Source: Les Manuscrits de Uonard_ de Vin ci, publi hed by berg ' s formulation of quantum mechani c i in term of f2. Today, th 1 pace
Ravai sson-Mollien, Pari 1890.) Hi11ts: Re olve F 1 into a horizonral and i used in many other applicat ion , including econom ics ( ee, fo r examp1e,
a ertical component" do the ame for F2. Since the y tem i at re t, the the work of the Catalan econorni t Andreu M as-Co lel l).
equation F1 + F2 + ~V = Öhold . Expres the ratio 19. For a Jine L in JP. 2 , draw a ketch to interpret the foll win g tran Formation
geometrical ly:
EA a. T(x) = ; - proh.X .
and
EB b. T (x) = x- 2 proj'"i.
c. T; = 2 proj'"; - i .
in terms of a and ß. u ing trigonometric function , a nd compare the re ult .
15. Con ider the ector 20. Refer to F igure 13.

~
- lil
Cancer rate
(deviation
from mean
Un ited States
4 •
Find a basis of the sub pace of 1!( con isting of all vectors perpendicular to ü.
16. Con ider the vectors
10

-= 1/2j2l
1 lj2l
1/2 ' - = -1/2
1/2 •
r
VI Great Britain

ru
VJ Meat con umption
1/2 - 1/2
(deviai io n from mean

4
4
in Ii{ . Can you find a vector u4 in 1!( such that the vectors 1
, i:i2 , iJ3 ü4 are - 100 • 100
orthonormaJ? If o, how many such vectors are there? Israel
17. Find a basis for w.L where
•
Finland
.. - 10
Japan

18. Here i_s an _"infinite-dimensional" version of Euclidean space: in the space

of all mfi~Jte sequences, consider the subspace e2 of ". quare-summable" se-
quences, I.e., those sequences (x1, x2, . .. ) for which the infinite series xf +
xi + · · · converges. For .X and ji in e2 we define The Ieast-squares /in e for tbe e data is the line y = mx that fit the da_ta best,
in rhat the um of t.he square of the verticaJ distances between the Im and
II.XII = V/x +x 22 + · · ·, x- . y- -x
- I y I + x2Y2 +
2

1
.• .. the data point is minimal. We want to minimi ze the sum
(Why does the series x1 Y1 + x2Y 2 + ... converge?) (mx 1 - Y1 )
2
+ ( m x2 - n)
2
+ · · · + (mxs 2
- Ys) ·

'-------
196 • Chap. 4 Orthogonality and Least Squares Sec. 4.1 Orthonormal Basesa nd Orth gonal Projections • 19 7
25. a. Con ider a vector ü in IR", and a ca lar k. Show that
V = /IJ.X
llkvll = lkl llüll .
11LY; -- b. Show th at if ü is a non zero vector jn lR", then ü = 1 ~ 1 ü is a unit vector.
!}
:~ ] onto the sub pace of 111
tn.r;- Y;
... - -- --- -
- - -----
:"
: (x;.)';)
26. F;nd the orthogonal pmjecüon of [ 3
spanned by

[-H
X;

and

In vector notation, to minimize the sum mean to find the scalar m such that
27. Find the orthogonal projection of 9e1 onto the sub pace of JR 4 spanned by
llmx - .YII 2
is minimal. Arguing geometrically, explain how you can fi nd m. U e the
accompanying sketch, which is not to scale. and

28. Find the orthogonal projection of

Find m numerically, and explain the relationship between m and the correla- onto the ub pace of JR 4 panned by
tion coefficient r. You may find the following information helpful :
x·y= 4182.9, 11x11 ~ 198.53, II.YII
To check whether your solution m is reasonable, draw the line y = mx in the
scatter plot reproduced at the beginning of thi exercise. (A more thorough
discussion of Ieast-squares approximations will follow in Section 4.4.)
~ 21.539.

[!J. Ul [=lJ
29. Consider the mthonormal vectors VI, Li2 , Li 3 , Ü4, ü5 in JR 10 . Find the length of
21. Find scalars a, b, c, d, e, j, g uch that the vectors the vector

30. Consider a subspace V of I!(", and a vector in I!(". Let x .v = projvx· What
are orthonormal. is the relation shjp between the following quantitie ?
22. Consider a basi Üt, ~ •... , ü", of a subspace V of IR". Show that II.YII 2 and y ·x
V .L = { x in I!(" : Ü; · x ~ 0 for all i = I , . .. , m} . 31. Con .ider the orthonormal vectors VI, Ü2 •... , Ü111 in IR" , and an arbitrary vector
23. Complete the proof of Fact 4.1.5: the orthogonal complement v.t of a ub- xin lR". What is the relationship between the two quantities below?

+ cV2- c-
space V of IR" is a subspace of lR" as weil. -)2 + · · · + _)ry
p= ( -VI ·X)
- 2
·X V", ·X - and
24. Complete the proof of Fact 4.1.6: orthogonal projections are linear Irans-
formations. When are the two quantities equal?
198 • Cha p. 4 Orthogonality and Least quares
Sec. 4.2 Gram-Schmidt Processand QR Factorizati on • 199
V V
32. Con ider two vector ii 1 and ü1 in IR!" . Fonn the matrix

For wluch choice of ü1 and ii2 i the mattix G invertible?

33. Con ider a plane E in JR! 3 witb orthonormal ba i ü,, ii2. Let ."t be a vector in
JR!3 . Find a formula for the reflection R(x ) of -~ in the plane E . 0
34. Con ider three unü vectors ü1• ü2 , ü3 in IR!" . We are told that ii , · ü:! = ii, · ii 3 =
1/2. What are the po sible value of Ü2 · Ü3? What cou ld the angle between Figure 1
the vector ii2 and ü3 be? Gi e examples; draw ketche for the ca e n = 2
and n = 3.
35. Can you find a line L in IR!" and a vector x in IR!" such thar V V

is negative? Explain , arguing aJgebraically.

0 0

Figure 2
2 GRAM-SCHMIDT PROCESS
AND QR FACTORIZATION
In tbe last section we have seen that it is sametime useful to have an orthonormal
The last tep is easy : we divide tbe vector il2 - projL ii2 by its ]ength to get
basis of a subspace of IR!" . Now we how how to construct such a basis. We pre ent
an algorittun that allows us to convert any ba ·is ii1. ü2, . . . . Ü111 of a subspace V
w
the second vector 2 of an orthonormal ba i (see Figure 3).

of IR!" into an orthononnal basis ÜJ 1 ÜJ2 .. .. , W111 of V .

Let us first think about simple cases. If V is a line wi th basis ii 1, we can
find an orthonormal basis w1 simply by dividing ii 1 by its length:

Figure 3
When V is a plane with basis ii 1 ü2 , we first divide the vector ii 1 by its length to
V V
get a unit vector

(see Figure 1).

Now comes the crucial step: we have to find a vector in V orthogonal to
ÜJ , (initially, we will not insist that th is vector be a unit vector). Recalling our
work on orthogonal projections, we realize that ii 2 - projL ii 2 = ii2 - (w 1 . ii 2)ÜJ, i
w
a natural choice, where L is the line spanned by 1• See F.igure 2.
L0
200 • hap. 4 Orthogonality and Lea t q uare · Sec. 4.2 Gram-Schmidt Process and QR Factorizatio n • 201
EXAlVIPLE 1.... Fin d an orLhonorma l ba i of the ubspa e Now that we l<Jlow how to fi nd an o rthonorm al basis of a plane, how wo ul d
we proceed in the case of a three-dimensio nal subspace V of lR" with basis v1, ü2.
w
Ü3? We can first find an orthonorm al bas is w1 , 2 of the pl ane E = span(ü 1 , ~)
as above. T he n, we consider the vector ü3 - proj Eü3 and di vide it by its Jength to
get W3, as show n in Figure 4. (How do we know that ü3 - proj Eü3 is no nzero?)
Reca ll fro m Fact 4.1.6 that
of IR?.\ with bas is Ü3 - proj Eü3 = Ü3- (w 1 · Ü3) w 1 - w2· Ü3)ÜI2.
U ing the same method, we can construct an o rtho normaJ ba i of any ub-
space of lR". Unfo rtunate ly, the notatio n gets a bit heavy in the general case.

Algorithm 4.2.1 The Gram-Schmidt process 1

Solution
Con ·ide r a subspace V of lR" with bas is ü1, ü2 , • •• , Ü111 • We wish to construct
U ing the termino logy in trodu ced above, we find the fol lowing re ul t
w
an orlhonormal basis w1, ÜJ2, . .. , 111 of V . Let Ül 1 = 0/llv lii)ÜI. As
we define w; fo r i = 2, . .. , m, we may ass ume that an orthonormal basis
_ 1 _ 1I //22 ] üi 1, ÜJ2 , . .. , ÜJ;- 1 of V; - 1 = span (ÜJ ü2, .. . , Ür - I) ha already been found .
Let
Wi = ll vdi UI
[ l/2
= 1/2 .
I -
W; = IIV; - proj V,_, v;ll ( U; - proj v,_,Ü;) .

Note that
u;- proj v,_,v; = ü; - (w 1 · v;)w l - · · · - (w; - 1 · v;)w; - ).
by Fact 4. 1.6.

Beca use the computati on of ü2 - proj Lü 2 is rather messy and an error i If you are confused by these fo rmul as, go back to the cases where V i a
ea y to make, it is a good idea to check that the vector yo u fo und is indeed two- or three-dimensional pace.
perpendicular to w1 (or, equi valently, to u1) :
Figure 4

F inall y,

E E E
We have found an orthonormal basis of V:

- 1/2
1/2 ]
Wi = 1/2 '
[ ' amed aftc r the Danis h ac tuary Jörge n Gram ( 1850-19 16) and lhe Genna n mmhematician rhardt
1/ 2 Schmidt ( 1876-1959).
202 • Chap. 4 Orthogonality and Least Squ ares Sec. 4.2 Gram-Schm idt Process and QR Factorization • 203
+ The QR Factorization The verification of the unigueness of the Q R factotization is left as E xer-
The Gram-Schmjdt proces s can be presented succi nctly in m atrix fon~. U sin? the cise ~. 3. 31. To find the QR factorization of a matrix M we pe1form the Gram-
tenninology introduced in Algorithm 4.2.1, let us express the vector v; as a hnear Schnudt process on the columns of M, constructing Q and R column by column.
No extra computations are required: all the information necessary to build M and
combination of tü1.. .. , w;.
R .is pro:'ided by the Gram-Schmidt process. Q R factorization is an effective way
v1= l! v11iw1 and to orgamze and record the work performed in the Gram-Schmidt process; it useful
v; = pro.i v,_1v; + )I Ü; - pro.i v,_1 v; llw; for many computational and tbeoretical purposes .
= (w 1 . v; )w1 + ... + (w;- 1 . i:i;)w;-1+ llv; - pro.i v,_1 ü; llw; EXAMPLE 2 ... Find the QR f actorization for
for i = 2, ... , m.. Let r 11= ll viii. r;; = llv; - pro.i v,_1 ü;ll for i > 1, and l'j; = iiJj ·v;
for i > j. Then
v1= r 11 iü t
v2 = r 12w 1 + rnw2
Solution
We can use tbe work done in Example 1.
We can write tbese equations in matrix form:

l
llii i l/
-!.
- J[ I"J0J
u~
rl" M = QR = [·w1 W2

WJ lV2
-
Wm
1"22 ,..1111
r 2m

0 rn~m ll iiz - proj v, ii2ll

-
1/2
1/ 2 -5/~44l
3/.JM 2 12 J
- 1/2 3/.JM [ 0 .J44
[

M=QR
\/ 1/ 2 -1 //44
Note that you can check this answer.

Note that M is an n x m matrix with linearly iJ1dependent columns, Q is an n x m EX E RC I S ES

matrix with orthonormal columns, and R is a:n upper triangular m x m matrix witb
GOALS Perform the Gram-Schmidt process, and thus find the QR factori zation
positive entries on the diagonal.
of a matrix .

Fact 4.2.2 QR factorization Using paper and pencil, perfom1 the Gram-Schmidt process on the equences of
vectors given in Exercises I to 14.
Consider an n x m matrix M with linearly independent columns 1, . . . , V111 • v
Tben there is an n x m matr ix Q whose column s are orthonormal and an
upper ttiangular m x m matrix R with positive diagonal e ntries such that
M= QR.
This representation is unique. Furthermore,

r 11 = IIÜI II and r;; = llv; - proj v,_1 Ü; II (for i > 1),

where
204 • Chap. 4 Orthogonality and Least Squares Sec. 4.2 Gram-Schmidt Process and QR Factorizat ion • 205

s. [il UJ 30. Consider two linem:ly i~dependent

v
vec tors v1 = [ : ] and
Draw sketches (as 111 Ftgures 1 to 3 of this section) to illustrate the Gram-
Schmidt process for v1, 2 • You need not peiform the process algebraically.
31. Perform the Gram-Schmidt process on the follo wing bas is of lll3 :
v2 = [ ~ ] in 11l2.

10. rll m
I~ [1]· [~] · [_~] ·
Here, a, c, and f arepositive constants and the other constants are arbitrary.
Hlustrate your work with a sketch, as in Figure 4 of this section .
32. Find an Ot1honormal basis of the plane

Xt + x2 +x3 =0.
33. Find an orthononnal basis of the kemel of the matrix

~l
Using paper and pencil, find the QR factorizations of the matrices lD Exer-
cises 15 to 28 (compare with Exercises 1 to 14). A=U -i - 1

-n
34. Find an orthonormal basis of the kerne! of the matrix

16. [~ 17.
u_:n A=[i ~ ~ ~l

19. un 20.
[~ ~ n 35. Find an ortbonormal basis of the image of the matrix

A=[; 2 -2
70 i].
r~ ~J . 23.
[~
1 - 5 !] .
36. Consider tbe matrix

~ [~
22. I
2 -2 1 3 - 1
M=
2 1 -1

~- [~ j] 26.
[
2 4]
3
0
6 13
4
2 .
1
Find the QR factorization of M.
37. Consider the matrix
1

~- rt ~ n M ~ ~ [i
1 1
- 1
- 1
- 1
I
-1
-l] [HJ.
- 1 0 0
2
29. Perform the Gram-Schmidt process on tbe basis of l!l below: Find tbe QR factorization of M .
38. Find the QR factorization of

- [-3]
VJ = 4 ' -3
0
0
Illustrate your work with sketches, as in Figures 1 to 3 of this section. 0
206 • Chap. 4 Orthogonality and Least Squares
Sec. 4.3 Orthogonal Transfo rmations and Orthogona l Matrices • 20 7
39. Find an orthonormal ba is w1. w2. w3 of IR3such thal 3 ORTHOGONAL TRANSFORMATIONS
AND ORTHOGONAL MATRICES

fn geometry, we are particularly interested in those linear Iran sformations which

preserve the le ngth of vectors.
and

span (W,, W2) ~ span (UJ. [_

[]) Definition 4.3.1 Orthogonal transformations and orthogonal matrices
A linear transformation T from IR" to IR" is called orthogonal if it preserves
40. Consider an invertible n x n matrix A whose column are orthogonal but not the le ngth of vectors:
neces arily orthonormal . What doe the Q R factorization of A Iook like?
41. Consider an invertible upper triangular n x n matrix A. What does the Q R fac- IIT(x) ll = llx ll,
torization of A Iook like? for all .K in IR".
42. The two column vectors ü1 and ü2 of a 2 x 2 matri x A are shown in the figure lf T (x) = Ax is an orthogonal tran format.ion, we say tbat A an
below. Let A = Q R be the Q R factorization of A. Represent tbe diagonal orthogonal malrix.
entries r 11 and r22 of R a length in the figure. Interpret the product ~"11r22
as an area.
EXAMPLE 1 ..... The rotation

is an orthogonal transformation from IR2 to IR2 , and

A = [c?sifY - inqy]
Slll f/Y cosqy
an orthogonal matrix, for all angles f/Y.
43. Consider a partitioned marrix
EXAMPLE 2 ..... Consider a ub pace x Jvx -x
V of ~" . Fora vector in IR" , the vector R(,r) = 2 pr
A= [ AI A2 ] i called the reflection of .X in V (compare with Defi nition 2 .2 .6; ee Figure I .
with linearly independent column (A 1 is an n x m 1 matrix and A 2 is n x m2). Show that reflection are orthogonal tran formations .
Suppose you know tbe QR factorization of A. Explain how this allows you
to find the QR factorization of A 1•
Figure 1
44. Consider an n x m matrix A with rank(A) < m. Is it alway possible to write
the matrix A as
A = QR
where Q is an n x m matrix with ortbonormal columns and R is upper trian-
gular? Explain.
45. Consider an n x m matrix A with rank(A) = m. Is it always possible to write
Aas

A = QL

where Q is an n x m matrix with ortbonormal columns and L is a Jower

triangular m x m matrix with positive diagonal entries? Explain.
 Sec. 4.3 Orthogonal Transformations and Ort hogona l Matri ces • 209
208 • Chap. 4 Orthogonality and Lea t Squares
The two shaded triangles are congruent, because corresponding ides are the
Solution a.me length (s ince T preserves length). Since D 1 is a ri ght tri ang le, o is D2 •
we can w1i te R(.r ) = proj 11 x + ( proj 11 x- .r) and .r = proj 11.r + (x - proj 11 .r). By Here is an alternative characteri zalion of orthogonal transformations:
the theorem of Pythagoras, we have
2
II R(x) \1 2 = II proj 11 xll 2 + II proj v-r - .rll Fact 4.3.3
2 a. A li near transformation T from IR" to IR" is orthogonal if (and onl y if) the
= II proj 11 .r ll- + 11-r - proj .r f = 11-rll .
vectors T(e,), T (e2) , .. . , T (e11 ) form an orthonormal basis of IR11 •
As the name sugge ts orthogonal transformations preserve right ang les. In b. An n x n. matrix A is orthogonal if (and only if) its columns form an
fact, orthogonal tran fo rmations preserve all angles (Exerci e 3). orthonormal basis of IR".

Figure 3 illustrates part a for a li near transformation from JR2 to JR2 .

Fact 4.3.2 Con ider an orthogonal transformati on T fro m IR to IR". If the vectors
11
v and
iü in IR" are orthogonal, then o are T(u) and T (w). Proof We prove part a; part b then follows from Fact 2.1 .2. If T is orthogonal, then
by de finition the T (e;) are unit vectors and by Fact 4.3. 2 they are orthogonal.
Conversely, suppose the T (e;) form an orthonormal basis. Consider a vector
Proof By the theorem of Pythagoras, we have to sbow that
x = x,e, + x2 e2 + · · · +xnen in IR". Tben

IIT (ü) + T (w)ll 2 = II T (ü)ll

2
+ IIT(w)ll 2 . llx ,T(e,) + x2 T Ce2) + .. · +x"T (en)i1 2
llx , T(e,)ll 2 + \\x2T Ce2)ll 2 + · · · + ll xnT (en)li 2 (by Pythagoras)
Let's see.
x~ +xi + · ·· + x;
!I T Cü) + T Cw)ll 2 II T (v+w f (T is linear)

llü + wll 2 (T is orthogonal) •

= lliill 2 + Hwf (ü and ware 01t hogonal) Wa m in.g: A matrix with orthogonal columns need not be an orthogonal
ii T(v)li 2 + IIT (w)\1 2 (T is orthogonal) • matrix. As an example, consider the matrix A = [ ~ -!l
Fact 4.3 .2 is perhaps berter explained with a sketch (see Figure 2).
EXAMPLE 3 ~ Show that the matrix A is orthogonal.

~~
-1 - 1
Figure 2

T
A
2
[:
l
l
-1
1
1
-1
1
-ll
- 1

üi (translated
-------------- I Figure 3

e2
.....--------..
T
T(ii 2)
210 • hapo 4 Orthogonality and Least Squares Sec. 4°3 Orthogonal Transformation s and Orthogonal Matrices • 211
B.'i This result i no coincidence: the ijth entry of BA is the dot product of the ith
roow of B and the jth column of A By definition of B , this is just tbe dot product
0

of the 1th colum.n of A and the jth column of Ao Since A is orthogonal, this
product is 1 if i = j and 0 otherwiseo -<1111

preserves preserves
Let us generalize:
leng th leng th
ABX

Definition 4.3.5 The transpose of a matrix; symmetric matrices

Figure 4
Consider an m x n matrix A The t ranspose Ar of A is the n x m matrix
0

whose ijth entry is the j ith entry of A 0

Solution We say that a square matrix A is symmetric if AT = A 0

4
Check that the column of A form an orthonormal basi of lR

un
0

Fact 4.3.4
He re are some algebraic properlies of orthogonal matrice

a. The product AB of two orthogonal 11

0

x n matrice A and B is orthogonal.

EXAMPLE S ... If A ~ u; n then A' ~
b. Theinverse A- 1 of an orthogonaln x n matrix Ai oithogonal. EXAMPLE 6 ..... The matrix

Proof In part a, the linear transformation T Ct) = A B.x preserve length because II T (i) II =
1
II A( Bi ) ll = IIB.xll = llx ll o In part b the linear tran formation T(x) = A- x
preserves lengtb because IIA - xll = II A(A- x)ll = 11-tll o Figure 4 illustrates prop-
1 1
is symmetric because AT = Ao
~ao •
Note that the transpese of a (column) vector ü is a row vector: if
+ The Transpose of a Matrix
EXAMPLE 4 ..... Con ider the orthogonal matrix then VT = [ l 2 3] 0

-3
6
2
-n
Form another 3 x 3 matrix B whose ijth entry is the jith entry of A, ioeo, the rows
The transpese gives us a convenient way to express the dot product of two (col-
umn) vectors as a matrix product.

of B correspond to the columns of A 0

Fact 4.3.6 If ü and oÜJ are two (column) vector in IR", then

1[2 3

-n
V o W

B =7 ~ 2
-6 dot product matrix product

Compute BA and explain the result.

Solution For example,

BA=- 1[2
49
6
3 -6
3
2
-n[~
6
2
-3
-~]2 ~ ~f~
49 0
0
49
0
0]
0
49
= /3
 ·-

212 • Chap. 4 Orthogonality and Least Squares Sec. 4.3 Orthogonal Tra nsforma ti onsa nd Orthogonal Matrices • 213
Now we can succinctly state the observation made in Example 4 . Here are some algebraic propertie of transposes:

Fact 4.3.9 a. If A is an m x n matri x and B an n x p matri x, then

Fact 4.3.7 Consider an n x n matrix A . The matrix A is orthogonal if (and only if)
1
AT A = I" or, equivalently, if A- = AT. (A B )T = B T AT.

Note the order of the factor .

b. Jf an n x n matrix Ais invertible, then so is AT, aod
To justify this fact , write A in terms of its column :
(AT) - 1 = (A- t )T.

c. For any matrix A,

rank (A) = rank (AT) .
.A = VI t12

Proof a. Campare entries:

ijth entry of (A B )T = jith entry of AB

Then = (j th row of A) · (i th col umn of B )
-T
u, ijth entry of s T AT= (ith row of B T). (j th column of AT)

l~' ~· - -ü" ]
V I · V2 = (i th column of B ) · (j th row of A)
-r
v2 V2 ·V I V2 · V2 ti1
V2 ·· Vn
A7 A = U1 v2 v" . . b. We know that

v" · v1 v" · v2 Vn · V 11
-r
vn Transpo ing both sides and using patt a we find that
By Fact 4.3.3b this product is I" if (and only if) A is orthogonal. • (AA - I)T = (A-t)TAT = 1" .
Later in this text, we will frequently work with matrices of the fonn AT A. By Fact 2.4.9, it follows that
lt is helpful to think of ArA as a table displaying the dot products v; ·
Vj among
the columns of A, as shown above.
I
We surnmarize the various characterizations we have found of orthogonal
c. Consider the row space of A, i.e., the span of the row of A. It is not hard
matrices.
to show that the dimension of thi s space is rank(A) (see Exercises 49- 52,
Section 3.3):

Summary 4.3.8 Orthogonal matrices rank (A 7) = dimension of the pan of the colum.n of AT
Consider an n x n matrix A. Then the following Statements are equivalent: = dimen ion of the pan of the row of A

i. A is an orthogonal matrix .
= rank(A) •
ii. The transformation L (x) = Ai preserves length, that is, IIAill = llill
for all i in !Rn. + The Matrix of an Orthogonal Projection
iii. The columns of A fom1 an orthonormal basis of IR". The transpese al lows us to write a formula for the matrix of an orthogonal pro-
iv. AT A = I" . jection. Consider first the orthogonal projection
v. A - 1 =AT . proj L,r = (ii 1 . •r)ü 1
214 • Chap. 4 Orthogonality and Least Squares Sec. 4.3 Orthogonal Transformationsand Orthogona l Matrices • 215
v
onto a Jine L in JR", where 1 i a unit vector in L . l f we view ~e vector v1 as Solution
an 11 x 1 matrix and the scalar ü1 . .i as a 1 x 1 matrix, we can wn te
v and v are orthonormal.

l[
Note that the vectors 1 2 Tberefore, the mat:ri x is
proj L.i= = v1(v 1 · .1:)

-~
- -T- 1 0 0
= V] VJ X
I 1 1] 1 0 ] 1
= M Jr -1 11 -1 - 1 1 =2 0 1 1
[
1
where M = V] üf. Note tbat V] is an n X I matrix and vi is 1 X n, so tbat M is 1 0 0
n x n as expected.
More generally, consider the projection EXER C I S E S
proj v.X = (v 1 · x )v1 + · · · + (vm · x)v"' GOALS Use the various characteri zations of orthogonal transformations and or-
thogonal matrices. Find the matrix of an orthogonal projection. Use the properties
onto a subspace V of lR11 with orthonormal basis v1, ... , Vm. We can write of the transpose.
· - - -T- - -T-
prOJ vX = V] VJ X +··· + VmVmX 1. Con ider an m x n matrix A , a vector v in lR", and a vector win JRm . Show
- -T - -T)-:
= (VJ V 1 + ·· · + Vm V X
111 that

( Av). w = v. (Arw) .

2. Con ider an mthogonal transformation L from lR11 to lR" . Show that L pre-
serves the dot product:

v · w= L ( v) · L (w),

fo r all v and win JR".

We bave shown tbe following result:
Fact 4.3.10 The matrix of an orthogonal projection
Consider a subspace V of lR" with orthonormal basis ii1, v2.
matrix of the orthogonal projection onto V is
Pay attention to tbe order of the factors (AAT as opposed to A 7 A).
EXAMPLE 7 ..... Find the matrix of the orthogonal projection onto the subspace of JR4 spanned by
3. Show that an orthogonal transformation L from lR" to lR" preserves angles: the
angle between two nonzero vectors and v w
in lR" equals the angle between
L (v) and L (w). Conversely, is any li near transformation that preserves angles
orthogonal? ·
Vm· The 4. Con ider a linear transform ation L from lR" to JRm which preserves length .
W11at can you say about the kerne! of L? Wllat is the dimension of the image?
What can you say about the relationship between n. and m? If A is the matrix
of L, what can you say about the columns of A? Wbat is A T A? What about
AAT? Jllustrate your answers with an example, where n = 2 and m = 3.
5. If a matrix A is orthogon al , i A 2 orthogonal as weil ?
6. If a matrix A is orthogonal, i A 7 orthogonal as weil ?
7. Are the rows of an orthogonal matrix A orthonormal?
8. a. Consider an n x m matrix A uch tbat A7 A = 1111 • Is it necessarily true
that AA T = 111 ? Explai n.
b. Consider an n x n matrix A such tbat A T A = 111 • Is it necessarily true that
AA T = 111 ? Explain.
9. Find all orthogonal 2 x 2 matrices.
10. Find a11 orthogonal 3 x 3 matrices of the form
a
c
[
e
 Sec. 4.3 Orthogonal Transformations and Orthogona l Matrices • 215
214 • Chap. 4 Orthogonality and Least Squares
onto a line L in JR", where ü1 is a unit vector in L. . 1f we view t~1e vector v 1 a Solution
x
an n x l matrix and the calar ü, . as a I x 1 matr1x, we can wnte Note that the vector v1 and ü2 are orthononn al. Therefore, the matrix i
prohi = ü, (v, · _r)

=
~
= VI VI X
M_i
~r -:
_ .: ] [ 1
- l 1 - 1 - 1
I
l] ~ [~ ~ ~ ~]
1
=
201101'
1 l 0 0
where M = ~, üf. Note that v, i an /1 X l matrix and v{ i 1 X 11, so that M i
n x n as expected. EX ERC I S ES
More generally con ider the projection
GOALS U e the various characterizations of orthogonal transformations and or-
proj 11 -r = (v 1 • i)ü , + · · · + Vm · .r)vm
thogonal matrices. Find the matri x of an orthogonal projection . Use the propertie
onto a subspace V of IR" with orthonormal ba i ii,, . . . , Vm· We can write of the tran spose.

proJ· 11x- = -u,u- r- - -r-

1 X+···+ VmVmX 1. Con ider an m x n matrix A , a vector v in IR", and a vector win IR"' . Show
- -r - -r)- that
= (v 1v 1 + · · · + Vm V 111 X

Av)·ÜJ=v·(ATÜJ) .

2. Consider an orthogonal tran fonnation L from IR" to IR" . Show that L pre-
erves the dot product:
Ü · ÜJ = L(ii) · L (ÜJ),

for aU ü and ÜJ in JR".

3. Show that an orthogonal transformation L from IR" to IR" preserves angle : the
We have shown the following result: v
angle between two nonzero vectors and w in IR" equals the angle between
L (v) and L (ÜJ . Conversely, is any linear transfonnation that preserve angJes
The matrix of an orthogonal projection orthogonal?
Fact 4.3.10
4. Consider a linear tran forrnation L from 1R11 to IR111 which preserves length.
Consider a subspace V of IR" with orthonormal basis ii 1, u2 , Whar can you ay about the kerne! of L? What is the dimension of tbe image?
matrix of the orthogonal projection onto V is What can you say about the relation hip between n and m? If A is the matrix
of L , what can you say about the colurnn of A? What is ArA? What about
AAr? Illustrate your answers with an example, where n = 2 and m = 3.
S. If a matrix A i orthogonal, is A 2 orthogonal as weU?
6. lf a marrix A is orthogonal., is A 7 orthogonal as well?
7. Are the rows of an orthogonal matrix A orthonormal?
8. a. Consider an n x m matrix A such that ArA = / 111 • ls it necessarily true
that AA 7 = 1"? E plain.
Pay attention to the order of the factors (AAr as opposed to ArA). b. Con ider an 11 x n matrix A such that AT A = 111 • Is it neces arily true that
AA T = 111 ? Explain.

EXAMPLE 7 .... Find the matrix of the orthogonal projection onto the subspace of IR 4 spanned by 9. Find all onhogonal 2 x 2 matrices.
10. Find all orthogonal 3 x 3 matrice of the form
216 • hap. 4 Orthogona lity and Least Squares Sec. 4.3 Orthogonal Tra nsformati onsand Orthogonal Matrices • 217
11. Find an orthogonal tran formati on T : ~3 ---+ ~3 uch that 22. Let A be the matrix. of an orthogonal projection . Find A 2 in two ways:

12. Find an
rU~J ~
orthogonal matri x of the form
m a. Geometrically (what happens when you apply an orthogonal projection
twice?).
b. By computation, using the formula given in Fact 4.3.10.
23. Consider a unit vector ü in IR: 3 . We define the matrices
A = 2U.üT - / 3 and B = 13 - 2üüT .
2/ 3 1/-12
2/ 3 - 1/-12 Describe the linear Iransform ations defined by these matrices geometrically.
[ l/3 0 24. Consider an m x n matrix A . Find
13. Is there an orthogonal tran form ation T: ~ ---+ ~ su h that
3 3
dim(im (A)) + dim(ker(AT)) ,

Tm ~ m rnJ ~ HJ ?
in term of m and n .
and 25. For which m x n matrices A does the equation
dim(ker(A)) = dim(ker(A T))
r14. 1f A IS · s11ow that
· any m x n matnx, · the matrix AT A i ynunetric. What about
hold ? Explain.
AA T?
15_•..- If two n x matrice A and B are symmetric, is A B neces arily synunetiic? 26. Consider a Q R factorization
11
16 If an n x n matrix A i ymmetric, is A2 nece arily sy nunetric? M=QR.
17. f an invertible n x n matri.x Ais symmetric, i A- 1 neces aril y symmetric? Show that
18. An n x n matrix A is called skew-syrnmetric if A T = - A.
R = QTM .
a. Give an example of a skew-symmetric 3 x 3 matrix.
2
b. If A is skew-symmetiic, what can you say about A ? 27. If A = QR is a QR factorization, what is the relationship between AT A and
19. Consider a line L in ~~~ , spanned by a unit vector RTR?
28. Consider an invertible n x n matrix A. Can you write A as A = LQ, where
L is a lower triangular matrix and Q is orthogonal? Hint: Consider the
Q R factorization of A T.
29. Consider an invertible n x n matrix A . Can you write A = R Q, where R is
an upper triangular matrix and Q is orthogonal?
Consider the matrix A of the orthogonal projection onto L. Describe the ijth 30. a. Find all n x n matrices whicb are both orthogonal and upper triangular,
entry of A, in terms of the components v; of ii. with positive diagonal entries.
20. Consider the subspace W of ~ 4 spanned by the vectors b. Show that the Q R factorization of an invertible n x n matrix is unique.
1 1
Hint: If A = Q1R1 = Q 2R2, then the matrix Q2 Q1 = R2RI is both
orthogonal and upper triangular, with positive diagonal entries.
and 31. a. Con ider the matrix product Q1 = Q2S , where both Q1 and Q 2 are n x m
matrices with orthonom1al columns. Show that S is an orthogonal matrix.
Hin.t: Compute Q[ Q 1 = (Q 2S l Q2S. Note that Qf Q I = QJ Q2 = Im ·
Find the matrix of the orthogonal projection onto W. b. Show that tbe QR factori zation of an n x m matrix M is unique. Hint:
[f M = Q 1 R 1 = Q2R 2, then Q , = Q2 R2 R~ • Now use part a and Exer-
1
21. Find the matrix A of the orthogonal projection onto the line in ~~~ spanned
by the vector below: cise 30a.
32. Consider the matrix

l~~~- llJ all n components are 1 A=[;; =!]

2 2 0
 Sec. j.4 Least Sq uaresand Data Fitting • 219
218. h ap. 4 Ort hogo nality and Lea t Squares
/
with LDU factorizati on
+ Another Characterization of Orthogonal

: -1]
Complements
I 0I 00 ][10 -~ ~] [~ 2 . Consider a subspace V = im (A ) of IR", where A = [ ij 1 v2
A=
[ 3
2 0 I 0 0 2 0 0 1
V .L = {x i n IR": v· x = 0, for all v in v }
Vm ]. Then

Find the LDU factorizati on of Ar ( ee Exerci e 2.4.58d). = {x in IR": v; · x = 0 , for i = 1, . . . m }

33. Con ider a symmetr ic invertible n x n matrix A whi ch adm it an LDU fac tor- = {.r in lR": vTx = 0 for i = 1
l ' '
,.,1}
• •• ' •
izarion A = LD U ee Exerci e 58, 6 1, and 62 of ection 2.4). Recall that
thi fac torization is uniqu ( e Exer ise 2.4.62). Show that U = LT. (Th.i. In other words v.L is the kerne! of the matri x
1 o metimes called the L D LT Ja ctoriza tion of a y mmetric matrix A.) v,- r
34. T hi exercise shows one way to defi ne the quatemions. d iscovered in 1843
by the lrish mathe mati cian Sir W . R. Hamitton ( 1805- 1865). Consider the
set H of all 4 x 4 matrice M of the form
-q
p
-s
-r
s
p
-sj
- r
1 '
r -q p

where p . q, r, s are arbitrary real numbers. We can write M more uccinctly Fact 4.4.1 . For any matrix A,
in partitioned forn1 a

M = [~
Here is a very simp le example : cons ider tl1e line
where A and B are rotati on-diJ ation matrices.
a. Show that H is clo ed under addition: If M and N are in H , tl1en so i
M+N.
b. Show tl1at H i clo ed under sca1ar multiplicati on: If M is in H and k is
an arbitrary scalar, then k M is in H.
c. . Show that H is closed under m ultipli cation: lf M and N a.re in H , then Then
so is MN . V .L = ker [ 1 2 3]
d. Show tl1at if M is in H , then so is MT.
e. For a matri x M in H , compute MT M. is the plane w ith equation x 1 + 2x 2 + 3x 3 = 0 (see F igure 1).
1
f. Which matrices M in H are invertible? lf M is invertib1e, is M- alway s
in H?
Figure 1 V.L = ker [ I 2 3]
g. lf M and N are in H , doe the equation MN = NM always ho ld ? the plane .r 1 + 2r2 + 3x =0

LEAST SQUARESAND DATA FITTING - the line panned by [ ~3

]

In thi ection, we wi ll pre ent an impo11ant application o f the ideas introd uced in
t11i.. ch~pter. First we ta.ke another Iook at orthogo nal co mpleme nts and orthogonal
prOJeCtJOn .
220 • Chap. 4 Orthogonality and Least Squares ec. 4.4 Least quar a nd Data foittin • 221
Here ar some prope1tie of the orthogonal compl ement which are obvious b. Note that A.,. A is an n x n matrix . By part a, kcr(AT A = {Öl, and AT A i
for ubspaces of ~ 2 and ~ 3 : there fore invertible ( ee Su mmary 3.3. 11 ). •

+ An Alternative Ch.aracterization
Fact 4.4.2 Con ider a subspace V of ~~~. Tbeo of Orthogonal Projections
a. dim ( \f + dim (V .L) = 11 ,

b. (V .L ).L =V
Fact 4.4.4 onsider a vector .X in lR" and a sub pace V of IR". Th n the orthogonal
c. v n v .L = {Ö). projection proj v.X .is thc vector in V closesl to :X, in that
II .X - pr ~ V-II < u- - VII.
for all ü in V different from proj v - .
Proof a. Let T (i = proj vJ
r be the orthogonal projection onto V. Note that im(T) =
V a.nd ker(T) = V .L . Fact 3.3.9 tell u that n = dim (ker(T)) +dlm (im T )
= dim V)+ dim (V.L .
To justify thi s fact , app ly the theorem of Pythagora t th . hacled right
b. First observe that V ~ V .L ).L, ince a vector in V i orthogonal to every
vector in v .L (by definition of V .L ). Furthermore, the dimen ions of the two triangle in Figure 2.
spaces are equaJ, by part a:

dim (V.L .L = n - dlm (V.L) + Least-Squares Approximations

= n- (n- dim (V)) Con. ider an in consistent linear y. te m Ax = b. The fac t that this . y tc m i.
= dim (V) inconsistent. means that the vector b is not in the imagc of !I (sce igur ).
Though thi .. y tem can not be solved wc might be intercstcd in findin g a
It foUows that the two spaces are equaJ (Exercise 3.3.41). good approximate o.lu tion. We ca n try to find a vector .X* such that !l x• i "a'
close as possible" tob . ln other words, we try to minimize the error llh - Ax ll.
x
c. Jf is in V andin v.L, then xisorthogonal to itself; that is, .X.; = Jlxll2 = 0,
aod thus .i = Ö. •
Figure 2
(rranslated)
The following somewhat technica1 result wiU be u eful later:

Fact 4.4.3 a. Jf A is an m x n matrix, then

ker(A) = ker(A TA).
b. Jf A is an m x n matrix with ker(A) = {0}, then AT A is invertible.
Figure 3

Proof a. Clearly, the kerne! of A is contained in the kerne! of AT A. Conversely,

x
consider a vector in the kerne! of AT A, o that AT Ax = Ö. Theo Ax is
in the image of A andin the kerne! of AT . Since ker(Ar) is the orthogonal
complement of im(A) by Fact 4.4.1, the vector Ax is Ö by Fact 4.4.2c, that
is, .X is in the kerne! of A.
222 • Chap. 4 Orthogon ality and Least Squares Sec. 4.4 Least Squares and Data Fitting • 223

Definition 4.4.5 Least-squares solution Fact 4.4.6 The Ieast-squares solutions of the system
Consider a linear syste m Ai =b
Ar = &. are the exact solutions of the (coosistent) system
where A is an m X /1. matrix. A vector x* in IR" is call ed a Least-squares ATAx = A r b.
sohaion of this system if ll b - Ax*ll ~ Jlb- Axll fo r all ~r. in IR".

The system A r Ax = AT b is referred to as the normal equation of Ax = b.

See Figure 4. The case when ker(A) = {Ö} is of particular importance. Then the matrix
The tenn "Ieast-squ ares so lution" reflects the fact _that we are rnio imizing
ArA is invertible (by Fact 4.4.3), and we can give a closed formula for the Ieast-
the sum of the squares of the components of the vector b - Ai.
squares solution.
If the system A.x = b happens to be consistent, then the " least-squm·es
solutions" are its exact solutions: the error llb - A.t ll is zero.
How can we find the Ieast-squares solutions of a linear system kr = b?
Fact 4.4.7 If ker(A) = {Ö}, then the linear system
Consider the foUo wing stri ng of equivalent Statements.
Ax =h
The vector X.* is a Ieast-squares solution
of the system Ai = b. has the unique Ieast-squares solution

~ Def. 4.4.5
x* = (A TA )- IATb.

l b- Ai" II ~ Jlb - Aill for all x in IR". From a computational point of view it may be more efficient to solve the
~ Fact 4.4.4 normal equation A TAi = Arb by Gauss-Jordan elimination rather than by using
Fact 4.4.7.
Ai* = proj vb, where V = im (A )
EXAMPLE 1.... Use Fact 4.4.7 to find tbe Ieast-squares solution x* of tbe system

b-
~

~
Facts 4.1.6 aud 4.4.1

A1* is in v.t = ( im(A)).l = ker(Ar) AX = b, where A = [: n and b= m

Wbat is the geometric relationship between Ax* and b?
Solution
We compute

Take another Iook at Figures 2 and 4.

and

Figure 4
As ctiscussed above, b* = Ai • is the orthogonal projection of b onto the image of
A. Check tbat

im(A )

is indeed perpendicu1ar to the two column vectors of A (see Figure 5).

224 • Chap. 4 Orthogonallty and Least Squares Sec. 4.4 Least Squaresand Data Fitting • 225
b- k i' * (translated) Solution
Let

and compute
Figure S

If _;* is a Ieast-squares solution of the system Ax = b, then b* = Ai* is the

orthogonal projection of b onto im(A) . We can use this fact to fi nd a new forrnula
for orthogonal projections (compare with Fact 4.1.6 and Fact 4. 3.10). Consider + Data Fitting
v
a subspace V of !Rn and a vector b in ]Rn . Choose a basis 1, . • . , Vm in V, and
form the matrix A = [ v1 Vm ]. Note that ker(A) = {0), since the colurnns Seienrists are often interested in fitting a function of a certain type to data they
of A are linearly independent. The Ieast-squares solution of the system A-t = b is have gathered. The functions considered could be linear, polynomial, rational,
x* = 1AT A)- 1A rb. As observ~d above, the orthogonal projection of b onto V is trigonometric, or exponential. The equations we have to solve as we fit data are
proj yb = A.~* = A (AT A ) - I ATb. frequently linear (see Exercises 29 and 30 of Section 1.1, and Exercises 30 to 33
of Section 1.2).
EXAMPLE 3 .... Find a cubi.c polynornial whose graph passes through the points (1 , 3), (- 1, 13),
(2, 1), (-2, 33).
Fact 4.4.8 The matrix of an orthogonal projection
Solution
Consider a subspace V of lR" with basis v1, ii2, ••. , vm. Set
We are looking for a function
f(t ) =Co+ C j ( + c2 t 2 + c3 t 3
Then the matrix of the orthogonal projection onto V is
such that .f (l) = 3, .f( -1) = 13, .f(2) = 1, .f(- 2) = 33; that is, we have to solve
A (AT A )- IAT .
the linear system below.
Co + CJ + C2 + C3 = 3
Co - Ct + C2 - C3 = 13
We are not required to find an orthonormal basis of V here. If the vectors co + 2ct + 4c2 + 8c3 = l
v; happen to be orthonormal, then AT A = 1111 and the formula simplifies to A AT co - 2c t +4c2 - 8c3 = 33
(see Fact 4.3 .10). This linear system has the unique solution

EXAMPLE 2 ..... Find the matrix of the orthogonal projection onto the subspace of .JR4 spanned by
the vectors

mm and
that is, the cubic polynornial whose graph passes through the four given data
points is .f(t) = 5 - 4t + 3t 2 - t\ as shown in Figure 6. ....
226 • Chap. 4 Orthogonality and Least Squares Sec. 4.4 Least Squ ares an d Data Fitting • 22 7
This qu adratic function f* (t) fi ts the data points best, in that the vector

( I, 3) (2, I)

Figure 6

is as close as possible to
Frequently, a data-fitting problern Ieads to a linear system with more equa-
tions tban variabl es (thi s happens wben tbe number of d ata points exceeds the
number of parameters in the function we seek). Such a system is usually incon-
sistent, and we willlook for the Ieast-squares solution(s).

EXAMPLE 4 .... Fit a quadr atic function to the four data points (a , , b ,) = ( - 1, 8), (a2, b2) = (0, 8),
(a3, b3) = (1, 4), and (a4, b4) = (2, 16). This means that

Solution
We are looking for a function f( t ) = c0 + c 1t + c2 t 2 suchthat is minimal: the sum of the squares of the vertical distances between graph and
Figure 8 data points is minimal (compare with Figure 8). ~
j(a 1) = b1 Co - CJ +
= 8 C2
f(a.2) = b~ c0 = 8
or or
f(a.3) = b3 c0 + c 1 + c2 = 4
j(a4) = b4 co + 2c 1 + 4c2 = 16 EXAMPLE 5 .... Find the linear function c0 + c 1t whicb best fits the data points (a 1 , b 1), (a2 , b 2 ) ,
. .. , (an , b" ), using least squares. Assurne that a 1 -1= a2 .
~
where
._ 4

[ /II
I]
1a1
0-1
and
Solution
We have to attempt to solve the system

co + c ,a, = b1
We have four equations, corresponding to tbe four data points, but only three co + c1a2 = bz
unknowns, tbe three coefficients of a quadratic polynomial. Piease check that this
system is indeed inconsistent. The Ieast-squares solution is

or

The Ieast-squares approximation is j*(t) = 5- t + 3t 2 , as shown in Figure 7.

Figure 7
or

Note that rank (A) = 2 since a , i= a2.

228 • Chap. 4 Orthogonality and Least Squares Sec. 4.4 Least Squaresan d Data Fitting • 229
The Ieast-squares solution is Solution
We have to attempt to solve the system

co + 76c 1 + 48c2 = 43
co + 92cl + 92c2 = 90
co + 68c, + 82c2 = 64
co + 86c, + 68c2 = 69
co + 54c, + 70c2 = 50
The Ieast-squares solution is

[ ~~ ] = (AT A )- iA Tb ~ [ - ~::39].
4

ci 0.799
The function which gives the best fit is approximately

We have found:
f = -42.4 + 0.639h + 0.799m .
This formula predicts the score

f = -42.4 + 0.639. 92 + 0.799 . 72 ~ 74

for Selina.

These formulas are well known to statisticians. You do not need to memorize
iliem. ~ EX ER C I S ES

GOALS Use the formula (im (A))..L = ker (AT) . Apply ilie characterization of
We conclude this section with an example for multivariate data fitting.
proj vx as the vector in V "close t to x."
Find the least- quares olution of a
EXAMPLE 6 ..... In ilie accompanying table we list the scores of five students in ilie tbree exams linear system Ai = b using the normal equation A T Ai = AT b.
given in a dass.
1. Consider the subspace im(A ) of JR2 , where

A=[; :].
Find a basis of ker(A T) and draw a sketch illustrating tbe formula
Gabriel 76 48 43
Kanya 92 92 90
Je sica 68 82 64 in this ca e.
Janelle 86 68 69
Wynn 54 2. Consider the subspace im(A) of IR 3 , where
70 50

A= [ ~1 3~] .

Find ilie function of ilie form f = co + c1h + c2 m whicb best fits tbese data,
using least squares. What score f does your formula predict for Selina, another Find a basi of ker(A T) , and draw a sketch illustrating the formula ( im(A) )..L =
student, whose scores in tbe first two exams were h = 92 and m = 72? ker(AT) in this case.
230 • Chap. 4 Orthogonality and Least Squares ec. 4.4 Leas t Squares and Da ta Fitting • 231
3. Consider a subspace V of lR" . Let ü1, ... • ü" be a basis of V and iü ,, ... , w" b. What relationship do you observe between ker(A) and im (A7 )? Exp lain .
w ,wq a basis oflR"?
a basis of y.L _ Is ü1 , . •• ,ü", 1 •• •• Explain. c. What relation hip do you ob erve berween ker(A) a nd S? Explain.
4. Let A be an m x n matrix. Is the formula x
d. Find the unique vector 0 in the intersec tion of Sand (ker(A)) .L . Show 0 x
o n your sketch .
x
e. What can you say abo ut the le ngth of 0 , com pared to Lhe le ngth of all
other vectors in S?
necessarily true? Explain. 10. Co nsi der a consistent syste m Ai = b.
5. Let V be the solution space of the linear system a. Show that thi s system has a olution >to in (ker(A)) .L . Hint: An arbitrary
so luti on x of the system can be written as ~-r = ,(" + _( 0 , where x" i in
x 1 + x 2 + x3 + ..r4 = 0 I
x
ker(A) ancl 0 is in (ker(A )).L .
Ix 1 + 2x2 + Sx3 + 4x4 = 0 ·
b. Show that the system Ax = b has onJy one soluti on in (ker(A )) .L . Hint:
Find a basis of V .L . If x0 and ~-r , are two solutions in (ker(A) ).L , think about x1 - ,v0 •
6. If A is an rn x n matrix , .i the formula c. lf .1:0 is the solution in (ker(A) ) .L and ; , is another solu tion of the system
im (A) = im (AAr).
Ax = b, show that l!ioll < l!i 1 1!. The vector i 0 is ca lled the minimal
solution of the linear ystem A,r = b.
necessarily true? Explain. 11. Consider a linear transformatio n L (x) = A,'i: from " to IR111 , where
rank(A) = m. The pseudo-inverse L of L is the transformation fro m lR117 to
7. Consider a symmetric n x n matrix A. What is the relationship between im(A)
lR" given by
and ker(A)?
8. Consider a linear transformation L (i) = A.t from lR." to JR.m, with ker (L) = {0}.
The pseudo-inverse L + of L is the transformation from 1R111 to lR" given by
L + (y) = (the minim al solution of the syste m L (x) = ji)

( ee Exercise 10).
L + (Y) = ( the Ieast-squares solution of L (x) = ji).
a. Show that the tran formation L + is linear.
b. What i L ( L + ()i)) , for ji in lR. 111 ?
a. Show that the transformation L + is linear. Find the matrix A+ of L +, in c. What is L + ( L (x)) , for x in IR"?
terms of the matrix A of L. d. Derermine image and kerne! of L +.
b. If L is invertible, what is the relationship between L + and L - t? e. Find L + fo r the linear tran formation
c. What is L +(L (i)), for i in IR"?
d. Wbat is L (L +(ji)), for ji in lR111 ?
e. Find L + for the linear transformation
- = [ 1
L (x)
0
0 0]-
1 0 X.

9. Consider the linear system
12. Usi ng Exercise lO as a g uide de fin e the tenn "min.im allea t- quare solution"
of a linear system. Explain why the minimal Ieast-square SO luti o n x* of a
linear sy tem A-i = bi in (k r(A)) .L .
Ax = b, where 13. Con .ider a linear transformation L(x) = A.Y from lR" to lR111 • The p e udo-
in ver e L + of L is the tran formation from 1R111 to lR" given by

L + (y) = (the minimal least- quru·es solution of the sy tem L(.'i) = .Y)
a. Draw a sketch showing the following subsets of JR 2 :
( ee Exerci es 8, 11 , and 12 for spec ial cases).
• the kerne! of A, and (ker(A )).L. a. Show Lhat the tran fonnation L + i linear.
• the image of Ar . b. Wh at ~ L + (L (-~)) , for ~ ~n lR"?
• the solution set S of the system Ai = b. c. What 1 L (L + (y) ) , for y w lR111 ? (continued
232 • Chap. 4 Orthogonality and Least Squares Sec. 4.4 Least Squares and Data Fitting • 233
d. Derermineimage and kerne! of L +(in tenns of ün (Ar ) and ker(Ar)). 20. By using paper and pencil, find the Ieast-squares Solution x* of the system
e. Find L + for the linear transfonna6on

- [20 00 0]0 -
L (x) = x. AX = b, where A = [l ~] and b= m
14. In the figure below we show the kerne! and the image of a linear transfor- Verify that the vector b- Ax* is perpendicular to the image of A.
w
mation L from JR2 to JR2, tagether with some vectors ·u h 1, w2, w3. We are
21. Find the Ieast-squares solution x* of the system
told that L (v 1) = w 1 • For i = 1, 2, 3, find the vectors L + (wi), where L + is
the pseudo-inverse of L defined in Exercise 13 . Show your solutions in the
figure, and explain how you found them.
AX = b, where A =[ ~ Ii] and b= [ 4n
Determine tbe error llh- Ai* II.
22. Find the Ieast-squares solution x* of the system

0
AX = b, where A = un m and b=

Deterrnine the error llb- Ax* ll.

23. Find the Ieast-squares solution x* of the system

15. Consider an m x n matrix A with ker(A) = {0]. Show that tbere is an n x m

matrix B such that BA = In . Hint: ArA is invertible.
16. Use the formula ( im(A) )j_ = ker(Ar) to prove the equation
Ax =h, where A = [i1 1]
~

Explain.
rank:(A) = rank(Ar) . 24. Find the Ieast-squares Solution x* of the system
17. Does the equation

rank(A) = rank(A r A)
hold for all m x n matrices A? Explain.
Draw a sketch showing the vector b, the image of A, the vector Ax*, and the
18. Does the equation
vector b - Ax*.
25. Find the Ieast-squares solutions .\:* of the system Ar
= b, where

hold for all m x n matrices A? Explain. Hint: Exercise 17 is useful.

19. Find the Ieast-squares Solution x" of the system

[1 0b]
Use only paper and penciJ. Draw a sketcb.
Ax = b, where A= ~ 26. Find the Ieast-squares solutions .\:* of the system Ax = b, where
1 2 3]
Use paper and pencil. Draw a sketch showing the vector
the vector Ax*, and the vector b - Ax* .
b, the image of A, A=
[7 8 9
4 5 6
234 • Chap. 4 Orthogonality and Least Squares Sec. 4.4 Least Squares and Data Fitting • 235
27. Consider an inconsistent linear system Ax = D, where A is a 3 x 2 matrix. We
are told tllat tl1e Ieast-squares Solution of this system is x* = [ 1~ Considerl to a given continuous function g(t) on tbe closed interval fro m 0 to 2rr.
One approach is to choose n equally spaced points a; between 0 and 2rr
(a; = i · (2rrjn ), for i = 1, . . . , n, say). We can fit a function
an oriliogonal 3 x 3 matrix S. Find tl1e least-squares so lution(s) of the system
SAx = Sb. ,f,, (t) = C11 +p 11 sin(t) +q 11 cos(t)
28. Consider an ortl1onormal basis u1, u2, ..., Ün in IR". Find the Ieas t-squares to the data points (a; , g(a;)), for i = I , . . . , n. Now examine what happens
solution(s) of the system to the coeffic.ients c11 , p 11 , q11 of f,, (t) as n approaches irrfinity.
Ax = ii",
\Vhere g(t )

29. Find the Ieast-squares solution of the system To find fn (t), we have to rnake an attempt to sol ve the equations

Ax = h, where A ~ [ 10~ 00 !OL] and b= [ 10~ 10 ] . or

for i = 1, ... , n

10 -10
Descrlbe and explain the difficulties you may encounter if you use technology. c11 + p" sin(a 1) + q 11 cos(a 1 ) = g(at)
Then fi nd ilie solution using paper and pencil. c" + p" sin(a2) + q 11 cos(a2) = g(a2)
30. Fit a linear function of the form f(t) = c0 + c 1t to ili.e data points (0, 0),
(0, 1), (1 , 1), using least squares. Use only paper and pencil. Sketch your Cn + p" sin(a") + q" cos(a") = g(a 11 )

solution, and explain why it makes sense.

or
31. Fit a linear function of the form f(t) = c0 + c 1t to the data points (0, 3),

[c" ] =
(1, 3), (1, 6), using least squares. Sketch the solution.
All Pn bn
32. Fit a quadratic polynornial to the data points (0, 27), ( J, 0), (2, 0), (3, 0),
q/1
using least squares. Sketch the solution.
33. Find the trigonometric function of the fonn f(t) = c0 + c 1 sin(t) + c2 cos(t) where
which best fits the data points (0, 0), (1, 1), (2, 2), (3, 3 ), using least squares.
Sketch the solution together with the function g(t) = t.
34. Find the function of the fonn

f(t) = co + c 1 sin(t) + c2 cos(t) + c3 sin(2t) + c4 cos(2t)

A" ~ [I ;
sin(a 1)
sin(a2)

sin(a")
cos(a,)]
cos(a2)

cos~an) '
b/1 = [ ;~~~~]
g(~n)
.

which best fits the data points (0, 0), (0.5, 0.5), (1, l), (1.5 , 1.5), (2, 2), a. Find the entries of the matrix A?," A" and the components of the vector
(2.5 , 2.5), (3, 3), using least squares. Sketch the solution, together with the Tb-
function g (t) = t . A" "'
b. Find
35. Suppose you wish to fit a function of the form
Iim ( -2rrAT
n "An
) and . (2rr
hm - A 11 b . T-)
j(t) = c + p sin(t) + q cos(t) n-4-oo n~ n
236 • Chap. 4 Orthogonality and Least Squares Sec. 4.4 Least Squaresand Data Fitting • 23 7
Hint: Interpret tbe entries of the matrix (2rrjn)A~ A11 and the compo- 37. The table be!ow lists several commercial airplanes, the year they were intro-
nents of tbe vector (2rr jn) A Tb as Riemann sums. Then the limits are the duced, and the number of di splays in the cockpit.
corresponding Riemann integrals. Evaluate as m any integrals as you can.
Note that
Plane Year t Displays d
. (2rr
lun T
- A"A" )
n-+ oo n Douglas DC-3 '35 35
Lockheed Constellation '46 46
is a diagonal matrix. Boei ng 707 '59 77
c. Find Concorde ' 69 133

lim [ ;: ]
.
= lim T
(A A")
11 ~ 00
11
-I A 11T b"
- .
= n-+
ltm - A A" )-I(.2rr
[(21T
n
T
11
T- ) ]
-A" b"
n
n-+oo
qll a. Fit a linear function of the form log(d) = c0 + c1t to the data po.ints
. (2rr
= [ lim
n~OO
- A 11T A"
n
)]-I . lim ( -A,.
ll~ OO
2rr r b- 11 ) .
n
(t;, log(d;)), usi ng least squares.
b. Use your answer in part a to fit an expo nenti al functio n d = ka' to the
data points 1; , d;).
c. The Airbus A320 was introduced in 1988 . Ba ed on yow· an wer in part b,
how many di splays do you expect in the c ckp it of thi s pl ane? (Ther are
The resulting vector [ : ] gives you the coefficient of tlte desired ft10ction
93 displ ays in the cockpit of an Airbus A320. Explain.)

f(t) = lim J,,(t). 38. In the table below, we Ust tbe height h the gender g , and the weight w of
ll~OO
some young adults.
Write f(t). The function f(t) is called tbe first Fourierapproximation
of g(t). The Fourier approximation satisfies a 'continuous" Ieast-squares
condition, which we wiU make more precise in Chapter 9. Height h Gender g
(in inches above (1 = "female," Weightw
5 feet) 0 = ''male'') (in pounds)
36. Let S (t) be the number of daylight hours on the tth day of the year 1997, in
Rome, Italy. We are given the following data fo r S{t): 2 I 110
12 0 180
5 I 120
11 I 160
Day t S(t) 6 0 160

January 28 28 10
March 17 77 12
May 3 124 14 Fit a function of the form
June 16 168 15

to these data, u ing least squares. Before you do the computations, !hink
about the signs of c 1 and c2 . What igns would you expect if the e data were
We wish to fit a trigonometric function of the fo.rm representative of the generat popu lation? Why? What i, the ign of c0 ? What
is the practical significance of c0 ?
f(t) = a + b srn
. ( 2rr t ) + ccos ( 2rr r)
365 365 39. In the table below, we Iist the estimated number g of gene and the e timated
to these data. Find the best approximation of this form, using least squares. number z of cell types for various organisms.
238 • h ap . 4 rth ogona Uty an d Least Sq uares
r l

Number of Nurober of
Organism genes, g cell type ·, z

Humans 600,000 250

Anne1 id warm _oo.ooo 60
Jellyfi h 60,000 25
Sponges 10,000 12
Yea t 2.500 5

a. Fit a function of the fo rm log(z = co + c, log(g) to the d ata points

log g;), log(.:;) using least quares.
b. Use your an wer in part a to fi t a power fu nction z = kg" to the data poi nts DETERMINANTS
(g;. Z;) .
c. Using the theor of self- regulato ry sy te m c ie nti t developed a model
tbat predi cts th at z i a square- root function of g, that i , a = k.jg, fo r
some constant k. Is your an wer in part: b rea o na bl y c lo e to thi fo rm ?
.1 INTRODUCTION TO DETERMINANTS
40. Consider the data in the tabl e below.
In Chapter 2 we found a criterion for tbe invertibility of a 2 x 2 matrix: the matrix
a D
mean d.istance from
tbe Sun (in
period of
revolution A=[~ ~]
Planet astronomical units) (in Earth years)
is invertible if (and only if)
Mercury 0.387 0.24 1 ad - bc =f 0,
Earth 1 1
Jupiter 5.20 J 1.86 by Fact 2.3.6. The quantity ad - bc is called the detemzinant of tbe matrix A ,
Uranu 19. 18 84.0 denoted by det(A). lf the matrix A is invertible, tben its inverse can be expressed
P1uto 39.53 248.5 in terms of the determinant:

Use the metbad discussed in Exerci e 39 to fit a power functio n of the fo m1

A-l- 1 [ d
ad- bc - c
-b] = 1 [
a det (A)
d
-c
D = ka" to the e data. Expl ain , in tem1s of Ke pler s laws of pl a netary motion.
It is natural to ask whether the concept of a determinant can be generalized
Explain why the constant k is clo e to 1.
to square matrices of arbitrary size. Can we assign a number det(A) to any quare
41. In tbe table below '.ve Iist the publi c debt D of the U ni ted State (i11 billions
matrix A (expressed in terms of the entries of A) , such that A is invertible if (and
of dollars), in the year t (a of Septe mber 30).
only if) det(A) =f 0?

'70 ' 75 '80 '85 ' 90 ' 95

+ The Detenninant of a 3 x 3 Matrix
D 370 533 908 1,823 3,233 4 ,871 Let
(11 2
(122
a. Fit a linear function of the form log (D ) = c0 + c 1t to the data points (1 32
(1;, log( D;)), using lea t quares. Use the res ult to fit an ex po nenti al func-
tion to the data points (1;, D;) . Let us denote the three column ü, vand w. Tbe matrix A is not invertible if the
b. What debt does your formula in part a predict for the year 2020? three vectors ü, v,and ware contained in a plane E. (This means that the image
of A is not all of JR3 .)
239
. 240 • Chap. 5 Determinants Sec. 5.1 Introduction to Determinants • 241
1 In this case, the vector ü is perpendicul ar to the cross produ ct v x 1v, i.e., iixw
11 · cv x w) = o
(compare with Fact A.lOa in the Appendix). See Fig.ure I. .
What does the condition ü · (v x w) = 0 mean m terms of the entnes aij of
S the matrix A?

t;· (vxtv) = [~~; ] · ([:~~ J [~~! ]) [ ~~:] [ ~~~~~~ = ~~:~~~~ ]

a 31 o 32
x
a 33
=
C13 1
·
OJ2CI23 - CI22C1 13
E

= a 11 (o22 a 33 - o 32 o 23 ) + a2 I (a32 a 13 - a 12a33) + GJ J (G J2G23 - a·na13)

= a 11 a22 a 33 - a 11 a 32 a 2 3 + Clz J0 32 CI J3 - CI2JCIJ2CI33 + CIJ JCIJ2CI23- G3 JCl22 0 JJ.
Figure 1

We define the last expression as the determinant of A. The work above

shows that the 3 x 3 matrix A is invertible if (and only if) its determinant is EXAMPLE 2 ... Find the determinant of the upper triangular matrix below.
nonzero.

A=[~ ~ ;]
Since the formula for the determinant of a 3 x 3 rnatrix is rather long, you
may wonder how you can memorize it. Here is a convenient rule:

Fact 5.1.1 Sarrus's rule 1

To find the determinant of a 3 x 3 matrix A, write the first two columns of A to Solution
the right of A. Then multiply the entries along the six diagonals sbown below. We find that det(A) = adf because all other contributions in Sarrus's formula are
zero. The determinant of an upper (or lower) triangular 3 x 3 matrix is the product
of its diagonal entries. <11111

EXAMPLE 3 .... For whicb choices of the scalar A. is the matrix

+ + + I-.\ I
Add or subtract these diagonal products as sbown in the diagram. A = 1 1 - .\
[
I 1
det(A ) = G J JCI22CI33 +a !2Cl23 CI31 + Cl 13 Cl'2 JCl3'2

- CIJ3CI22CI3! - GJlCI23 CI32 - GJ 2G2J CI33 invertible?

Solution
EXAMPLE 1... Find
3]
6 .
det(A) = (1 - A.) 3 + 1 + 1 - (1 - .\) - (1 - A.) - (1 - A.)
= 1 - 3.\ + 3>..2 - .\3 + 2- 3 + 3A. = - ).3 + 3A.2
10 = }•.?(3 - A.)
Solution The determinant is 0 if A. = 0 or A = 3. The matrix A is invertible if A. is different
from 0 and 3.
1 · 5 · 10 + 2 · 6 · 7 + 3 · 4 - 18- 3 · 5 · 7 - l · 6 . 8 - 2 . 4 . 10 = - 3.
<11111
det(A) =
This matrix is invertible. ~
+ The Determinant of an n x n Matrix
We may be tempted to define the determinant of an n x n matrix by generaJizing
1
Staled by Pierre Frederic Sarrus (1798- '1861 ) of Strasbourg, c. 1820, Sarrus's ru le (Fact 5.1.1). Fora 4 x 4 matrix, a naive generalization of Sarrus' s

L
242 • Chap. 5 Determinants Sec. 5.1 Introduction to Determinants • 243
rule produces the expression conclude that there are n(n - l )(n - 2) . . . 3. 2. ] patterns in an n x n matrix. The
quantity 1 · 2 · 3 · · · (n - 2) . (n - 1) . n is called n! (read "n factorial").
a 11 a 2 2 a 33 a 44 + · · · + a 14 a 2 1a32G43 - l7 J4G2 a 3:!a41 - ... - a 13a 22 a 31a 44 ·
We observed that the detenninant of a 3 x 3 matrix is obtained by adding up
the proclucts associated with some of the patterns in the matrix and then subtracting
the products associated with some other patterns. We note first th at the product
a11a22a33 associated with the diagonal pattern is added. To find out whether the
product associated with another pattern is added or su btracted, we bave to deter-
mine how much this pattern deviates from the diagonal pattern. More specifically,
+ + + + we say that two numbers in a pattern are inverted if one of thern is to the right
For example, for the invertible matrix and above tbe other. Below, we indicate the inversions for each of the six patterns
in a 3 x 3 matrix. Note that the diagonal pattern is the only pattern without any
l 0 inversions.
0 1

l [~@ @: : j
A = 0 0
0 0
the expression given by a generalization of Sarrus' s rule is 0. Thls shows that a31 a32
we crumot define the determinant by generalizing Sarrus' s rule in thls way: recall no inversion 2 invers ions 2 invers ions
that the detenninant of an invertible matrix must be nonzero.
Wehave to Iook for a more subtle structure in the formula @
l
(/ 12
det (A) = GJJG22033 + G J2G23G31 + GJ)G2 JG32 a21 a22
- a 13 a 22 a 31 - a 11a23a32 - a12a21a33 03 1 a32

for the determinant of a 3 x 3 matrix. Note that each of the six terms in this 3 i.nversions J inversion J inversion
expression is a product of three factors involving exactly one entry from each row Wben we compute the determinru1t of a 3 x 3 matrix , tbe product associated
and each column of tbe matrix: with a pattern is added if the number of inversions in the pattem is even and is
subtracted if this number is odd. Using this observation as a guide, we now define
tbe detenninant of a larger square matrix.

Definition 5.1.2 Determinan t

For Iack of a better ward, we call such a choice of a number in each row
and column of a square matrix a pattern in the rnatrix. The simplest pattem is
the diagonal pattem, where we choose all numbers a;; on the main diagonal. For
you chess players, a pattem in ru1 8 x 8 matrix corresponds to placing 8 rooks on
a cbessboard so that none of them can attack another.
How many patterns are there in an n x n matrix? Let us see how we can
construct a pattem column by column. In the first column we have n choices. For
each of these, we then have n - I choices left in the second column. Therefore,
we have n(n - 1) choices for the numbers in the first two columns. For each of
these, there are n - 2 cboices in the thlrd column, and so on. When we come
to the last column, we have no choice because there is only one row left. We
A pattern in an n x n matrix is a way to choose n entries of tbe matrix such
that there is Q!le chosen entry in each row and in each colunl!! of the matrix.
Two numbers in a pattern are inverted if one ortnem is located to the
rioht and above the other in the matrix.
0
We obtain the deterrninant 1 of an rz x n matrl.x A by adding up the
products associated witb all patterns with an even number of inversions and
subtracting the products associated with all patterns with an odd number of
inversions.
--~ lt appears that determinants were first considered by the Japanese mathematician Seki Kowa (1642-
t 708). Seki may have known that the detenninant of an n x n matnx has n !_ terms and that row ' a~d
columns are interchangeable (compare with Fact 5.2. 1). TI1e French mathemattelall Ale.xandre-Theoph•le
244 • Chap. 5 Determinants Sec. 5.1 Introduction to Determinants • 245
EXAMPLE 4 ~ Count the number of inversions in the pattern below. EXAMPLE 6 ~ Find det(A) for

103 4
56 2
0
0
3
0
0
0
0
0
0
7 8 907 6 A= 0 0 5 0 0
5 4 3 2 1 2 0 0 0 7 0
3 4 5 6 7 8 0 0 0 0 9
9 8 7 604 Solution
3 2Q)2 3 4
The diagonal pattern (with no inversions) makes the contribution 2 · 3 · 5 · 7 · 9 =
1890. All other patterns contain at least one zero and will therefore make no
Solution contribution toward the determinant. We can conclude that
There are seven inversions, as indicated by the bars. det(A) = 2 · 3 · 5 · 7 · 9 = 1890.

More generally, we observe that the determinant of a diagonal matrix is the

product of the diagonal entries.
EXAMPLE 7 ~ Find det(A) for
0 2 0 0 0 0
0 0 0 8 0 0
0 0 0 0 0 2
A=
3 0 0 0 0 0
0 0 0 0 5 0
EXAMPLE 5 ~ Apply Definition 5 .1.2 to a 2 x 2 matrix and verify that the result agrees with the 0 0 1 0 0 0
defi.nition given on page 239.
Solution
Solution As in Example 6, only one pattern makes a nonzero contribution toward the

There are two patterns in the 2 x 2 matrix A = [ ; !]. determinant.

no inversions one inversion

Therefore, det(A) = ad- bc.

7 inversions

Thus, det(A) = - 3 · 2 · 1 · 8 · 5 · 2 = - 480 .

.EXAMPLE 8 ~ Find det(A) for
Vandermonde (1735-1796) was the first to give a coherenl and systematic exposition of the theory of
determinants. Throughout the 19th century determinants were considered the ultirnate tool in linear algebra,
6 0 1 0 0
used extensively by Cauchy, Jacobi, Kronecker, and others. Recently, determinants have gone somewhat 9 3 2 3 7
out of fashion, and some people would like to see them eliminated altogether from linear algebra. (See. A= 8 0 3 2 9
for example, Sheldon Axler' s article ''Down with Determinants" in The American Marhematical Monthly,
February 1995, where we read: "This paper will show how linear algebra can be done better without
0 0 4 0 0
determinants." Read it and see what you think.) 5 0 5 0 1
246 • Chap. 5 Determinants
Sec. 5.1 Introduction to Determinants • 247
EXAMPLE 10 ~ Find det(A) for
Solution
Again, only one pattern makes a nonzero contribution , but this fact is not as
obvious here as it was in Examples 6 and 7. In the second column, we must
choose the second component, 3. Then, in the fourth column we must choose the
third component, 2. Next, think about the last column; and so on.

@o 1 o o Solution
902 3 7 Most of the 4! = 24 patterns in this matrix contain zeros. To get a pattern that
8 02_(2)9 could be nonzero we have to choose the entry n in the last row and the k in the
o o(9o o third. In the first two rows we are then left with two choices:
o s o(D
Cf><]) ;~ ] ~]
[
5
J inver ion
o o([)m and @;
o o([)m .
det(A) = - 6 · 3 · 4 · 2 · 1 = -144.
0 0 0 0 0 0 0 0
no inversions one inversion

EXAMPLE 9~ Find det(A) for Thus, det (A) = afkn - ebkn = (af - eb)kn. Note that the first factor is the

1 2 3 4 5 determinant of the matrix [: ~]. ~

0 2 3 4 5
A= 0 0 3 4 5
0 0 0 4 5
0 0 0 0 5
EX ER C I S ES
GOALS Use pattems and inversions to find the determinant of an n x n matrix.
Solution
Use detenninants to check the invertibility of 2 x 2 and 3 x 3 matrices.
Note that A is an upper triangular matrix. The diagonal pattem makes the contri-
bution
Find the determinants of the matrices in Exercises 1 to 20.

n n
1 . 2. 3. 4 . 5 = 120.

[~
2 0
Any pattem other than the diagonal pattern contains at least one entry below the
diagonal (Exercise 39), and the contribution the pattem makes to the determinant
1.
u ~l 2.
[~ 11]
11 . 3. 2
0
4.
[! 2
3
is therefore 0. We conclude that

det(A) = product of diagonal entries = 1 · 2 · 3 · 4 · 5 = 120. ~ 5.

[~
0
3
0 n n n[l n
6.
[:
3
3
0
7. [ ~
2
5
8
8. [;
2
5
8

We can generalize this result: 9.

[~
f;
0
,6 ~l un n 10.
8
8
5
11.
1
1
1

n [~ ;] [~ n
3 4 0 0
h
6 7 14.
0 1
The determinant of an (upper or lower) triangular matrix is the product of the 12. b 13.
Fact 5.1.3
diagonal entries of the matrix. [: e
0
0
3
0
1
0
0
0
248 • Chap. 5 Determinants Sec. 5.1 lntrodu cti on to Determinants • 249
0 0 2 3 1 35J Find the detem1inant of the 100 x lOO matri x A below.

[~ n
2 3 0 0 0 2 2
2 3 0 ... 0

[!
15. 16. 0 9 7 9 3
I

n
0 0 0 5 0 . ..
0 0 0
0 3 3 4 5 8 5
A= l 0
0 0 l 0 0 a b c d e
0 0 0 0 0 0
0 0 0 1 0 0
17. I 0 0 0 0 18. .f g h j The ijth entry of A is 1 if i + j = 101 ; all other entries are 0.
0 0 0 0 1 k I m tl p 36. Find the deterrninant of the n x n matrix A below.
0 0 0 0 q r s u 0 0 0 1
'
l 1 1 1 0 0 0

n
1 l 1 1 A=

~
b
1 1 I 1 0 l
p 20. 0 0
19. [ I 1 1 1
r 0 0 0
I I I 1 1
1 1 1 I 1 The ijth entry of A is l if i + j = n + l; all other entries are 0. Give your
answer in terms of the remainder you get when you divide n by 4 .
37. For 2 x 2 matrices A , B , C, form the partitioned 4 x 4 matrix
Use the deterrninant to find out which matrices in Exerci ses 21 to 25 are invertible.

~ ~l
[~0 0~ c~] .
M =[
22. [ ~ ~]. 23.
Express det(M) in terms of the determinants of A, B , and C.
38. For 2 x 2 matrices A, 8 , C, D , form the partitioned 4 x 4 matrix
I 25 63] .
24.
[ 4
7 8 9
25.u~n M=[~ ~l
Is it necessarily true that det(M) = det(A) det(D ) - det(B) det (C) ?
For the matrices A in Exercises 26 to 32, find all (real ) number A. such that the 39. Explain why any pattem in a matrix A, other than the diagonal pattem, con-
matrix ).[n - A is not invertible. tains at least one entry below the diagonal and at least one entry above the

26. [6 ;J. 27. [~ ;J. 28. [~ -~J. 29. [ _ ; -~ l

diagonal.
4ö. For which choices of CL i the matrix

30. u~ n 31. [ ~~ : l 32. [

33. If a matrix A hac; a row of zeros or a colurnn of zeros, what can you say
~~ Jl invertible?
A =
[
COSCL
0
sin CL

about its determinant? 41. For which choices of x is the matrix

,. W
.
------
A square matrix is called a ermutation matrix if each row and each column
-:::;; contains exactly one entry I, with all otber entries being 0. Examples are
A=[l; ~]
0 1 0]
[ 0
1 0
0 1
0
. invertib1e?

In Exerci es 42 to 45, Iet A be an arbitrary 2 x 2 matrix with det. A) = k.

What are the possib1e values of the deterrninant of a pemmtation matrix?
Explain. 42. If B is obtained from A by multiplying the first row by 3, what is det(B)?
250 • Chap. S Determinants Sec. 5.2 Properti es o f th e Determinant • 251
43. If c is obta.ined by swappi ng tbe two row of A , what. i. det(C)? + The Dete1minant of the Transpose
44. lf D i obta.ined from A by adding 6 times the econd row to the fir t, what
EXAMPLE 1 ~ Let
is det(D)?
45. What i det(A r)? I 2 3 4 5
46. Con ider two vector ü a11d ÜJ in 3 . Form d1e matri 6 7 8 9 8
A= 7 6 5 4 3
A =[ Ü x w Ü w]
2 l 2 3 4
w
Expres det(A) i n rerms of II ü x 11. For which cho ices o f Ü and w i A 5 6 7 8 9
invertible?
Express det(AT) in terms ofdet(A). You need not compute det (A) .
47. If A is an 11 x 11 matrix , what is the relation hip betwee n det (A) and det( - A)?
48. lf A is an 11 x n rn atrix and k is a constant, what i the relat.ion hip berween
det(A) and det(kA )?
Solution
49. lf A is an invertible 2 x 2 matrix, what is the relationship berween det (A) and
det(A - 1)? For each pattern in A we can consider the conesponding (tran posed) pattem in
50. Does the matrix A below have an LU factorization (see Exercises 2.4.58 and A T· for example,
2.4.6 1)?

7 4 2]
A =
[3 1 3
5 3 I A=

51. Find tbe detenninant of ilie (2n) x (2n) matri x

/" ]
0 . Th two pattems (in A and AT) involve the same numbers, and they contai n the
sa me number of inversion , but the role of the two numbers in each inversion is
52. Js the determinant of the matrix reversed . Therefore, the two pattern make the same contribution to the re pective
determinants. Since these observation apply to all pattern of A, we can conclude
l 1000 2 3 4 that det(Ar) = det(A). <1111111

5 6 7 1000 8
A = 1000 9 8 7 6
5 4 3 2 1000 Since wehavenot used any special prope1ties of the matrix A in Example l ,
2 1000 3 4 we can state more generally:

positive or negative? How can you tell? Do not use technology.

Fact 5.2.1 Determinant of the transpose

ff A is a square matrix , then
det (A T) = det (A) .
2 PROPERTIES OF THE DETERMINANT

The main goal of thi s section is to show iliat a square matrix of any size is
invertible if (and only if) its determin ant is nonzero. As we work toward this
goal, we will discuss a number of otber prope1ties of the determinant that are of
interest in ilieir own ri ght.
Thi s y mmetry property will prove very useful. Any property of the deter-
min ant expressed in terms of the rows hold for the column a weil. and vice
versa.
 Sec. 5.2 Properties of th e Determina nt • 253
252 • Chap. 5 Determinants

? Linearity Properdes of the Detenninant

For example, the transfo rmation

,EXAMPLE 2 ~ Consider tbe transformation

T X
x t] = det [ lJ
2
1
xX2 ] [ 41 [ X3
X4
X1
1
T x = det 7
3
[
X4 4
is linear (by linearity in the third row).
Consider the linear transfo rmati on
from IR.4 to IR.. Is this transformation linear?

Solution
Each pattem in the matrix T (x ) = det Li"

A-
}2
4 5
XI
X2
3]
6
-
[47 6
3
X3

X4
5
1
introduced in Fact 5.2.2.
involves three numbers and one of the variables x;; that is, the product associated Using the alternative characterization of a linear transformation (Fact 2. 2. 1),
with a pattem has the form kx; , for some constant k. The determinant itself, we infer that
obtained by adding and subtracting such procJ,ucts, has the form + y) = T (x) + T (Y)
T (x and T (k.'i) = kT (X),
for allvectors x and y in IR." and all scalars k.
We can write these results more explicitly:
for some constants c;. Therefore, the transformation T is indeed linear, by Defi-
nition 2.1.1. (What is the matrix of T ?) ~
Fact 5.2.3 Linearity of the determinant in the columns
~ a. lf three n x n matrices A , B , C are the same except for the jth colurnn
We can generalize:
and the jth column of C is the sum of the jth colurnns of A and B ,
then det(C) = det(A) + det(B).
r-

Fact 5.2.2 Linearity of the determinant in the columns

Consider the vectors i:i,, ... , Vj -l , Üj+l , .. . , v in IR." . Then the transformation
11
det Üt .i + y Vn

c
T (x) = det v1

= det v1 X V11 +det Vt y

from Rn to IR. is linear. This property is called linearity ofthe dete1minant in

the jth column. Likewise, the determinant is linear in the row s. A 8
254 • Chap. 5 Determinants Sec. 5.2 Properlies of the Determinant • 255
b. If two 11 x n matrices A and B are the same except fo r the j th column EX.AMPLE 3 .... Consider the matrices
and the jth column of B is k times the j th column of A, then det(B) = 1 2
k det(A).
3 4 5
:> 6 7 8 9

n
6 7 8 9 8 1 2 3 4
A= 7 6 5 4

de{~·
3 and B= 7 6 5 4
2 1 2 3 4 2 1 2 3
kx VII = kdet Vi X VII 5 6 7 8 9 5 6 7 8
?bserve th at B is obtained from A by swapping the first two row . Express det(B)
111 term of det(A).

B A
Solu tion

For each pattern in A we can consider the corresponding pattern in B ; for example,
The same holds for the rows. For example,
8 9 8
2
2
x 2 + y2 x3 +
3 ] = det [ x1 Y2
3 1
5 6 4 5 A= and B= 7
2
5
+ Determinants and Gauss-fordan Elimination
Suppose you bave to find the determinant of a 20 x 20 matrix. Since there are The two patterns (in A and B) involve the same nurnbers, but the number of
20! ::::;; 2 . 10 18 pattems in this matrix, you would have to perform more than inversions for the pattern in B is I less than for A (we lose the inversion formed
10 19 rnultiplications to compute the determinant using Definition 5.1.2. Even if by the numbers in the first two rows of A). This implies that the respective
a computer performed l billiou multiplications a second, it would still take over contributions tbe two patterns make to the determinants of A and B bave the same
1000 years to carry out tbese computations. Clearly, we have to look for more absolute value but opposite signs. Since these remarks apply to all patterns of A ,
efficient ways to compute tbe determinant. we can conclude tbat ·
So far in this text, we bave found Gauss-Jordan elirninati on to be a powerful det(B) = - det(A)
tool for solving numerical problems in linear algebra. lf we could understand wbat
bappens to tbe determinant of a matrix as we row-reduce it, we could use Gauss- (if the numbers in tbe first two rows of A do not form an inversion, tben an
Jordan elimination to compute deterrninants as weil. We bave to understand what inversion is created in B). <11111

bappens to tbe determinant of a matrix as we perform the three e lementary row lf the matrix B is obtained from A by swapping any two rows ( rather tban tbe
operations: (a) dividing a row by a scalar, (b) swapping two rows, aud (c) adding first two) , keeping track of the number of inversions is a little trickier. However,
a multiple of a row to anotber row. it is still true that the number of inversions changes by an odd number .for eacb
pattern (exercise), so that det(B) = - det(A) , as above.
a. lf
EXAMPLE 4 .... If a matrix A has two equal rows, what can you say about det(A) ?

Solution
A= V; and B= v;/k Swap the two equal rows and call the resulting matrix B. Since we have wapped
two equal rows, we have B = A. Now
V11 Vn det(A) = det(B) = - det(A) ,

then det (B) = (1 /k) det(A ) , by linearity in the ith row. so that
b. Refer to Example 3. det(A) = 0.
- - 256 • Chap. 5 Determinants Sec. 5.2 Properties of the Determinant • 25 7
c. FinaUy, what happens to the determ:inant if we add k times the ith row to ./' Then
the jth row? 1
det( rref(A)) = ( - 1)' det(A)
k tkz . . . k,
or
V;
det(A) = ( - lYk 1k 2 ... k, det( rref(A)) ,
A= -+ B=
by Fact 5.2.4.
V_;
Let us examine ilie cases when A is invertible and when it is not.
If A is invertible, then rref(A) = / 11 , so that det(rref(A)) = 1, and
det (A) = (- l)sktkz . . . k,.
By linearity in ilie j th row, we find that
Note that this quantity is not 0 because all the scalars k; are different from 0.
----
If A is not invertible, then nef(A) is an upper triangular matrix with some
~
zeros on the diagonal, so iliat det ( rref(A)) = 0 and det(A) = 0.
V; v·/, We have established the following fundamental result:

~act5.2.5
det(B) = det +kdet = det(A)
V;
Vj
A square matrix A is inve.rtible if and only if det(A) # 0.

If A is invertible, the discussion above also produces a convenient method to

by Example 4. compute the deterrninant, using Gauss-Jordan elimination.

Fact 5.2.4 Elementary row operations and determinants

0 Algorithm 5.2.6
I
Consider an invertible matrix .4. Suppose you swap rows s times as you
row-reduce 1 A and you divide various rows by the scalars kt , kz , .. . , k, .
Then
a. If B is obtained from A by dividing a row of A by a scalar k , then
det(B) = (l jk ) det(A).
Suppose you have to find ilie determinant of a 20 x 20 matrix. Gauss-Jordan
b. If B is obtained from A by a row swap, then elimination of an n x n matrix requires less than n 3 operations, that is, less than
det(B ) = - det(A). 8000 for a 20 x 20 matrix. lf a computer performed 1 billion operations a se-cond,
iliis would take less than 100 ~ 000 of a second (compared to the more than 1000 years
c. If B is obtained from A by adding a multiple of a row of A to another it takes when we use Definition 5.1.2).
row, ilien Here is a much briefer example.
det(B) = det(A). L) EXAMPLE 5 ... Find

: ~ _!].
Analogaus results hold for elementary column operations.

I 1 2
Now that we understand how elementary row operations affect determinants,
we can describe the relationship between the determinant of a matrix and that of its
reduced row echelon form. Suppose that in the course of Gauss-Jordan elimination I Here, it is not necessary to reduce A all the way to rref. It suffices to bring A into upper triangu lar
we swap rows s times and divide various rows by the scalars k 1, k 2 , •. • , k,. form with l 's on the diagonal. (Why?)
258 • Chap. 5 Determinants Sec. 5.2 Properties of the Determinant • 259
Solution +- The Determinant of an Inverse
We perform Gauss-Jordan elimination , keeping a note of all row swaps and row If A is an in vertible n x n matrix, what is the relationship bet.ween det(A) and
divisions we do . det(A - 1)?
By definition of the inverse matri x, we have

l ~ ~ ~]~ lb ~ ~ ~] . ;-2 l~ ~ ~] ~
2 AA - 1 = ! 11 •
I 2 - 1 I l 2 - 1 --7 1 2 - 1 - (!)
By taking the deterrrunants of both side and using Fact 5.2.7, we find that
I I 2 I I 2 L 2 - (I )
det(AA - 1) = det (A) det(A - 1) = det(/ = 1. 11
)

lI~ l ~ jl J~ Il~0 l j iJ +;:) l~ 6 ~

1

0 0 - 1 _; 0 0 -2 ..;- (-2) 0 0 0
_i J
I
Fact 5.2.8 Determinant of an inverse
If A is an in vert.ible matri x, then

Wemade two swaps and performed row divi sions by 2, - .L , -2, so that 1
ctet(A - ' ) = (ctet(A)r ' = - - .
det(A)
det(A ) = ( - 1) 2 · 2 · (- 1) · ( - 2) = 4. <11111

EXAMPLE 6 .... If S is an in vertible n x n maui x, and A an arbitrary n x n matri x, what is the

+ The Determinant of a Product relationship between det(A ) and det(S- 1 AS)?
Con ider two n x n maui ces A and B . What is the relation hip between det (A B ),
Solution
det(A) , and det(B)? First suppo e A is invertible. Our work applies the following
auxiliary result (the proof of which is left for Exercise 20). det cs - ' AS) = detcs- ' ) det(A) det(S) (Fact 5.2.7)
Consider an invertible n x 11 matrix A and an arbitrary n x n matrix B . Then, r
= ( det(S) I det (A) det(S) (Fact 5.2.8)
= det (A) (The determinants commute,
rref[ A : A B ] = [In B]. because they are scalars.)
~-. Thus, det(S- 1 AS) = det (A).
Suppo e we swap rows s times and divide rows by k 1, k_, ... , kr as we perform
. / v\ ::\l P ! I I • .::_
~ r
this elirnination.
Considerj'ng tbe left and right ·'hal ves" of the matrices separately, we con- 0 +- Minors and Laplace Expansion 1
clude that Recall the formula
det(A) = a 11a22a33 + a n a 23a3 1 + a13a2 1a32 - a13a22a3 1 - a,,a23a32 - a,2a2 1a 33

and for the determinant of a 3 x 3 matrix (Fact 5.1.1). Collecting the two terms
involving a 11 and then those involvin g a 2 1 and a 3 , , we can write
det (A B ) = (-I Y k ,k 2 ... k, det (B) = det(A) det(B).

Therefore, det(A B) = det(A) det ( B ) when A is invertible. If A is not invert- det(A) = a 11 (a22a33 - a3_a23)
ible, then neither is A B (exerci e), so that det(AB ) = det(A) det(B) = 0 .
+ a 2 1 a 2a 13 - a 12a 3)

+ a 1 (a 12a23 - a22a13)
Fact 5.2.7 Determinant of a product
lf A and B are 11 x n matrices, then (Where have we seen this fonnula before?)

det(A B) = det(A ) det(B ).

I Named aftcr the French mathematicia n Pierre-Simon Marqu i de l a pl acc ( 1749-1827). l aplace is
perhaps be st known f r his investigation inlo the stab ilily of the solar system. He was also a prominent
An alternative proof of thi s re ult is outlined in Exerci se 42. member of 1he co mmitt ee whi ch aided in the organization of the metric syste m.
260 • Chap. 5 Determinants Sec. 5.2 Properties of the Determinant • 261
Note that computing the determi nant this way require o nl y 9 multiplications, We ca n now represent the determinant of a 3 x 3 matrix more succinctly:
compared to 12 for Sarrus ' s formul a. Let us analyze the tructure oftbis formu ht
a Ii ttle more closely . Tbe tenns (022 0 33 - a 32a 23 , (a 32a 13 - a ,2a33) , and (a 12a 23 -
a 22 o 13 ) can be thought of as the determinants of submatrice of A, as follows . The
expression a 22 a 33 - a 32a 23 is the detemtinant of the mat:rix we get w hen we omit
the first row and the first column of A:

- T hi s representati on of the determi nant is called the Laplace expansion of

[a'
a ,
a 22
0 32
a"
0 33
J det(A) do wn the .first column. Likewise, we can expand along the first row ( since
det(AT ) = det(A)) :
L ikewise for the other sununands:

det(A ) ~ au det [a ,
a,
an
032
a,;
0 33
J - a def '
21
a. 1
G t2

0 32
an
0 33
j In fac t, we can expand along any row or dow n any colurnn (we can verify this
directly, or argue in terms of row or column swaps). For example, the Laplace
expansio n down the second colurnn is

+ 031 det [ a0 'I

G t2
0 22
an]
(/23 .
and tbe L aplace expansion along the thi rd row is

To state these observations more succinctly, we introduce som e tenni nology.

The m le for the signs is as follo ws: The summand aij det (Aij) has a negative sign
Definition 5.2.9 if the sum of the two indices i + j is odd. The signs follow a Checkerboard
Minors
' pattern :
For an n x n matrix A , Iet Aij be the matrix obtained when we omit the itb

~ ~]
row and the jth column of A . The (n - 1) x (n - l ) matrix A;j is called a
minor of A.
[ +
The following example will help us to generali ze Laplace expansion to larger
matrices.
G I! 01 2 Gt j 0 111
0 2! an 0 2j (/211

EXAMPLE 7 .... Consider an n x n matrix A whose jth column is e;:

A=
an Gi2 G;j a;"
G I! (/ 12 0 a ,"
(/2 1 ac2 0 a2n
a", 0112 a"j a " ll
A=
an 0;2 au,
o,, 0 12 lj a1 11
o,z, an 2j G2n
Clll i (/11 2 0 a""
A ij = /
j th column

What is the relationship between det (A) and det(A;j)?

a", On 2 nj
262 • Chap. 5- Determ.inants Sec. 5.2 Properlies of the Determinant • 263
Solution Proof Sin ce det(Ar) = det(A), it suffices to dernon trate Fact 5.2. 10 for the colum.ns.
Consid er an n x n matrix
As we compute det(A ), we need to con ider only tho e pattems in A that involve
the 1 in the jth colurru1, since all other pattems will contai.n a zero. Each such
pattern corresponds to a pattern of Aij . involving the same numbers except for the
1 we have omitted. For example,
VII
5
8
4 ~ i = 3
A=
3 4 3 4 with jth column
8 9 8 9
7 0 5 ~ 7 58
'\
j =4

8 inversion 5 inversions

The products associated with the two pattems are the same, but what happen By linearity of the determinant in the jth colurnn (Fact 5 .2.3), we can write
to tbe nurnber of inversions? We lose aU the inversion involving the 1 in the jth
colurnn. We leave it as Exercise 36 to show that the number of such inversion
is even if i + j is even and odd if i + j is odd (in our example, both numbers are
odd). Since these remarks apply to all the patterns of A involving the 1 in the jth
column, we can conclude that det(A) = ± det(A ;j ) , taking the plus sign if i + j is det(A)=O tjdet Vt Vn + Clz j det VJ VII

even and the negative sign if i + j is odd. We can write more succinctly:
det(A) = (- J)i+j det(A ij).

Now we are ready to show how to compute the detenninant of any n x n

rnatrix by Laplace expansion.
+ · ·· + G11j det VJ VII

Fact 5.2.10 Laplace expansion

We can compute the determinant of an n x n matrix by Laplace expansion
along any row or down any column.
Expansion along the ith row:
n
(by Example 7).
•
Again, the signs follow a Checkerboard pattern:
det(A) = L (- J) i+ ja;jdet (A;j)
j=J
+ +
Expansion down the jth colurnn: + +
n + +
det(A) = L (-l)i+j aijdet(Aij) + +
i= l
264 • Chap. 5 Determinants
Sec. 5.2 Prope rties of the Determlnant • 265
EXAMPLE 8 .... Use Laplace expan ion to compute det(A) for 0 2 1 0 1
-1 2

A= [ : !~ 3.
[-l
-2
2

6
1
I
14
10
-2]
6
10
33
. 4.
0
0
0 7
0
5
2
3
4
0
9
0
2
9
1
5 0 0 3 9 5 4 8
1 1 1
Solution 2 2 2 2
Looking for rows or colurnns with niany zeros to simplify the computations, we 5. 1 3 3 3
can choo e the econd col umn. 1 1 4 4
1 1 I 5
det(A) = -a 12 det(A 12 ) +an det(A22 -an det (A32) + a42 det(A42)
6. Find the deterrninant of the matrix
= 1 det [~l_ ~ r ~ ~] 2det [ 1 1 1 1

;t~ ~J ~ I 2 2 2

[I 1 2] M"= I 2 3 3
I 1 2]
= det
[5 0 3
9 2 0 - 2 det 9
5
3
0
0
3
1 2 3
for arbitrary n (the ijth entry of M~ is the rninimum of i and j).
n

= _ det [ ; ~ ] + 3 det [ ~ ; ] - 2 ( 2 det [ ; ~ ] + 3 det U~ ]) 7. Find the determinant of the following matrix using paper and pencil:
I 0 0 9 8
/'
2 1 0 7 6
Expand down
the last A= 3 2 I 5 3
column 0 0 0 7 4
0 0 0 5 3
= -20 - 21 - 2(-30- 18) =55.
8. Consider the linear transformation
Computing the determinant usi ng Laplace expansion is a bit more efficient
than using the definition of the determ.inant, but a Iot less efficient tha.n Gau s-
Jordan eümination (exerci e).
T (x) = det x v2 v3 v"
EX ER C I S ES

GOALS Compute the determinant by Gauss-Jordan eümination or by Laplace

expansion. Find the detenninants of products, inverses, and transposes. Apply from !R" to JR, where ii2. ii3, . . . , v" are linearly independent vector in
the linearity of the determinant in the rows and columns. Use the determinant to !R". Describe image and kerne) of this transformation. and determine their
check whether a matrix i invertible. dimensions.

Find the determinants of the matrices in Exercises 1 to 5, using either Gauss-Jordan

elünination or Laplace expansion, but no technology. Consider a 4 x 4 matrix A with rows ii,, V2, V), V4. If det (A) = 8, find the
deterrninants in Exercises 9 to 14.

1. [;
2
5
8 ~l
10
2.
[~
1 0
9
9
9
0
9
I

n 9. det [ -9H • det [U 11. det [q V4

266 • Chap. 5 Determinants Sec. 5.2 Properties of the Determinant • 26 7

u. det l"' ~; "l

9
r
13.
Explai n why f(t) is a polynomial of n th degree. Find the coefficient k of
t" using Yandermonde 's formula for a0

Conclude that
. . . , a" _ 1 • Explain why

f(ao) = f(a J) = · · · = f(a" _ J) = 0.

f(t) = k(t - ao)(t - a1) ... (r - a,. _ 1)

for the calar k you found above. Substitute t = a,. to dernon trate Van-
dennonde 's fo rmula.
15. Ju tify the following statement: if a square mau·ix A ha two identical
18. Use Exercise 17 to find
columns, then det(A) = 0.
16. Consider two distinct number , a and b. We define the function 1 I 1 1 1
1 2 3 4 5
1 1] det 1 4 16 25 9
f(t)= det [ ; a b . 1 8 64 125
27
'2 a2 b- 16 81 256 625
a. Show that f( r) is a quadrati c function . What is the coefficient of t 2 ? without u ing technology.
b. Explain why f(a) = f(b) = 0. Conclude that f( t ) = k(r - a)(r - b) , for 19. For n distinct scalars al> a 2 , . .., a 11 , find
ome constant k . Find k, using your work in part a.
c. For which values of r i the matrix invertible? 02
17. Vandermonde determinants [introduced by Alexandre Theo phile Vander-
monde]. Consider distinct calars a0 , a 1, •• • , a,.. We defi ne the (n + 1) x (n + 1) det
[ 0a,
:T
2
C/2
a" ]
a2
n

matrix
a"1 ci}_ ... a"n
1
ao 20. a. For an invertible n x n matrix A and an arbitrary n x n matrix B show
Cl]
a"
a2 02 a; that
A= 0 1

rref[A AB] = (I,. B] .

o"0 a] a"n
Yandermonde showed that Hint: The left part of rref[A AB] is rref(A) = /". Write

det (A) = ni>j

(a; - aj) ,
rref[ A : A B] = [111 : M]; we have to show that M = B . To demonstrate
this note that the vector

B ~; ]
the product of all difference (a; - aj). where i exceed j .
a. Yerify this formula in the ca e of n = I.
[-e;
b. Suppose the Vandermonde formula holds for n - l. You are asked to
dernarrstrate it for n . Consider the function is in the kerne! of [A AB] and therefore in the kerne! of (I" M], for

1
all i = I, .. . , n.
b. What does the formula
ao C/ j a" _ J
f(t) = det
a2
0
C/2
1
(/ 2
11 - 1 '2 rref[A : AB]= [I" : B]

a"0 a"I a"n- t" teil you if you set B = A-I?

1
268 • Chap. 5 Determinants Sec. 5.2 Properties of the Oeterminant • 269
ZV Consider two distinct point [ :~ ] and [ ~~ ] in the plane. Ex plain why the 30. The cross product in IR". Consider the vectors ü2 ü3 , ... , ü" in IR". The
u·ansformatio n - -
solutions [ ~~ ] of the equation

T (i) = det x -
VII

form a Iine and why thi line goe through the two poin ts [ :~ ] and [ ~~ ] . linear. Therefore, there i a unique vector ü in IR" uch that

22. Consider three disti nct points [ :~ l [~~ J[~~ ] in the plane. Describe the
T(x ) = x · ü
for all x in IR" (compare with Exercise 2.1.43c). This vector ü is called the
set of all points [ ~~ ] atisfying the equation cross product of ü2 , ü3 , ... , ü,., written a

l
- = V2- X V]-
U
-
X · · · X Vn.
1
In other words, the cross product is defined by th fact th at
det x, c,1 = 0.
l
x~
X2
+ xi c~
C?
+ c~
-
VJ
23. Consider an 11 x n matrix A such that both A and A- t have integer entries.
Wbat are the possible values of det(A)?
24.If det(A) = 3 for some 11 x 11 matrix, what is det(ATA)?
iS)If A is an invertible matrix, what can you say about the sign of det (AT A)? x
for all in IR" . Note that the cross product in IR" is defined only for n - 1
26. lf A is an orthogonal matrix, what are the possible values of det(A)? vectors (for example, you cannot form the cross product of just 2 vectors in
27. We say that an 11 x n matrix A is skew-symmetric if AT = -A .
IR4 ). Since the ith component of a vector w is e; · w,
we can find the cross
product by components as follows:
a. Give an example of a nonzero kew-symmetric 2 x 2 matrix .
b. What can you say about the diagonal elements of a skew-symmetric ma- ith component of V2 X VJ X . .. X Vn = e; . (v2 X ... X v/1)
trix?
c. Consider a skew-symmetric n x n matrix A, where n is odd. Show that A
is noninvertible, by showing that det(A) = 0.
28. Consider an m x n matrix
~ ~; ~
det [ v, v,

A = QR
a. When is ii2 x ü3 x · · · x Ü11 = 0? Give yorn answer in term of linear
where Q is an m x n matrix with orthonormal columns, and R is an upper independence.
triangular n x n matrix with positive diagonal entries r 11 , ••• , r 1111 • Express b. Find e2x e3 x ... x e".
det(A TA) in terms of the scalars ru. What can you say about the sign of c. Show that ii2 x ü3 x · · · x ü" is orthogonal to all the vectors ii;, for i = 2
det(AT A)? ... 'n.
d. What is the relationship between Ü2 x Ü3 x · · · x ü" and ii3 x ii2 x · · · x ü"?
29. Consider two vectors ü and w
in IR". Form the matrix A = [ v w].
Expre s (We have swapped the first two factors .)
det(A T A) in terms of lliill, llwll , and ü. w. What can you say about the sign e. Express det [ii2 x ii3 x · · · x ii" ii2 v3 v/1 ] in terms of llii2 V] X
of the result?
···X V" II. (continued)
2 70 • Chap. S Determinants Sec. 5.2 Prope rties of the Determinant • 271
3
f. How do we k:now that the cro product of two vector in IR as defined 35. Con ider the n x n matrix Qn defined the sarne way as Mn in Exerci e 34
here, is the same as the usual cro s product in IR 3 ( ee Defi nition A.9 in except that the nnth entry is 1 (instead of 2). For example,
the Appendix)?
31. Find the derivative of the function
1 1 2 3 4
0 9 2 3 4
f(x) = det 0 9 0 3 4 Find a formula for det(Q 11 ), first in terms of det(Q 11 _ J) and det(Q 11 _ 2 ), and
X 1 2 9 1 then in terms of n.
7 0 0 0 4
36. Consider a pattern in an n x n matrix, and choose an entry aij in thi s pattem.
Show that the number of inversions involvi ng a;j is even if (i + j ) is even

n
32. I am thinking of ome numbers a, b c d, e, and f such that
and odd if (i + j) is odd. Hint: Suppose there are k numbers in the pattem

~
1 to tbe left and above aij . Express the number of inversions involving aij in
1 ; ] =7 and 2 terms of k.
det[: det [ = ll
1 3 37. Let P" be the n x n matrix whose entries are all ones, except for zero directly
below the main diagonal; for example,
a. Find

d]
e .
Ps =
1 1
0 1
1 0 1
1
1 1
1
1
1
1
I
f
1 1 0 1 1
b. Find 1 1 0 1

Find the determinant of Pn.

38. A zero-one matrix is a rnatrix whose entries are all zeros and ones. What is
the smallest possible nurober of zeros you can have in an invertible zero-one
33. Is the function matrix of size n x n? Exercise 37 is helpful.

T[ ~ !] = ad + bc
39. Con ider an invertible 2 x 2 matrix A with integer entries.
a. Show that if the entries of A-I are integers, then det(A)
-1.
= 1 or det (A) =

linear in the rows and columns of the matrix? b. Show the converse: if det(A) = 1 or det (A) = -1, then the entries of A-I
34. Let M 11 be the n x n matrix with 2' s on the main diagonal , 1' directly above are integers.
and below the main diagonal, and O's elsewhere. For example, 40. Let A and B be 2 x 2 matrices with integer entries such that A , A + B A + 28.
A + 38, and A + 4B are all invertible matrices whose inverses have integer
2 1 0 0 0 entries. Show that A +58 is invertible and that its inverse has integer entries.
1 2 1 0 0 (Th:is question was in the William Lowell Putnam Mathematical Competition
Ms= 0 1 2 0 2
in 1994.) Hint: Consider the function f(t) = (det(A + r8)) - 1. Show that
0 0 1 2 I thi is a polynomial; what can you say about its degree? Find the value f(O),
0 0 0 1 2 f(l), f(2), f(3), f( 4 ), using Exercise 39. Now you can determine f( r) by
using a familiar result: if a polynornial f(t) of degree s m has more than m
We define Dn = det(M11 ).
a. Find D1 , D2 , D 3 , and D4 . zeros, then f( t) = 0 for al l t.
b . If n exceeds 2, find a formula for Dn in terms of D 11 _ and D 11 _ 2 • Justify 41. Fora fixed positive integer n , Iet D be a function which as ign to any n x n
1
your answer. matrix A a number D(A) such that
c. Find a formula for D 11 in terms of n. a. D is linear in the rows (see Facts 5.2.2 and 5.2.3),
 Sec. 5.3 Geometrical Interpreta tions of the Determin ant; Cramer's Rule • 2 73
2 72 • Chap. 5 Determinan ts
b. D ( B ) = -D (A) if B i obtai ned fro m A by a row w~p, and (Fact 4.3.7). Taking the determinants of both sides and using Facts 5.2. 1 and
5.2.7 , we find that
c. D 1" ) = 1.
Show that D (A) = det(A) for all 11 x n matrices A . Hint: Consider E = det(AT A ) = det(AT ) det(A) = (det(A)) 2 = 1.
rref(A). Think about the relationhip between D (A) and D(E), mirnicklog
Algorithm 5.2.6. Note: The point of this exerci se is that det.(A) can be char- Therefore, det(A) is either 1 or - 1.
acteri zed by the three properlies a, b, and c above ; the determin ant can in
facr be de.fined in term of these properties. Ever ince rhis approach wa
fir t presented in the 1880s by the German mathematician Kar] Weierstra Fact 5.3.1 The deterrninant of an orthogonal matri x is either 1 or -1 .
( 1817-1897), this definition ha been generally u ed in advanced linear al -
gebra courses because it allow a more elegant presentation of the theory of For example,
determinants.
42. U e the characterization of the determinant given in Exercise 4 1 to show that -sina] = 1
cos a
det(AM ) = det(A) det(M).
for rotation matrices, and
Hint: For a fixed invertible matri x M consider the function
det(A M)
D(A) = det(M) .
det [ c?s a
s1n a
sin a
- cos a
J = -1.
Show that this function ha the three properlies a, b and c li sted in Exer- For matrices of !arger sizes, we define:
cise 42, and therefore D (A) = det (A) .
43. Consider a linear transformation T from ~n+m to ~~~~ . The matrix A of T can
be written in partitioned form as A = [ A1 A2 ] , where A 1 is m x n and A2 is Definition 5.3.2 An orthogonal matrix A with det(A) = I is called a rotation rnatrix, and the
x
m x m. Suppose that det(A2) =1= 0. Show that for every vector in ~" there linear transformation T(x) = A-t is called a rotation.
is a unique y in ~m such that

+ The Determinant as Area and Volume

We will now examine what the Gram-Schrnidt process (or, equivalently, the
Show that the transformation
QR factorization) tells us about the determinant of a matrix.
_t -4 y
EXAMPLE 2 .... Below, weillustrate the Gram-Schrnidt process for two linearly independent vec-
from IR" to R m is linear, and find its matrix M (in terms of A 1 and A2). tors ii 1 and ü2 in JR 2 . As usual, we define V1 = span(vJ).
(This is the linear version of the implicit function theorem of multivariable
calculus.)

3 GEOMETRICAL INTERPRETATIONS w- , =w
_
OF THE DETERMINANT; CRAMER'S RULE - _ Uuill

We now present several ways to think about the detenninant in geometric terms.
L w,
Here is a preliminary exercise.
EXAMPLE 1 .... What are the possible values of the determinant of an orthogonal matrix A?

Solution
We know that What is the relationship between the determinants of the matrice A , B. C,
AT A = 111 and Q?
274 • Chap. 5 Determinants Sec . 5.3 GeomelTical Interpretations of the Determinant; Cramer's Rule • 2 75
Solution Now coosider an invertible n x n matrix
Since we ar dividing the fi rst column by lliiiii
det(A)
det B) = ~·

Since we are subtracti ng a multiple of tl1e first column from the econd,

det(C) = det(B).

Since we are dividing the . econd colu11111 by llwll = llii2 - proj v, il2ll . By Fact 4.2.2 we can write

det (C) A = QR
det(Q) = ~·
where Q is an orthogonal matrix , and R is an upper tri.angular matrix whose
We can conclude that diagonal enrries are

det (A = llii,ll ll ~- proj 11, ii2ll det( Q). and for i :::: 2.

Since det(Q = 1 or det( Q) = -1 , by Fact 5.3. 1, we find that We conclude that

det (A) = det ( Q) det(R)
det (A = ±llii, ll llii2 - proj 11, ii2ll
= ±llii1ll llii2- proj 11, ii2ll ... llii"- proj 11"_ , Ünll
or
because det(Q) = ±I by Fact 5.3.1 (since Q is orthogonal). Since R is triangular,
its determinant is the product of the diagonal entries (by Fact 5.1.3).

Tbis formula provides us with a simple geomenic interpretation of the de-

tenninant:
The area of the shaded parallelogram in Figure l is
(base)(height) = llii !llll ii2 -
Fact 5.3.4 If A is an n x n matrix with colu0111s ii 1, ... , 11 , then v
I det(A) I = IIu, ll llÜ2 - proj v, ii2 ll . . . IIV11 - proj v._, Ünll
proj 11, ii2ll = I det (A) I.
where Vi = span(u, u2 , . . . , üi).

Fact 5.3.3 Consider a 2 x 2 matrix A = [ u1 ii2 ]. Then the area of the parallelegram Above, we verified this result for an invertible n x n matrix A; we Jeave the
v
defined by u1 and 2 is I det(A) 1. noninvertible case as Exercise 8.
As an example consider the 3 x 3 matrix

There are easier ways to derive this fact, but they do not generalize to
higher-dimensional cases as well as the approach taken above.

Figure 1

12:~ .7 with
Height = IIii2 - projvhll ldet(A)I = lliiiii ii 'Ü2 - proj 11, ii2ll llii3 - proj 111 Ü 11 .
v, As we observed above, II ii, ll IIÜ2 - proj v,ii2ll is the area of the parallelogram defined
Base = ll iidl
v v
by 1 and 2 (see Figure 1). Now consider the parallelepiped defined by ii,, Ü2,
2 76 • Chap. 5 Determinants Sec. 5.3 Geometr i.cal Interpretations of the Determinant; Cra mer's Rute • 277
Alternative ly, we can write the formula for the k-volume as

v
Let A be the n x k matrix whose columns are 1, ... , ük . We will show that the
k-vol ume V(Ü t , ... , vk) is closely related to the quantity det(AT A) . 1f the col umns
of A are linearly independent, we can consi der the Q R factorization A = QR.
Height The n AT A = R r QT QR = R T R, because QT Q = l k (since the column of Q are
orthonormal). Therefore,

Figure 2
l det(AT A) = det (R T R) = (det(R)) 2
= (rt t r22 ... rkk) 2

v
and 3 , that is. the set of all vectors of the form Ct V t + c2ii2 + c3u3, where the c; = ( 11 iitll ll ii2 - rroj v,ii2 11 ... ll ih - projvk-l iik ll r
are between 0 and 1, as shown in Figure 2.
The volume of this parallelepiped is = (v(v], ... ,Vk)r.
volume = (base)(height) We can conclude:
= IIvtll II ii2 - proj v, ii2 II II u3 - proj v2 u3 II
= I det(A) I Fact 5.3.7 Consider the vectors ü1, ii 2 , . . . , ük in IR". Then the k-volume of the k-
by Fact 5.3.4.
parallelepiped defined by the vectors 1, . .. , ük is v
) det (A 7 A),
where A is the n x k matrix witb colurnns ii 1, ii2 , • •• , vk.
Fact 5.3.5 Consider a 3 x 3 matrix A = [v1 ii2 ü3 ]. Then the volume of the paral- In particular, if k = n, thi s volume is
v
lelepiped defined by 1, ü2, and Ü3 is I det(A) I.
I det (A) I
(compare with Fact 5.3.4).
Let us generalize these Observations to higher dimensions.

We Jeave it to the reader to verify Fact 5.3 .7 for linearly dependent vectors
Definition 5.3.6 k-parallelepipeds and k-volurne v1, . . . , ük
(Exercise 15).
As a simple example, consider tbe 2-volume (i.e. , area) of the 2-parallelepi-
Consider the vectors v1, v2 , . . . , vk in IR". The k-parallelepiped defined by tbe ped (i.e., parallelogran1) defined by the vectors

m
vectors Ü1, . . . , iik is tbe set of all vectors in IR" of the form c 1ü2 + c2 ü2 + · · ·+
Ck vk. where 0 :::: c; :::: I. The k-volume V (v 1, ... , ük) of this k-parallelepiped
is defined recursively by V(ii 1) = llii!ll and
V(ii t •.. . , vk) = V(Ü t, ... , ük- t) ll ih - projv._. vkll·
"·=[:] and V, =

in ~R 3 . By Fact 5.3 .7, this area is

Note tbat this formula for the k-volume mirnies the formula

(base )(height)
det ( [ ~ 11] [I m=
2 3
1
1
det [~ ~~J = ~.
which we used to compute the area of a parallelogram (that is, a 2-parallelepiped) In this special case, we can also determine the area as the norm II v1 x Ü2 II of the
and the volume of a 3-parallelepiped in IR3. cross product of the two vectors (exercise).
2 78 • Chap. 5 Determinants Sec. 5.3 Geometrical Interpretations of the Deterrninant; Cramer's Rule • 2 79
+ The Determinant as Expansion Factor
Consider a linear transformation T from iR2 to IR 2 . In Chapter 4 we exam.ined T (x) =
Ax
how a )jnear transfom1ation T affects various geometric quantities such as lengths ~
and angles. For exarnple, we ob erved that a rotation preserves both the lengtb
of vectors and the angle between vector . Sim.ilarl y, we can ask how a linear Figure 5
transformation affects t.he area of a region Q in the plane (see Figure 3).
We migbt be interested in finding the expansion factor , the ratio
lt is remarkable that the bnear Iransformation T (x) = Ax expands tbe area of all
area of T (Q)
parallelograms by the same factor (namely, 1 det(A) I).
area of Q ·

The simplest example is the case when Q is the unit square ( ee Figure 4). Fact 5.3.8 Expansion factor
Since the area of Q i 1 here, tbe expansion factor is simply the area of the
parallelograrn T (Q) , which is Idet(A)I, by Fact 5.3.3. Consider a linear transformation T (x) = Ax from JR 2 to iR2 . Then 1 det(A ) 1 is
More generally Iet Q be the parallelogram defined by iit and ii2 as shown the expansionfactor
in Figure 5. area of T(Q)
Let B = [ii t Ü2]. Then
area of Q
of T on parallelograms Q .
area of Q = I det(B) I, and
Likewise, foralinear transformation T (X) = A.X from lR" to IR11 , 1 det (A) 1
area of T (Q) = l det[Aii 1 Aii2] I = ldet(A.B) I = l det(A)IIdet(B) I, is the expansion factor of T on n-parallelepipeds:

and the expan ion factor is V(Aiit • . .. , Aiin) = I det(A)I V(iit •... , ii,.)
for all vectors ü1 , • • • ii,. in IR".
area of T(Q) Idet(A.) I I det(B)I
- -- - = =I det (A)I.
area of Q I det(B) \
This interpretation allows us tothink about tbe formulas det(A - t) = lj det(A)
and det(AB ) = det(A) det(B) from a geometric point of view. See Figure 6
and 7.
Figure 3 The expansion factor I det(A - 1)1 is tbe reciprocaJ of the expansion factor
I det(A )I :
1
I det(A - I)I = I det(A)I

The expansion factor I det(AB) I of the composite transformation is the prod-

uct of the expansion factors I det(A) I and I det(B) I:
Idet(AB) I = I det(A)I I det(B) I
Figure 4
T(x ) =Ai=[: :]x Figure 6

.i' =A- ')i

....
280 • hap. 5 D termin ants Sec. 5.3 Geometrical Interpretations of the Determinant; Cramer's Rule • 281
Tbis formula can be used to hnd a closed-formula solurion for a linear sy tem
:: = AB .r
OllX ! +a1 2X2 = b11
I a 2 1X1 + a22x2 = b 2
when the coefficient matrix is invertible. We write the system as A;( = b, where

Then

Figure 7
To write th.i s form ula more succi nctly, we observe that

Using techniques of calculus, you can verify that I det(A) I gives us the ex-
a22 b1 - a12b2 = det [ :~ replace the first colum.n of A by b.
pansion factor of the transfonnation T (x) = Ax on any regi on Q in the plane. 011
The approach uses in cribed parallelograrn (or even square ) to approximate the a 11 b2 - a21b 1 = det [ replace the econd column of A by b.
02 1
area of the region, as shown in Figure 8. Note that the expansion factor of T
on each of these squares is I det(A) I. Choosing smaUer and smaller squares and Let Ai be the matrix obtajned by replaci ng the ith colurnn of A by b:
applying calculus you can conclude that the expan ion factor of T on Q itself is
I det(A) I.

We will conclude this chapter with the discussion of a closed-form solurion The olution of the systern Ax = b can now be written a
for tbe linear system Ax =bin the case wben the coefficient matrix A is invettible.
det(A 1) det(A 2 )
X ---- X2 =
! - det (A) ' det(A)
EXAMPLE 3 ..... Use the formula above to solve the system
lf a matrix
2x, +3x2= 7 1
a.12
a·n
J 14x 1 + Sx2 = 13 ·

is invertible, we can express its inverse in terms of its detenninant: Solution

A- ' = _ 1_ [ a22
det(A) - a21
- al2
a11
J
x, =
det [ 1 ~ ~] =2, X2 =
det [;
~~ J=1
ragure 8
det [;
n det [; ;J
T(!l) This rnethod is not particularly helpful for . olving numerically given linear
rcx> =Ai systems; Gauss-Jordan elirnination is preferable in this case. However, in many
~
applications we have to deal with systems whose coefficients contain parameters.
a sq uare S
T(S)
area (T(S))
Often we want to know how the solution changes as we change the parame-
area (S) = [det (A)l ters. The closed-fonnula solution given above is weH suited to deal with such
questions.
 Sec. 5.3 Geometrical Interpretations of the Determi nant; Cra mer's Rule • 283
282 • Chap. 5 Determinants

EXMIPLE 4 .... Solve the y tem I I I

b b
(b- I )x i + ax2 = 0 I b

I - ax 1 + (b - l )x2 = C ·
I I I
-
where a, b C are arbitrary positive con rants. d d d

Solution
a c a c a c
der [ ~ b: 1] - aC (a) {b) (c)
XI = =
+ a2 Figure 9 (o) 8oth components x ond y of the solution ore positive. (b) ax;aa < 0: os a increoses, the component
~ 1]
(b- 1)2
der [ b- 1 b x of the solution decreoses. (c) ax;ac > 0: os c increoses, the component x of the solution increoses.
-a

X2 =
der [ b - I
-a ~] b - 1)C
(b - 1) 2 + a 2
der [ b- 1
-a b: l] Fact 5.3.9 Cramer's rule
Con ider the linear system

EXAMPLE 5 .... Consider the linear sy tem Ax = h,

ax +by =
l cx +dy = l .
ll where d > b > 0 and a > c > 0.
w here A is an in vertible n x n matri x. The components x; of the olution
vecto r -"r are
det (A;)
r· -
How doe the solution x c hange as we change the parameter a a nd c? More ' ' - det (A) '
precisely, find ax ;aa and ax ;ac, and determine the igns of these quantities. w here A; is the matiix obtained from A by replacing the i th column of
A by b.

Solution

det [ ~ ~] d_ b ax - d(d - b) This result is due to the S wi s mathematician Gabriet Cramer ( 1704- 1752).
X = - --=------':=- = > 0 - - - -2 < 0,
a db ] ad - bc ' aa (ad - bc) T he rule appeared in an appendix to his 1750 book, Introducrion l 'analy e des a
det [ c lignes courbes algebriques.

ax b(d - b)
Proof Write A = ( WI ÜJ2 ... w; ... W J. 11 Tf ,'Y i the so lution of the sy te m A ,r = b,
_ _ _ ...:_
2 > 0. th e n
ac (a d - bc)

See Figure 9 . det( ;) = det[ Üit w2 b .. . w"] = der[ w, w2 .. . A.i . . . w" ]

= del[w, w 2 +x2w2 + ·· · +x;w; + · · · +x"w") ... w"]
(.r t WI
An interesting application of these simple results in biology is to tbe srudy x;w; . .. wll ] = X; det[iv, iü_ .. ' W; ' '. wll ]
= det[Üit w2
of castes [see E. 0. Wilson, "The Ergonomics of Caste in the Socia l ln sects,"
American Naturalist, 102 (923): 41 -66 ( 1968)]. = X; det(A).
The closed formula for solving linear syst.ems of two equ ations with rwo
unknowns generalizes easily to !arger system s. Note that we have used the linearity of the de r rmin ant in the ith column .
284 • Chap. S Determ.inants Sec. 5.3 Geo metri ca l In terpretations o f the Determjnant; ramer's Rule • 285
Therefore, We have shown the following result:

det(A;)
X; = det.(A) . • Fact 5.3.10 Corollary to Cramer's rule
Con sider an in verti ble n x n matri x A. The classical adjoint adj(A) the
Cramer· ru le allows us to find a closed formula for A - J, generalizing the
n x n matrix whose ijth entry is (- l )i+j det(Aj;). Then
result
1
A- 1 = -d - - adj (A).
a b ]-J L [ d et(A)
[c d = det(A) -~

for 2 x 2 matrice .
Consider an invertible n x 11 matrix A and write
For an invertible 2x 2matrix A = [ ac db] , we find
111 12 tn) j
m - 1.
adj (A) = [ d
-c
-b]
a
and A-I - 1 [ d
ad - bc - c
- ab ]

A- 1 = [m".
n1 21 117 22

'" '"]
tn 2n

(compare with Fact 2.3 .6).

11l:llll . For an invertible 3 x 3 matrix
m" J m "2 m"j ...

1
We know that AA- 1 = / 11 • Pieking out the jth column of A- , we find that

lntj ] the formula i

A
[
. =
11! 2j -
ej·
A- 1 = - -
1 [ek- fh
fg -dk
eh - bk
ak- cg
bf -
cd- af
ce] .
m"j det(A) dh _ eg bg- ah ae - bd

By Cramer's rule, m;j = det(A;)/ det(A), where the ith column of A is replaced
by ej. EX ER <I S E S
GOALS Interpret the determinant a an area or volume and a an e xpansion
GJJ an 0 G(n
factor. Use Cramer' s rule.
a22 0
l
02 1 a2n
1. Find the area of the parallelogram defi ned by [ ~] and [ ~
A;=
GjJ Gj2 Gjll ~ jth row
2. Find the area of the triangle defined by [ ~] and [ ~].
a"J a11 2 0 a""
t
ith
column [;]
By Example 7 of Section 5.2, det(A;) = (- J)i+j det (Aji ), so that

. .. _ ( l)i+j det(Aj; )
m,J - - .
det(A)
286 • Chap. 5 Determinants Sec. 5.3 Geometrical In terpretations of the Determina nt; Cramer's Ru te • 287
3. Find the area of tbe triangle sketched below. 8. Demonstrate the equ ati on

I det(A) I = llii, llllii2 - proj v, ii2l l .. . ll iin- proj v"_' ii"ll

for a noninvertible n x n matrix A = ( jj 1 i:i 2 Ü11 ] (Fact 5.3 .4).
2v
9. Suppose two nonzero vector ü1 and 2 in JR enclose an angle a (with 0 :::::
a ::=:: n). Explai n why

I det( ü, ü2 ] I= ll iJ I!I IIi:i2ll sin (a ) .

10. Consider an n x 11 matrix A = ( i:i 1 iJ 2 v
11 ]. What is the relationship

between the product lliJI!III ~II ... IJÜ11 II and 1det(A) I? When is 1det(A) I =

·
4. Consider tbe area A of the tnangle ·
w1·tb vertlce [a'] [b' ] [c']
02
, b
2
, c
2
. E xpress
llii, ll lli:i2 ll ... llii"IJ?
11. Consider a linear transformation T (-"r) = Ai from IR.2 to JR2 . Suppose for two
A in terms of vectors ü, and v2 in IR.2 we have T (v 1) = 3ii 1 and T (ü2 ) = 4ü2. What can you
say about I det(A) I? Justify your answer carefully. Draw a sketch.
12. Consider those 4 x 4 matri ce whose entries are all 1, - 1, or 0 . What is the
max imal value of the determinant of a matrix of this type ? Give an example
5. The tetrahedron defined by tbree vectors ii" ii2, ii3 in JR 3 is the set of all of a matrix whose determinant has this maximal value.
vectors of the form c 1ii 1 + c2ii2 + c3Ü3, wbere c; 2: 0 and c , + c2 + c3 ::=:: l. 13. Find the area (or 2-volume) of the parallelogram (or 2-parallelepiped) defined
Draw a ketch to explain why tbe volume of this letrahedra n i o ne-sixth of by the vectors
the volume of tbe parallelepiped defined by i:i, ii2, Ü3.
6. What is tbe relationship between the volume of the tetrahedron defined by
the vectors and

14. Find the 3-volume of the 3-parallelepiped defined by the vectors

l~l· l~l-
and tbe area of the triangle with vertices

(see Exercises 4 and 5)? Explain thi s relationship geometrically. Hint: Con-
l1l ·
sider the top face of the tetrahedron. 15. Demonstrate Fact 5.3.7 for linearly dependent vectors ü" .. . , vk.
7. Find the area of the region sketched below. 16. True or fals e? If Q is a parallelogram in IR 3 and T (.r ) = Ai is a linear
transformation from IR. 3 to IR. 3, then
area of T (Q) = I det (A) I (area of Q).
17. (For some background on the cross product in IR.", ee Exercise 5 .2.30.) Con-
sider three linear! y independent vectors ü" Ü1 , ü3 in IR.4 .
a. What is the relationship between V (ü, . ~. Ü3) and V (ii 1, i:i2 , ü3 Ü1 x ~ x 3 )? v
See Definition 5 .3.6. Exerci e 5.2.30c i belpful.
b. Express V (Ü1, Üz, v3, ii, x Ü2 x Ü3) in terms of II ü, x Ü2 x Ü3IJ .
c. Use parts a and b to express V(i:i ,, ii2 Ü3) in term of llii, x Ü2 x u3 11. I
your result till true when the Ü; are linearly dependent ?
(Note the analogy to the fact that for two vectors ü 1 and ii2 in IR. 3 , IJÜ 1 x ii2 11
is the area of the parallelogram defined by ü, and ii2.)
288 • Chap. 5 Determinants Sec. 5.3 Geometrica!Jnterpretations of the Determinant; ramer's Ru le • 289
18. lf T Cx) = Ax i an invertible linear tran formation from IR 2 to IR 2 • then the 25. Find the class ical adjoint of the matrix
image T (Q) of the unit ircle Q is an ellipse (see E erci e 2.2.50).
a. Sketch th.is ellip e when A = [ ~ ~ J where p and q are positive. What
is it area?
b. For an arbitrary invertible transfom1ation T (x) = Ax, denote the length of and use the result to find A - J.
the semi-major and the semi-minor axis of T (Q) by a and b, respectively.
26. Consider an 11 x n matrix A with integer entries such that det(A) = 1. Are
What is the relat.ion hip between a, b, and det(A )?
the entries of A- I necessarily integers? Explain.
27. Consider two positive numbers a and b. Solve the system below.

~ ~~~!; :~ 1
What are the signs of the solutions x and y? How does x change as b
increases?
28. In an economics text 1 we find the system below:
sY +ar = JO +G
m Y - hr = Ms -Mo
Solve for Y and r.
29. ln an economics text2 we find the system below:

c. For the transformation T(x) = [i ;];, sketch this ellipse and determine
Rl
1- a ~g ~:12 ] [ d y 1J _
- (1 _ a ) 2
[
dx l
-
o0 J
its axes. Hint: Con ider T [ ~] and T [ _ ~]. - R2
Ci
dp -R2 de2

19. A basis Ü1, Ü2 , Ü3 of IR 3 is calledpositively orien.ted if ii 1 encloses an acute
angle with ii2 x v3. Illustrate th.is definition with a sketch. Show that the basis
is positively oriented if (and only if) det [ ü1 v2 v3 ] is positive.
20. We say that a linear transformation T from IR 3 to JR 3 preserves orientation. if it
transfonns any positively oriented bas is into another positively oriented basis
(see Exercise 19). Explain why a linear transformation T (x) = Ax preserves
orientation if (and only if) det(A) is positive.
21. Arguing geometrically determine whether the followino- orthoo-onal transfor-
. f Tll>3 Tll>3 b b
maLions rom m. to ~ preserve or reverse orientation (see Exercise 20).
a. Reftection in a plane.
b. Reftection in a line.
c. Reftection in the origin .
Sol ve for dx1, dy1 , and dp. ln your answer, you may refer to the detenninant
of the coefficient matrix a D (you need not compute D). The quantities R 1,
R 2 and D are positive, and a is between 0 and 1. lf de 2 is positive, what
can you say about the signs of dy 1 and dp?
30. (For those who have studied multivariable calculus.) Let T be an invertible
linear transformation from IR 2 to IR.2 , repre ented by the matrix M . Let Q 1
be the unit sq uare in IR. 2 and Q 2 its image under T (see figure on page 290).
Consider a continuous function f(x , y) from IR 2 to ~. and define the func-
tion g(u, u) = f( T (u, v)). What is the relationship between the two double
integrals below?
IInz
f(x, y)d A a.nd II n1
g(u, v)dA

Use Cramer's rule to solve the systems in Exercises 22 to 24. Your answer will involve the matrix M . Hinr: What happens when f(x , ) =
3x+ 7y = 1 1
1, for all x, y?
22.
14x+ 1l y = 3 ·
2x+3y = 8
24. 4y + 5z = 3 1Simon and Blume, Matlr ematics for Economists, Norton. 1994.
6x + 7z=-1 2 Simon and Blume, op. cit.
290 • Chapo S Determinants

V y

T
~

EIGENVALUES AND
EIGENVECTORS
31. True or False?
a. U A i an n x n matrixo then det (2A ) = 2det(A)o 1 DYNAMICAL SYSTEMS AND EIGENVECTORS:
b. Suppose A and B are 11 x 11 matrices, and A is invertibleo Then der (A BA - I)
AN INTRODUCTORY EXAMPLE
= det(B)o
c. The transformation T (x) = [ ~ ; ] ; preserve the area of a parallelo- A tretch of de ert in northwestern Mexico i populated mai.nly by two pecies
gramo of anima ls: coyote and roadrunnerso We wi h to model the population c(t) and
d. If A is an n x n matrix, then det(AA T) = det(A T A)o r (t ) of coyotes and roadrunners t years from now if the current populations c 0 and
e. lf all diagonal entries of a square matrix A are odd integers, and all other ro are known o1
entries are even, then A is inverribleo For thi s habitat, the following equations model the transformation that this
32. True or Fa/se? system wi ll undergo from one year to the next , from timet to time (t + 1) :
a. The matrix c(t + 1) = 0086c(t ) + 0008r (t ) I
I + 1. 14r(t)

j]
0

127 3 r (t +I) = -0 012c(t )

4 153 Why is the coefficient of c(t ) in the first equation less than I. while the coefficient
9 8 of r (t ) in the second equation exceed I? What i the practical significance of the
5 4 s1gn of the other two coefficient , 0 008 and - 0012?
is inve11ibleo The two equations above can be •vritten in matrix form :
b. If all entries of a square matrix A are zeros and ones, then det(A) is 1, 0,
or - I.
c(t + I)]
[ r (t+ l )
= [ Oo86c (t) + 0 008r (f) ] = [ 0 086
-0012c(t +1.14r(t) -0012
0
0008 ] [ c(r) ]
1.14 r (l )
0

c. lf A and B are 11 x n matrices, then det(A + B ) = det(A ) + det(B )o

The vector
d. Suppo e A is an n x n matrix and B is obtained by swapping two rows of
Ao lf det(B) < det(A), then A is inve11ibleo x(t) = [ c(t)
r (t)
J
e. If all diagonal entries of a square matrix A are even integers, and all other
entries are odd, then A is invertibleo is caJled the state vecror of the y tem at time t becau e it completely de cribe
thi ystem at time t 0 lf we Iet

A = [ 0086 0 008] ,
-0012 1.14

1Thc point of' this lighthcarted stor io 10 present an introd uctory examp lc where neithcr mcsoy data

nor a compli cated scenario dis tracl us from thc mathematica l ideas we w ish 10 d<::velopo
291
292 • hap. 6 Eigenvalue and Eigenvectors Sec. 6.1 Dynamica l Systems and Eigenvecto rs: An Introductory Example • 293
we can write the matrix equation abo e more succinctly ac x( I)= [I330
IO] = 11 -
· xo

l ."tCt + I ) = Ax(t ) I· - [100

300 ]
xo =
The transformation the system undergoes over the period of one year is linear,
represenred by the matrix A.

x(t )~xCt + I)

Suppo e we know that the initial state i

Figure 1
- = [ co
- 0) = xo
x( ro ] .
lt is now easy to compute x(t ), for arbitrary t:
We wish to find x(t), for an arbitrary po üive integer t .
; (2) = Ax ( 1) = 1.1 Axo = 1. I 2.t 0
_ ,1 _ A _ A _( ) A A -( ) A
.x(0)----*.l."(l)---+X(2) -'- H 3 -----+· · ·---*X f -----* · · · x(3 ) = Ax(2) = 1.1 2 A;to = l.l 3io

We can find x(t) by applying the transformation t time to x(O):

x(t) = l.l'io
I x(t ) = A' x(O) = 1
A -'to I
We keep multiplying the state vector by 1.1 each time we apply the transforma-
Although it is extremely tediou s to find x(t) with paper and pencil for large t , we tion A.
can easily compute x(t) using technology. For example, given Recall that our goal is to find closed formulas for c(t and r(t). Wehave

- = [ 100 ~( ) = [ c(t) ] = 1. 1
' '- = I . I' f\l3 10 , L~.)v
J
xo 100 ] ' x f r (t ) xo

so that
we find that
c(f) = 100(1.1)'

x(IO) = A xo
10
~ [ 1 ~~]. and
r(t) = 300(1.1)' .
To under tand the long-term behavior of this system and how it depends on the
initial values we must go beyond numerical experimentation. It would be useful Both populations will grow exponentially, by 10% each year.
to have closedformulas for c(t) and r(t), expressing the e quantities as function s
oft. We will at first do this for certain (carefully chosen) initial state vectors. Case 2 • Suppose we have c0 = 200 and ro = 100. Then

- 1 - [ 0.86 0 .08] [200] [ 180] 0 9-

x() = Axo = -0. 12 1.14 100 = 90 = · xa.
Case 1 • Suppose we have co = 100 and r0 = 300. lnitiaLiy, there are 100
coyote and 300 roadrunners, . o that x 0 = [ ~~ ] . Then Both populations decline by 10% in the first year and will therefore decline another
10% each ubsequent year:

xO) = Ax0 =[ - 00.86

. 12
o.o8J [ 'oo ]=[' ' o]
1.14 300 330 .
so that
Note that each population has grown by 10%. This means that the state vector c(t) = 200(0.9)'
x(l) is a scalar multiple of x0 (see Figure 1): and
x(l) = Axo = l.lio r(t) = 100(0.9)'.
294 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.1 Dynamical Systems and Ei.genvectors: An Introductory Example • 295
The initial populations seem to be mismatched: ~oo many coyotes chasing too few Note that the ratio r (t)jc t) can be interpreted as the slope of the state vector
roadrunners, a bad state of affairs for both spectes. x(t ) = [ c(t) ]
x (t), as shown in Figure 2.
1 r (t) H.ow can we represent the computations petformed on the preceding page
I graphically?
I
Case 3 • Suppose we bave c0 = ro = 1000. Then I x
Figure 3 shows the representation 0 = 2iJ 1 +4ü2 of 10 as the sum of a vector
I in L1 and a vector in L 2 • The formula
I r (l )
- - [ 0.86 0.08] [ 1000] - [ 940] I
x(I) = Axo = - 0.12 1.14 1000 - 1020 · I
I
Things are not working out as nicely as in the ~r~t. two cas:s we considered: ~he I
now teils us tbat tbe component in L 1 grows by 10% each year, while the compo-
I
state vector x(l) is not a scalar multiple of the tmttal state xo. Just by computmg _ _ ______ I
nent in L2 shrinks by 10%. The component (0.9)'4ih in L2 approaches Ö, which
x(2), -~(3), . . . , we could not easily detect a trend that would allow us to generate c(t} means that the tip of the state vector .X (t) approaches the line L 1, so that the slope
a closed formula for c(r) and r(t). Wehave to look foranother approach. Figure 2 of the state vector will approach 3, the slope of L 1•
The idea is to work with the two vectors To make a clearer picture of the evolution of the system, we can sketch just
the endpoints of the state vectors .:t(t). Then the changing state of the systemwill
- [ 300
100] and -
v2 = [200
100 ] be traced out as a sequence of points in the e-r plane.
VI=
It is natural to connect the dots to create the illusion of a continuous trajectory
v;
considered in the first two cases, for which A 1 was easy to compute. Since (although, of course, we do not know what really happens between times t and
v 1
and ü2 form a basis of iR2 , any vector in iR2 can be written uniquely as a t + 1).
v
linear combination of 1 and ü2 . This holds in particular for the initial state Sometimes we are interested in the state of the system in the past, at times
vector - 1., -2, .... Note that x(O) = Ax(- 1), so that i(-1) = A - 1.Xo if A is in-
vertible (as in our exa:rnple). Likewise, x(-t) = (N) - 1.r0 , for t = 2, 3, ....
- [ 1000]
xo = 1000

Figure 3 r
of the coyote-roadrunner system:

A straighttorward computation shows that c1 = 2, c2 = 4:

Recall that A1 v = (1.1) Ü1 and N il2 = (0.9Y Ü2. Therefore,

1
1

x(t) = A xo = A (2ü1 +4Üz) = 2A Üt +4Nv2

1 1 1

= 2(1.1) VI 1
+ 4(0.9) ü2 1
1000

= 2(1.1)
1
,[ ;~~ ] + 4(0.9) i~~ ] .
1
[

Considering the components of this equation, we can find formulas for c(t) and
r(t):

c(t) = 200(l.lY + 800(0.9)'

r(t) = 600(1.1) 1
+ 400(0.9) 1
Since the terms involving 0.9 1 approach zero as t increases, both populations even-
tually grow by about 10% a year, and their ratio r(t)jc(t) a:pproaches 600/200 = 3. c
296 • hap. 6 Eigenvalues and Ei en ectors Sec. 6.1 Dynamical Systems and Eigenvectors: An Introducto ry Example • 29 7
r
The trajectory (future and pa t) for our coyote-roadrunner ystem is shown in LI = Span [1 00]
300
Figure 4 .
To 0oet a feelino0 for the lon 0o-tenn behavior of thi s sy tem and how it depends
on th ini tial state, we can draw a rough sketch that shows a number of different
trajectories, representing the vruious qualitative types of behavior. Such a sketch
is called a phase portrait of the system. In our example, a phase portrait might
how the three cases di cu ed above, a weil as a traje tory that tarts above L 1
and one that starts below L 2 . See Figure 5.
To ketch these trajectorie . express the initiaJ ta te vector xo as the sum of
w w
a vector 1 in L 1 and a vector 2 in L2 . Then see how the e two vectors change
over time. If .r0 = w1 + ÜJ2, then ~- = span [z100oo]

We see that tbe two population will prosper over the long term provided that
the ratio r 0 fc 0 of tbe initiaJ population exceeds 1(2; otherwi e, botb populatio n
will die out.
From a matbematical point of view , it is informati ve to ketch a phase portrait
of this sy tem in the whole e-r plane, even though the trajectories outside tbe first
quadrant are meaningles in term of our population tudy ( ee Figure 6).
Figure S c

Figure 4 r

Figure 6 r

1000

1000 c
298 • Chap. 6 Eigenvalues and Eigenvectors ,Se · 6.1 Dynamical Systems and Eigenvectors: An Introd uctory Example • 299
+ Eigenvectors and Eigimvalues
In the example di scu sed above. it turned out to be very u eful to have ome L
nonzero vectors 1 such that A ü i a sca lar multipl e of v, that is , tÜ

for some scalar A..

Such vectors are important in many otber context as we ll.

Definition 6.1.1 Eigenvectors 1 and eigenvalues T(tÜ) = Ö= Ow

Figure 8
Con ider an n x n matrix A. A nonzero vector ü in IR" i called an eige-nvector
of A if A ü is a scaJar multiple of ü, that is, if
Aü = A.ü , EXAMPLE 2~ Let T be the ortho/cmaJ projection onto a Iine L in IR.2 • Describe the eigenvectors
of T geo_metrically and find all eigenvaJ ues of T.
for some scalar A.. (Note tbm this calar A. may be zero.
l]le scalar A. above is caJ ied tbe eigenvalue associated with tbe eigen-
vector u. Solution
We find the e igenvectors by inspection : can you think-of any nonzero vectors ü
. A nonzero vector ü is an eigenvector for A if the ectors and Au are v in the plane uch that T (v) i a caJar,. multiple of ü? Clearly, any vector ü in
parallel , as shown in Figure 7 (see Definition A.3 in the Appendix). L will do (wi th T(v) = lil), as wei l as any vector w
perpendicular to L (witb
T(w) = Ö= Ow). See'Figure 8.
EXAMPLE 1 ~ Find all eigenvectors and eigenvalues of the identity matrix /" . The eigenvalue are 1 and 0. <11111

EXAMPLE 3 ~ Let T: IR 2 --+ ~2 be the rotation in the plane through an angle of 90° ·in the
Solution counterclockwise direction . Find all eigenvalues and eigenvectors of T .
Su1ce l"ü = ü = lii for all ü in ~~~. al l nonzero vectors in IR" are e igenvectors,
wüh eigenvalue I . ~ Solution
v
lf ü i any nonzero vector in IR2 , tben T(Ü) is not parallel to (it.s perpendicular).
Flgure 7 See Figure 9. There are no eigenvectors and eigenvalues here. <11111

Aii
ii

Aii= ii - Figure 9

Aii = 2ii Aii

/
= ii = l ii AÜ = -ii = (- l )ii Aii = Ö= Oii
T(ii)

eigenva lue: 2 eigenvaiÜe: I eigenvalue: - 1 eigenvaluc: 0

1
From Eierman eigen: proper, characleristic.
300 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.1 Dyn amica l Systems and Eigen vectors: An ln troduct01y Examp le • 301
Then

Aii = ii = l ii x(t) = A'xo.

/
Such a system is called a discrete linear dy_MtmiJ::aLSY- · "m ("discrete" indicates
that we model the change the sy tem undergoes from time t to time t + l , rather
than modeling the continuous rate of change, which is described by diffe rential
Figure 10 (o) A = 1. equations).
(b) A = - 1. (al (b)
x
For an initial state 0, it i often our goal to find closed f o rm.ulas for x 1(l ),
x2 (t ), ... , x,. (t ), that is, formula s express ing x;(l ) as a function oft alone (as
EXAMPLE 4 ~ What are the possible real 1 eigenvalues of an orthogonal matr.ix A? oppo ed to a recursive fonnula , for example, whi ch wouJd merely express x; (t + 1)
in tenn of x, (t ), x 2 (t ), . . . , x,.(t )).
Solution
Recall that the linear Iransformation T (x) = kr pre erves length : II T (x) II = Fact 6.1.3 Discrete dynamical systems ;x
I!Ai ll = 11-rl!, for all vectors ; (Definition 4.3.1 ). Consider an eigenvector ü of A ,
with eigenvalue ),: Con ider the dyo arnical system
Aü = A.ü. x(t +o= A x(t )
Then with initial state
lllill = IIAli ll = IIA.li ll = IA.IIIli l! ,
so that A. = l or A. = -1. The two possibilities are illustrated in Figure 10. Suppose we can find a basi ü1, li2 , •.• ü,. of IR" coosisting of eigenvector
of A , with
AÜ; = >,,ü;.
Fact 6.1.2 The possible real eigenvalues of an orthogonal matrix are 1 and - 1.
Write i 0 as a linear combioation of the Ü;:

As an example, consider the reflection in a Line in IR2 (Exercise 15).

+ Dynamical Systems and Eigenvectors
Consider a physical system whose state at any given time t is described by some
quantities x 1(r) , x 2 (t) , . . . , x,.(t). (In our introductory example, there were two
such quantities, the populations c(t) and r(t).) We can represent the quantities
x 1(t), x 2 (t), .. . , x,.(t) by the state vector
Then
We are left with two questions: How can we find the eigenvalues and eigen-
vectors of an n x n matrix A? When is there a basis of ~~~ consi ting of eigenvectors
of A ? These issues will keep us bu y for the re t of this chapter.
If A is a 2 x 2 matrix, we can sketch a phase portmit for the system

x(t) = [~~~~~ ] . .t(t + 1) = A.t (t ,

(t )
showing the trajectories x(t) for various initial states .ro.
Even though each tra-
Xn
x
jectory is just a sequence of points, .t (O) = 0 • ,r(l ), .r (2) , .. . , we often gain a
Suppose the state of the system at time t + 1 is detennined by the state at time t, better sense of the evolution of the ystem if we "connect the dots" and ketch
and the transformation the system undergoes from time r to time t + 1 is linear, continuous trajectories.
represented by an n x n matrix A: In Figure 11 we sketch phase portraits for the case when A has two eigenval-
ues A. 1 > A.2 > 0 with associated eigenvectors ü, and v 2 . We leave out the pecial
x(r + t) = Ax(r ) . case when one of the eigenvalues is 1. Start by sketching the trajectorie along
the line L 1 = span(ü 1) and L 2 = span (Ü2) . A you sketch the other trajectories
1
ln Section 6.4 we will cons ider complex eigcnvalues. x(r) = c, A1il1
1
+ c2A.~ ü2 ,
302 • hap. 6 Eigenvalue and Eigenvectors
Sec. 6.1 Dynam ica l Systems and Eigenvectors: An Introductory Example • 303
. Characteri zati on (x) was stated in Fact 5.2.5. The equiva lence of (v i) and
(X1) fo ll ows from the definition of an eigenva lue. (Note that an eigenvector of A
with eigenva lue 0 is a nonzero vector in ker(A).)

EX E R C I S E S
GOALS Apply the concept of eigenvalues and eigenvectors. Use eigenvectors
to analyze di screte dynarnical systems.

In Exerci e 1 to 4, Iet A be an in vert.ible n x n matrix and v an eigenvector of A

with as ociated eigenvalue A.

(a) (b) (c) 1. Is ii an eigen vector of A 3 ? lf so, what is the eigenvalue?

Figure II
th.ink about tbe summand c 1A. 11ü1 and c2 A.;~. Note that fo r a large positi ve t the
vector x(r ) will be almostparall el to L 1, since ),11 will be much )arger than ;,.;.
Likewi e for large negati ve t the vector x(t ) will be almo t para llel to L 2.
We conclude th.is section with a third (and final) ver ion of the summing-up
theorem (compare with Summary 3.3. 11).
2. Is ii an eigenvector of A-I? lf so, what is the eigenvalue?
3. l s ii an eigenvector of A + 2/" ? If so, what is the eigenvalue?
4. Is ii an eigenvector of 7 A? If so, what i the eigenvalue?
5. If a vec tor ·v is an eigenvector of both A and B , is ii an eigenvector of A + B?
6. Tru e or fa lse? A square matrix A i invert.ible if (and only if) 0 is not an
eigenvalue of A .
7. If ii is an eigenvector of the n x n matrix A with associated eigenvalue A.,
what can you say about
ker(A/" - A) ?

Summary 6.1.4 Con ider an n. x n. matrix ls the matrix Al" - A invert.ible?

8. F:ind all 2 x 2 matrices for whicb e = [ ~]

1 is an eigenvector with associated
e1gen value 5.
u" 9. Find all 2 x 2 matrices for wh.ich e1 i an eigenvector.

10. F_ind all 2 x 2 matrices for wh.ich [ ~] is an eigenvector with corresponding

e1genvalue 5.
Then the following Statements are equivalent:
(i) A is invertible. 11. Find an 2 x 2 matrices for which [ ~ ] is an eigenvector.
(ii) The linear system Ai = b has a unique solution x, for all b in IR".
(iii) rref(A) = 1".
12. Consider the matrix A = [; ~]. Show that 2 and 4 are eigenvalues of A

(iv) rank(A) = n. and find all corresponding eigenvectors.

(v) im(A) = IR".
~]
13. S_how that 4 i an eigenvalue of A = [ _ ~~ 1 and find all corresponding
(vi) ker(A) = {0}. eJgenvector .
(vii) The ii; are a basis of IR". 14. Find all 4 x 4 matrices for which e2 i an eigenvector.
(viii) The v; span IR".
Arguing geometrically, find all eigenvectors and eigenvalues of the linear trans-
(ix) The ii; are linearly independent.
formation s ti sted in Exercises 15 to 22. Find a basis consisting of eigenvectors if
(x) det(A) =1= 0. possible.
(xi) 0 is not an eigenvalue of A.
15. Reflection in a line L in IR2 .
304 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.1 Dynamica l Systems and Eige nvectors: An Introdu ctory Exa mpl e • 305
2
16. Rotation through an angle of 180° in lR . In Exercises 30 to 32, consider the dynarnical system
17. Counterclockwise rotation through an angle of 45o followed by di lation by 2
0 ~ J .xcr) .
11
in lR2 . .t(t + l ) = [
18. Reftection in a plane E in !R3 .
3
19. Orthogonal projection onto a line L in lR . Sketch a phase portrait of tbis system for the given val ues of A:
20. Rotation about the e3-axi through an angle of 90°, coun terclockwise as r 30. A = 1.2. ;: 31. A = I. K32. )" = 0.9.
3
viewed from tbe positive e3-axi in lR 0

/ 33. Consider the coyotes and roadrunners system discus ed in thi ection. Find
21. Dilation by 5 in JR 3 . clo ed formtdas for c(t ) and r (t ), for the initial popula6on c0 = 100, r 0 =
22. The sbear witb T(v) = v and T(w) = v + w for the vector v and w in 1R2 800.
sketcbed below.
34. Find a 2 x 2 matrix A such that [ i] and [ ~] are eigenvectors of A , witb
eigenvalues 5 and 10, respectively.
Y 35. Find a 2 x 2 matrix: A such that

is a trajectory of the dynarnical system

23. a. Consider an invertible matrix: S = [ii1 u2 v11 ]. Find s- 1ii;. Hilll :
Se;= v;. .x cr + 1) = Ax(t).
b. Consider an n x n matrix A and an invertible n x n matrix: S = :X 36. Imagine that you are diabe6c, and you have to pay close attention to bow your
[ii 1 ii2 vn] wbose columns are eigenvectors of A with Av; = A;v;. body metabolizes glucose. Letg_(L) r~j:lresent tbe excess glucose concentration
Finds- i AS. Hint. Think about the product s- 1 AS column by column. in your blood, usually measured in rnilligrams of glucose per 100 milliliters of
b d Ex:cess means that we mea re_b.l v much the alucose concentration
In Ex:ercises 24 to 29, consider a dynamical system de · tes from our fasting Ievel, i.e., tbe Ievel 0ur s stem a roacbes ·er1

x(r + 1) = Ax(r) many hours of fasting .) A negative value of g(t) indicates that tbe glucose
concentration is below fasting Ievel at time t . Sbortly after you eat a heavy
witb two components. Below, we sketch the initial state vector .Xo and two eigen- meal , the function g(t) will reach a peak, and then it will slowly return to 0.
vectors, ii 1 and ti 2, of A (with eigenvalues Ai and A2, respectively). For the given Certain hormones belp regulate glucose, especially the hormone insulin . l,&!
values of Ai and )..2 , sketch a rough trajectory. Think about the future and tbe past h t re resent the excess hormone concentration in your blood . Researchers
of the system. have developed mathematical models for the glucose regu atory system. Here
is one such model, in slightly simplified form (these formulas apply between
meals; obviously, tbe system is disturbed during and right after a meal):

g(t + 1) = ag(t) - bh (t ) I
l
h(t + 1) = c g(t) + d h (t)

where time t is measured in minutes; a and d are constant lightly less than
I ; and b and c are small positive constants. For your system the equations
might be ,,_"
..... "1 "'

g(t + 1) = 0 .978g(t) _ 0 .006h (t) I

+ +
0

I h(t
)<I - !LJ
1) = 0 .9,04g(t)
1
'H! ( I, .!-.)
0.992h(t )
:)!Ue:t-r Ir • 1)
The term - 0 .006h (t) in the first equation is negative because insulin helps
24. AJ = 1.1 , A2 = 0.9. Y25 . .A 1 = 1, A2 = 0.9. Y 26. A1 = 1.1, .A2 = 1. your body absorb glucose. The term 0.004g(t ) is pos itive becau e glucose in
1. 27. A1 = 0.9, )"2 = 0.8. 'I 28. AJ = 1.2, A2 = 1.1. y 29. AJ = 0.9, A2 = 0.9. your blood timulates the cells of the pancreas to ecrete in ulin (for a more
306 • Chap. 6 Eigen val ues and Eigen ectors Sec. 6.2 Fi nding the Eigen va lu es of a Ma trix • 30 7
thorough di cu io n of this model, read E. Ackerm an et al. . " Blood gluco e
regul ado n and diabete ,' Chapter 4 in Concept and Mode ls of Biomathemat-
b. Yerify that [ ~] and [ _ ~] are eigenvectors of A . Find the as ociated
ics Marcel Dekker, 1969): eigenvalues.
Con ider the coeffi cient matri x c. Find closed formulas for n (t ) and a (t).
39. Consider two n x n matrices A and S, where S is invertible. Show rhat the
A - [0.978 - 0.006 ]
- 0.004 0.992 matTices A and s- 1 AS have the same eigenvalues. Hint: lf v is an eigenvector
of s- 1ASthen Sv is an eigenvector of A.
of th is dynarnical system. 40. Suppose v is an eigenvector of the n x n mattix A, with eigenvalue 4. Explain
a. We are told tbat [ - ~] and [ _ ~ ] are eigenvectors of A. Find the a so- v
why is an eigenvector of A 2 + 2A + 3/11 • What is rbe associated eigenvalue?
41. True or fa lse?
ciated eigen alue .
b. After you ha e consumed a heavy meal, the concentTati on in your blood
a. If 0 is an eigenvalue of the matrix A, then det (A) = 0.
are g0 = 100 and ho = 0. Find closed fo rmul a for g(t ) and h (r) . Sketch
the trajectory. Briefl y de cribe the evolution of this sy tem in practi cai b. lf [ ~] is an eigenvector of [ ~ ~ ]. then [ ~ ] is an eigenvector of
tenn .

[ ~ ~ ~] .
c. For the case di cus ed in part b how long does it take fo r the gluco e
concentration to fall below fas ting Ievel? (Thi s qu anti ty i useful in di-
agno ing diabetes: a period of more than fo ur hours may indi cate mild 0 0 0
di abetes.) c. If A is a 4 x 4 matrix with A4 = 0, then the only possible eigenvalue of
A is 0.
37. Con ider the matrix
42. True or fa lse?
A = [~ -~ l v v
a. lf is a nonzero vector in lR", then is an eigenvector of the matrix
b. If v is an eigenvector of A and B , then v is an eigenvector of AB.
vv 7 .
a. Use the geometric interpretation of this tran Formation to fi nd tbe eigen- v v
c. If is an eigenvector of A, then is in ker(A) or in im(A).
value of A.
b. Find tw o linearly independent eigenvecrors fo r A.
x 38. Consider the growth of a lilac bush. The state of this lil ac bush for several
years (at year' s end) is sketched below. Let n (t) be the number of new
branches (grown in the year t ) and a(t) tbe number of old branches. In the FINDING THE EIGENVALUES
sketch, the new branches are represented by shorter lines. Each old branch will OF A MATRIX
gww two new branches in the foUowing year. We assume that no branche
die. ln the last section, we saw how eigenvalues help us analyze a dynarnical system

~ru + 1) = A-r(t ).
year 0 year I year 2 year 3 year 4

n(O) = I
a(OJ = 0
n(l) =0
a(l) = I
n(2)
+
=2
a(2) = I
_±_
n(3)
a(3)
=2
=3
±
n(4)
a (4)
=6
=5
Now we will see how we can actual ly find them.
Consider an n x n matrix A and a scalar >.. . By definition, A. is an eigenvalue
v
of A i f there is a nonzero vector in lR" such that

Av = >.. ü
or

a. Find the matrix A such that

n(t
[ a(t
+ I)] = A
+ I)
[n(t)]
a(l) ·
or
(Al" - A )v = Ö.
308 • hap. 6 Eigenvalues and Eigenvectors Sec. 6. 2 Finding the Eigenvalues of a Matrix • 309
This means, by defirution of the kernel , that EXAMPLE 2 ..... Find the eigenva lues of
ker(Al" - A) =I= {0} . 1 2 3 4 5
0 2 3 4 5
(i.e .. there are other vectors in the kernet. besides the zero vector).
A= 0 0 3 4 5
This is the case if (and only it) the matrix AI" - A is not in vertible (by
0 0 0 4 5
Fact . 1.7) tbat is , if det(A l" - A) = 0 (by Fact 5.2.5).
0 0 0 0 5

Solution
Fact 6.2.1 Con ider an n x n matrix A and a scalar A. Then A i an eigenvalue of A if
(and only it) det(H11 - A) = 0. Again , we have to so lve the equ ation det(A./ 5 - A) = 0.

A. - 1 - 2 -3 -4 -5
To clarify things, we represent the observations made above as a series of 0 A-2 - 3 -4 - 5
equivalent Statements: det(,l,. / 5 - A) = det 0 0 .l..-3 -4 -5
0 0 0 ), -4 - 5
). is an eigenvalue of A . 0 0 0 0 A.-5
~ = ().. - 1) ), - 2)(A. - 3)(), - 4)()..- 5) = 0
There is a nonzero vector - ucb that Aü = AÜ or (,\ /" - A)v = Ö. Reca11 that the determinant of a triangular matrix is just the product of the diagonal
~ entries (Fact 5. 1.3). Tbe solution are I, 2, 3, 4, 5 ; tbese are the eigenvalues
ker(U11 - A ) =/= {0 }.
of A . ~

~
Hn - A is not invertible. Fact 6.2.2 The eigenvalues of a triangular matrix are its diagonal entrie .
~
det(H" - A ) = 0. By Fact 6.2. 1, we can think of the eigenvalues of an n x 11 matrix A as tbe
zeros of the function
EXAMPLE 1 ..... Find the eigenvalues of tbe matrix
I [4 (A.) = det (A.I" - A) I·
A= [! ~l
EXAMPLE 3 ..... Find .fA(A.) for tbe 2 x 2 matrix
Solution
By Fact 6.2. 1, we have to Iook for numbers A. such that det(A./ 2 - A) = 0. A=[~ !].
det(A./2 - A) = det ([ ~ ~] - [! ;]) Solution
A Straightforward computation show that
), _- 1 A. -2 ]
= det [ 4
_
3 -b ]
= (). - l )(A.- 3)- 8
.fA A.) = det(A/2 - A) = det [ A.-
- c
a
).. _ d = A. 2 - (a + d)).. + (ad- bc) .
= A.2 - 4A. - 5 This is a quadratic polynomial with constant tenn det(A) = ad - bc. This makes
= (A. - 5)(A. + 1) sense becau e the constant term is j~, (0) = det(Oh - A) = det(- A) = det(A).
The coefficient of A. is the negative of the sum of the diagonal entrie of A.
The equation det(U2 - A) = (A. - 5)(A. + 1) = 0 holds for ,\ 1 = 5 and A.2 = -1. Since thi sum is important in many other contexts as weil, we introduce a nam~
These two scalars are the eigenvalues of A. <1111
for it.
310 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.2 Fi n di ng t he Eigenvalues of a Matrix • 311
is po ible to describe the coefficients of >.., A. 2 , . . . , ;._n- 2, but these fo rmula are
Definition 6.2.3 Trace
com pl icated, and we do not need the m bere.
The sum of the diagonal elements of an n x n matrix A i ca lled the trace of
A, denoted by tr(A ).
Fact 6.2.5 Characteristic polynomial

We an now wri te the function JA (A.) associated wi th a 2 x 2 matrix A Consider an n x n matri x A. Then JA(>.. ) = det(A/11 - A) is a polyn01rual of
degree n, of the fo rm
uccinctly a fo llows:
J A(Ä) =A
11
- tr(A).l.." - 1 + · · · + (- 1)" det(A) .

If A is a 2 x 2 matri x. tben
!A ()..) is ca1led tl1e characteristic polynomial of A.
Fact 6.2.4
/A (A.) = A.'!. - tr(A)A. + det A) .
Note tbat Fact 6.2.4 represents a special case of Fact 6.2.5 .
What does Fact 6.2 .5 teil us about the number of eigenvalues of an n x n
For the matrix A = [! ~] i.n Exan1ple 1 we bave tr(A) = 4 and det(A) = matrix A? We know from elementary algebra that a polynomial of degree n has
at mo t n zeros. Therefore an n x n matrix has at most n eigenvalue . lf the
- 5, so that characteri sti c polynomial / ;r.(.l..) is of odd degree, then tbere is at lea t one zero,
by tbe intermedi ate value tbeore m, since

tim JA(),) = and li m f 4 (>..) = - oo.

as we found earlier. ).~ A.-+-
Wbat is tbe format of ! A(A) for an 11 x n matrix A? See Figure l .
f_.. (A.) = det(Al 11 - A)
EXAMPLE 4 ..... Find all eigen value of
).. - a 11
- a ln ] l 2 3 4 5
- a2 1 -a2n
= det . 0 2 3 4 5
[ A= 0 0 1 2 3
A _: Clnll .
-On I - On2 0 0 0 2 3
0 0 0 0 l
Tbe contribution a pattem makes to thi s determinant i a polynomi al in >.. of degree
less than or equal to 11. Thi s implies that the determina nt itself, being tbe sum of
all tbese contributions, is a polynomial of degree les than or equal to n .
We can be more precise: the diagonal pattem of tl1e matrix A/11 - A makes Figure 1
the contribution
().. - 0 11 )(). - 022) · · · (). - Clnn)

= )..n - (a ll + a22 + · ·· + 0 1111 A11 - 1 + (a polynor.rual of degree _::: (n - 2))

= )," - tr(A)>.."- 1
+ (a polynomial of degree _::: (n - 2)).
Any other pattem involves at least two calars off the diagonal (Exerci se 5.1.39),
and its contribution toward the determinant is therefore a polynomial of degree
less than or equal to n - 2. This impli es that
An eigenva lue of A
!A(>..) = ).!' - tr A)A."- 1 + (a polynomial of degree _::: (n - 2)).

The constant tenn of thi s polynomia l is f A(O) = det (-A) = (- 1)11 det(A) . lt
312 • Chap . 6 Eigen va lue and Eigenvec tors Sec. 6.2 Find ing the Eigenvalu es qf a Matrix • 313
Solution
The characteri tic polynomial is f 11 (J...) = (J... - l ) 3 (J... - 2) ~ . ! he ei gen v~lue are 1
and 2. Since 1 is a root of multiplicity 3 of the characten st:Jc poly no mtal, we say
that the eigenvalue 1 ha algebraic mu/riplicity 3. Likew ise, the eigenvalue 2 has
al gebraic multiplicity 2. <11111

Definition 6.2.6 Algebraic multiplicity

We say that an ei genval ue J.. 0 of a quare matrix A has algebraic mulriplicity
k if
Figure 2
fA ('J. . ) = ('J... - J...o)k g (A)
for some polynomial g (J...) with g (J.. o) i= 0, that is, if Ao is a root of multiplicity
k of j;~(A) . f f n is even, an n x n matrix A need not have any real ei genvalue . Perhaps
the . implest example is

In Example 4, we had
k
A= [~ -b]
+ with !A (A) + I. See Figure 2.
= A2
f;~('A) = (A- 1)3 (J... - 2)2 GeometricaJi y it make sen e that A ha no real eigenvalue : the tran for-
'-v-"'
t g(A.) mati on T (x ) = Ax i a counterclockwise rotati on through an angle of 90"' .
>-o
EXAMPLE 6 ..... Describe all po ible cases fo r the number of real eigenvalues of a 3 x 3 matrix
The algebraic multipliciry of the eigenvalue Ao = I is k = 3. and thei.r algebraic multi plicitie . Give an example in each case and graph the
EXAMPLE 5 ..... Find the eigenvalues of characteristic poJynomi al.

A = [ -~-I
-1
2
- I
-1]
- 1
2
Solution
The characteri tic po lynorni al either factor completely
with their algebraic multiplicities.

or it has a quadrati c fac tor without real zeros:

Solution
fA(A) = (A - Ai p ('A .
)... - ?
fA(A) = det ~ A- 2 In the fir Lca e, the Ai could all be distinct two of them could be equ al, or they
[ I could all be equ al. This Ieads to the following possibilitie :
We find two di stinct ei genvalues, 3 and 0, with al gebraic multiplicities 2 and ~
respectively.
No. of Distinct Algebraic
Let us summari ze: Case Eigenvalues Multipllcities

I 3 I each
Fact 6.2.7 An n x n matrix has at most n eigenvalues, even if they are counted with 2 2 _ and I
their algebraic multiplicities. 3 3
If n is odd, then an n x n matrix has at least one eigenva lue. 4
314 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.2 Find ing th e Eigenvalues of a Matrix • 315
Examples for each ca e follow.

Case 1 •

See Figure 3.
A ~ [~
0
2
0 n f11 (A) = (A - l )(A - 2)(A - 3)

Case 2 •

See Figure 4.
A ~u n
0
1
0
Figure 4

Case 3 •
A = / 3,

See Figure 5.

Case 4 •

Figure S
f A(A) = (A- 1) ().2 + 1)

See Figure 6.

You can recognize an eigenvalue ;.0 whose algebraic multiplicity exceeds I

on the graph of /A (l ) by rhe fact that /A (lo) = f~ (), 0) = 0 (the deri vative is zero ).
The verification of this observation is left as Exercise 37. <11111

figure 3

Figure 6

• Finding the Eigenvalues of a Matrix in Practice

To find the eigenvalues of an 11 x 11 matrix, we have to find the zero of the
characteristic function , a polynomial of degree 11. For n = 2, this isatrivial matter:
316 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.2 Finding the Eigenvalues of a Matrix • 317
[-1 -1]
u ;]
we can e ither factor the pol ynomial by in pecti on or use the quadratic formu la. - 1 -2
The problern of finding the zeros of a po lynomial of higher degree is nontri via l: 7. /3 8. - ] - 1 - 1 9. 0
it has been of considerable interest throughout the hi tory of math emati c . In - ] - l - 1 0

[-~ u
the early 1500s ltalian mat.hematic ians found fonnu las in the cases 11 = 3 and

l~
- 2

J]
0

n -~]
11 = 4 , published in the Ars Magna by Gerolamo Cardano.t See Exerci ·e 38
0 1
10. - I - 1 0
for the ca e 11 = 3. Duri ng the ne ·r 300 years. people tried to find a general 11. 1
-2 7 12. 0 3
fonnula to ol e the qui nt.ic (a polynomial equation of fifr.b orde r). [n 1824 the 2 -7
0 2 -3
Norweoian mathematician Niels Henrik Abel (1802- 1829) showed that no uch

[~ !]
0
1
general solution is possible, putting an end ro the long search. The first numerica l
13. 0
example of a quintic which cannot be so lved by radica ls was given by the French
0
mathematician Evariste Galoi s (1811-1832). (Note the short life pans of these
two brilliant mathematician . Abel died from t:uberculosis; Galoi. , in a duel.)
It is usuaJiy impo sible to find the exact eigenvalues of a matrix. To find
14. Consi~er a 4 x 4 matrix A = [ g ; ],where B, C, and D are 2 x 2 matrices.
approxim ations for the eigenvalues, you could graph tbe cha:racteristic po lynomial What 1s the relationship between the e igenvalues of A B, C, and D?
(using technology). The grapb may give you an idea of the number of eigenva lue
15. C ons t'der t he matnx
. A = [ 11 k ] , where k ts
. an arbttrary
. constant.
For which
and their approximate values. Numerical analyst tel! us that thi i not a very 1
efficien t way to go about find.ing tbe eigenvalues; other techniques are used in val ues of k does the matrix have two disti nct real eigen values? When is there
practice (see Exercise 6.4.33 for an exa:mple; another approach uses QR facto riza- no real e i.genvalue?
tion).2 lf you have to fi nd the eigenva lues of a matrix , it may be worth trying out a
few small integers, such a ±1 and ±2 (those matJice considered in introductory
li11ear algebra texts often happen to have such eigenvalues).
16. Consider the matrix A = [ ~ ~ l where a, b, and c are nonzero constants .
For which choices of a, b , and c does A have two distinct eigenvalues?

EXERCISES (_
17. Consider the matrix A = [ ~ _ : ] , where a and b are arbitrary constants.

GOALS U e tbe characteristic polynomial JA (,l,.) = det (A/11 - A) to find Lhe Fi nd all e igenva lues of A. Explain in terms of the geometric interpretation of
eigenval ues of a matrix A. with their algebraic multiplicities . the li near transformation T (.r ) = iLt.

For each of the matti.ces in Exercises I to 13, find all real eigenvalues, wit h their
18. Consider the matrix A = [~ !], where a and b are arbitrary constants.
algebraic multiplicities. Show your work. Do not u e technology. Find all e igenvalues of A.
19. Tr-ue or false? If the determi nant of a 2 x 2 matrix A is negative, then A has
two distinct real eigenvalues.
1. [~ ~] 20. Consider a 2 x 2 matr ix A with two distinct real eigenvaJues, At and )..2.
Express det(A) in terms of At and A2 . Do the same for the trace of A. Hint:

5. [ 161 - 15 ]
-7 6. u~ ] The characteristic polynomiaJ is

1Cardano ( 150 1- 1576) was a humanist wi th a wide range of interest . In hi s book Liber de ludo
aleae he pre ents Lhe firs t sy tcmatic compu tation of probabilities. Trained as a phy ician. he gave thc
Yerify your answers in the case of the matrix A = [! 2] 3 .
first clinical description of typhus fever. In hj s book Somn ionmr Synesionmr (Basel, 1562) he ex plores the 21. Consider an n x n matr ix A with 1z d.isti nct real e igenvalues, At, )"2 , ... , An.
meaning of dreams (sample: "To dream of living in a new and unknown ciry mea n immi ne nt death''). Express the determi nant of A in terms of the A;. Do the same fo r the trace
Still, he is best known today as Lhe most outstanding rnathematician of his time and the author of' the Ars
Mag na. In 1570 he was arTested on accusation of heresy: he lost his academic position a nd the righ t to of A.
pub li sh. 22. Consi der an arbi trary n x n matrix A . What is the re lationship between the
For an Engl ish tran slation of pans of Chapter XI of the Ars Magna (dealing wilh cubic eq ua!ions)
see D. J. Struik (editor). A Source Book in Ma riremmies 1200-1800, Princeton Universi ty Press, 1986.
characteri stic po lynomials of A and A 7 ? What doe your answer tell you
2
See, for example, J. Stoer and R. Bulisch , ln!roductionro Nwnerical Analysis, Springer Verl ag, J980. about the eigenvalues of the two m atr ices?
318 • Chap. 6 Eigenvalues and Eigenvectors
Sec. 6.2 Finding the EigenvaJues of a Matrix • 319
23. Consider two 11 x n mau·ice A and B, where B = s- A S for an in verti ble n x 11
1
the fo ll owing week. The state of this town (as far a grocery shopping is
matrix s. What i the rel ationship between the c haracte ri tic po lynomial · of concerned) can be represented by the vector
A and ß? What does your an wer tell you about the eigenva lue of the two
matr.J. ces.?
24. Find all eigenvalues of t.he mat1ix
x(t ) = [w(t) ]'
m (t)

0.5 where w(t) and m(t) are the numbers of farnilies shopping at Wipfs and at
0.25]
A = [ 0.5 0.75 . Migro , re pectively, t weeks after Migro opens. Suppose w(O) = 1200 and
m(O) = 0.
25. Con ider a 2 x 2 matrix of the form a. Find a 2 x 2 matrix A suchth at ," i(t + I ) = Ax(t). Yeri fy that Ais a regular

A= [~ ~ l
tran ition matrix .
b. How many farniües will shop at each store after t weeks? Give closed
form ul a .
where a, b, c, d arepositive numbers such that a + b = c + d = l (the matrix
c. The Wipfs expect that they mu st close down when they have less than 250
in Exercise 24 has this format). A matrix of this fo rm i called a regt.t!ar
customers a week. Wben does that happen ?
rransition m.arrix._ Verify that [ ~] and [ _ ~] are eigenvecto rs o~ A~hat 29. Consider an n x n matrix A uch that the sum of the entries in each row is 1.
are tbe associated eigenvalue ? I tbe ab olute value of the e e1genva lue Show that the vector
more or Jess than I ? Sketch a phase portrait.
26. Based on your answer to E xer i e 25, ketch a phase portrair of the dynamical
ystem
_ 1 [o.5 o.25]- c)
x(t + ) = 0.5 0.75 X I . in IR" i an eigenvector of A. What is the corresponding eigenvaJue?
27. a. Based on your answer to Exercise 25, find closed formulas for the com- 30. a. Consider an n x n matrix A uch that the sum of the entries in each row is
ponents of the dynarnical y tem v
I and all entries are positive. Consider an eigenvector of A with positive
components. Show that the associated eigenvalue is les than or equal to
- + l )= [0.5
x(l _ o.25]- c) I . Hin t: Consider the !arge t entry of v. What can you ay about the
05 0.75 X I .
corresponding entry of A ii?
x
with initial value 0 = e1. Then do the ame for the initial value xo = e2. b. lf we drop the requirement that the components of the eigen vector ü be
Sketch the two trajectorie . po itive, i it still true that the associated eigenvalue i Je than or equal
b. Consider the matrix to 1 in ab olute value? Justify your answer.
A = [0.5 0.25] 31. Con ider a matrix A with po itive entrie such that the entries in each column
0.5 0.75 . add up to 1. Ex.plain why 1 is an eigenvaJue of A. What can you say about
the other eigenvalues? Is
Using technology , compute some powers of the matri x A: say A2, A5
A 10 , . ... What do you ob erve? Explain your an wer carefully.

c. Tf A = [ ~ ~] is an arbitrary regular transition marrix, what can you ay

about the powers A 1 as 1 goes to infinity?
28. Con ider the isolated Swiss town of Andelfin gen, inhabited by 1200 families. nece sarily an eigenvector? Hint: Consider Exercises 22 29, and 30.
Each farnily takes a weekly shopping trip to the only grocery store in town, 32. Consider the matrix
run by Mr. and Mrs. Wipf, until the day when a new, fancier (and cheaper) 0 1
chain store, Migros, opens its doors. It is not expected that everybody will A = 0 0
[ k 3
immediately run to the new store, but we do anticipate that 20% of those
shopping at Wipfs one week switch to Migros the following week . . Same wher k i an arbitrary constant. For whi ch values of k does A have three
people who do switch mi . s the personal service (and the goss ip) and switch di tinct real eigenvalues? What happen in the other case ? Hint : Graph the
back: we expect that 10% of tho e shopping at Migros one week go to Wipf' s fun ction g(A.) = A. 3 - 3A.. Find it local maxima and rninima.
320 • hap . 6 Eigenvalue and Eigenvectors Sec. 6. 3 Finding the Eigenvectors of a Matrix • 321
33. a. Find th e characteri tic polynomial of the matrix Thi s solution can also be wri tten as

I 0]I .
0 3 q
b c 2
b. Can yo u find a 3 x 3 matr.ix M whose characteri tic polynorn:iaJ is What can go wrong when p is negative?
e. Con ider an arbitrary cubi c equation
>.. 3 - l7 A.2 + 5>.. - rr ?
3
34. Suppo e a certain 4 4 matrix A ha two. distinct real eige.nvalues. What
x + ax 2 + bx + c = 0.
cou ld the algebraic multipl icitie of these e1genvalue be? G1ve an example Show that the Substitu tion x = t - (a / 3 allows you to write thi s equation
for each poss ible case and ketch the characteristic polynomial. as
35. Give an exan1ple of a 4 x 4 matrix A without real eigenva lues.
t3 + pt = q.
36. For an arbitrary positive integer n, give a 2n x 2n matr.ix A without real
eigenvalues.
37. Con ider an eigenvalue A{) of an n x n matrix A. We are told that the algebraic
multiplicity of A{) exceeds 1. Show that f~ (>..o) = 0, that i , the derivative of FINDING TBE EIGENVECTORS
the characteristic polynomiaJ of A vanishes at A{). OF A MATRIX
38. In hi groundbreaking text Ars Magna (Nuremberg, 1545), the ltalian mathe-
matician Gerolamo Cardano explains how to solve cubic equations. In Chap- Afterwehave found an eigenvalue >.. of an n x n matrix A, we may be interested
ter XI, he considers the following example: in tbe corresponding eigenvectors. We bave to find the vectors ü in IR" such that
x 3 + 6x = 20. or (Aln- A)Ü = 0.
a. Explain why this equation has exactly one (real) so lution . Here, this In other words, we have to find the kernel of the matrix >..!" - A. The following
solution is easy to find by inspection. The point of the exercise is to show notation i useful:
a systematic way to find it.
b. Cardano explains bis method as follows (we are usi ng modern notation
for the variables): " J take two cubes v3 and u3 whose difference shall be Definition 6.3.1 Eigenspace
20, so that the product vu shall be 2, that is, a third of the coefficient of
Consider an eigenvalue A. of an n x n matri x A. Then tbe kerne! of the matrix
the unknown x . Then, I say that v - u is the value of the unknown x ."
A./ 11 - A is called the eigenspace associated with A., denoted by E>.. :
Show that if v and u are chosen as stated by Cardano, then x = v - u is
indeed the solution of the equation x 3 + 6x = 20. EJ.. = ker(A/11 - A)
c. Salve the system
Note that E>.. consists of aJI solutions ii of the equation
3 3
v - u = 20 I Aü = A.u.
I vu = 2
to find u and v.
d. Consider the equation In other words , the eigenspace E>.. consists of all eigenvectors with eigenvalue
A., tagether with the zero vector.
x3 + px = q
EXAMPLE 1 ~ Let T (x) = A.i be the orthogonal projection onto a plane E in IR . Describe the
where p is positive. Using your work in parts a, b, and c as a guide, show eigenspaces geometricaUy.
that the unique solution of this equation is
Solution
3 q
X = - -+
2
We can find the eigenvectors and eigenvalue by inspecüon: Th nonzero vector
ü in E are eigenvectors with eigenvalue 1, because A ü = ü = 1ü. The eigenspace
322 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.3 Finding the Eigenvectors of a Matrix • 323

Eo: 1l1e vectors ii _

such lhal Aii = Oii = 0
'

[7-\- p -----,-
-> ' E1: Lhe vectors ii

L-----------:------~
/
such lhal Au = 1u = u

Figure 1

Figure 2
E 1 consists of all vectors ü in IR such that A Ü = I Ü = ü, that is, E 1 is ju t the
plane E. Likewise, Eo i the line EJ. perpendicular to E . Note that Eo is ~i mply
the kerne! of A, that i , Eo consists of all solutions of the equation A ü = 0. See
We can (and should) check that the vectors we found are indeed eiaenvectors of
Figure l. ~ A, with the eigenvalues we claim:
ö

To find the eigenvectors a ociated with a known eigenvalue A. algebraically,

we seek a basis of the eigenspace E>.· Since E>. = ker(A/" - A) th i amounts to
finding a basi of a kerne!, a problern we can handle (see Section 3.3) .

EXAMPLE 2 .... Find the eigenvector of the matrix A = [! ~ J. Both eigenspaces are lines, as shown in Figure 2.

EXAMPLE 3 .... Find the eigenvectors of

Solution
In Section 6.2, Example 1, we saw that the eigenvalues are 5 and - I. Then

E 5 = ker(S / 2 - A) = ker [ _: -2]2 .

Solution
It is easy enough to find this kernet algebraically, using the reduced row echelon
The eigenvalues are l and 0 (the diagonal entries of A), with algebraic multiplic-
form:
ities 2 and I, respectively. Then
Es=ker[ _ : -;J=ker[~ -~2 ] = span[ i
12
]= span[;J. -I
-1]
1 - 1 .
We can also think about this problern geometrically: we are looking for a vector 0 0
that is perpendicular to the two rows (which are parallel, of course). A vector
To find thi kerne! , bring the matrix into reduced row echelon fom1 and olve the
perpendicular to [ -~ J is [;] (swap the two components and change one of the corresponding system.
signs). Therefore,

Es = ker [ _ : - ;] = span [ ~] = span [; J [~

- I

0
1 -1]
-1
0
rref
-+
[~
1
0
0 !]
and The general solution of the system

-2 IX2 = 01
E-1 = ker [ _
-) 4 X3 = 0
324 • ha p. 6 Eigenvalues and Eigenvectors Sec. 6. 3 Fi nding the Eigen vectors of a Ma trix • 325
To discu s these different case , the following termin ology is useful :

Definition 6.3.2 Geometrie multiplicity

C 01~ sid e r an eigenvalue Ä of a matri x A. Then the dimension of the eigenspace
Therefore E;,_ IS ca lled the geometric multiplicity of Ä.

Example 3 shows that the geometri c multiplicity of an eigenvalue may be

di fferent from the al gebraic multipli city: we have
Now Iet us think about the eigenspace Eo:
(algebraic multip licity of eigenvalue I) = 2,
I l
Eo = ker 0 0 but
[ 0 0
(geometric mu ltiplicity of eigenvalue 1) = 1.

I
G+ X2 f.:\ = 0
~ = 0
I However, the following inequality always holds:

The generat solution is

[-X?]
~2 = X2
[-1]
~
Fact 6.3.3 Consider an eigenvalue ), of a matrix A . Then
(geometric multiplicity of Ä) :::: (algebraic multiplicity of Ä).

Therefore, To give an elegant proof, we need some additional macrunery, wruch we

will develop in Chapter 7· the proof will then be left as Exercise 7.2.60.
The following example illustrate the relationsrup between the algebraic and
geometri c multiplicity of an eigenvalue.

Both eigenspaces are lines in the x 1x 2-plane, as shown in Figure 3. EXAMPLE 4 .... Consider an upper triangular matrix of the form
Note that Example 3 i different from Example l , where we studied the l • • • •
orthogonal projection onto a plane in ~3 . There too, we have two eigenvalues, I 0 2 • • •
and 0, but one of the eigenspaces, E,, is a plane. A= 0 0 4 • •
0 0 0 4 •
0 0 0 0 4
Figure 3
where the bullets are placeholders for arbitrary entries. Without referring to
Fact 6.3.3 , what can you say about the geometric multiplicity of the eigenvalue 4 ?

Solution
3 • • • •
0 2 • • •
E4 = ker 0 0 0 • •
0 0 0 0 •
0 0 0 0 0
326 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.3 Finding the Eigenvectors of a Matrix • 327
The reduced row echelon fo rm is E0 = EJ..

CD o •••
0 0 ...
0 0 0 • •
0 0 0 0 •
0 0 0 0 0

where any of the bullet in rows 3 and 4 could be leading 1 's. Figure 4
The number of leading variables wi II be between 2 and 4, and the dimension
ofthe kernel (i.e., the dimen ion of E4 ) will be between 3 (=5 - 2) and 1 (=5 - 4).
We can conclude that the geometric multiplicity of d1e eigenvalue 4 is between J
and 3. This result agrees wi th Fact 6.3.3, since d1e algebrai c multiplicity of d1e
eigenvalue 4 is 3. <11111
Example 2 Revisited • A = [! ~].
Using Example 4 a a guide, can you give a proof of Fact 6.3.3 for triangular The vector [ ~] and [ - ~]form an eigenbasis for A, as sbown in Figure 5.
matrices? See Exercise 30.

When we analyze a dynamical system

x(t + 1) = Ax(t ) ,
Example 3 Revisited • A ~ [~ ~ ~] .
There are not enough eigenvectors to form an eigenbasis. For example, we
wbere A is an n x n matrix, we may be interested in finding a ba is Vt. v2, e
are unable to express 3 as a linear combination of eigenvectors. See Figure 6.
v" of IR" that consists of eigenvectors of A (recall the introductory example in
Section 6.1, and Fact 6.1.3). Such a basis deserves a name.
Based on Example 3, ü is easy to find a criterion that guarantees there will
be no eigenbasis for a given n x n matrix A: if the sum of the dimensions of
the eigenspaces is Jess tban n, then there are not enough linearly independent
Definition 6.3.4 Eigenbasis eigenvectors to form an eigenbasis.
Conversely, suppose the dimensions of the eigenspaces do add up to n, as
Consider an n x n matrix A. A basis of IR" consisting of eigenvectors of A in the first two examples discussed above. Can we con truct an eigenbasis for
is called an eigenbasis for A. A simply by picking a basis of each eigenspace and combining the e vectors?

Below we discuss the following questions: Figure S

• How can we teil whether there is an eigenbasis for a given matrix A?

• How can we find an eigenbasis if there is one?

Recall Examples 1 to 3 above.

Example 1 Revisited • Projection onto a plane E in JR 3 . Pick a basis 1, Ü2 ofv [_:]

v v v
E and a nonzero Ü3 in E l. . The vectors 1, 2 , 3 form an eigenbasis. See Figure 4.
328 • Chap. 6 Eigenva lues and Eigenvectors Sec. 6.3 Finding th e Eigenvector · of a Matri x • 329
Fact 6.3.5 v
Con ider the eigenvector ii 1, 2 , . . . , Ü111 of an n x n matrix A, with d i tin ct
e igenvalues AI, A2, . . . , A111 • The n the Ü; are linearly indepe ndent.

Proof T he proofs of Fact · 6.3.5 and 6.3.7 are somewhat technjcaJ. You may wi h to
kip the m in a fi r t reading of this text.
We arg ue by ind uction on m . We Jeave the case m = 1 as an exercise, and
assume that the claim holds for m- 1. Consider a relation

Figure 6 C! VI + · · · + Cm - l V
111 - l + CmV111 = Ö.
Fir t, we apply the transforrn ation T (x ) = Ax to both side of eq uation (I , keeping
in mi nd that Av; = A; ii;:
Thi s approach certainly works in the two example. above. However, in more
compli cated cases we mu t worry about the linear independence of the n vectors
we fi nd by combining ba e of the various eigenspace . Let us first thi nk about a Next, we multip ly both sides of equation (I) by Am:
simple case.
(III)
EXAMPLE 5 ..... Consider a 3 x 3 matrix A with three eigenvalues. I , 2 , and 3 . Let ii " ii2 and Ü3
be corresponding eigen ector . Are the Ü; necessaril y linearly independent? Now subtract equ ation (III) from equation (11).

Solution
Consider the plane E pan ned by ii 1 and ~ . We have to examine whe ther Ü3 could
be contained in this plane. More generall y, we examine whether the plane E could
contain any eigenvectors besides the multiples of ü, and Ü2. Consider a vector =
c 1ü1 + c 2 ~ in E (with c 1 =f. 0 and c2 =f. 0). Then Ax = c, Aii , + c2 Ai:i2 = c, i:i, +2cz iiz.
Tru s vector is not a scalar multiple of that is, is not an eigenvector of A. See x,
Figure 7.
We have shown that the plane E does not contain any eigenvect.ors beside.
the multiples of ii 1 and v2 ; in particul ar, v3 is not co ntained in E. We conclude
that the vecto rs ii 1, ii2, and v3 are Linearly independe nt. <11111
T he o b ervation we made in Example 5 generalizes to hi ghe r dimensions.
(IV)
B y ind uction, the v; in equation (IV ) are linearly independent. Therefore, equation
(IV) mu t represent the tri via! relation, that i , c; (A; - A111 ) = 0, fo r i = I , . . . ,
m - J. The eigenvalues were assumed to be d istinct; therefore A; - A111 =f. 0, for
x i = 1, .. . , m- I. We conclude that c; = 0, for i = 1, . .. , m - 1. Eq uation (I)
x now te ils us that C111 Üm = Ö, so that C111 = 0 a well.
•
The proof above gives us the foll owin g re ult.
Fact 6.3.6 If an n x n m atrix A has n distinct eigenva lues, then there is an e igenbasis
for A: we can construct an e igenbasis by choos ing an eigenvector fo r each
eigenvalue.

Figure 7 , - - - -- --- - AX = c 1ii 1 + 2c2 ii 2 EXAMPLE 6 ..... Ts there an eigenba is for the fo ll owing matrix?
I I
I I
I I
1 2 3 4 5 6
I I-X 0 2 3 4 5 6
~- -- - - = c 1-u 1 + c2 u- 2
I I 0 0 3 4 5 6
I A= 0 0 0 4 5 6
I
_ __ _ _ _; 0 0 0 0 5 6
ii , 0 0 0 0 0 6
330 • hap. 6 Eigenva lue and Eigenvectors
Sec. 6.3 Findin g th e Eigenvectors of a Ma trix • 331
Solution We leave it a : an exerci se to in terpret the entri es of A in terms of reprodu ction
Ye . because there are 6 di tin ct igenvalues, the di agonal entrie of A: 1, 2, 3, rares and surv1va l rares.
4, 5. 6. ~ Supp~se the initi al popul ations are j 0 = 750 and mo = a0 = 200. What will
the popul at10ns be after t years, accordin g to this model? What will happen in
What if an n x n mau'ix A has Jess than n di stin ct eig · nvalues? Then we the long tenn ?
have to consider the geo metric multiplicities of the eigenvalue . .
Solution

Fact 6.3.7 Con ider an 11 x 11 matrix A. If the geometric multi plic itie of tbe eigenval - To an wer the e que tions, we have to find an eigenbasis for A (see Fact 6.1.3).
ue of A add up to 11 , then there i an eigenbasi fo r A: we can construct
an eigenba i- by choo ing a ba i of each eigen pace and com bini ng these Step I Find the eigenvalues of A. The characteristic polynomial i
vector . ). - 0.95 - 0 6]
f11 (!..) = det - 0 .8
[ 0
1.. ? = ),3 - 0.76). - 0.24.
Suppose the eigenvalues are 1.. 1, 1..2 , .. • • A"" with dim (E>J = d;. We first choose - 0.5 "
Proof
a ba i ü1 ii 2, . . . , ud, of E>., , then a basis üd,+r. . . . , iit~, +d2 of E-J.. 2 and so on. Note that AJ = 1 is an eigenvalue. To find the others, we factor:
We have to show th at the vector ü1 ••• , ii" found in thi way are linearly
f11 (A) -7- (). - ÄJ) = (1.. 3 - 0.761.. - 0.24) -7- (A- 1)
independent. Consider a relati on
= 1..
2
+ 1.. + 0.24
c1ii1 + · · · + cd, - d, + ··· + cd,+d2 iid,+d1 + ·· · + · · · + c" ü" = 6. = (!.. + 0.6)().. + 0.4)
"----v--"
w1 in E,., The characteri ti c polynomial is
Each of tl1e vectors ÜJ; above eitl1er is an eigen vector with eigenva lue A.; or else
fA (!..) = (!.. - J)(!.. + 0.6)(1.. + 0.4)
is Ö. We wi sh to sbow that the ÜJ; a:re in fact all 6. We argue indirectly, assuming
that some of the w;
are nonzero and showing that thi s assumpti on Ieads to a con- and the eigenv alues of A are
tradiction. Those ÜJ; which are nonzero are eigenvector with di tinct eigenvalues; !..1 = 1, !..2 = - 0.6.
iliey are linearly _0dependent, by Fact 6.3.5. But here i our contradiction: the
sum of the ÜJ; is 0; t~erefore they are linearly dependent. Now we know that an eigenbasis exists for A, by Fact 6.3.6 (or Example 5).
w
Because 1 = 0, it follo w that c1 = c1 = · · · = cd, = 0, ince ü1, v2 , ... Step 2 Construct an eigenbasis by finding a basis of each eigenspace.
vd , are linearly independent. Likew ise, all the other Cj are zero. •
1
- 0.8
- 0.95
1
06]
We conclude this section by workin g another exampl e of a dynamical system:
E 1 = ker
[ 0 - 0.5
- 0.
1
EXAI\IPLE 7 ..... Con ider an Albanian mountain farmer who raises goats. Thi particular breed of
~ =;·
5
goat has a life span of three years. At the end of each year t ilie farmer conducts
= ker[b ] (use technology to find rref)
a censu of his goats. He counts the number of youn g goats j (t) (those born in
0 0 0
the year t ), the middle-aged ones m (t ) (bom the year before), and tl1e old one
a (t ) (born in the year t - 2). The tate of the herd can be repre ented by the vector IXJ - 2.5X3= 0 I
X1 - 2x3=0
x(t ) = [~~~/) ] . The general solution is:
a(t )
2.5x3] [2.5]
2x3 = 2
How do we expect the population to change from year to year? Suppose that for
thi s breed and this environment the evolution of the system ca n be n~.odeled by
[ X3
x3
J

[~s
0.95
x(t + t) = Ax(t ) , where A = 0 00.6 ] . E 1 = span 7
2.5] = pan [5]
~
0.5 0 [
332 • hap . 6 Eigenvalues and Eigenvecto rs Sec. 6.3 Fin d ing the Eigen vecto r o f a Ma tri x • 333
Similarly, we find
[ -~ _;J.
Lo.6 ~ pan [-:~ l 5.

8. [~ ~ n
n-2]
We have constructed the eig nba i

'·~ m -, ~ [-:~l l
n
11. [ : : :

-~ ~
with a ociated eigenvaJue
A.3 = - 0.4.
4 0

),, =I , A.2 = - 0.6, 14. [

-3
Step 3 Express the initial tate vector as a lin ear combination of eigenvec-
tor . Wehave to solve the linear . y tem .X:o = c,ü, + c2Ü2+c3Ü3. U in g technology,
we find that c 1 = 100, Ce =50 and c3 = 100.
Step 4 Write tbe clo ed formula and cliscu s tJ1e long-term behav ior. See
Fact 6.1.3 .
-n l
19. Con ider tbe matrix
17•
~ 0~ 0~ 0~l -
0
0 0 0
°

Then
1
=
1 l b] a
x(t ) =A'.t o = c, A' ii , + c2A' ü2 +c3A Ü3
= c I).\ ü, + C2A~ V2 + C3 >-~ V)
A
[0 0 c
0 I

~100m + 50(-0.6)' [ _ :n + 100(-0.4)' [ ~n .

where a, b, c are arbitrary con tants. How does tbe geometric multipJjcity
of the eigenvalue 1 depend on the con tants a b, c? When doe A have an
eigenba i ?
The individual populations are given by 20. Con ider the matrix
j (1) = 500 + 450( - 0.6)' - 200 (- 0.4)' '
= 400 - 600( -0.6)' + 400( - 0.4)' ,
1 b] a
m (r )
a (t ) = 200 + 500( - 0.6)1 - 500( - 0.4)' .
A=
[00 01 2c ,
In the long term, the populations approacb the equilibrium value
where a, b, c are arbitrary constants. How do the geometric multiplicities of
j = 500, m = 400, a = 200. the eigenvalues 1 and 2 depend on the constants a , b c? When is there an
eigenbasis for A?
21. Find a 2 x 2 matrix A for which
EXERCISES
GOALS For a given eigenvalue, find a basis of the a. ociated eigenspace. Use and
the geometric multiplicilies of the eigenvalues to determine whether there i an
eigenbasis for a matrix . How many such matrices are there?
22. Find all 2 x 2 matrices A for which
For each of the matrices in Exercises l to 18 below, find all (real) eigenvalues.
Then find a ba<;is of each eigenspace, and find an eigenbasis if you can. Do not E7 = IR2.

J
use techno1ogy.
~ ~
2. [ ~ ~ l 23. Find aJI eigenvalues and eigenvectors of A = [
Interpret your result geometricaJly.
I ther an eigenba is?
334 • Chap. 6 Eigenvalue and Eigenvectors
Sec. 6.3 Finding the Eigenvectors of a Matrix • 335
33. Con ider a ymmetric n x n matrix A.
24. Find a 2 x 2 matrix A for wlüch
a. Show that if ü and w
are two vectors in Rn, then

Av · w= v · Aw.
b . Show_tl~at if ü andw are two eigenvectors of A, with distinct eigenvalues
then w IS orthogonal to v.
i the onl y eigeospace.
34. Consider a rotation T (x) = Ax in iR3 (that is, A is an orthogonal 3 x 3
25. What can you say about the geometric mul ti pli c ity of the e igenval ues of a
mat1ix with deternünant 1) . Show that T has a nonzero fixed point (i.e., a
matri x of tbe form vector v with T (ü) = v). T his result is known as Euler's theorem after the
great Swi s mathematician Leonhard Euler (1707-1783). Hint: Co~sider the

[~ ~]c .
characteristic polynonüal JA. Pay attention to the intercepts with both axes.
A = b Use Fact 6.1.2.
b 35. Consider a s ubspace V of R " with dim(V) = m.
a. Suppose the n x n matrix A represents the orthogonal projection onto V .
where a , b. c are arbitrary coo tant ? What can you say about the eigenvalues of A and tbeir algebraic and
geometric multiplicities?
26. Tru e or fa lse? l f a 6 x 6 matri x A has a negative determinant, then A ha
b. Suppose the n x n matrix B represents the ref:lection in V. What can
at Jea t one positive eigenvalue. Hint: Sketch the graph of the characteristic
you say about the eigenvalues of B and their algebraic and geometric
polynonüal. multiplicities?
27. Consider a 2 x 2 matrix A. Suppo e that tr(A) = 5 and det (A) = 6. Find the
36. Let x(t) and y(t) be the annual defense budgets of two antagonistic nations
eigenvalues of A . (expressed in billions of US dollars). The change of these budgets i modeled
28. Consider the rnatrix by the following equations:

A. 0 0 x(t + l) = a x(t) + b y(t)

0 A. 0 y(t + 1) = bx (r ) + a y(t ),
J/1(),) = where a is a constant slightly less than l , expressing the fact that defense
budgets tend to decline when there i no perceived threat. The constant b is
0 0 0 a mall positi ve number. You may a sume that a exceeds b.
0 0 0 Suppo e x(O) = 3 and y( O) = 0.5. What will happen in the long term?
Tbere are three possible ca e , depending on the numerical value of a and b.
(all A.'s on the diagonal and I s directly above), where A. is an arbitrary Sketchf'h trajectory for each case, and di cu the outcome in practical terms .
constant. Find the eigenvalue(s) of 111 (A.), and deternüne their algebraic and Tnclude the eigenspaces in all your s ketche .
geometric multiplicity . 37. Con ider a modification of Ex ample 7 : uppo e the transforrnation matrix is
29. Cons ider a diagonal n x n matrix A with rank (A) = r < n. Find the algebraic
and the geometric multiplicity of the eigeovalue 0 of A in terms of r and n.
30. Consider an upper triangular n x n matrix A with a;; -:j:. 0 for i = 1, 2, .. . , A = [~8 1.4 1.2]
0 0
0.4 0
.
m and a;; = 0 for i = m + 1, ... , n . Find the algebraic multiplicity of the
eigenvalue 0 of A. Without u ing Fact 6.3.3, what can you ay about the The initial populations are j (0) = 600, 111 (0) = 100, and a (0 = 250. Find
geometric multiplicity ? clo ed formulas for j(t) , m(t), and a(t). Describe the Iang-term be havior.
31. Suppose there is an eigenbasis for a matrix A. What is the rel ationship What can you say about the proportion j (t ): m(t): a(t) in the lang tem1?
betwee n the algebraic and geometric multiplicities of its eigenvalues? 38. A street magician at Montmartre begin · to perform at 11 :00 p.m . on Saturday
32. Consider an eigenvalue A. of an n x n matrix A. We know that A. is an eigen- night. He tarts out with no onlooker , but he attract pa ersby at a rate of
value of AT as weil (since A and AT have the same characteri . tic polynomi al). 10 per minute. Same get bored and wander off: of the people pre ent t
Campare the geometric multiplicities of ), as an eigenvalue of A and AT. nünutes after 11 :00 p.m ., 20% will have left a minute later (but everybody
336 • Chap. 6 Eigenva lues and Eigenvector
Sec. 6.3 Finding the Eigenvectors of a Matrix • 33 7
stays for at least a minute). Let C(l ) be the ize of the cro>vd I minutes after a. Explain the significance of the entries of tbe tran formation matrix in
practical terms.
11 :00 p.m. Find a 2 x 2 matrix A uch that
b. Find closed formulas for the amount of pollutant in each of the tbree lakes
[ C(t ~ 1)] = A [ C
1r)]. t weeks after the accident. Graph the three functions against time (on the
same axes) . When does the poUution in Lake Sils reach a maxünum?
Find a closed formula for C (r), and graph thi s function . What is t.he lon g-term
behavior of C (1) ?
39. Tbree fr.iends , Alberich, Brunnhilde, and Carl , play a number game together:
each think of a (real) number and announces it to the others. In the fir t round,
each player find the average of the number chosen by the two others· that Ri ver lnn
is bis or her new score. In the second round , the corresponding averages of
the cores in the fir t round are taken , and so on. He re i an example:

A B C

Initial choice 7 II 5
After Ist round 8 6 9
After 2nd round 7.5 8.5 7

Wboever is ahead after 100 l rounds wins.

a. Tbe state of the garne after r rounds can be represented as a vector:
a(t)] Alberich 's score 41. Consider a dynarnical system
x (t ) =
[ b (1)
c(t)
BrunnhiJde' score
Carl ' s score x(t) = [x1 (t(t )) J
X2
Find the matrix A such that i (l + I ) = Ai 1).
b. With the initial values mentioned above (a 0 = 7, bo = 11 , co = 5), what whose Iransformation from time t to time t + 1 is given by the following
is the score after 10 rounds? After 50 rounds? Use technology. equations:
c. Now suppose tbat Alberich and Brunnbilde initiaUy pick the numbers I Xt(t + l)=O.lx,(t) +0.2x2(l) + 1,
and 2, respectively. If Carl picks the number c0 , what is the state of the x2(t + 1) = 0.4x 1 (t ) + 0.3x2 (t) + 2.
game after r rounds? (Find closed formulas for a (1), b (t ), c(t), in terms
of c0 .) For whicb choices of c0 does Carl win the game? Such a system, with constant terms in ~e equations is not linear but affine .
a. Find a 2 x 2 matrix A and a vecror b in ~2 such tbat
40. In an unfortunate accident involving an Austri an truck, I 00 kg of a highly
toxic substance are spilled into Lake Silvaplana, in the Swiss Engactine Val- xCt + 1) = A-"i(r) + E.
ley. The river lnn carrie tbe poUutant down to Lake Sils and later to Lake
St. Moritz. This sorry state, t weeks after the accident, can be described by b. Introduce a new state vector
the vector

[;~~~~]
pollutant in Lake Silvaplana }
x(r) = pollutant in Lake Sils (in kg)
X3(l) poll utant in Lake St. Moritz w.ith a "dummy" 1 in the last component. Find a 3 x 3 matrix 8 such that
Suppose
y (t + 1) = By (t) .
~.6 ~
0.7
.xcr). How is 8 related to the matrix A and the vector b in patt a? Can you
x(r +I) = ~.I ]
[ 0.2 0.8 write B as a partitioned matrix involving A and b? (continued)
338 • Chap. 6 Eigenvalu es and Eige nvectors Sec. 6.3 Finding the Eigenvectors of a Matrix • 339
c. What is the relation hip between the eigenvaJ ue of A a nd B? What about b. lntrod uce the state vector
eigenvectors?
d. For arbitrary values of x 1(0 and x2(0) , what can you say about the long-
term behavior of x , (t ) and x2 (t ) ? y(t) = [ ~~i~~ J
T3(t ) '
42. A machine contains the grid of wires sketched be low. A t the se ven indicated 1
points tbe temperature is kept fixed at the given value (in °C) . Consider tbe
wi th a "dummy ' I as the last component. Find a 4 x 4 matrix B such that
temperature-s T1(t) , T2 (t ), and T3 (t ) at the other three me h po ints. Because
o f heat ft ow aJong the w ires, the temperatures T;(t ) changes according to the .Y(t + I) = B y(t ) .
formula (T hi technique for converting an affine system in to a li near y tem is
1
!0 L ( T;(t ) -
introd uced in Exercise 41 ; ee al o Exercise 38.)
T;(t +I ) = T;(t ) - Tactj Ct))
c. S uppo e the initial temperature are T1(0) = T2 (0) = T3 (0) = 0. Using
technology, find the temperatures at the three points at t = 10 and t = 30.
where the sum is taken over the fo ur adjacent points in the grid and time is
Wh at long-tenn behav ior do you expect?
measured in rnin utes. For example,
d . U ing technology, fi nd numerical approximations for the e igenvalues of
the matrix B . Find an e igenvector fo r the largest eigenva lue . Use the
re ults to confirm your co njecture in part c.
43. The co lor of snapdragons is determined by a pair of genes. w hich we designate
by tbe Ietter A and a. The pair of genes is caJied the flower' genof) pe.
Genotype A A produces red fl owers, genotype Aa pink one and genotype
Note that eacb of the fo ur terms we subtract repre e nts the cooling caused by
aa white o ne . A bi ologist undertakes a breeding prog ram , starring with a
heat flowing aJong one of the wires . Let
]arge population of ftowers of genotype AA. E acb flowe r is fe rtiJi zed with
pollen fro m a plant of genotype Aa (taken from another popuJati on), and one
offs pring is produced. Since it is a matter of chance which of the genes a
parent pa se on, we expect half of the. fl owers in the next generation to be
red (genotype AA) and the otber half pink (genotype Aa). A ll the fto wers
in thi generati on are now fe11ilized w ith po llen from plant of genotype Aa
(taken fro m another popuJ ation), and o on.
0 a . Find c lo ed formul as for the frac tion of red, pink, and w hite fl owers in
the t th generati on. We know that r (O) = l and p(O) = w (O) = 0 , and we
fo und that r ( l ) = p(l ) = ~ and w( l ) = 0 .
200 b. What is the proportion r(t) : p (t ): w(t ) in the long run?
0 44. Leonardo of Pisa: The rabbit p rob fem. Leonardo of Pi sa (c. 1170- 1240),
also kn ow n as Fibo oacc i. was the fir t o utstanding Euro pea n mathematician
after the ancient Greeks. He trave led wide ly in the Islamic world and tudied
Arabic mathematicaJ wri ting. Hi work i in the spirit of the Arabic rn atbe-
0 matics of hi . day. Fibo nacci brought the decimal-positio n y te m to Europe.
200
Tn hi book Liber abaci (1202), Fi bonacci di scusses the fo llow ing problern:

How many pairs of rabbits can be bred from o ne pair in o ne year? A

man has o ne pair of rabbit at a certain pl ace e ntire ly urro unded by
0 400
a wall. We wi sh to know how many p airs can be bred fro m it in one
year if the nature of the. e ra bbits is s uch that they breed e very month
one other pai r and begin to breed in the second month aftcr their birth .
a. Find a 3 x 3 matrix A and a vector b in JR 3 such that
Let the first pair breed a pair in the fi rst month , the n d upli cate it and
x(t + 1) = Ax (t ) + E. there w ill be 2 pairs in a mo nth . Fro m these pair o ne, namely the
340 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.4 Ca mplex Eigenvalues • 341
first, breeds a pair in the s cond month. and thus there are 3 pa:irs Ln b. In hi s paper "On the Measurement of the Circle' the great Greek mathe-
th e econd month. From these i n one month two will become pregnant, matician Arehirnedes (c. 280-210 sc) uses the approx imation
o that in the third month 2 pairs of rabbit wiii be born. Thu s there
are 5 pairs in this month. From these in the same month 3 will be 265 < .J3 < 1351
153 780
pregnant, so that in the fourth month there will be 8 pair ' . From these
pair 5 will breed 5 other pair . which added to the 8 pairs gives 13 to estimate cos(30°) . He does not expl ai n bow he arri ved at these esti-
pair in the fifth month, from which 5 pairs which were bred in that mates . Explain how we ca n obtain these approximations from the dyoam-
same month) will not conceive in that mooth , but the other 8 will be ical system in part a. Hint :
pregnant. T im there will be 2 1 pairs in the sixth month . When we add
to these the 13 pair that are bred in the 7th month, then there will be 4 [ 97 56] A6 _ [ 1351 780]
A = 168 97 ' - 2340 1351 .
in that month 34 pairs [and o on, 55, 89, 144, 233, 377, .. .]. Fi.nally
there will be 377 . And thi nurnber of pairs has been born from the c. Without using technology, explain why
first-mentioned pair at the given place in one year.
1351 - .J3 < 10- 6.
Let j (t) be the number of juvenile pairs and a 1) the number of adult pair 780
after t months. Fibonacci st.art his thought experiment in rabbit breeding Hint : Consider det (A 6 ).
witb one adult pair o j (0) = 0 and a (0 = l. At r = 1, the adult pair will d. Basedon the data in part b, give an underestimate of the form pjq of -/3
bave bred a (juveni le) pair, so a(l) = 1 and j(l) = 1. At t = 2, the in:itial that is bett:er than the one given by Archimedes.
adult pair will bave bred another (juvenile) pair, and last month's juvenile
pair will have grown up, so a (2) = 2 and j (2) = 1.
a. Find formulas expressing a(t + 1) and j(t + 1) in terms of a(t) and j(t) .
Find the matrix A s uch that
.xcc+ 1) = A-r(r), 4 COMPLEX EIGENVALUES
where In Section 6.1 we considered the rotation matrix

x(l) = [~~~~ J. A =[ ~ -~ ]
b. Find closed formulas for a(t) and j (t). Note: You will have to deai with
irrational quant ities here. with characteristic polynornial
c. Find the Iimit of the ratio a(t) / j (r ) as t approaches infinity. The result is
known as the golden section. The goiden section of a li.ne segment Aß is
We observed that this matrix bas no real eigenvalues, because the equation
given by tbe point P such that
A.2 +I=0 or A.
2
=- 1
AP PB has no real solutions. However, if we allow complex solutions, then the equation
)...2 = - 1
A p B
has the two solutions A. 1, 2 = ±i; the matrix A has the complex eigenvalues ±i.

45. Consider a "random" n x n matri x A with zeros everywhere on the diagonal

and below tbe diagonal. What is the geometric m ultiplicity of the eigenvalue • Complex Numbers: A Brief Review
0 likely to be? Let us review some basic facts about complex numbers. We trust that you have
46. a. Sketch a phase portrait for the dynamical system x( t + I) = Ax(t), where at .least a fl.eeting acquaintance with complex numbers. Without attempting a

l
defi nition, we recall that a complex number can be expressed as
A= [; ~ z =a+ib.
342 • hap. 6 Eigenval ues and Eigenvectors Sec. 6.4 Camplex Eigenvalues • 343
lmaginary
axis
3i - -- - - - = ~ + 3i
iz = - b + ia

2i

z= (l + ib

Figure 2
Rea l axis
Figure 1

where a and b are real number . 1 Addüion of complex number defined in a

: = a + ib
natural way. by the ruJe
(a + ib) + (c + id ) = (a + c) + i (b + d ),
and multipli ation is defined by th e ru le
(a + ib )(c + id ) =
that i , we Jet i · i = -1 and di tribute.
(ac - bd) +i ad + bc), ~~z = a- ib

lf z = a +ib is a complex nurnber we caJI a it real part (denoted by Re(z))

Figure 3
and b its imaginary part (denoted by lm(z)). A complex number of tbe form ib
(with a = 0) is called imaginmy .
The set of all complex numbers is denoted by C. The real numbers, ~' fonn
a subset of C (namely , tho e complex numbers w ith imaginary part 0).
Complex numbers can be represented graphically as vectors (or points) in
the complex plane? as shown in F igure 1.
EXAMPLE 1 ~ Con ider a nonzero complex number z. What is the geometric relationship between - = a + ib
z and i- in the complex plane?

Solu tion
4> = arg(z)
If z = a + i b, then iz = -b + ia. We obtain the vector [ - b ] (representing
a

i z) by rotating the vector [ ~] (repre enting z) through an angle of 90° in the Figure 4
counterclockwise direction. See Figure 2. ~

The conjugate of a complex number z = a + ib i defined by (the ign of rhe imaginary part is re ersed). We say tbat z and -= form a conjugare
z= a - ib pair of complex numbers. Geometrically, the conjugate: i the reflection of in 7

the rea l axi , a shown in Figure 3.

1
The !e uer i for the imaginary unit was imroduced by Leonhard Eul er. the mos t prolific mathematician
in hi story. Fora fascinating gli mpse at the hi story of the complex numbers see Tob ias Dantzig, Number:
The Lang uage of Science. Macmillan. 1954.
2 AI o ca lled "Argand plane; · after the Sw iss mathematician Jean Roben Arga nd ( 1768- 1822). The
representation of complex numbers in the plane was introduced independently by Argand , by Gaus . and
by the orwegian mathematician Caspa r We sei (1745- 1818).
Sometimes it is useful to de crib a complex number .in polar coordinates
as shown in Figure 4.
The length r of the vector i called the modulus of .::, denoted by lzl. The
angle qy is ca lled an argument of z; note that the argument i determined only up
to a multiple of 2rr (mathemaücian ay ·' modulo 'hr" . For example, fo r <: = - I
we can choo e tbe argumenr rr or - rr or 3rr.
344 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.4 Camplex Eigenvalues • 345
w

Figure S - 2

Figure 7

In general, if z = r (cos a + i sin a) and w = s(cos ß + i sin ß ), tben

zw = rs( cos(a + ß) + i sin(a + ß )) .

r (cos f/J)
When we multiply two complex numbers, we mu1tiply the moduli and we add the
Figure 6 arguments:

EXAMPLE 2 ..... Find the modulus and argument of z = -2 + 2i .

Solution
lzw l = lzl lwl
arg (zw) =arg z +arg w (modulo 2rr )
lzl = J 22 + 22 = J8. Representing z in the complex plane, we see tbat ~ rr is an
argument of z. See Figure 5. ~
If z is a complex nurober with modulus r and argument </J, we can write z EXAMPLE 4 ..... Describe the transformation T (z) = (3 + 4i )z from C to C geometrically.
as
z = r (cos </> ) + ir (sin<jJ) = r (cos<jJ + i sin<jJ) , Solution
as shown in Figure 6. IT(z)l = 13 + 4i l lzl = 51zj,
The representation
Iz= r(cos <jJ + i sin<jJ) arg(T (z)) = arg(3 + 4i ) + arg(z) = aretau ( ~) + arg (z) ~ 53° + arg (z).
is called the polar form of tbe complex number z.
The transformation T is a rotation-dilation in the complex plane . See Figure 8 .~
EXAMPLE 3 ..... Consider the complex numbers z = cos a + i sina and w = cos ß + i sinß. Find
the polar form of tbe product zw.

Solution Figure 8 Rotale through about 53 o

ond streich the vector by a factor of 5.
Apply the addition formulas from trigonometry (see Exercise 2.2.32):
zw = (cos a + i sin a ) (cos ß + i sin ß )
= (cos a cos ß - sina sin ß ) + i(sin a cos ß + cos a sin ß)
= cos(a + ß ) + i sin(a + ß)
)
T(z2 )
We conclude that the modulus of zw is 1, and a + ß is an argument of z w. See
Rgure7. ~
346 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.4 Camplex Eigenvalues • 34 7
Tbe polar form is convenient for finding powers of a complex number z: if

z = r(cosrj> + i si n rj>)
then
2
2 = r 2 ( cos(2r/>) + i sin(2r/>))

7
11
= r 11 ( cos(nrj>) + i sin (nr/>)) -I

for any positive integer n. Each time we multiply by z the modulus is multiplied
by r and the argument increases by rj>. The formula above is due to the French
mathematician Abraham de tvfoivre (1667-1754).

Fact 6.4.1 De Moivre's formula -i

(cos rj> + i sin 1> )" = cos(nrj>) + i sin(nr/>) Figure 9

EXA.M PLE S ~ Consider the complex number z = 0.5 + 0.8i. Represent the powers z2 , z3 , .. . in Fact 6.4.2 Fundamental theorem of algebra
the complex plane. What is lim -"? Any polynornial p (A) with complex coefficients can be written as a product
1)400

of linear fac tors:

Solution
p(),) = k(),- AJ)(A- A2) ... ( l - A11 ) ,

To study the powers, write z in polar form:

for some complex. numbers A1, A2, . .. , A", and k. (Tbe A; need not be
z = r (cos 1> + i sin rj> ) , distinct.)
Therefore, a polynomial p (Ä) of degree n has precisely n roots if tbey
where are properly counted with tbeir multiplicities.
r = Jo.52 + 0.8 2 :::::: 0.943
For example, the polynornial
and
0.8
p(A) = A2 + 1,
</> = arctan - :::::: 58°.
0.5 which does not bave any real zeros, can be factared over C:
Wehave p (A.) = (A. + i)(A. - i).

z" = r" ( cos(n</>) + i sin(n</>)). More generally, for a quadratic polynornial

The vector representation of z"+ 1 is a little sborter than that of z" (by about q().)=),2 + bA.+c,
5.7%), and z" + 1 makes an angle ljJ :::::: 58° with z". If we connect the tips of where b and c are real, we can find the complex roots
consecutive vectors, we see a trajectory that spirals in toward the origin, as shown
in Figure 9. Note that lim i ' = 0, since r = lzl < 1. ~ -b ± ,Yb2 - 4a c
n---+- oo AJ .2= - - - - - -
2
Perhaps the most remarkable property of the complex numbers is expressed
and
in the fundan1ental theorem of algebra, first: demonstrated by Carl Friedrich Gauss
(in his thesis, at age 22).
 Sec. 6.4 Camplex Eigenvalue • 349
348 • hap. 6 Eigenvalues and Eigenvector

Proving the fundamental theorem would Iead us too far afield. Read any intro-
duction to complex analy i or check Gauss ' s ori g in al proof.
1
The vectors [ ~ l [-~] form a complex eigenbasis (of C 2 ) for A. We can check
thi · result:

+ Complex Eigenvalues and Eigenvectors

The complex mtmber bare some basic algebraic properties with the rea l number ;
mathematician ummarize these properties by say in g that both the real nurober
IR and the complex numbe r C form a field.
2 The great advantage in working with complex eigenvalue is that the char-
To what extent do the re ult and rechnique derived in this text thu far acteri sü c pol ynomial always factors completel y (by the fund amental theorem ,
still apply when we work in C, i.e., when we co n ider comp lex calars, vectors Fact 6.4.2) :
with complex components, and matrice with compl ex e ntri es? We observe that
everything work in <C (or any oth er field without modifi catio ns except for tho e
geometrical concepts which are based on the idea of the dot product Le ngth , angles,
orthogonality, and so on). The dot product in C" i defined in a way that we will
not di cuss in thi s introductory tex t. The whole body of' core Lin ear algebra" can
Fact 6.4.3 A complex n x n matri x has n complex eigenvalues if e igenvalue are c unted
with their algebraic mul tipli cities.
be generalized without diffi cultie , however: ecbelon fo rm , linear tran Formation,
kerne!, image, Linear independence, basi , dimensio n, determinant , eigenvalues,
eigenvectors. Here is a imple example: Although a complex n x n matrix may have fewer than n distinct complex

EXAMPLE 6 ..... Find a complex eigenba is for A = [ ~ -10 ]. eigenvalues (exarnples are / 11 and [ ~ ~ ] ), thi s is literally a coincidence (some
of the A; in the factorization of the characteri tic polynomial coincide). "Most' '
Solution complex n x n matrices do have n di tinct complex eigenvalue . We can conclude
that there is a complex eigenbasis for most complex n x n matrices (by Fact 6.3.6).
We have already observed that the complex eigenvalues are Al.2 = ±i. Then [n thi text , we focu on those complex n x n matrices for wh.ich there is a

~]
complex eigenbasi and often dismi s the other as aberration . Much attention
E; = ker(i h - A) = ker [ _ : = pan [ ; ] is given to these other matrices in more advanced linear algebra course , where
ub tit:ute for eigenba e. are constructed.
and
EXAMPLE 7 ..... Consider an n x n matrix A with complex eigenvalue At. A2, ... , A", Ii ted with
E_; = ker( -i/ 2 - A) = ker [ _
-i
1
-i1] = span [-i ]
1 .
their algebraic multiplicities. What is the relationship betwee n the A; and the
determinant of A? Hint: Evaluate the characteri tic polynomial at A = 0.

Solution
1C. F. Gau s: Werke , lll , 3-56. For an English trans lation ee D. J. Struik (ed itor): A Source ßook /A(A) = det (A /" - A) = (A - At)(A - A2) . . . (A - A")
in Motlzemarics 1200-1800, Princeton University Press, 1986.
2 Here is a Ii t of these properties: .f.4 (0) = det -A) = (- t)"AIA2 . . . A"
I . Addition is commutati ve. Recall that det( -A) = ( - 1)" det(A) by linearity in the row . We can conclude
2. Add ition is as ociati ve.
3. There is a unique number 11 ·uch that a + 11 = a , for all numbers a. Thi s number n i denoted that
by 0.
4. Foreach number o there is a unique number b such that a + b = 0. This number b is denotcd
by -o . (Comment: Thi s propeny says that we can ubtract in thi s number ystem .)
5. Mu ltipli cation i commutative.
6. Multiplication i a sociative. Can you interpret thi re ult geometrically when A i, a 3 x 3 matrix with a
7. There is a unique number e such that ea = a, for a ll numbers a . This number e is denoted real eigenbasis? Hint: Think about the expan ion factor. See Exerci e 18.
by I.
8. For each nonzero number a thcre is a unique number b such that ab = I . This number b is Above we have found that the determin ant of a matrix i the product of
denoted by a - 1. (Comm enr: Thi s propen y ays that we can divide by a no nzero nurnbcr. ) .it complex eigenvalues. Likewise, the trace i the surn of the eigenva lue . The
9. Mullipli cation disuibutes over addition: a(b + c) = ab+ ac .
verification is left as Exerci e 35.
10. The nurnbers 0 and I introduced above arenot eq ual.
350 • Chap. 6 Eigenvalues and Eigenvertors Sec. 6.4 Complex Eigenval ues • 351
Consider an n n matrix A with cornplex eigenvalues A1, A2, .. . , A", listed For Exercises 13 to 17 : Which of the follow ing- are jields (with the customary
Fact 6.4.4 addi tion and multiplication)?
wi th their algebraic rnultiplicities. Then
13. The rational numbers Ql.
tr(A) = A1 + A.2 + · · · + A"
14. The integers Z .
and 15. The binary digits (introduced in Exercises 3.1.53 and 3.1.54).
det(A) =At · A2 · · · · · A.".
16. The rotati on-dil ation matrices of the form [ P. -q] , wbere p and q are real
q p
numbers.
Note that this result is obvious for a triangul ar matri x: 1n thi s case the 17. The set H considered in Exercise 4 .3 .34.
eigenvalues are the diagonal entiies. 18. Consider a real 2 x 2 matrix A with two distinct real eigenvalues, )._ I and A.2 .
Explain the formula det(A) = A1A2 (or at least Idet(A) I = IA.1A21) geometri-
cally, think:ing of I det(A) I as an expansion factor. Illustrate your explanation
EXERCISES with a sk.etch . Is tbere a similar geometric interpretation for a 3 x 3 matrix?
GOALS Use the basic properties of complex numbers. Write products and 19. Consider a subspace V of IR", with dim ( V) = m < n .
powers of complex numbers in polar form . Appl y the fund amental tbeorem of a. If the n x n matrix A represents the orthogonal projection onto V , what is
tr(A)? Wbat is det(A)?
algebra.
b. If the n x n matrix B represents the refiection in V , what is tr(A) ? What
1. Write the complex number z = 3 - 3i in polar form. is det(A)?
2. Find all complex numbers z such that z4 = I. Represent your answers graph-
Find all complex eigenvalues of the matrices in Exercises 20 to 26 (includi ng the
ically in the complex plane.
real ones, of course). Do not use techuology. Show all your work.

n
3. For an arbitrary positive integer n, find all complex numbers z such that
z" = 1 (in polar form). Represent your answers graphically.
4. Show that if z is a nonzero complex number, then there are exactly two 20. [32 -5 ]
-3 . 21. [ 1 ~ - 15 ]
-7 . 22. [_! 1~ J. 23.
[0 0 1 0
complex numbers w such that w2 = z. If z is in polar fom1, describe w in 0 1

n [~ n -:l
polar form. 0 0

[~
I [ 1 - 1 1
5. Show that if z is a nonzero complex number, then there are exactl y n complex 0 0 1 1 1
24. 0 25. 26 1 .
numbers w such that w" = z. If z is in polar form, wiite w in polar form. 1 0 · 0 0 l
-7 0 1 0 0 l 1
Represent the vectors w in the complex plane.
6. If z is a nonzero complex number in polar form , describe 1/z in polar fmm. 27. Suppose a real 3 x 3 matrix A has only two distinct eigenvalues. Suppose
z
What is the relationship between the complex conjugate and l jz.? Represent that tr(A) = 1 and det (A) = 3. Find the eigenvalues of A with tbeir algebraic
the numbers z, z, and 1/z in the complex plane. multiplicities.
7. Describe the transfonnation T (z) = (1 - i )z fron1 C to C geometrically. 28. Suppose a 3 x 3 matrix A has the real eigenvalue 2 and two complex conjugate
8. Use de Moivre' s formula to express cos(3<j>) and sin (3<j>) in terms of cosq) eigenvalues. Also, suppose that det (A) = 50 and tr(A) = 8. Find the complex
and sin <j>. eigenvalues.
9. Consider tbe complex number z = 0 .8 - 0.7i . Represent the numbers z2, z\ 29. Consider a matrix of the form
.. . in the complex plane and explain their long-term behavior. 0 a
10. Prove the fundamental theorem of algebra for cubic pol ynomi als with real A= [ ~ ~
coefficients.
11. Express the polynomial f ().) = A3 - 3A2 + 7 A- 5 as a product of linear factors where a, b, c, d are positive real numbers. Suppose the matrix A has three
over C. distinct real eigenvalues. Wbat can you say about the signs of the eigenvalues
12. Consider a polynomial f (.!.) with real coefficients. Show that if a complex (how many of them are positive, negative, zero)? ls the e.igenval ue with the
number .l.o is a root of f (.l.), then so is its complex conjugate, I 0 . largest absolute value positive or negative?
352 • Ch ap. 6 Eigen values an d Eigenvect o rs Sec. 6.4 Ca m plex Eigenva lues • 353
30. A real n x 11 matrix A i al!ed a regul ar tran ition matrix if a ll en~ri es of A For examp le, 20% of the peo ple w ho u e AT&T go to S print one month
are positive, and the e ntries in each co lu mn add up to 1 ( ee Exerc1.ses 24 to later.
3 1 of Section 6.2). A n example i how n be low. a. We introd uce the state vector

0.4 0. 3 0 .1 ]
A= 0.5 0.1 0.2
[ 0.1 0.6 0.7
Yo u may take the fo llowing properties o~ a r g ul ar tra nsitio n matri x fo r
granted (a partial proof i outJined in Exerc1se 6.2.3 1):
• 1 i a n ei oenvalu of A , with d im ( E 1) = 1.
• If )... i a Zomplex eigenvalue of A o ther than I, the n IA.I < I.
a. Consider a reo uJ ar n x n tran iti o n matrix A a nd a vecto r x in IR" w hose
entries add u; to 1. Show that the e ntrie of A.i wi ll also add up to 1 ..
b. Pi ck a regular tra nsition matrix A. and com pute ome power of A u s1~ g
technology): A 2 , •.. . A 10 , .. . , A 100 , •• .. Wbar do you o bser ve? Expla.!Jl
33. Th e power method fo r fin ding eigenvalu.es. Conside r E xercises 30 an d 31 fo r
your ob ervatio n . He re, you may assume that there is a com plex eigenba is
fo r A . Hint: Think abo ut N column by column ; the ith column of N ts
A1 e;.
31. Fo rm a 5 x 5 m arrix by w ri ting the integers I, 2 , 3 , 4 , 5 into each co lumn in
auy order you want. He re is an example:
5 l 2 2 because the e ntri es of B are probably rather large .
l 4 3 3 2
A= 3 3 5 4 3
2 5 4 l 4
4 2 1 5 5
(Optional question for combinatori cs afici o nados: how m a ny suc h matrices
are there?) Take higher and higher power of the m atri x you have chosen
(using technology) and compare the columns of the matrices yo u get. What
do you observe ? Explain the result (Exercise 30 i helpful).
32. M ost long-distance te lephone service in the United S tates is prov ided by
three companies, AT&T, MCI, and Sprint. The tbree companies are in fierce
competition, affering discounts or even ca. h to those w ho sw itch . If the
figure-s advertised by the companies are to be believed , people are switching
their long distance provider from o ne mo nth to the nex t according to the
following pattern :
34. Exercise 33 illu trates how you can use the powers of a m atrix to find it
SPRINT
x(t ) =
a t)]
m (l )
fractio n us ing AT&T
fraction us in g MCl
[
s( t ) fraction us ing Sprint
Find the matri x A such that x(r + I) = Ax(t ) , ass uming that the c u to mer
base re mai ns unch anged. Note that A is a reg ul ar tra ns itio n matri x.
b. Whi c h fract ion of the cusro mer will be with each company in the long
term ? Do yo u have to know the curre nt market shares to answer thi s
q uestio n? U e the power method introduced in E xercise 30.
some background . Usi ng techn ology, gene rate a ra ndo m 5 x 5 matrix A with
no nnegative entries (de pending on the technology yo u are using, the e ntrie
co uJd be integers between 0 and 9, o r number be tween 0 and 1). Usin g
technology, compute B = A 20 (or another hi g h po wer of A). We w ish to
compare the co lurnns of B . This i. hard to do by inspecti o n, particularly
To get a be tter hold on B , form the diagonal 5 x 5 m atrix D whose
ith diagonal element i b 1; , the itb eleme nt of the first row of B . Co mpute
C = BD - 1 •
a. How i C o btai ned fro m B? Gi ve your a nswer in tenn of e le mentary row
or column Operations.
b. Take a Iook at the co lumns of the matrix C you get. What do you o bserve?
W hat does yo ur an wer tell you abo ut the colurnns o f B = A 20 ?
c. Explain tbe o bservation you m ade in part b. You m ay assume that A
has 5 di stinct (complex) eigenvalues and th at the eigenvalue w ith max i-
mal modulu s is real and positi ve (we cannot explain here why thi w ill
" us ually" be the case).
d. Compute AC. What is the sig nificance of the e ntries in tbe to p row of thi
matr ix in terms of the ei genvalu e of A? What is the sig nificance of the
columns of C (or B ) in te nns o f the e igenvectors of A?
dominant eigenvaJue (i.e., the ei genvalue with m axim al modulu s), at lea t
whe n this eige nvalue is real. But what about the other ei genvalues?
a. C o n ider an 11 x n matri x A w ith 11 di tinct complex e igenvalues A., , A._ ,

~ 10%

10%
... , A.," where A. 1 i real. Suppose you have a good (real) approx imatio n A.
of )... 1 (good in tha t IA. - A. 1 1 < IA. - A.; 1. for i = 2 . . . . , n ). Con ider the matri x
AI" - A. What are it e igenvalue ? Wh ic h ha the ma llest modulu . No
MCI
I AT&T consider the matrix (Al" - A) - 1 • What are it · e igenvalues? Whi ch has the
!arge t modulus? What i the re lationship be tween lhe eigenvecto rs of A
20%
354 • Chap. 6 Eigen alues and Eigen ectors Sec. 6.4 Camplex Eigenva lues • 355
and (A/11 - A) - 1? Consid r higher and high er po~ers ?f (A l" - A.) - I. How lf current age-dependent birth and death rates are extrapolated, we have the
does thi help you to find an eigenvector of A wllh e1genvalue A. 1, and A. 1 following model :
itself? U e tbe re ults of Exerci e 33. 1.1 1.6 0.6 0 0 0
b. As an example of part a. consider the matrix
0.82 0 0 0 0 0
0 0 .89 0 0 0 0

~].
I 2 x(t + 1) = x(t ) = Ax(r).
0 0 0.8 L 0 0 0
A = 4 5
0 0
[ 7 0 0 .53 0 0
8 10
0 0 0 0 0.29 0
We wish to find the eigenvector and eigenvalues of A without using the a . Explain the significance of all the entries in the matrix A in terms of
corresponding command on tbe computer (which i , after all, a "black population dynamics.
box'' . First. we find approximations for the eigenvalues by graphing the b. Find the eigenvalue of A with largest modulus and an a sociated eigenvec-
characteristic polynomial (use technology). Approximate the three real tor (use technology). What is the significance of tbese quantities in terms
eioenvalues
0
of A to the nearest integer. One of the three eigenvalues of population dynarnics? (For a summary on matrix technique used in
of A i negative. Find a good approx.imation for thi s eigenvalue and a the study of age-structured populations, see Drnitrü 0. Logofet. Motrices
corre ponding eigenvector by using the procedure outlined in part a. You and Graphs: Stability Problems in Mathematical Ecology, Chapters 2 and
are not asked to do the ame for the two other eigenvalues. 3, CRC Press, 1993.)
35. Demonstrate the formula 37. Con ider the et IHI of all complex 2 x 2 matrice of the form

tr(A ) = )q + Al + · · · + A" A = [ ~ -;J,

where the A; are the complex eigenvalues of the matrix A, counted with their where w and z are arbitrary complex numbers.
1
algebraic multiplicities. Hint: Consider the coefficient of A" - in !11 (A) = a. Show that IHI is closed under addition and multiplication, i.e. , the um and
(A - ),!)(A.- ),1 ) · · · (A. - A.11 ) and compare the result witb Fact 6.2.5. the product of two matrices in IHI are again in IHI.
36. ln 1990, tbe population of the African country Benin was about 4.6 rnillion b. Which matrices in IHI are invertible?
people. lts composition by age was as follows: c. If a matrix in IHI is invertible, is tbe inverse in IHI as weil?
d. Find two matrices A and B in IHI such that AB =1- BA .
IHI is an example of a skew fie/d: it saüsfies all axioms for a field except for
the commutativity of mulüplication. [The skew field IHI wa introduced
Age Bracket 0-15 15-30 30-45 45-60 60-75 75- 90 by the Irish mathematician Sir William Hamilton (1805- 1865); it ele-
Percent of Popu lation 46.6 25.7 14.7 8.4 3.8 0.8 ments are called the quatemions. Another way to define tbe quaternions
is discussed in Exercise 4.3.34.]

!n
38. Con ider tbe matrix

c,~ [~ ~
We represent tbe. e data in a state vector whose components are the populations
in the various age brackets, in rnillions:

0.466 2.14
0.257 1.18 a. Find the powers of C4: CJ C~, C! .... .
0 . 147 0.68
x(O) = 4.6 :::::::: b. Find all complex eigenvaJue of C4, and con truct a complex eigenbasis.
0 .084 0.39 c. A 4 x 4 matrix is called circu/ont if it i of the form

n
0 .038 0.17

~ l~ ~ ~
0.008 0.04

We measure time in increments of 15 years , with t = 0 in 1990. For ex- M

ample, x(3) gives the age composition in the year 2035 (1990 + 3 · 15).
(

356 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.5 Stability • 35 7

Circulant matrices plav an important role in statistics. Show that any circulant In our case:
4 x 4 matrix M can be expressed as a linear combination of / 4, C4, C~ , CJ.
Use this representation to find an eigenbasis for M. W hat are the eigenvaJues 14p 2 + 12p 3 - 1 = 0.
(in terms of a, b , c, d )? Now Iet
39. Consider tbe n x n matrix C11 which has ones clirectly below the main diagonal
1
and in the rigbt upper corner, and zeros everywhere else (see Exercise 38 for p =-.X
a discussion of C4).
a. Describe the powers of C". At this point we inten"Upt Einstein 's work and ask you to finish the job. Hint:
b. Find aJI complex eigenvalues of C11 , and construct a comp lex eigenbasis. Exercise 40 is helpful. Find the exact solution (in tenns of trigonometric and
c. GeneraJize part c of Exercise 38. inverse trigonometric functions), and give a numerical approximation as weiL
40. Consider a cubic equation (By the way, Einstein, who was allowed to use a logarithm table, solved the
problem correctly.) Source: The Collected Papers of Albert Einstein, Val. 1,
x 3 + px = q, Princeton University Press, 1987.
where (p/ 3)3 + (q/ 2)2 is negative. Show that thi s equation has three rea l
solutions; write the solutions in the fonn xi = A cos(c/>J) for j = l , 2, 3,
expressing A and cPi in terms of p and q . How many of the solutions are in
the intervaJ <J-p/3, 2/ -p/ 3)? Can there be solutions !arger than 2J- p j3?
Hint: Cardano's fonnula derived in Exercise 6.2.38 is useful.
41. In hishigh school final examination (Aarau, Switzerland, 1896), young Albert STABILITY
Einstein ( 1879-1955) was given the following problem: In a triangle ABC
Iet P be the center of the inscribed circle. We are told that A P = 1, B P = !, In applications, the long-term behavior is often the most important qualitative fea-
and C P = t.
Find the radius p of the inscribed circle. Einstein worked ture of a ~ynarnicaJ system. We are frequently faced \Vith the following situation:
through this problern as follows: the state 0 represents an equilibrium of the system (in pbysics, ecology, or eco-

(~) = p ,
nomics, for example). If the system is disturbed (moved into another state, away
sin from the equilibrium _9) and then left to its own devices, will it always return to

sin (~) =2p ,

the equilibcium state 0?
EXAMPLE 1 ..... Cons.ider a dynamical system x (t+ 1) = A x(t) where Ais an n xn matrix . Suppose
sin (~) = 3p. an initial state vector i o is given. We are told that A bas n distinct complex
eigenvalues and that the modulus of eacb eigenvalue is less than I. What can you
say about the Iang-tenn behavior of the system, that is, about lim x(t )?
1-+ 00
c
(not to scale) Solution
For each complex eigenvalue A; , we can choose a complex eigenvector Then v;.
the v;form a complex eigenbasis for A (by Fact 6.3 .6). We can write _{0 as a
complex linear combination of the ii; :

Then

For every triangle the following equation holds:

B y Example 5 of Section 6.4
. 2
sm (a)
2 +
. (ß)2 + . (Y)
sm2
2
+ . (a) . (ß) . (y) =
2
sm2
2 2
2 sm sm Sill 1. lim :>-;
1 -->00
= 0, since lA; I < 1.
358 • Chap. 6 Eigen values and Eigenvectors Sec. 6.5 Stabili ty • 359
Therefore ,
Jim .r (l) = o .
t~

For the di scussion o f the long- term behavio r of a dynami ca1 ystem the
fo llowing defi nition i u eful :

Definition 6.5.1 Stahle equilibrium

Consider a dynami ca l system
Figure Ib Not osymptoticolly stoble .
.x cr + I ) = Ax(t ).

We say that Öis an (asymptoticall y) stable equilibrium for thi s yste m if Genera li zing the result of Example 1 above, we have the foll ow ing result:
lim
,_,. x t) = 0
Fact 6.5.2 Consider a dynami cal sy tem .r(f + I ) = Ax(t ) . The zero state is asymptot-
fo r all it trajectories. 1
ica ll y stable if (and onl y if) the modulus of all complex eigenva lue of A is
le than l.
ote tbat the zero state is stable if (and only if)
lim A 1 = 0 Ex ampl e I illust.rate thi fac t onl y w hen there i a complex eigenbas is fo r
1->
A . Recall th at this is the case fo r " mosf' matrices A. In Exercises 37 to 42 o f
(meaning that all entries of N approach zero). See E xercise 36. Secti on 7.3 we wi ll di scu s the case whe n there is no complex eigenbasi .
Consider tbe examples shown in Fi gure 1. For an il lustrati on of Fact 6.5.2 see Fig ure 11 of Secti on 6. 1. where we
ketched the phase portraits fo r 2 x 2 m atrice wüh two di tinct po iti ve eigen-
va tue .
Figure 1a Asymptoficolly stoble.
We w ill now turn our atte nti.on to the phase portraits fo r 2 x 2 m atrices with
eigenvalues p ± i q (where q =/= 0).
EXAMPLE 2 ... Con ider the dynamical y tem

,i:(t + I ) = [ ~ -q]p
.X (t ),

w he re p is an arbitrary real number and q is nonzero. Examine the tability of

thi y tem. Sketc h phase portra it . Discus your re ult in term of Fact 6.5.2.

Solu tion
We ca n interpr t the linear tran form atio n

T (.x) = [ P
q
-q]x
p

a a rotation-dil ati o n with a di lati 11 factor f r = J p - q 2 . I 11 Fig urc 2 we dra\!

so me trajectories fo r thi y tem, for vari o u va lu e of p 2 + q 2 . N t th at

1
In th is tex t, '·stabl e" will always mean asymptotically stablc. Seve ral othe r not io ns of stability are
p -q]
x (f) = [q p
1
-
Xo -
_
I
.t [ CO (rj> f
in (rj> t )
- i n(r/> t)
CO(r/>t)
JXo.
_
used in applied mathematics.
360 • Chap. 6 Eigenvalue an d Eigen vectors Sec. 6.5 Stabili ty • 361
gi~e a somewhat ro undabout expressio n fo r the trajectory x(t): the vector x(t ,
Wil h rea l ~o m po n e nts, is expressed in tenn of complex qu antities. We may be
lO terested In a " rea l formuJ a" for x(t), that is, a formuJ a ex pressing x(t) in tern1s
of rea l quantiti es alone. S uch a fo rmula i useful , fo r exampl e, when ketchi ng a
phase portrait.
It is he lpful to wri te Ä 1 in po lar fo rm : ).. 1 = r (co (</>) + i in </>)) · the n
Ä 1 ~ r' (cos(cjJt) + i sin (<f>t)). Si nce the vectors c 1)..'1ü1 and c 2 ~ ü2 are compl ex
1

conJu gates, we ca n write

x(t) = 2 Re (c 1)..'1ü1)
(b) (c) = 2 Re((/z + ik)r'(cos(<f>t ) + i in (<f>t ))(ü + iw))
(al
1: trojectories spiral inward. (b) p2 + q = 1: trajectories are circles. (c) p + q2 >
2
Figure 2 (o) p2 + q1 <
2 1: trajectories
= 2r ' (h cos(cjJl)ü - h sin (<f>t)w- k cos(</>t)w- k sin (<f>t)ü)
spiral outward.
= 2 ,., [ w ü] [-- kk c?s(</>t ) - h sin (c/Jt ) ]
sm(</> t ) + h cos(<f>t )
2 2
Fi gure 2 illustrate that the zero tate is stabl e if p + q < 1.
Al tern ati vely, we can find the eigenvalues and apply Fact 6.5.2. = .2r' [ w v][ c?s(<f>t )
sm(<f>t)
- sin (</>t) ] [
cos(</>t )
-k]
h ·
JA()...) = )..?. - 2 p'A + p2 + q 2
Note that x(O) = - 2kw + 2hü. Let us set a = - 2k and b = 2h for impli city.
2p±)4 p2 - 4p 2 - 4q 2 .
AJ.2 = = p ± rq The computati ons above may seem o mewhat contri ved; we have carefull y
2 arranged things in s uch a way that the resuJt involves the rotation matrix
I'A II = I'A2l = J P + q2
2

COS(</>t ) - sin (</>t ) ]

Again, we o bserve stabili ty if p 2 + q 2 < I , by Fact 6.5.2. [ in cjJ r) CO (c/J t ) .
In E xarnple 2 we have seen that the matrix [ P - q ] ha the eigenvalue
q p Let u summarize.
p ± i q , and we have sketched phase portraits for the dyna:mical system

.xcr + l ) =[P
q
-q]
p
.xcr).
Fact 6.5.3 Co nsider a real 2 x 2 matrix A with eigenvalues
More generall y, if A is any real 2 x 2 matrix with e igenvalues 'A 1, 2 = p ± i q, what Ä1,2 = J7 ± iq = r(cos(</>) ± i sin</>)
does the phase portrait of the dynamical system x(t + 1) = Ai 7(t) Iook like?
Consider the traj ectory x(t ) = A'.xo for a given (real) initi al state .Xo. By and corresponding e igenvector ü ± i~. Consider the dynarnical y te m
Fact 6.1.3 we have
:xcr + 1) = Ax(t )
1
x(r ) = c,'A',ü , + c?.Ä 2V2 ,
1
with initial state x(O) = xo. Write x 0 = aw + bv.
where Üu = ü ± iw are ei genvectors with the ei genvalues Ä. 1, 2 (see Exercise 31). The n
We leave it to tbe reader to show that c 1 and c2 are complex conju gate numbers:
c~, 2 = h ± i k see Exerci se 32). x(t ) = r' [ w - sin (<f>t ) ] [
cos(</>t ) b
a]·
Note that the formula

1Convemion: Whenever we consider eigenvectors ii ± i 7v with assoc iated eigenvalues p
± i q, it is
understood th at p and q are real; q is nonzero; and ii and ÜJ are vectors with rea l components.
In Exa mple 5 of Section 7.2, we will present a more conceptu al deri vaü on of thi
re ult . There is no need to memorize the formula in Fact 6.5 .3, but make s ure
you understand its implications tated in Fact 6.5.4 below.
362 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.5 Stability • 363
t = ':!

Figure 3

Figure 5
Wbat does the trajectory ."i(t given in F act 6.5 .3 Iook like? Let u fir t think
about the trajectory
If r = I, we have fo und the trajectory x(l) . If r exceeds 1, then the expo-
cos(</Jt)
[ in (</Jr)
- sin(</J r)
CO (</Jl)
J[ab J. nential growth fac tor r 1 will produce Ionger and langer vectors as t increases: the
trajecto ry will spiral outward, as shown in Fi gure 5.
Likew i e, if r is Ies than 1, then the trajectory spirals in ward.
Since the matrix represents a rotation, these po int will be located o n a circle as
shown in Figure 3.
The invertible matri x [w v] transforms the circle in Figure 3 into an ellipse Fact 6.5.4 Co nsider a dynamical sy tem
(compare with Exercise 2.2.50). See Figure 4. x(r + I) = Ax(r ),
Let u summarize our work thus far:
w here A is a real 2 x 2 matrix with eigen values
- in (</Jl) ][a]
CO (</Jt) b A.1.2 = p ± iq = r(cos(</J) ± i sin (</J)).
;r
If r = 1. the points (t) are located on an ellipse; if r exceeds 1, the
trajectory pirals outward; and if r is le s than 1, the trajectory spirals inward.
points on an ell ipse

Fact 6.5.4 provides another illustration of Fact 6.5.2: The zero state i stable if
(and only if) r = /A.ll = /A.2I < l.
I = 2
Figure4
EXAMPLE 3 ..... Consider the dynamical system

x(t + I) = U -5] _
1
x (r ),

. h trutta
wtt - = [ 0 ] . Find real closed formulas for the components of x(l),
. . . I state xo
1
and sketch the trajectory.

Solution
We wi ll use the terminology introduced in Fact 6.5.3 throughout. The characteristic

polynomial of A = [ ~ =n is A.2 - 2A. + 2, with eigenvalues

2±J4 - 8 .
A. 1 ? =
·- 2
= I ± t.
364 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.5 Stab.i lity • 365
We write A. 1 in polar form
I r= 2

o that
J[
r= ../2 a.nd rp -- -4 '

Next we need to find an eigenvector ü + iw for the eigenval ue I + i:

cos ('.fr) -sin('.f r)] [ 1J J[1J
- 2 +i [ sin (%r) cos ('.fr) 0 [0 -5] [ cos
I - 2
('.f r)
sin(%r)
- sin (%r)
cos (t r) 0
E 1+i = ker [ _
1 (a) (b)

We write the initial state vector .x0 = [ ~] as a linear comb in ation of ÜJ

[ ~] and ü = [ =;l Since x=0 ÜJ we have simply

xo= lw + Oü, x(r ) = (·f21'

VL}
[01 -5][ cosSin('.f('.ft)t)
- 2
- sin('.ft )
CO (%t)
J[1J
0
Figure 6 (c)
so that

a =I and b = 0.
Tbe traj ecto ry spirals outward, since r = .J2 exceeds 1. In Figure 6, we
Now we are ready to u. e the fom1Ula stated in Fact 6.5.3 . develop the trajectory step by step as we have done in Fig ure 3 to 5 above. Note
that we are using different scales in Fi gures 6a 6b, and 6c. ....

If you have to sketch a trajectory as in Fig ure 6c without the aid of tecbnol-
ogy, it he lps to compute and plot the first few poin ts .r(O), x(l), .1:(2) ,.. . until
you can see a trend . Then use Fact 6.5.4 to continue the trajectory.

Tbe components of x (t) are
x , (r) = - 5(.J2)1 si n
EX ER C I S E S
GOALS Use eigenva lues to determine the stability of a dynamical yste m. An-
alyze rhe dynarni cal system ~r(r + 1) = A-r(t) , where A is a real 2 x 2 matrix wi th
eigenvalues p ± iq.
For t:he mat:rices A in Exercises 1 to 10 deterrnine whether the zero state i a stable
(~c) equilibrium of the dynarnical system .r(t + 1) = A.x(r) .

x2 (r ) = C.J2)' (cos ( ~~) - 2 si n (~t) ) . 1• A = [ 00.9 0.8

0
.
J 0.8 0.7
3. A = [ - 0.7 0.8 .
J
 Sec. 6.5 Stabili ty • 36 7
366 • hap. 6 Eigenvalue and Eigenvecto r

~.6 ].
[- 1 a. Show that if A and B are compl ex m x n and n x p matrices , re pectively
[ - 0.9 -0.4 ] 5. - [ 0.5 0.6] 6. A = - 1.2
4. A = 0.4 - - 0.3 1.4 . then
- 0.9 .
A B = A B.
7. A = [2.4
I
- 2.5 ]
- 0.6 .
8. A = [ b. l - 0.2 ]
0.7 . b. Let A be a real n x n matrix and v+iw an eigenvector of A with eiaenval ue

9. A =
[08
0
0
0.7
-06]
0 . 10. A =
[03 03]
0.3
0 .3
0.3 0.3 .
p+iq. Show that the vector v-iw is an eigenvector of A with ei;en value
p- iq.
0.3 0.3 0.3 32. Consider a real 2 x 2 matrix A with eigenvalues p ± iq and corre pond-
0.6 0 0.8
v
ing eigenvectors ± iw . Show that if a real vector .X0 is written as -~o =
c 1 ( v + i ÜJ) + c2 ( v - i ÜJ), then c 2 = c 1•
Consider the matrices A in Exercise II to 16. For which real nu mbers k is the 33. The glucose regulatory system of a certain patieot cao be described by the
zero state a stable equilibri um of the dynamical sy tem .r(r + 1) = kr(t)? equation s
I
g(t + 1) = 0.99 g(t) _ 0.0 1 h (t )
11. A = [kO 0~9] . 12. A = [~~ k ]
0.6 .
13. A = [007 k ]
- 0.9 . I h(t + 1) = 0.01 g( r) + 0.99 h (t)
(see Exercise 6.1.36 for some background). Sketch tbe trajectory for the
14. A = [ ~ k] k . 15. A =[o.~l k1 ]. 16. A = [ ~:~ k ]
0.3 . initial state g(O) = 100 and h (O) = 0. Wbat does the trajectory tell you in
practical terms? Be as specific as possible.
34. Consider a dynamical system x(t +I) = Ax(t) , where Aisareal n x n matrix.
For the matrice A in Exercises 17 to 24 find real clo ed form ulas for tbe trajectory a. If ldet(A)I ::: 1, wbat can you say about the stability of the zero state?
.r(t + l = Ai t), where .t(O = [ ~ l Draw a rough sketch.
b. lf ldet(A) I < 1, what can you say about tbe stability of the zero state?
35. a. Consider a real n x n matrix with n distinct real eigenvalues A. 1 • • • A.",
where lA.; I ::: 1 for all i = 1, .. . , n. Let x(t) be a trajectory of the

[ ~:~ -0. 8 ] [ - 0.8 0.6 ] 19. A = [ ; - 32 ] . dynarnical system x(t + 1) = Ax(t). Show tbat this trajectory is bounded:
17. A = 0.6 . 18. A = -0 .8 - 0.8 . that is, there isa positive number M such that llx(t)ll ::: M for allpositive

[~
integers t.
~] · ~l
- 15]
20. A = [ _ 4 21. A = [ I 22. A = -11 . b. Are all trajectories of the dynarnical system
3 -2

23. A =
[ -0.5
-0. 6
1.5 ]
1.3 . 24. A =U .2
-3 ]
- 2.6 .
x(r+ l) =[~ i].t(t)
bounded? Explain .
Consider an invertible n x n matrix A such that the zero state is a stable equilibrium 36. Show that the zero state is a stable equilibrium of the dynarnical ystem
x
of the dynarnical sy tem (t + 1) = Ax(t ) . What can you ay about the tability
;r(t +I)= Ax(t) if (and only if)
of the systems Listed in Exercises 25 to 30?
( -'>
25. ;er+ I)= A- ' .xcr). 26. x(t + I ) = A Tx(t).
(meaning that all entries of N approach zero).
21 . .xcr + 1) = - A.t(t). 28. x(r + I ) = (A - 2 /" )x(r) .
37. Consider the national income of a country, which consi t of con umption ,
29. x(r + 1) = (A + I" )x(t ).
inve tment, and govemment expenditures. Here we assume the govemment
30. Let A be a rea l 2 x 2 matrix. Show that the zero state is a stable eq uilibrium expenditure tobe constant, at Go, while the national income Y (t), consumption
of the dynarni cal system x(r + I) = Ax(t) if (and only if) C(r), and investment I (I) change over time. Accord.ing to a imple model ,
itr(A) I - 1 < det(A) < I. we have
Y(t) = C(t) + I (t ) +Go
31. Consider a complex m x n matrix A. The conju gate A is defi ned by taking C(t + l ) = y Y (r) (0 < y < 1)
the conjugate of each entry of A . For example, if I(r. + 1) = et(C(t + 1) - C(t)) (et > 0)

A=[2+3i
2i
5]
9 '
then - [2 -
A=
-
3i
.
2L
where y is the marginal propensity to con ume and et is the acceleration
coefficient. (See Paul E. Samuel · on, "Imeractions between the Multiplier
368 • Chap. 6 Eigenvalues and Eigenvectors Sec. 6.5 Stability • 369
Analy i and the Principle of Aceeieration " Re1iew of Economic Statisrics, 39. Consider the dynarn:ical system
May 1939 . pp. 75-78 .) .
a. Find the equilibrium solution of these equ atwns, when Y (t + 1 = Y (1) , X1(t + 1) = O. lx 1(t )+ 0. 2x 2 (t ) + I ,
C (1 + 1) = C(l ) , and I (r + 1) = l (t ) .
b. Let y (1) , c(r), i(r) be the dev iations of Y(1), C(r), I (1) from the equilibriurn x2(t + I ) = 0.4x l (t) + 0 .3x2(t ) + 2
state you fo und in part a. T he e quantities are related by the equations
(See Exercise 6.3 .41 ). Find the equilibrium state of th.i s system and determine
y (t ) = c(l ) +i (t ) its stability (see Exercise 38). Sketch a phase portrait.
c (t + l ) = yy(t) 40. Consider the matri x
i (l + 1) = a(c (t + 1) - c (t ))

(Verify this!) By substituling y(1) into the econd equat.ion, set up equa-
tions of the form
-q
p
-s
- r
s
p
-sl
- r
q
c(t+ l ) = p c(t) + q .i (r) l · r -q p
l i(t + 1) = r c(t + s 1(1)
c. When a = 5 and y = 0.2, determine the stability of the zero state of th.i s where p, q , r, s are arbitrary real numbers (compare with Exerci e 4.3.34).
system. a. Compute AT A.
d. When a = 1 (and y is arbitrary , 0 < y < 1), deterrnine the stability of the b. For which choices of p , q , r, s is A invertible? Find the inverse if it
zero state. ex.ists.
e. For eacb of the four sector in the a - y plane below, determine the c. Find the determinant of A .
stability of the zero state. d. Find the complex eigenvalues of A .
e. If .X is a vector in IR4 , what is the relationship between Jlx II and IIAi II?
f. Consider the numbers
4a
'Y = ( I + a )2

and

Ci
Express the number
2 3 4

2 183

Discuss the various cases, in practical terms.

as the sum of the squares of four integers,
38. Consider an affine transformation

T (i) = Ai + b,
where A is an n x n matrix and b is a vector in !Rn (compare with Exer-
cise 6.3.41 ). Suppose that 1 is not an eigenvalue of A .
a. Find the vector ii in !Rn such that T(ii ) = ii; this vector is called tbe
equilibrium state of the dynarnical system x(t + 1) = T (x(t )) .
b. When is the equilibrium ii in part a stable (meaning that lim x (t ) = ii for
all trajectories)? ~--+
Hint: Parteis useful. Note that 2183 = 59 · 37.
g. The French mathematician Joseph-Louis Lagrange (1736-1 8 1 ) howed
that any prime number can be expres ed a the um of the square of four
integers. Using this fact and your work in pmt f as a guide, show that any
positive integer can be expressed in thi s way.
3 70 • Chap. 6 Eigenva lues and Eigenvectors
41. End a 2 x 2 matii · A without real eigenvalue · and a vector -~o in JR uch
2

that fo r all positivei nt ger r the point A1.xoi lo ated on the e llipse sketched
below.

COORDINATE
SYSTEMS

COORDINATE SYSTEMS IN JRn

42. We quote from a texr on computer graphic (M. Beeler et al., "HAKMEM,"
.rvm Artjficial Intelligence Report AIM-239 , 1972): We now tak:e a fresh Iook at the example in Section 6.1 about coyotes and road-
runners. Consider the matrix
Here is an elegant way to draw almost circles on a point-plotting
di splay. A = [ 0.86 0.08]
-0.12 1.14
CIRCLE ALGORITHM :
=
NEW X
NEW Y =
OLD X
OLD Y +
K
K
* OLD
* NEW
Y;
X.
with eigenvectors ii, = [ 7] and Ü2 = [ ; ] . Suppose the initial state is x=
Thi makes a very round ellipse centered at the origin with its ize
determined by the initial point. Tbe circle algorithm was invented by
[i~] to make the numbers more manageable, we measure the populaüons of
animals in hundreds) . We wish to analyze the dynamical system .r(t+ l ) = A.x(t) .
mistake when 1 tried to save a register in a display hack!
v
First, we express ,r as a linear combination of the vectors 1 and ü2 :
(In the formula above, k is a small number.) Here, a dynamical system is
defined in "computer lingo." In our terminology, the fonnulas are or

x(t+ I =x(t) - k y(l)

The unique solution of this linear system is c, = 4, c2 = 2. We can imagine that
+ 1) = + kx(t + 1).
y(l y( t)
a. Find the matrix of th.is transformation (note the entry x(t + I) in the second
the vectors ii 1 = [ 7] and Ü2 = [ ; ] define a coordinate system in the plane, as

formula). sketched in Figure I.

b. Explain why the trajectories are elljpses, as claimed above. The figure confirms our computation: the coordinates of ,{, with respect to
the basis iil> ii2, are c 1 = 4 and c2 = 2. In the c,-c2 coordinate system, we can
represent ; by its coordirwte vecror

371
3 72 • Chap. 7 oordinate ystems
Sec. 7.1 Coordinate ystem in lR 11 • 373

EXAMPLE 2 .... Find [XJ8 for x= [ ~]

1
(for the bas is ß introduced above).

Solution
To find the coordin ates of x, we solve the linear system
10 or

A Jjttle computation hows that c 1 = 2 and c2 = 3, so that

It is important to point out that a ba is is a sequence of vectors and not ju L

5
a set (the order matters). Consider the ba is

UJ. [7]
of JR2, denoted by n
(we have rever ed the order of tbe vectors Ü1. Ü2 above) .
Tben the order of the coordinates i reversed as well ; for example:
Figure 1 5
[xJn = [;] for ,t = L~ J.
2 1
Let us denote the basis ü1, v2 by ß. Then the coordin ate vector of .X with . ß.., cons1.stmg
Now return to the bas1s . o f the vectors v- 1 = [ ] an d-v2 = [ ] .
1 3
respect to ß is denoted by [x] 8 :
The equation

definino0 the coordinates of a vector .r in IR 2 can be written in matri x form as

-
X = [I-I]
~I ~2
[CJ [I
-
~I Ci
=
-

Solution
lf we denote the matrix
In Figure 1, we can read off the coordinates of ü1: they are c1 = 1 and c2 = 0.
Algebraically, we have

Either way, we find that by S, the equalion takes the compact form

and, likewise, 1 .-r = s [.r] 8 1·

3 74 • Chap. 7 oordin ate ystems Sec. 7.1 Coordinate Systems in R" • 3 75
W can now " rite a c losed fonnula for [x]6 in term of x and

[x-] ß = s-1X-
For example, if x= [ ~ ] , we find that
1

-
[x ]ß= S
- 1-
x= [21 31]- [7
1
]_1[ 3
11 - S - 1 10

Note tbat this agree with the re ult in Example 2.

Now we go a tep further. We wish to tudy the linear u·ansformatio n

- - [ 0.86 0.08 ] -
T(x) = Ax = -0.12 1.14 x
5

in the oordinate sy tem defined by the ba is ß.

More precisely. we wi h to find the mat1ix B th at transforms [.q 6 into
[r<x)] 8 .
Recall fir t tbat ü1 = [ ~] and ü2 = [ ~] are eigenvector of A with A ii1 =
0.9ü 1 and A~ = l.lÜ2.
If .J; = c 1ü1 +c2 ü2 , rhen T(X) = 0.9c 1ii 1 + U c2 ii2 . Or, expressed in terms of Figure 2
coordinate vectors: if

Because ü1, ü2 is an eigenbasi , w ith as ociated eigenva1 ues 0.9 and 1.1 ,
tben respectively, B is a diagonal matrix , wirb the eigen value on the diagonaL We
wil l rerurn to tbis point in tbe next ection (Fact 7 .2.1 ).
0] [Ci ]= [0.09
l.l C2
Whm is the relation hip between the " tandard matrix

The matrix A = [ 0.86 0.08]

-0.12 1.14
B _ [0.9 0 ]
- 0 1.1
ofT (whi ch tran forms .X into T (.r)): the matri
which transforms [x] 8 into [T (x) ]8 i called the rnatrix ofthe linear transformation
T with respect to the basis ß. 1 Tbi means that in the c 1-c2 coordi nate sysrem
defined by the basis ß , the first component is contracted by 10% and the seco nd
B=[o0.9 1.l0]
component is stretched by 10%, as illustrated in Figure 2.
ofT with respecr to ß (which transform [x]8 into [T(x)] 6 ) : and the matri x

s -- [2 1]
1 3

1
Some authors introduce the notation [Tla for thi s matrix.
which repre ent the basis ß?
 Sec. 7.1 Coord inate Systems in iR'' • 377
3 76 • ha p. 7 oord inate Systems

A
We can gene rali ze the concepts introduced in the example above:
X
rc.v)

Definition 7 .1.1 Coordinates

Co nsider a basi ß of IR", consis ting of the vectors ü1, ii2 , •..• Ün. 1 T he n any
s vector >i: in IR" can be written uniq uely a

x= c1ii1 + c2ii2 + · · · + c"li".

Figure 3
B The c; are called the coordinates o f x w ith respec t to tbe bas is ß , and the
vector

r ~:l
C on ider the diagram in Figure 3.
After a Iittle ''diagram-chas ing" we can conclude that
c"
is called the coordinate vector of x denoted by [x]8 .
More explicitly:

B [x ] = [T(.t)] 8 = s- 1T x) = s- 1Ax = s- 1AS [.K) 6 ,

6
Consider the Standard basis A o f Ji{"' consisting of tbe vectors el' e2, .. ..
so tba t e". A vector
B = s- 1AS.
Verify thi s formula by computing s- 1AS fo r the matii ces A a nd S given in our
example.
The formula

B = s- 1AS or A = s s s- 1 in Ji{" can be represented a

give us a new way of thinking about the dyna mi cal system

x(r + 1) = A>r (r) or

1
.r(r) = A xo.
i.e. tbe coorilinates of x
with respec t to the tanda rd ba i A are just the com-
x
pone nts of the vector (the x; ). Tm
means that the coordinate vector [.r ]A i x
Fir t note that itself:

t times In m any applications, in physic , stati stics, and geome try in particu lar, it i
ofte n use ful to work in a coorilinate syste m th at i '·weil adju ted" to a proble rn

l
so rhat
at band. For example, if there i a n eige nba i for a Linear tran formati on T

~ [0.9
from IR" to IR" , it makes sense to work in the coordinate syste m defined by thi
v2 0
0 J[Cic2 ]
1.1
1

1
eigenbasis (a we have do nein C hapte r 6 , without fo rm all y introducing the co ncept
I of coordi nates) .

1 For simplicity we stiife the defi nitions of this ecti on ··over IR." Everything works thc ame way

over C (or uny other tield).

Thi agrees with the fonnula we fo und in Fact 6 . 1.3.
3 78 • Chap. 7 Coordinate Systems
Sec. 7.1 Coordinate Systems in JR'' • 379
--------------------------------~
Fact 7.1.2 Consider a basis ß of IR". Then Solution

; = s [.t]8
for any vector ; in IR" , where S is th~ matr~ x wh? e columns ar~ the basis
B = S- 1 AS=[l
. 1 2 ] -'
l
[5 -6] [I
3 - 4 I
vector (in the given order). Since S 1 an mvertJble n x n matn x, we can Work out the details of this computation yourself.
write thi equation a
. <=:onsider a linear transformation T from lR" to lR" and a bas i ß of JR"
[x-]ß = s-'-x. consJstmg of the vectors ü, , Ü2, ... , ü". Let B be the matrix of T with respect to
ß , defined by the equation

Again note that the formul a x = S [x)z3 is simpl y a compact way to write [r cx>]8 = s [x]8
the equation for all x in lR". Applying thi s equation to x = Ü;, we find tbat
[T (v;)]8 = B [ü;] 8 =Be;= ith column of B.
which is the definition of coordinates.
Thi s Observation allows us to find the matrix of a linear transfonnation column
by column :
Definition 7.1.3 The matrix of a linear transformation
Consider a linear transformation T from !R" to IR", and a basi ß of IR". The Fact 7.1.5 Consjder a linear transformation T from JR" to IR." and a bas i ß of JR"
11x n matrix B which transforms [xL,
into [T (x) ]8 is called the m atrixof T consjsting of the vectors ii,, ü2 , .•. , Ü11 • Then the matrix B ofT with respect
w ith respect to ß: to ß is

for all x in JRn.

that is, the ith column of B is the coordinate vector of T (Ü; ) with respect to
the basis ß.
Fact 7.1.4 If A is the standard matrix of a linear transformation T (that is, T (;"t ) = Ax ,
for all ; in IR"), and B is the matrix of T witl1 respect to some basis ß , then
B = s- 1As, Note that Fact 7.1.5 generalizes the fonnula

where S is the matrix whose columns are tlle vectors of the basis ß .

Note that the "srandard matrix" A ofT, introduced in Chapter 2, is simply

the matrix of T with respect to the Standard basis A , because the equation
rce")
[T (x)]A = A [x]A
siroplifies to
for the standard matrix A of a linear transformation T (Fact 2.1.2).
T(x ) = Ax. ln the introductory example, we have
EXAMPLE 3 ..... Find the matrix B of the linear transformation

T(.r) = U=~ J ; 8 ~ [ (T(V,JJn (T(V,)Jn] ~ [ (09V,Js (I.IV,Js] ~ [009 101].

EXAMPLE 4 ..... Consider two orthogonal unit vectors v1 and ü2 in JR 3 • Form the ba is ü1, i),
with respect to the basis ß consisting of the vect.ors [ : ] and [ 7]. v3 = ü, x ii2 of JR.l , denoted by ß . Find the matrix B of the linear tran formatio~
T(x) = ü1 x ;r. with respect to the basis ß.
380 • hap. 7 Coordjnate Sy tems
ec. 7.1 Coordinate ystem in 11
• 381
~ he~oe the first component lists the height in meters and the econd one the mass
Solution
111 kilogram . For my friend s in the United States, I need to ex press these mea-
U e Fact 7 .1.5 to construct th sure me nt in their system:

I foot ~ 0.305 meters,

1 pound ~ 0.454 kilograms.

=
-V ( X
-
Vz =
-3 0
-!] lf I in sist on using matrix techniques to perfmm these unit change , I can in troduce
the basi. ß of IR 2 consisting of the vectors

and
[ ~.454 ] 0

T hen

draw a sketch
[x]a = s-'; = [ 1~5 J
gives the height in feet and the weight in pou nds, as desired.
Draw a sketch superimposing the coordi nate system in term of feet and
EXMIPLE 5 ..... Con ider a real 2 x 2 matrix A with eigenvalues p±iq and corresponding eigenvec-
pound over the metric coordinate y tem. Use two different co lor
tor - ±iw. It can be hown that the vector u and ÜJ in IR2 are linearly independent
0

(Exercise 29). Find t11e matrix B of the linear Lransfom1ation T (x) = Ax with
respect to t11e basis ß consi ti ng of ÜJ and ü. EX ER C I S ES
GOALS Use the concept of coordinates. Apply ilie defi ni tion of the matrix of
Solution a linear tra nsformation with respect to a basis. Relate this rnatrix to the standard
Because matrix of the transformation. Find the matrix of a linear tran formation (with
respect to any bas is) column by column.
ACü + iw ) = <P + iq ) ü + iw ) = (pv - qw) + i(qü + pw),
we have ~ 1. Find the coordinate vector of [
1
~] with respect to the ba i [; ] . [ ; ] .
1
Aw = pw+qv 2. Find the coordinate vector of
AÜ = - qw + pÜ

and

with res pect to the basis

[i]
~

l~J · l~J· l~J· l~J

ote that B is a rotation-dilation matrix . Compare wi th Exercise 30.
The problern of coordinate changes is a multidi men ional version of the
probl ern of unit changes which you may have encountered .in math or physics
courses (or in your dai ly lives, when dealing with fo re.ign systems of measurement
or fo reign currencies). 3. Find the matrix of the linear tran fo rmati on
We illustrate th.is poi nt with a simple example. Suppose I collect data re-
garding the height and weight of people. For each person, 1 represent the data as T (_v) = [ ~ ; ] x
a vector; fo r example:

-- [1.83]
X- 84
with res pect to ilie ba i [ ~ l [~ l
382 • hap . 7 Coordinate System Sec. 7.1 Coordinate Sys tems in IP. 11 • 383
4. Find the matrix of t.he linear mm formation
12. ln the figure below, sketch the vector .X with [.XJ 8 = [ - ~ l where ß i the

~ = [ _7
T (x)
6
-1]-
8 X
basis of JR 2 consisting of the vectors ii, w.

with respect to the ba i [21] [-3

1]·
s. Let T: JR2 -* JR 2 be the orthogonal projection onto the line spanned by [ ~ l
a. Find the matrixofT with r pect to tbe bas1s
1
,
3
. [3] [-1]
. Draw a ketch. 0

b. Use you r answer in part a to find the standard matrix of T .

13. Consider the vectors ii, v, w sketched below . Find the coordinate vector of
6. Let T: JR3 ---7 !Pl3 be tbe reflection in tbe plane given by the equation ÜJ with respect to the basis ii , ii.

a. Find the matrix of thi transformation with re pect to the bas i ii (Iran ·Ja!Cd)

0 /(

Draw a sketch.
b. Use your answer in part a to fi nd the standard matrix of T.

l
14. Given a hexagonal tiling of the plane. such as you might find a n a kitche n
7. Con ider tbe linear tran formation T: IR 2
-* 2
iR with tandard matrix [ ~ : floor, con ider the ba i.s ß of ~2 con isting of the vector ii ÜJ ketched
below.
R
Fi nd the matrix oftb is transfonnation with respect to the ba i [ ~] , [ ~ ] .
8. Consider a linear tran sforma lion T: IR 2 --+ iR2 • We are told that the matrix of

T with respect to the ba i.

ofT.
Dl U] is [ ~ ~ J. Fi nd the standard matrix

9. Con ider a linear tran formation T : 2

--+ ~ 2 . We are told th at the matrix of

T with respect to the ba i · [ ~] , [ b] is [ ~ !]. Find the Standard matrix

ofT in tenns of a , b, c, d .

10. Consider the basis ß of ~2 con isting of the vectors [ ~] and [ - ~]. Find a. Fi nd the coordinate vector [ 0 P ] and [ O.....Q ] .

UJ
8 8

[.XJ 8 for x= [ ~]. II Iu strate the re ult with a sketch. b. We are to ld that [ OR ]
8
= Sketch the po in t R . ls R a vertex r a

11 . Da Exerci e 10 for x= [ ~ J ce nter of a ti Je?

e. We are to ld thar [ ÖS ]
8
= [ :;]. a ccnte r r a vertc f a til ?
384 • hap. 7 Coord inate y -tem
Sec. 7.1 oordinate Systems in IR" • 385
15. Find tJ1e coorclinate vector of e1 with respect to the ba i 0.6 ]
23. Let L be the line in JR 3 spanned by the vector ü =
[~-8 . Let T: R 3 R 3 be

of ~ 3 .
Ul nl m the rotation about thi s line through an angle of rr j2, in the direction indicated
in the sketch be.low. Find the matrix A suchthat T (x) =Ai .

16. If ß i a ba is of IR!", i the tran formation T from IR!" to IR!" given by

T (x) = [x ] 8
L
linear? Ju tify your an wer.

17. Con ider the ba i ß of R 2 con i t.ing of the vectors [ ; ] and [ ~]. We are , ... "' ... '
06 ii •• ••

1 ~]
XI . • ••• • •• • • ••••••••••••••• • ~
told that [.x] 8 = [ for a certain vector ,r in R- . Fi nd x.
18. Let ß be the ba is of IR!" consisting of the ector Ü1, Ü2, .. . , ü" , and Iet T
be some other basis of IR!". Is 24. Con ider the regular tetrahedron ketched below, whose center i at the origin.

a basis of ~n as well? Explain.

19. Consider the ba is ß of ~2 con isting of the vector [ ~] and [ ; l an d Iet

n be the basis consisting of [ ~ ] , [ ~] . Find a matri x P such that

for all .X in iR2 .

2.0. Find a basis ß of IR- uch that

and

21. Consider two orthogonal unit vectors ü1 and ü2 in JR 3 and form the basis ß
consisting of the vectors ü1, ü2 , ü3 = ü1 x ü2 • Find the matrix of the linear
transformation
Let ii 0 , ü1, ü2 , ii3 be the position vector of the four vertice. of the tetrahedr n:
.....
ü0 = OP o, .. . , ii. = OP 3.
with respect to the ba is ß. a. Find the sum iio + ii1 + Ü2 + Ü3 .
b. Find the coordinate vector of Üo with re pect to the ba i VJ. IJ:!, VJ.
22. Consider a 3 x 3 matrix A and a vector ü in IR uch th at A 3 ü = 0, but
c. Let T be the linear transformat.ion with T(Üo) = Ü3, T(v3) = v1 . and
A 2 ü =/= Ö.
T(ü 1) = ü0 . What is T (v 2 )? De. cribe t.he transformation T geometrically
a. Show that the vectors A2 ü, AÜ ü form a bas is of JR: 3 . Hint : Demonstrate
(as a reflection, rotation, projection , or whatever). Fi nd the matri 8 ofT
linear independence.
b. Find the matrix of the transformation T x) = Ax with re pect to the ba i
with respecr to the basi ii 1, ii2 , v. Find the complex eigenval u of 8.
3
What is 8 ? Expl.ain.
A 2 ü, AÜ , ü.
386 • Chap. 7 Coordinate Systems Sec. 7.2 Diagonalizati on and Similarity • 387
25. Con ider a rotation T (x) = Ax in ffil. 3 (that is, T is an orthogonal tran forma- 29. Consider a real 2 x 2 matrix A with eigenvalues p ± iq and corresponding
tion. and det(A) = I). v
eigenvectors ± iw. Show that the matri x [ w ii ] i invertible. Hint: We
a. Consider an orthonormal basis ß of ffil. 3 . Is the matrix B of T with respect ca n write
. . . ?
to ß orthoaonaJ? Js it necessanly a rotatwn matnx.
b. Now supp~se ß i an ?rthon~rmal ~asi Ü1, Ü2, V3 of IR\_ where Ü1 is_,a
fixed point of T , that ts, T v 1) = v 1 • (Such a vector ex tsts by _Eu Iet s
30. Consider a real 2 x 2 matri x A with eigenvalue p ± iq and corre ponding
theorem· see Exercise 6.3.34.) What can you say about Lhe matr1x 8 of
eigenvectors v ± iw. Find the matrix B of the linear Iransformation T(x) =
T with ~espect to ß? Describe the first row and the tirst column of 8 ,
Ax with respect to the basis consisting of v and w. Campare with Example 5.
and explain why the rninor 8 11 is a 2 x 2 rotation matrix . ExpJain the
significance of this result in geometric terms.
26. Consider a linear tran formation T (,\-) = kr from IR" to IR". Let B be the ma-
trix ofT with re pect to the ba is -e1, e2,
- e3, ... , (- l )"e" of IR" . Describe
DIAGONALIZATION AND SIMILARITY
the entries of B in terms of the entries of A.
27. Consider a linear transformation T (x) = Ax from IR" to IR". Let B be the The introductory example of the last section (pp. 374 and 375) sugge t the fol-
matrix ofT with respect to the basis e1" en-1, ... , e2, e1 of IR". Describe the lowing resu!t:
entrie of B in terms of the entrie of A.
28. This problern refers to Leontief s input-output model , first di cussed in the
Fact 7.2.1 The matrix of a linear transformation with respect to an eigenbasis is diagonaL
Exerci es 1.1.20 and 1.2.37 . Consider three industries /1 , h / 3 each of which
More specifically, consider a linear Iransformation T (.r) = Ax, where A is an
produces only one good, with unü prices P I = 2 P2 = 5, PJ = LO (in doiJars),
n x n matrix . Suppose ß is an eigenbasi for T consisting of the vectors v1,
respectively. Let tbe three products be good 1, good 2, and good 3. Let
v v;
ü2 , ..• , 11 , with A = .A; v;.
Then the matrix B of T with respect to ß is

012 013] = [0.3

o 22
0 32
a 23
0 33
0.1
0.2
0.2
0.3
0.2
0.1 ]
0.3
0.1 8 ~ s-' AS ~ !l'' JJ 0
)'2

0
be the matrix that lists the interindustry demand in terms of dollar amounts.
The entry oij teils us how many doiJars' worth of good i are required to where
produce one doUar's worth of good j. AJtematively, the interindustry demand
can be measured in units of goods by means of the matrix

where bij teils us how many units of good i are required to produce one unit
of good j. Find the mat.rix B for the economy discussed here. Also wri.te an To justify this fact, consider the column of B (see Fact 7.1.5):
equation relating the tbree matrices A, B, and S, where 0
0

~~J
2 0
S= 0 5 (ith column of B) = [T(Ü;)] 6 = [.A;v;]6 = .A; +- i th component
[0 0

is the diagonal matrix listing the unit prices on the diagonal. Justify your 0
answer carefully. Applying this observation to i = 1, 2, .. . , 11 , we prove our claim. •
388 • Chap. 7 Coordin ate Systems Sec. 7.2 Diagonaliza tion and Sirrti la rity • 389
The con verse of Fact 7 .2.1 is true as weil : if the matrix of a li near tra n - Asking
. .
whether. a matrix A is diaoo
b
naJizable
.
is the ame as a kin bo whethe r
fo m1 ation T with respect to a basis ß i diagonal, lhe n ß is an eigenbasis fo r T there IS an e tgenbasts fo r A. l n C hapter 6 we outli ned a method for answering
(Exercise 17). thi s questio n. We summarize th is proce s below:
Fact 7.2. I motivates the followin g detinition:
Algorithm 7.2.4 Diagonalization
Definition 7.2.2 Diagonalizable matrices Suppose we are asked to decide whe ther an n x n mat rix A is diagonali zable,
and, if so, to fi nd an invertibl e S such that s-' AS is di agonal. Th is process
An n x 11 matrix A is called diagonalizable if there is an invertibl e matri x S can be carried out over IR, over C , or over any other fie ld.
such that s-' AS is di agonal.
a. Fi nd the eigenvalues of A , that is, olve the eq uatio n / A(A.) = 0.
b. For each eigenvaJ ue A., fi nd a bas i of the eigen pace
As we observed above, s- 1 AS is diagonal if (and only il:) the column. of S
E 1. = ker(AI" - A ).
form an eigenbasis for A. This impli es the following result:
c. T he matrix A is diagonaJi zable if (and only if) the dimens ion of tbe
eigenspaces add up to n . In thi case, we find an eigenbas is ti 1 ti2,
Fact 7.2.3 The matrix A is diagonalizable if (and only il:) there is an eigenbasis for . . . , ü" for A by combining the bases of the eigenspaces we fo und in
A . In particula r, if an 11 x n matri x A has n di stinct eigenvalues, then A .is u
step b. Let S = [ 1 ti2 iJ"]. Then s-' AS is diagonal with the
diagonah zable (by Fact 6.3.6). cotTes ponding eigenvalues on the di agonal.

EXAMPLE 1 ~ Diagonali ze the matri x

A matrix with real entries may be di agonaliza ble over C, but not over 1lt
For example, the rotation matrix

Solution
has no real eigenvectors, so that A is not diagonalizable over IR. However, there We proceed step by step as outlined above:
is a complex eigenbasis for A, namel y,
a. Find the eigenvalues:

~1 ~
1
JA (A.) = det [ A. - 0 ), - ]
with associated eigenvalues i, - i (see Example 6, Section 6.4). T herefore, A is - 1 0 A. - I
diagonalizable over C. The matrix A. - 1) 3 - (A. - I ) = (A. - I ) ( A. - 1)2 - 1)

S= u -n = (A. - I )(A. 2
- 2A.) = (A.- I )(A. - 2)A.
The eigenva lu es are 1 2, and 0, so that A is diago nali zab le (over IR).
=0 .

does the job: b. Find an eigenvector for eac h eigenva lue :

s- ' A S = [ ~
-~J E 1 = ker
0
0
[ - 1 0
0
0 -l] [I 00]
~ = ker ~ ~ ~ ;

Note that "most" n x n matrices (with real or complex entries) are diagonalizable
over C , because they have n distinct complex ei genvalues. An example of a
matrix which is not diagonalizable over C is the shear matrix [ b ~ ].
 Sec. 7.2 Diagona lization and Similarity • 391
390 • Chap. 7 Coordinate Sy tem
where
Si milarly, we find eigenvector for the eigenvalues 2 and 0:
l 2 0 -i]J .
t t
Diagonalization is a usefu l too l to study the powers of a matrix: if

mm[-n then
s- 1 AS = D,

c. Let

s~ u -1] 1
0 0
1
. and
A= sos- 1

Then 1 tim es

0 0]
2 0 .
Note that D' is easy to compu te: raise the calars on the di agonal to the t th
power. Tf
0 0

[~
0

n
Note that you need not actua lly comp ute s- 1 AS. The theoretical work above
0 A.~
guarantee that s- 1 AS is diagonal, wi th the eigenvalues on the diagonal. To check
yo ur work you may wish to verify that s- 1 AS = D , or, equivalently, that AS = SD
OJ
. ' then D'= .

(here we need not compu te the inverse of S). 0 A." 0 0 . ..

AS ~ [6
0
L
1 0 ~] [! -1] [0 ~]
1
0 0
1
= I
0
2
0
2
Consider a dynami cal system
.xcr + 1) = Ax(t).

-I][I 0~] ~ u ~] s- 1
where A is di agonali zable, with AS = D . Then
2
so~[!
l
0 0 0 2 0 .J I X(I) = A' Xo = s D' s- I io 1-

1 0 0 2
Thi is a more succinct way to write the formula fo r dynarnical y tems derived
EXAMPLE 2 ..... Diagonalize the rotation-dilation matrix in Chapter 6. To see why, write

-q]
[~
A = [ : p '
0
[,,: J.
J
A.'2 - 1- -
C?
-
where p and q are real numbers with q =1= 0. S= 'U J 'U2 v" '
D' = . S xo = [xo]8 =

Solution 0 0 c"

The eigenvalues of A are A. 1.2 = p ± iq (see Section 6.5, Example 2) . T hen

Then
1
C JA 1
· -- ker [ iq
E p+rq
-q
c2 A.~
and v"

E1,-;q = span [ -~ J. c"A.~,

Now = c 1A.11v1 + c_A.~Ü2 + · · · + c"A.~, i.i".

s-J[ ~7!} -Pq Js = [ P -lo
- iq o J' Thi is the formul a famili ar from Chapter 6 (Fact 6.1.3).
"' p-iq
392 • Chap. 7 oordinate System
Sec. 7.2 Diagonalization and Similarity • 393
+ Similarity EXAMPLE 5 ...... Consider a real 2 x 2 matri x A with eigenvalues p ± iq and correspondi ng ei gen-
So far we have focu ed o n the diagonalizable matrices. that is, those matrices fo r vectors v± iw. In Example 5 of Secti on 7. 1 we have seen that A is imi lar to
which there is an eigenbas is.
lf there i. no eigenba i for a linear transfo rmati on T (_r ) = A.r from C " to
B = [p
q -q]·
p More specifica lly, B = s- AS, where S = 1 [w ü ]. Use thi

C ". we may still Iook for a ba is ß of C" thar i "weil adj usted'' to T , in the sense simil ari ty to derive a real closed fo rmula for x (t ) = N x 0 .
thar rhe matrix B of T with r spe t to ß i ea ier to work wilh than A it elf.
To put it differently, for a (nondi agonalizable) marrix A we may wi h to Solution
fi nd an inve1tible S such that B = s- 1 AS is of a simpl er form than A itse lf. Ln
view of ou r intere t in dynamical sy tems, the main requ irement will be that the We can write
power 8 1 of B are ea to ompute. Then we can "do" the dynami cal y tem
- sin(<P) ]
x(t + I) = A,\- (1) : COS(cP) .

I x(r ) = N ,\-o = csss- 1Y-"ro = ss ~ s- 1.Xo I Then

For the di cu sion of this prob lem the fo llowi ng termi nology is useful : -()t = A1-xo = sBIS
X - 1-
xo = r 1[-
w - ] [ COS(cPI)
v sin(<Pt)
- sin (<Pt ) ] [
cos(<Pt )
a]
b '

where [ ~ ] is tbe coordinate vector of

Definition 7.2.5 Similar matrices x0 with respect to tbe basis w, v. Recall
Con ider two 11 x 11 matrice A and B. We say that A is similar to B if there that we fi rst stated thi formula in Fact 6.5.3. ~
is an invertible matrix S such that B = s- 1 A S.
As the term suggests, sirrülar matrices have many properties in common:

In terms of hnear tran Formations, this mean that A is similar to B if 8 is

tbe matrix of T (x) = Ax with re pect to some ba is ß. Note that a matrix by
Fact 7.2.6 Suppo e A is sirnilar to B. Then
definiti on is diagonaLizable if it i similar to a diagonal matri x.
The verification of tbe following fact is left a Exercise 20: a. !A ().. ) = f n ()...).
• An n x 11 matrix A is simi lar to itself. b. A and B have the same eigenvalues, with the same algebraic and geo-
metric multiplicities.
• If A is sirnilar to B , then B is irnilar to A.
c. det(A) = det( B) .
• l f A i sirnilar to B , and B i sirnilar to C, then A i im.il ar to C.
d. tr(A) = tr(B).
EXAMPLE 3 ...... Find all matrices B similar to !". e. rank (A) = rank(B).

Solution
We will demonstrate clai m a. The verification of the other claims i left to
If B i similar to l 11 , then B = s- 1 J"S = ! 11 • Therefore, the only marrix . imi lar to
the reader (Exercises 27, 28, 61). For all scalars A,
!" is 111 itself. ~

EXAMPLE 4 ...... Are the matrices A = [ ~ -n and B = [ =~ ~~ ] simi lar?

fa(A) = det(Al11 -

1
B ) = det (Al" -
= (det S) - det(A/11 -
s - I AS) = det (S- 1()...111
A) det(S) = det(Ain - A) =JA().) .
- A)S)
•

Solution Consider a nondiagonalizable matri x A. l s A irnilar to some matrix B " in

simple form "? ln thi s introductory text, we will not addres this issue in full
Find the eigenvalues of A and B . It turns out that both matrice have the eigen- aenerality. If you are interested, read the chapter on ' Jordan Normal Form "
values 2 and 3. This means that both A and B are diagonalizable, wi th diagonal f'n an advanced linear algebra text (for example, Section 6.2 in the text Linear
.
matnx D = [2 0]
0 3
. Because A and B are both sirnil ar to D = [2 0]
0 3
, they Algebra by Friedberg, In el, and Spence; Prentice Hall). . . .
We conclude thi s section with three examples concerrung nondiagonahzable
are sirnilar to each other. <11111
2 x 2 matrices.
 Sec. 7.2 Diagonalization and Similarity • 395
394 • Chap. 7 Coordinate Systems

Figure 1

2 Figure 2
EXAMPLE 6 .... Consider a real 2 x 2 matrix A (other tban /2) suchthat / A('A) = ('A - 1) . Show
that A represents a shear.
Solution
Solution
Wehave !A ('A) = ),2 -2}..+ 1 = (A.-1) 2, so that A represents a shear, by Example 6.
Cboose a nonzero vector ü in the one-dimensional eigenspace E 1 of A and a vector
By definüion of a shear, x(O), x(l), x(2), ... will be equally spaced points along a
w E
that is not contained in 1• By Definjtion 2.2.4, we have to how tbat ÜJ Aw-
is a scalar multiple of ü. See Figure l. straight li~e p~allel to E1• We compute .X ( I ) = Axo = [ ~] and find the trajectory
Consider tbe matrix B of tbe transformation T (x) = A-t with respect to the
sketched 1n F1gure 2.
basis ü, ÜJ. Since A ü = ü, the first col umn of B is [ ~}. Note that x(t) = io + t(x(l ) - xo) = xo +I (Axo - xo).
In our example, we bave

B=[~ ~ ]
Since fA(A.) = fB(A.), we have b = 1, so that
EXAMPLE 8 .... Consider the dynamical system
B=[~ ~] · x(t + 1) = Ax(t) with xo = [~] ,
The second column of B teils us that
where A = [ ~* ~ ].Interpret t~e linear transformatjon T (x) = Ax geometri-

or cally, and find a clo ed form ula for x 1).

Aw - w= aü,
Solution
as claimed.
Note tbat we can write the tran formation Tin even imp ler form: the matrix
We have JA A. = A.2 -).. + ~ = ('A- ~) 2 ; the eigenvalues of A are >.. 1 = .A2 = ~­
of T witb respect to the basis Aw- w, wis [ ~ ~]. <111
Therefore, the matrix M = 2A = [ ~t ; ] has the eigenvalue J with algebraic
multiplicity 2. By Example 6, the matrix M represents a hear. Thu the matrix
EXAMPLE 7 .... Consider the dynarrucal ystem A = 1M represents a shear-dilation , that i , a shear followed by a dilation by a
factor of 1·
x 1 + J) = A-t t) with x0 = [ ~] We can find M 1 x 0 as in Example 7:

wbere A = [ =! ~ l Find a closed formula for x(t) and sketch the trajecrory.
Mxo=xo+t(Mxo
1- - -
- xo
-
= [ 0] [2] [
1 +t 1 = l 21
+ t] ·
396 • Chap. 7 Coordinate Systems Sec. 7.2 Diago naliza ti o n and Simil arity • 39 7

7.
[~
0

0
I
-n 8.
[: l] 9.
[~ ~]
1
1
0

l~ l
I
10. [I 2 3]
0 2 3 II. [ ~
- 1
3 -~] 12.
1
0
l
I
0 0 3 -3 9 -5
0 0 j
13. Di agonali ze the matri x A = [ -~ ~] over C.

Figure 3 0
14. Diagonalize Lhe matrix A = [ =: 1
~] over C.

15. Consider the matrix A = [ _ ; -~ ] . Find an in vet1ible mat.rix S uch tbat

Then
_ I _
x(t) = A xo =
( 1) I I _
M xo =
( 1) I [
I
2t ]
+t .
s- 1 A S is of the form [ ~ ~ l
2 2
See Figure 3.
16. Consider the matrix A = [ ~ -~ J. Verify that A i a hear mat:rix , and find

Let us summarize our observation in Examples 6 and 7: consider an invert- an inverüble matrix S such that s- JAS = [ 6 ~ J.
ible real 2 x 2 matrix A that is nondiagonalizable over C. Then ! A(A.) = (Ä - .>. d 17. Show that if the matrix of a Linear tran formalion T with respect to a basi
for some nonzero A.Q, and A represents a shear-d.i lation , that i , a shear followed

-n.
ß i di agonal , then ß is an eigeobas is forT .
by a dilation by a factor of .>.. 0 . . .
Note that we now un~erstand the dynarnical ystem 18. Let A = [ ~ Find a clo ed formula for A 1 (entry by enrry). lf

.r. cr + I)= Ax(t ) ,

A1 _ [a (t ) b(t ) ]
for all real 2 x 2 matrices A (the case when JA (A.) = A. 2 is left as Exercise 56). - c(t ) d (t ) ·

what can you ay about the proportion a t ) : b (t ): c t ) : d (t ) as 1 goe to

E X E R C I S ES infinity?

GOALS Use the concept of a diagonalizable matrix and the idea of similarity 19. Let A = [ -~ ~]. Find a closed formula for N entry by entry .
of matrices. Analyze the dynarnical y tem x (t + 1) = Ax(t) for any reaJ 2 x 2
20. Justify the following facts : lf A B , and C are n x n matrice then
matrix A and sketch a pha e portrait.
• A is im.ilar to itself.
Decide which of the matrices A in Exercises I to 12 are diagonalizable (over JR).
lf possible, find an invertible S and a diagonal D uch that s- 1AS = D. Do not • lf A is sim.ilar to B , then B is similar to A.
use technology. • lf A i imilar to B , and B i im.ilar to C, then A i im.ilar to C .

1. [~ ~] 3. [ ~ =~] 21. Are the matrice [ ~ ~] and [ ~ ~] im.ilar?

~ ~] and [ =~ ~] sim.ilar?
4• [2 -1]
1 0 5• [--] 5]
2 5 6
. u=i ~] 22. Are the matrice [
23. Jf two matrices A and B both represent a hear in JR2
to A?
B nece arily imilar
398 • Chap. 7 Coordinate Systems Sec. 7.2 Diagonalization a nd Similarity • 399
24. In Exa mple 4 we found that the matrices A = [ ~ -~ ] a nd 8 = [ =~ 1 ~ ] 43. If A is diagonalizable, i A 2 diagonali zable as well?
44. If A is diagonalizable and r is an arbi trary positive integer, is A' diagonali zab le
are imilar. Find an invertibl e matri x S such that 8 = s- l A S. as weil?
25. Consider two 2 x2 matrices A and 8 with det(A) = det(B) and tr (A) = t.r(8).
45. lf A i diagonalizable and invertible, i A - I diagonali zable as well?
Are A and B nece sarily similar?
26. Consider two 2 x 2 matrice A and 8 with d t(A) = det(B = 7 and tr(A) = 46. lf A is in vertible, is A necessarily di agonalizable over C? Conver ely if A
is diagonalizable, is A neces arily invertible?
tr(B) = 7. Are A and B neces ari ly simil ar?
27. ·
a. Con ider tb e matnces A an d B = s- 1 AS . Show that i f ."K is in the kerne! 47. Jf A and 8 are di agonalizable, is A +B necessmily diagonalizable?
of 8 then S.r i in the kerne! of A. 48. ff A and 8 are diagonaJizable, is AB necessari ly di agonalizable?
b. Shm tbat rank (A) = rank(8 ). 2
49. lf A i diagonalizable, i A itself necessaril y di agonalizable?
28. Con ider the matrice A and 8 = s- 1 AS. Suppo e .r i an eigenvector of 8 , 50. Tf A and B are n x n matrice and A i in vertibl e, how th at AB imiJar
with a . ociated eigenvalue A.. Show that Si i aJ1 eigenvector of A wi.th the to BA.
same eigenvalue. Campare the geometric multiplicities of A. a. an eigenval ue
51. Give an exampl e of two (nonin vertible) 2 x 2 matrice A and B uch that A B
of A and B.
is not imilar to BA .
52. a. Consider two matrices A and 8 , where A i in vertible. Show that the
For the matrices A in Exerci e 29 to 36, find clo ed formula for the co mponent
matrices AB and BA have the same characteristic polynornial (and there-
of the dynamical system fore the same eigenvalues with the ame algebraic multiplicities). Hillt :
Exerci e 50 is helpfu l.
.r (l + 1) = A.~(t) wirb _r(O) = [~] . b. Now co n ider two n x n matlice A and B which may or may not be
invertible. Show tbat A 8 and BA have the same characteristic polynorniaL
and therefore the same eigenval ues. H int: For a fixed scalar A., consider
Sketch the trajectory. the function

29. A = [ ~ i] 30. A = [ 6 -i] 31. A = [ ;

n j(x) = det (Ain - (A - X ln) 8) - det (Hn- B (A - x l n)).

32. A = [~ ~] 33. A = [i -6] 34. A = [ -_ l

4 ~] Show that f x) is a polynornial in x. What can you say about it degree?

35. A = [i -i] 36. A = [ -~ !]

Explain why f(x) = 0 when x i not an eigenvalue of A. Conc lude tbat
f(x) = 0 for all x; in partic ular, /(0) = 0.
53. Find clo ed formulas for the entries of the hear matrix
37. Are the matli ces A and 8 below similar?

h [~ n s~ [~ ~l
l 0 I 0
0 0 0 l
0 0 0 0
0 0 0 0
54. Show that
Hint : Consider A2 and 8 2 .
38. Tru e or false? If two n x n matrice A and 8 have the ame eigenvalue , with
the ame algebraic and geometric multipliciti e , then A and B are imilar.
39. If A and B are invertible, and A is simiJar to B , is A- 1 imi lar to B- 1 ?
40. If A i similar to 8 , and A is invertible, is B necessarily in vertible? . 55. Consider a real 2 x 2 matri x A (other than / 2 ) uch that tr(A) = 2 and
41. If Ai s similar to 8 , i AT necessaril y sirni lar t.o B T? det (A) = l. 1 tbe linear t.ran form ation T(x) = Ai nece aril y a he ar .
42. lf A is diagonalizable, is AT similar to A? Explain.
400 • Chap. 7 oordinat ystems Sec. 7.3 Symmetrie Matrices • 401
2
56. Let A be a 2 2 matrix \ ith chara teri'tic polynorniaJ A . Show that A e. Show tbat if A B = BA and A has n distinct eigenvalue then A and B

similar to [ ~ ~ l What can you ay abou r A ?

2 are simultaneously diagonalizable. Hint: Part d is usefu l.
63. Consider a diagonalizable n x n matrix A with m di stinct eigenval ues A 1, . . . ,
57. Show that any ~ x 2 matrix i si milar to it transpose. A111 • Show that
58. ShO\ that any complex 3 x 3 matrix i imilar to an upper triangular 3 x 3 (A - Alln)(A- A.2/n) ... (A - A.111 / 11 ) = 0.
matri ·.
59. Con ider two real n x 11 matrices A and B whi h are "si milar over C": that 64. Consider a diagonalizable n x n matrix A witb characteristic polynorni al
i the re i a compl e invertib le 11 11 matTix S uch that B = - I AS. Show
that A and B are in fact '·simil ar over ~· · : that i , there is a real R such that
!A (A) = An + a"_l),n-l + · · · + a1A. + ao.

B = R - 1A R. Hints: Write S = S1 + iS2, where S1 and S2 are real. Con ider Show that
the function f(x) = det(S 1 +.rS2 ), where x is a compl ex variable . Show that
f(x) i a nonzero poly nornial. Conclude that rhere is a rea l number x 0 uch
that f(x o) f= 0. Show that R = S1 + xoS2 doe the j ob.
65. Consider a real 2 x 2 matrix A with fA(A) = (A- 1) 2 . Find fA(A). Campare
60. In this exercise we will show that the geometric multiplicity of an eigenvalue with Exercise 64.
is les than or eq ual to the algebraic multip li c ity . Suppose Ao i an e igenva lue
66. Is tbere a 3 x 3 matrix A with all of the following properties?
of an n x n matrix A with geometric multiplicity d.
a. Explain why there is a basi ß of ~~~ whose firs t d vector are eigenvectors • All eigenvaJues of A are integer .
of A, with e igenvalue Ao· • A is not diagonalizable over C.
b. Let B be the matrix of the linear tran formation T (x = A .~ with respect
• det(A TA) = 36.
to ß . What do the fir t d columns of B Iook like?
c. Explain why the characteri tic polynomi al of B i of the form Gi ve an exampJe or show that none can exist.
67. True orfalse? lf 2 + 3i is an eigenvalue of a real 3 x 3 matrix A, then A i
!s (),) = (A - Ao)d g (A) ,
diagonalizable over C.
68. For real 2 x 2 matrices A, give a complete proof of Fact 6.5.2.
for some polynomial g()..). Conclude that the algebraic multipli ity of A.o
as an eigenvalue of B (and A) i at lea t d.
61. Con ider two sirnilar matrices A and B . Show that
a. A and B have the . ame eigenvalues, with the same algebrai c multipliciti e .
b. det(A) = det (B). SYMMETRIC MATRICES
c. tr(A) = tr(B).
62. We say that two n x n matrices are simulraneously diagonalizable if there is In this section we will work over R except for a brief digression into C, in tbe
an n x n matrix S such that s- 1 AS and s- 1B S are both diagonal. discussion of Fact 7 .3.3. Our work in the last five sections dealt with tbe following
a. Are the matrices central question:
When is a given square matrix A diagonalizable; tbat is, when is there an
0 0]
1 0 and
eigenbasis for A?
0 I In geometry, we prefer to work with orthonormal bases, wbich raise the
question:
simultaneously diagonalizable? Explain.
b. Show that if A and B are simultaneously diagonalizable then AB = BA .
c. Give an example of two n x n matrice such that AB = BA , but A and B For wbich matrices is tbere an orthonormal eigenbasis? Or, equivalently for
are not simul taneously diagonalizable. which matrices A is there an orthogonal matrix S uch that s- 1AS = ST AS
d. Let D be a diagonal 11 x n matri x with 11 di stinct entries on the diagonal. is diagonal?
Find all n x 11 matrices B which commute with D.
402 • Chap. 7 Coordinate Systems
Sec. 7.3 Symmetrie Matrices • 403
(Recall that s- 1 = sT for orthogonal matrices, by Fact 4.3. 7.) We say that A
is orrhogonally diagonalizable if there is an orthogonal S uch that s- l A S = ST AS
is diagonal. Then the question is:

Whicb matrice are ort11ogonally di agonalizable?

Simple exam ples of ortllogonally diagonalizable matrice are diagonal matri-

ces (we can et S = !") and tlle matrices of ortllogonal projections and reflection
(exercise).
EXAMPLE 1 ~ If A is orthogonally diagonalizable, what is tlle relationship between AT and A?
Figure 1
Solution
Wehave Note that tl1e two eigenspaces, E3 and E 8 , are perpendicular (this is no
coincidence, as we wi ll see in Fact 7.3.2). Therefore, we can find an ortbonormal
s- 1AS = D or A = SDS- 1 = S DST. eigenbasis simply by dividing tbe given eigenvectors by their lengtbs:
for an orthogonal S and a diagonal D. Then - .J51 [-12], - .J51[1]
ul= V2 = 2 ·
AT = (S DST)T = SDTST = SDST = A.
U we define the orthogonal matrix
We find that
AT= A .

Surprisingly, the converse is true as weil:

h[i· ~,] ~ 5s[-~ n
Fact 7.3.1 Spectral theorem
then s- 1A s will be diagonal, namely, s- l A s = [~ ~ l
The key ob ervation we made in Example 2 generalizes as follows:
A matrix A is orthogonally diagonalizable (that is, there is an orthogonal S
such that s- 1 AS = srAS is diagonal) if and only if A is symmetric (that is,
AT= A). Fact 7.3.2 Consider a symmetric matrix A . lf ii1 and ii2 are eigenvectors of A, witl1
distinct eigenvalues, )q and A. 2, then ii1 · iiz = 0, that is, ii2 is orthogonal to ii1.

We will prove tllls theorem Iater in tllis section, based on two preliminary Proof We compute the product
results, Facts 7 .3.2 and 7.3.3 . First we will illustrate the spectral theorem witll an
example.
in two different ways :
EXAMPLE 2 ~ For tlle symmetric matrix A = [ ~ ; J. find an orthogonal s suchthat s- AS is
1
v- 1 = v-T1 (A.2v2
T Avz
- - ) = /\.2
' (- - )
v1 · u1
diagonal .
vi Av2 = üf ATÜ2 = (AÜJ)TÜ2 = ()..1Ü 1)Tv2 = A.1(Ü 1 · Ü2)

Solution Comparing the results, we find

A. 1Cü 1 · vz) = A2Cii1 · ii_)
We will first find an eigenbasis. The eigenvalues of A are 3 and 8, witll corre-
sponding eigenvectors [ _ iJ and [;]. respectively. See Figure 1. or
404 • Chap. 7 CoordinateS stems Sec. 7.3 Symmetri Matrices • 405
Since tbe fir t factor in Lhi product, A., - A.2 . is nonzero the ·e ond factor u1 . v2,
mu t be zero a claimed. •
Fact 7.3.2 tel! us that the eig nspace of a symmetri mati-ix are perpendic-
ul ar to one another. Here i another iUu tration of thi property:

EXAMPLE 3 .... For the ymmetric matri ·

find an orthogonal S such that s-' AS i diagonal.

Figure 3

Solution
In Figu re 3, the vectors ii 1, ii2 form an orthonormal bas i of E0 , and ii3 i a
The eigenvalues are 0 and 3, with unit vector in E3. Then ii,, ii 2, ii3 i. an orthonormal eigenbasi fo r A. We can Iet
S = [ii, ii2 ii3 ] to diagonaJize A orthogonall y.
If we apply Gram-Schm.idt 1 to the vector

Note tbat the two eigen paces are indeed perpendicul ar to one another, in accor-
dance with Fact 7.3.2. See Figure 2.
We can construct an orthononnal eigen ba i for A by picking an orthonorma l panning E0 , we find
ba i of each eigen pace (using Gram-Schmidt in the case of E 0 ) . See Figure 3.
and

f19ure 2 The eigenspoces fo ond f3 [I] The computations are left as an exercise. For E 3 we get
ore orthogonal complements. E3 = span ~

Therefore, the orthogon al matri x

- 1/../2 - l j -/6 I/J3

l j ./2 -l j -/6 l/J3 ]
[ 0 2! -16 t;J3

1Allernati ve ly, wc could fi nd a unit ve tor ii1 in Eo and :1 un it vec10r ii3 in EJ, :1nd then set
Ü2 = ii X ii1 .
406 • Chap. 7 Coordinate Systems
Sec. 7.3 Symmetrie Matrices • 40 7
diagonalizes the matrix A: For a I x 1 matrix A, we can Iet S = [1).
Now ass ume the claim is true for n - 1; we show that it holds for n . Pick a
s- 1 AS = 00 00 0]
:eal eigenvalue A. of A (this is possible by Fact 7 .3.3) and choose an eigenvector
0 v1 of length 1 for A.. We can find an orthonormal basis ii 1, ii 2 , . . • , iin of JRil (think
[0 0 3
about how you could construct such a basis). Form the orthogonal matrix

By Fact 7 .3.2, if a symmetric matrix is diagonalizable, tben it is orthogonally

diagonalizable. We till have to how that sy mmetric matrices are diagonalizable
in tbe first place (over IR). The key point is the following observation:
VII

Fact 7.3.3 I The eigenvalues of a symmetric matrix A are real.

Proof (This proof may be skipped in a first reading of this text without harming and compute
your understandlog of the concepts.) Consider two complex conjugate eigenvalues
w
p±iq of A with corresponding eigenvectors ii±i (compare with Exercise 6.5.31 b). P - 1A P .
We wish to show that these eigenvalues are in fact real, that is, q = 0. Note first Tbe fir t column of p- l AP is A.e 1 (why?). Also note tbat p - 1 AP = p T AP is
that symmetric: (P 7 AP) 7 = P 7 A 7 P = P 7 AP, because Ais symmetric. Combining
Cii + iw) 7 cii- iw) = lliill
2
+ !lwf these two statements we conclude that p - 1AP is of the form

(verify this). Now we compute the product p - 'AP - [ A.

- 0
0]
B '
Cii + iwl ACii - iw)
where B is a symmetric (n- 1) x n - 1) matrix. By induction, B is orthogonally
in two different ways: diagonalizable; that is, there is an orthogonal (n - 1) x (n - 1) matrix Q such that
Cii + iwl A(ii- iw ) Cii + iw) 7 CP - iq )(ii - iw ) Q- 1 BQ = D
<P- iq)(ll iif + llwll 2 ) is a diagonal (n - 1) x (n - 1) matrix. Now introduce the orthogonal n x n matrix
7
Cii + iw) 7 A(ii- iw) (ACii + iw)) Cii- iw ) = <P + iq)(ii + iw) 7 (ii - iw)
= <P + iq)CIIiill + llw ll )
2 2
R= [~ ~l
Comparing the results, we find that p + iq = p- iq, so that q = 0, as claimed. •
Then
The proof above is not very enlightening. A more transparent proof would
follow if we were to define the dot product for complex vectors, but to do so
would Iead us too far afield.
We are now ready to prove Fact 7.3.1: symmetric matrices are orthogonally is diagonal.
diagonalizable. Combining equations (1) and (TI) above, we find that
Even though this is not logically necessary, Iet us first examine the case of
a symmetric n x n matrix A with n distinct eigenvalues (recall that this is the R- 1 P- 1 APR=[~ ~] (lll)
case for " most" matrices). By Fact 7.3.3, the n distinct eigenvalues are real. For
each eigenvalue, we can choose an eigenvector of length 1. By Fact 7.3 .2, these is diagonaL Consider the orthogonal matrix S = PR (recall Fact 4.3.4a: the
1 1
product of orthogonal matrice is orthogonal). Note that s- 1 = ( P R)- = R- p - .
1
eigenvectors will form an orthonormal eigenbasis, that is, the matrix A will be
orthogonally diagonalizable, as claimed. Therefore equation (ill) can be written
Proof (of Fact 7.3.1): This proof is somewhat technical; it may be sk.ipped in a fi rst
reading of this text without harm. ·
We prove by induction on n that a synunetric n x n matrix A is orthogonally
s- 1 AS = [ ~ ~ l
diagonalizable. proving our claim. •
408 • hap . 7 Coord inate tems Sec. 7.3 Symmetrie Matrices • 409
The method outlined in the proof of Fact 7.3 .1 i not a en ible way to fi nd
the matrix S in a numerical exampJe. Ratl1er, we can proceed as in Example 3:

Algorithrn 7.3.4 Orthogonal diagonalization of a symmetric m atri x A

a. Fi nd the eigenva lue of A. and find a basi of each eigenspace.
b. Usi ng Gram-Schmid t, find an orthonormal ba i of each eigenspace.
c. Form an orthonormal eigenba is iJ, . Ü2, ... , ii" for A by combining the
vectors you fou nd in part b, and Iet Un it
circle

Figure 4

EXER CI S ES
GOALS Find orthonormal eigenba e fo r symmetric matrices. Appl y the pec-
tra l theorem.

5 i. orthogonal (by Fact 7.3 .2) and s-' AS will be diagonal (by Fact 7 .2 .1). For each of the matrices in Exerci es 1 to 6, find an orthonormal eigenba i . Do
not use technology.

We conclude thi sectio n with an example of a geometric nature: 3. [~ n

[~ ~ j]
EXAMPLE 4 ..... Con ider an in vertible symmetric 2 x 2 matrix A. Show that the linear Iransfor-
matio n T (.i) = A.t maps the unit circle into an ellipse, and find the lengths of the
emimajor and the semin:tinor axe of this ellipse in term of the eigenvalues of
6.
A . Compare with Exerci e 2.2.50.
For each of the matrice. A in tbe Exercise 7 to 11 fi nd an orthogonal matrix S
and a diagonal matrix D such that s-' AS = D . Do not use technology.
Solution

~ ~]
The spectral theorem teils us that there is an onhonormal eigenbasi.s Ü1, ii2 for
T , with associated real eigenvalues ), 1 and A. 2 . Suppose that IA.II ::::. IA.2 1. These 7. A = [
eigenvalues will be nonzero, since A is invertible. Tbe unit circle in ~2 consists
of all vectors of the form I -2
10. A = -2 4
ii = cos(t)v 1 + sin(t)ii2 . [ 2 -4

The image of the unit circle consists of the vectors 12. Let L: ~ ~ ~ 3 be a refl ecti on in the line spanned by

T(v) = cos(t) T(v,) + sin (t) T(v2 )

=cos(t)A. 1 u1 +sin(t)A.2 u2 ,
an ellipse whose semimajor ax is A. 1 v1 has the length IIA. 1 ü1 11 = IA. 1 1, while the
length of the semi mj nor axjs is IIA. 2 ii2 11 = IA.2 1. See Figure 4. a. Find an orthonormaJ eigenba i. ß fo r L .
ln the example illustrated in Figure 4 the eigenva lue A. 1 is positive and A.2 is b. Find the matri x B of L with re. pect to ß.
negati ve. ~ c. Find the matri x A of L with re pect to the tandard ba i of ~ 3 .
410 • Chap. 7 Coord.inate S tems
Sec. 7.3 Symmetrie Matrices • 411
. k = h r rhe
13. Cons1"der a ymrne tr"1c 3 X 3 l11atrix . A with . 3linear
?
transformaüon 19. Consider a linear transfonnation L from IR" to IR"'. Show that there i an
T (x) = A.r necessarily the reftectJOn m a subspace of ~ · orth onorm al basis ü1, ü2 , • .. , vn of IR" suchthat the vector L(ii 1) , L (v 2 ), • • • •
L (Ü") are orthogonal (note: some of the vectors L (v;) may be zero). Hint:
14. In Example 3 in this ection we diagonalized th matnx
Consider an orthonormal eigenba is ü1, ii2 , . . . , vn for the sy mmetric ma-
trix A TA.
1 20. Consider a linear transfonnation T fro m IR" to JRm, where n doe not exceed m.
1 v
Show that there is an orthononnal ba is 1, •• • , ii" of IR" and an orthonorrnal
by means of an orthogonal matri x S. Use t~i re ult to ~agonal ize the fo l-
ba is w, , ... ,
Ü!m of IR"' such that T (Ü;) is a scaJar multiple of w; ,
for i = 1.
.. . n . Hint: Exercise 19 is helpful.

:.w;[n~ TTJ' orthog:~a[JI-t~r and: D] 1D eac:. ca[r ~ !] 21. Consider a symmetric 3 x 3 matrix A with eigenvalues l , 2, 3. How many
different orthogonal matri ces S are there such that s- 1 AS is diagonal?
22. Con ider the matrix
2 2 2 I 1 -2 2 2 0
lf A is invertible and orthogonally diagonalizable, i A- t orthogonally diag-
15. o

~]
2 0
onalizable as well ?
16. a. Find tbe eigenvalue of rhe matrix

A=
1
J
1
1
1
1
1
1
J
1
1 where k i a constant.
A =
l k
~
0
k
0
2
0
k

1 1 1 1 1 a. Find a value of k such tbat the mattix A i diagonalizable.

1 1 1 1 b. Find a value of k such that A is not diagonalizable.
23. lf an n x n matrix A is both syrrunetric and orthogonal, what can you say
wirh their multip1icities (note that tbe algebraic mu1tiplicity agrees with about the eigenvalues of A? \\'bat about tbe eigenspaces? Interpret the linear
tbe geometric multiplicity; why?). Hint: What i the kerne! of A? transformation T (x) = Ax geometrically in the cases rz = 2 and n = 3.
b. Find tbe eigenvalues of tbe matrix 24. Con ider the matrix
3 1 1 J 1
J 3 1 J 1 0 0
0 0
B= l 1 3 1 1 A= 0 1
1 1 3 1 [
1 1 3
I 0

with their multiplicities. Do not use technology . Find an orthonormal eigenba i for A .
c. Use your result in part b to find det(B). 25. Consider the matrix
17. Use the approach of Exercise 16 to find the detenni nant of the n x n matrix
B that has p's on the diagonal and q 's elsewhere.
00 00 00 01 01]
0 0 1 0 0 .
0 I 0 0 0
I 0 0 0 0

Find an orthogonal 5 x 5 matrix S uch that s- tAS is diagonal.

18. Consider sorne unit vectors v1 , . • • , V11 in IR" such that the angle between Ü;
and üj is 60° for all i =f. j . Find the n-volume of the n-parallelepiped span~ed
by ü1, • •• , V11 • Hint: Let A = ( v1 v"] and think about the rnatrix ArA
and its detenninant. Exercise 17 is usefu l.
26. Let }11 be the n x n matri x with aJI ones on the '·other diagonal" and zero
elsewhere. (In Exercises 24 and 25 we tudied 14 and J . respectively.) Find
the eigenvalues of 111 , with their multiplicities.
412 • Chap. 7 Coordinate System Sec. 7.4 Quadra tic Forms • 413
27. Di agonati ze the n x n matri x b. Co_n ider a c? mp lex n x n matrix A that has zero a it only eigenva lue
(wtth algebraJc mul tipli city n). Use Exercise 37 to how that A is nilpotent
I 0 0 0 I 39. Let us first introdu ce two notati ons.
0 0 0
For a complex n x n matri x A , Iet lAI be the matrix whose ij th entry is lai;" I-
For two rea l n x n matrices A and B we wri te A < B if a-.1 < b·1. fo r all
0 0 1 0 0 entri e . Show that - ' - '
0 l 0 1 0 a. lA B I S IAII BI, for all corn plex n x n matrices A and B , and
0 0 0 b. lA' I S lAI' , fo r all co mpl ex n x n matrices A and allpos iti ve integer t_
(all one along botb di agonal and zeros el ew here). 40. Let U ::::: 0 be a real upper triangul ar n x n matri x with zero on the di agon al.
Show that
28. Diagonalize the 13 x 13 matrix
0 0 0 0 1 Un + u I .::: r"Un + u + U 2 + .. . + un- I)
0 0 0 0 ] for all pos iti ve integers t. See Exercises 38 and 39.
41. Let R be a complex upper tri angular n x n matrix with Ir;; 1< 1 for i = 1. . .. , n .
0 0 0 0 Show that
1
lim R' =0
r~
(all ones in the la t row and the last column, and zeros e l ewhere).
29. Con ider a symmetric matri x A. If the vector ü i in the image of A and w (meaning that the modulu s of all e ntri es of R' approaches zero . Hint : We
is in the kernel of A, is ü necessarily orthogonal to w? Ju li fy your answer. ca n write IRI S )..(in+ U ), fo r ome po iti ve real numbe r ).. < 1 and an upper
triangul ar matr ix U ::::: 0 with zero on the diagonal. Exerci es 39 and 40 are
30. Consider an orthogonal matri x R wbose first column is ü. Form the synunetric
helpful.
matrix A = vür. Find an orthogon al matiix S and a di agonal matrix D such
that s-t AS = D. Describe S in terms of R . 42. a . Let A be a complex n x n matrix such that 1)..1 < I for all eigenvalues ),
of A. Show that
31. True or false? If A is a symmetric matrix, then ran.k(A) = rank (A 2 ) .
32. Consider the n x 11 matrix with all ones on the main di agonal and all q 's lim A 1 = 0
/ -4
elsewhere. For which choice of q is this matri x in vertibl e? Hint: Exercise 17
i helpful. (meaning that the modu lu of all entries of A ' approache zero) .
b. Prove Fact 6.5 .2.
33. For whi ch angle(s) a cao you find three dislinct unit vectors in ~ 2 such that
the angle between any two of tbem is a ? Draw a ketch.
34. For which angle(s) a can you find four distinct unit vectors in ~ 3 such that
the angle between any two of them is a ? Draw a sketch .
QUADRATIC FORMS
35. Consider 11 + 1 distinct unit vectors in ~~~ such that the angle between any
two of them is a. Find a .
In thi s ection we will present an important apptication of the pectral theore m
36. Consider a syrnmetric 11 x n matrix A with .4 2 = A. Ts the Lineartransformation (Fact 7 .3. 1).
T (.r = A.i= necessaril y the orthogonal projection onto a subspace of IR"? In a multi vari abl e calculu text, we found the foll o\ ing proble m:
37. We ay that an 11 x n matrix A is triangulizable if A i simil ar to an upper
triangular n x n matrix B. EXAMPLE 1 .... Consider the fun ction
a. Gi ve an example of a matrix with real entries that i not trianguli zable q (x1. x2) = 8x~ - 4.x lx2 + 5x= .
over ~-
b. Show that any n x n matrix with complex entries is tri anoulizable over C. Note that q (0 0 = 0. Determine whether this fun ction ha a tri ct m1mmum at
Hint: Gi ve a proof by induction analogaus to the proof ~f Fact 7.3.1. (.x 1,.x2 = (0 , 0), that is, whether q (x1,x2) > 0, for all (.x 1, x 2) =1- (0 , 0).
There are a number of way to do thi. proble rn ome of which you may
38. a . Consider a complex upper triangular n x n matrix U with zeros on the
ha ve seen in previous courses. Here we pre ent an approach whi ch u e matrix
diagonal. Show that U is nilpotent, that is, un = 0 . (Compare with
techniques. We first develop some theory. and the n do the examp le.
Exercises 56 and 57 of Section 3.3.)
414 • Chap. 7 Coordinate Systems Sec. 7.4 Quadratic Forms • 415
Note tbat we can write

We "split" the contribution -4x 1x 2

I
/'
equally among tb.e two components.

More succinctly, we can write

qC-"i) = ;. Ax , where A = [ -~ -2]

5 ,

or Figure 1

q (x-) = x-r Ax- .

Tbe matrix A is symmetric by construction. By tbe spectral tbeorem (Fact 7 .3. 1), Let us present these ideas in greater generality:
v
there is an ortbonormal eigenbasis Ü1, 2 for A . We find

- = .j51 [- 2]}
Vt
- ./51[1]
v2 = 2 ,
Definition 7.4.1 Quadratic forms
A function q (XI. x 2 , . . . , Xn) from ~n to ~ is called a quadratic form if it
witb associated eigenva1ues ). 1 = 9 and )..2 = 4 (verify this). is a linear cornbination of functions of tbe form x;Xj (where i and j may be
x
If we write = c 1 ü1 + c2 ü2 , we can express the value of the function as equal). A quadratic form can be written as
follows:
q (x ) = ; . Ax = ;r Ai ,
for a syrnmetric n x n matrix A .

v
(Recall that 1 · ü, = I , ü, · Ü2 = 0, and ~ · ~ = 1, si nce ü1, ü2 is an ortbonormal
basis of ~2 .)
The formula q (x) = 9cf +4ci shows that q (x) > 0 fo r all nonzero x, because
EXAMPLE 2 ~ Consider the function
at least one of the terms 9cf and 4ci is positive.
q (X J, X2, X3) = 9.xf + 7xi + 3xj - 2X JX2 + 4XlX3- 6x2X3.
Our work above shows that tbe c 1-c 2 coordinate systern defi ned by an or-
thonormal eigenbasis for A is "well adjusted" to the function q. The formul a Find a symmetric matrix A such that q (x) = x · Ai for all i in R 3 .

9ci + 4c~
Solution
is easier to work with than tbe original formula
As pointed out in Exan1ple l , we Iet

a;; = (coefficient of xf),

because no term involves c 1c2 : aij = aj; = tccoeffi cient of x ;.xj). if i =I= j.

Therefore,
q (x,, x2) = 8xf - 4x ,x2 + Sxi
= 9cf + 4c~
The two coordinate systerns are sbown in Figure 1.
416 • Chap. 7 oord inate Systems
Sec. 7.4 Quadratic Form • 417
The ob en ation we made in Example 1 abov can now be generalized as By Fact 7.4.2, tbe definitenes of a ymmetric matri x A i ea y to determine
from it e igenval ues:
follows:

Fact 7.4.2 Con ider a quadratic form q ,\-) = ,\- . A.\- from IR" to lR. Let ß be an orthonor- Fact 7.4.4 A symmetric matrix A is positive definite if and on ly if) all of its eigenvalue
m al eigeobasis for A , witb a ociated eigenvalue At, . . . ,).." . T hen are positive. The matrix A is positive semidefinite if (and on ly if) al l of it
e igenvalue arepositive or zero.
q( i ) = ), , c j + )..2d + · · · + ).."c~ .
pect ro ß .1
These facts foiJow immediately from the formula

Aoain. note that we have been able to get rid of tbe mixed term : no
-
q (x ) = At C21 + · · · + AnC2
11
(Fact 7.4.2). •
umma:d i~volve c;c1 , itb i =j:. j) in the fonnula abo e. To j ustify tbe formula The determinant of a positive definite matrix is positive, ince the deter-
rated in Fact 7 .4.2. we can proceed as in Example 1. We leave the detail a an minant is the product of the eigenvalues. The converse is not true, however:
exercise. consider a symmetric 3 x 3 matrix A with one po itive and two negative eigen-
When we study a quadratic fo rm q we are often intere ted in fi nding out values. Then det(A) is positive, but q (x) = ,"Y · Ai is indefinite. In practice. the
\ herher q .\- ) > 0 fo r all nonzero .\- as in Example 1). In thi context it i u eful fo llowing criterion for positive definiteness is often used (a proof i outlined in
to introduce the following terminology : Exercise 38):

Definition 7 .4.3 Positive definite quadratic forms Fact 7.4.5 Consider a symmetric n x I! matrix A. F or m = 1, ... , 11 , Iet A (m ) be tbe
m x m matrix obtained by omitting all row and columns of A pa t the mtb.
Consider a quadratic form q (x) = ,\- -Ai, where A is a ymmetric 11 x n matrix. These matrices A (ml are called the principal submatrices of A.
We say that A i positive definite if q (x ) is positive fo r all nonzero x in The matrix A i positive definite if (and only if) det(A (ml ) > 0, for aJI
:IR", and we call A positive semidefinite if q(x ) 2: 0, for aH x in :IR". m = 1, ... , 11.
Negative definite and negative semidefinite ymmetric matrices are de-
.fined analogously.
Finally, we call A indefinite if q takes po. itive as weil as negative A an example, i::onsider the matrix
value. . -1

EXAMPLE 3 ..... Consider an m x n matrix A. Show that the functi on q (x ) =

form, find its matrix , and determine its definitene
II A~ f is a quadratic
from Example 2.
7
-3 -n
det(A ' l = det[9] = 9 > 0
Solution
9
\ e can \ rite q .\-) = Ax ) . (Ai ) = (Axl (A.r) = ; r AT Ai = ; . (AT Ai). This det(A <2> = det [ - l ] = 62 > 0
- l 7
shows that q i a quadratic form, with matrix AT A. This quadratic form i positi e det(A t 3>) = det A) = 89 > 0
emidefinite, becau e q (.r = II A.r ll 2 2: 0 for a!J vector ; in 2". ote that q (x = 0
We can conclude that A i po itive definite.
if and on ly if i i in tbe kerne! of A. Therefore, the quadratic form i po iti e
f A and u e Fact 7.4 . .
~
Alternatively, we could find the ei genva lu e in g
definite if and onl if ker(A) = {Ö}.
technology, we find that )q ~ 10.7, A2 ~ 7 .1, and A.J ~ J .2, c nfirming ur result.

+ Princip a l Axes
1The basic propert ie o f quadrati fonns were fi rst derived by the Dutchman Johan de Witt (1625- When we study a fu nction j(.x ,, x _, .. . , Xn) from IR" to IR . w are often intere. ted
1672) in his Eiementa curvamm linearum. De Witt was one of the leadin g state me n o f his time, guidin g
hi country lhrough two war against England . He consolidated his nati on· s com me rc ial and naval power. in the solution of the equation
De Witt met an unfortunate end when he w tom to piece by a pro-En gli h mob. (He . hou ld h, ve staycd
with math !).
j(Xt,X2, ... ,X11 ) =k
418 • Chap. 7 Coordinate Systems
Sec. 7.4 Quadratic Forms • 419
Now consider the Ievel curve

q (x) = x ·Ai = 1,

where A is an invertible symmetric 2 x 2 matrix. By Fact 7.4.2, we can write this

equation as

Ä1ct + Ä2d = 1,

where c1 , c2 are the coordinates of .X with respect to an orthonormal eigenbasis

for A, and AJ, Ä2 are the associated eigen values.
As we discussed above, this curve is an ellipse if both eigenvalues are
Figure 2 positive and a hyperbola if one eigenvalue is positive and one negative (what
happens when both eigenvalues are negative?).

for a fixed k in ~. called the Ievel sets of f (Ievel cur-ves for n = 2, Level swfaces EXAMPLE 4 .... Sketch the curve
for n = 3).
Here we will t.hink about the Ievel curves of a quadratic form q (x ~ 2 ) of
1•

two variables. For simplicity, we focu s on the level curve q (x 1, x 2) = I.

(see Example 1).
Let us first think about tbe case wben there i no mixed term in the formula.
We trust that you had at least a brief encounter with those Ievel curves in a previous
course. Let us discuss the two major cases: Solution
In Example 1 we found that we can write this equation as
Case 1 • q (x 1, x 2 ) = ax~ + bxi = I, where b > a > 0. This curve is an ellipse,
as shown in Figure 2. The lengths of the semimajor and the semiminor axes are 9cT + 4d = 1,
1j ..j0. and 1/.Jb, respectively. Thi s ellip e can be parameterized by
where c 1, c2 are the coordinates of x with respect to the orthonormal eigenbasis
[XI] = COS (t ) [ Jjfa] +sm. (t ) [ lj.Jb
0 ] .
X
2
Ü
- .j51 [-J2]
U] = V? =-1
- .../5
[1]
2

Case 2 • q (x1, x2) = a x ~ + bx~ =

is a hyperbola, _:,ith x 1 -intercepts [
I, where a is positive and b negative. This
±l~..jO.l as shown in Figure 3. What is the
for A = [ -~ -; J We sketch this ellipse in Figure 4.

slope of the asymptotes, in terms of a and b?

Figure 4
Figure 3
420 • Chap. 7 Coordinate Systems Sec. 7.4 Quad ratic Forms • 421
Th c 1- and the c 2 -ax ar called the prin ipal axes of the quadratic fo rm 9. Arealsquarematrix A i called skew-symmetric if Ar = -A .
,.
q.q ,.2 ·I·r 2 + 5·r 2·
1. ) -_ g·,.21 _ 4-r 2 ote that tlle - are the e igen pace of the matrix a. If A i skew-sy mmetric, is A 2 kew-symmetric as weil ? Or i A 2 ym-
metric ?
b. lf A is skew-symmetric, what can you say about the defin iteness of A 2 ?
What about the eigenvalues of A 2 ?
of the quadratic form . c. What ca n you say about the cornplex eigenvalue of a skew- ymmetric
matrix ? Which skew-sy mmetric matrices are diagonalizable over ~ ?
10. Consider a quadratic form q(x) = x . Ax on ~~~ a nd a fi xed vector v in iR" .
Definition 7.4.6 Principal axes Is the transformation
Con ider a quadratic form q x ) = X . Ax, where A i a ymmetric 11 X 11 L (.r) = q(,r + v)- q (.r) - q (v)
matrix with n distinct eigenvalue . Then tbe eigen pace of A a.re called the
principal axes of q (note that tllese wi ll be one-dimen iona l). linear? lf 0 what is its matrix ?
1

11. If A is an invertible symmetric matri x, what is the relationship betweeo the

defi ni tenes of A and A - 1 ?
Return to the case of a quadratic form of two variab le . We can ummarize
12. Show that a quadratic form q (x) = ,r -kr of two varia bles is indefinite if (and
our findings as follow :
on ly if) det(A) < 0. Here, A is a symmetric 2 x 2 m a trix .
13. Show that the diagonal e lements of a positive definite matrix A are positive.
2
~ J where a and det(A) are both positive.
Fact 7.4.7 Con ider the curve C in iR defined by
14. Con ider a 2 x 2 matrix A = [ :
q(x 1, x 2 =ax"j+bx
? 1x2+cx2=
' 1.
Without u ing Fact 7.4.5, show that A i positi ve definite. Hint: Show first
Let >.. 1 and >..2 be the eigenvalues of the matrix [ b~2 b~ ]
2
of q.
that c i positive, and thu tr(A) i positive. Then think about the si.gns of
the eigenvaJues.
lf both ), and >..2 are positive, then C is an ellipse. If there i a positive and
1

a negative eigenval ue, tben C is a hyperbo/a. Sketch tbe cu rve defined in Exercises 15 to 20. In each case, draw and Iabel the
principal axe , Iabel the intercepts of the curve with the principal axe , and give
the formula of the curve in the coordinate ystem defined by the principal axe
EXERCISES

GOALS Apply the concept of a quadratic fom1 . U e an ortbonormal eigenba i

15. 6x? + 4xJXz + 3xi = 1. 16. XJXz =1 .
for A to analyze the quadratic fonn q(x) = x · A.t. 17. 3x~ + 4XJX2 = J. 18. 9xT- 4x lx2 + 6x~ = l .
For each of the quadratic fonns q li ted in Exerci e 1 to 3 find a ymmetric 19. x? + 4x 1.rz + 4xi = 1. 20. -3xT + 6x1xz + 5xi = I.
matrix A suchthat q (x) = x . Ax.
21. a. Sketch the following three surfaces:
1. q (x1 x2) = 6xr - 7..r Jx2 + 8xi .
I
x? + 4xi + 9x j 1 I

2. q(X J X2) = XJX2·

xf + 4x 2 - 9xj 1,
3. q(x J x2 1x3) = 3x~ + 4x~ + 5xj
I + 6x1x3 + 7x2x3.
- xr - 4x~ + 9xj 1.
Detennine the definüeness of the quadrati c forms in Exercises 4 to 7 . Which of these are bounded? Which are connected? Label the point
close t to and farthe t from the origin (if there are any .
4. q (xJ x2) = 6xT
I + 4xlx2 + 3xi. b. Consider the surface
5. q(x 11x 2 ) = x~ +4x l x2+xi.
6. q (x J, x2 ) = 2xT + 6x1 x2 + 4xi . xr + 2xi + 3xj + XJ-'"2 + 2XJXJ + 3x2X3 = I.
7. q (x 1 x21 x3) = 3x~
I + 4x 1x3. Whi ch of the three surface in part a doe this surface qualitati vely re em-
8. If A is a symmetric matrix, what can you say about the definiteness of A2 ? ble most? Which point on thi urface are clo est to the origin? Gi ve a
When is A 2 positive definite? rough approximation ; u"e technology.
 Sec. 7.4 Quadratic Forms • 4 23
422 • Chap. 7 Coordinate Systems
32. Show that any positive definite 11 x n matrix A can be written a A = 8 B T ,
22. On the surface
where 8 is an n x n matrix with orthogonal columns. Hint: There is an
- xf + xi - X~ + l ÜX JX3 = 1, orthogonal matrix S suchthat s- 1 AS = sr AS = D is a diagonal matrix with
positive diagonal entries. Then A = S D ST. Now write D as the square of a
find the two points close t to the origin. diagonal matrix.
23. Consider an n x n matrix M that is not symmetric, and define the function
g(-~) = .~ . Mx from JR" to JR. Is g necessarily a qu adratic form? lf so, give a 33. For the matrix A = [ -~ - ;] write A = B8 T as discussed in Exercise 32.
symmetric matri x A (in terms of M) such that See Example 1.
34. Show that any positive definite matrix A can be written as A = 8 2 , where B
g(.~) = .~ · A,t .
is a positive definite matrix.
24. Consider a quadratic form 35. For the matrix A = [ -~ -;] write A = 8 2 as discussed in Exercise 34.
q(.~) =; . A x, See Example 1.
where A is a symmetric 11 x n matrix. Find q(eJ ) . Give your answer in tetms 36. Cholesky factorization for 2 x 2 matrices. Show that any positive definite
2 x 2 matrix A can be written uniquely as A = LLT, where L is a lower
of the entries of the matrix A .
triangular 2 x 2 matrix with positive entries on the diagonal. Hint: Solve the
25. Consider a quadratic form equation

[~ ~ ]=[~ ~][~ ~ l
q(x) = x · Ai ,
wbere A is a symmetric n x n matrix. Let ii be a unit eigen vector of A , witb
associated eigenvalue A.. Find q (ii) . 37. Find the Cholesky factorization (discussed in Exercise 36) for
26. Consider a quadratic form
q(x) = ;. Ax ,
A = [ -~ -;J.
38. A Cholesky fa ctorization of a symmetric matrix A is a factorization of the
where A is a symmetric n x 11 matrix . True o r fa lse? lf there is a nonzero form A = LLT , wbere L is lower triangular with positive diagonal entries.
vector ii in IR." such tbat q (ii) = 0 then A is not invertible. Show that for a symmetric rz x n matrix A the following are equivalent:
27. True o r false? lf A and B are two distinct symmetric n x n matrices, then the
quadratic forms q(x) = x. Ax and p (."i) = x· Bx
are di stinct as well. i. A is positive definite.
28. True or false? If A is a symmetric n x 11 matrix whose entries are all positive, ii. All principal submatrices A (m ) of A arepositive definite (see Fact 7 .4.5).
then the quadratic form iii. det(A (m)) > 0 for m = 1, ... , n.
iv. A has a Cholesky factorization A = L L r.
q(x) = x · Ai
Hints : Show that i implies ii, ii implies iü, iii impl:ies iv, and iv implies i.
is positive definite. The hardest step is the implication from iii to iv: arguing by induction on
29. True or false? If A is a symmetric n x n matrix such that the quadratic form n, you may assume that A<n- l) has a Cholesky factorization A("- 1) = BBT .
Now show that there is a vector .X in JR"- 1 and a calar t such that
q (i) = x · Ax
is positive definite, then all entries of A are positi ve.
A = [ A~; >
1
n [ffr ~] [B~r ~ ].
=

30. True or false? If q is an indefinite quadratic form from JR" to JR, then there Explain why the scalar t is positive. Therefore we have the Cholesky factor-
is a nonzero vector x in lR" such that q (x) = 0. ization
x
31. Consider a quadratic fonn q(i) = · Ai, where A is a symmetric n x n matrix
with the positive eigenvalues ;. 1 2: ;.2 2: . . . 2: ;." . Let S"- 1 be the set of
all unit vectors in lR". Describe the image of s"- 1 under q , in terms of the (continued )
eigen values of A.
424 • Ch ap. 7 Coord inate ystems Sec. 7.5 Singular Values • 425
Tlli rea onin o al o shows t.hat the C hole ky facto ri zatio n of A is un ique. 5 SINGULAR VALUES
Alternati vely,b_ ou can use the LDL T factorizatio n of A to show that ii i
imp lie iv ( ee E e rci e 4 .3.33). _ . We Start with a somewhat technical example.
To show that i implies ii consider a no nzero vector x tn IR"' and de fin e
EXAMPLE 1 .... Consider an m x n matrix A. Tbe n x n matrix AT A is symmetric; therefore,
there is an orthonorrnal eigenbasis ü1, ü2 , . . . , Ün for AT A, by the spectral theorem
(Fact 7 .3. 1). Denote the eigenvalue of A T A associated with Ü; by A.;. Wbat can
you say about the vectors A Ü1, • . • , A Ü11 ? Th.ink about their lengths and the angles
they enclose.
in IR" fi ll in 11 - 111 zeros). Then
.\-TA (111 ; = YT Ay > 0.
Solution

Let us compute the dot products A Ü; . A Üj.

39. Find the Cholesky factorizatio n of the matrix
{0
A~H ~~ 2n - A-
A V;· Vj=
(A -)T A-
V;
-T ATA-
Vj= U;
-T - . - -
Vj = V;AjVj = Aj(V; · Uj ) =

The vectors AÜt , .. . , AÜn are orthogonal , and the length of AÜj is
Aj
ifi=j=j
ifi=j

jfj. ~
40. Consider an invertible 11 x 11 matrix A. Wh at is the relatio n hip between the This observation motivates the following definition:
matrix R in the QR factorization of A and the matrix L in the Chole ·ky
factorizati on of AT A?
Definition 7.5.1 Singular Values
41. Consider the qu adratic form
The singular values of an m x n matrix A are the square roots of the eigen-
values of the symmetric n x n matrix AT A (listed with their algebraic multi -
We define pljcities). It is customary to denote the singular values by at. a~, .. . . a11 , and
to Ii st them in decreasing order:

Tbe discriminant D of q is defined as

We summarize the observation made in Example 1:
11
D = det [ q
Cf2 1

The second derivati ve test teils us that if D and q 11 are both positi ve, then Fact 7.5.2 Let L (x ) = Ai be a linear transformation from IR" to IRm. Then there is an
q (x 1, x 2) has a minimum at (0, 0). Justify th.i s fact, using the theory developed orthonormal basis ü1, ••• , Ü11 of IR11 such that
in thi s section. a. the vectors L (Ü1), •• •, L (vn ) are orthogonal, and
42. For wh.ich choices of the constant p and q i the n x n m atrix b. the lengtb of L(Üj) is aj, the jth singular value of A .
p q To construct ü1, ••• , Ü11 find an orthonormal eigenbasis for the matrix ArA .

l]
Cf p
B= .
[ Part a of Fact 7.5.2 is the key statement of thi ection ; make ure you
Cf q understand the significance of this claim. See Exercise 2.2.47 for a special case.
positive definite? (B has p ' s on the diagonal and q 's elsewhere.) Hint: Here are two numerical exa:mples:
Exercise 7.3.17 is he lpful. EXAMPLE 2 .... Let
43. For whi ch angles a can you fi nd a basis of IR" such that the angle between
any two vectors in thi s basis is a ? L(x) =Ai , where A = [ -~ ~J.
426 • Chap. 7 Coordinate Sy tems Sec. 7.5 Singular Values • 42 7
a. Find the si ngul ar value of A .
b. Find orthonormal vector -, and iJ! in IR 2 such that L (ü,) is orthogona l to
L(Üz).
LCi) = Ai
c. Sketch and descri be the image of the unit circle under the transformation L. ~
5

Solution
a. We need to find the eigen alues of the matrix. AT A fi rst.

AT A = [ ~ - ~ ] [ _~ ~ ] = [ -~~ -~~]
The characteristic polynomial of AT A is
;_2 - 125A. + 2500 = (A.- lOO)(A.- 25) . (Different scales are used for domain
Figure 1 and codomain.)
The eicrenvalues
ö
of AT A are A. 1 = 100 and A.z = 25. The singular value of
A are
and c. The unit circle consists of all vectors of the form .i = cos(t ü1 + sin(t)ii2 .
The image of the unit circle consisrs of the vectors L (.i) = cos(t L (ii 1) +
b. Find an orthonormal eigenbasi for AT A ( ee Example 1). in (r) L Üz); that is, the image is an ellipse whose semimajor and semiminor
axe are L (ü,) and L(Üz), respectively. Note that IIL(Ü 1) 11 = a, = 10 and
]5
E IOo = ker [ 30 30
60 ] = span [ - 2]
1 IIL(ii2)11 = a2 = 5. See Figure 2. <111111

E 25 = ker [ -~~ _ ~~ ] = span [ ~]

Therefore, the vector Fact 7.5.3 Let L (r) = A.i be an inverti ble linear transformation from IR2 to IR 2 • The
-u, = -J5l [- 2]
1 and
image of the unit circle under L i an elJipse E . The lengths of the ernimajor
and erniminor a.xes of E are the Singular values a 1 and a2 of A , respectively
(see Figure 2).
do the job.
We can check that the vectors

-
Au,=
1 [ - 20
.J5 10]
Figure 2
and

- = 1 [ 10] L(.i') = Ai
Avz -J5 5 ~

are perpendicular. Let us also checkthat the lengths of Aii, and Avz are tbe
singular values of A:

IIAii , ll = ~ I [ _;~J I =JsKoü = 10 = a ,

~ I [ ~] I = Js Jt2s
1 Uni I
IIA vzll = = 5 = az circle the unit
circle:
The vectors ii 1, iiz and their images L(ii i), L (Ü 2 ) are shown in Figure 1. an ellipse
428 • Chap. 7 Coordinate Sy tems Sec. 7.5 Singular Values • 429
EXAMPLE 3 .... Con ider the linear tran formation
L(."i) =Ai

= [~
I
~
L(.r ) = A.r, where A 1

a. Find the singular val ue ' of A.

b. Find orthonormal vectors -~> ii2 , ii3 in ~3 such that L(ii1 ) , L (ii2), and L(Ü3)
are orthogonal.
c. Sketch and describe the image of the unit sphere under the transforma-
tion L. Unil sphere
in ~3
Solution
Figure 3
a.
1
The image is tbe full e llipse shaded in Fig ure 3.
1
Example 3 shows that some of the si ngular value of a matrix may be zero.
The eigenvalues are >.. 1 = 3, >.. 2 = I , >.. 3 = 0. The ingular value of A are Suppose the singtllar values a 1, . .. , a,. of an m x I! matri x A are nonzero, while
as+ l ... , a" are zero. Choose vectors ü1, .. . , ii,., ü +I · . . . , ii" for A a introduced
in Fact 7.5 .2. Note that II A ii; II = a; = 0 and tberefore A Ü; = 6 for i = s + l ,
. . . , n. We claim that the vectors Aü 1 , •• • , Aüs form a basis of the image of A.
b. Find an orthonormal eigenbasis u1, u2 , u3 for AT A (we ornit the details). lndeed, these vectors are linearly independent (because they are orthogonal and
nonzero), and they span tbe image, since any vector in the image of A can be
Eo ~ ker ~ span [ - : ]
A)
written as
kr = A(c 1 ü + ... + c,. v + ... + c"Ü")
1 5

= c 1 Aü 1 + · · · + c AÜ5 .
Thi s show that s = dim(im A) = rank (A).
1
ii2 = - [
.J2 -1
~] •
Fact 7.5.4 lf A is an m x n matrix of rank r, then the singular vaJue a 1 , • • • ar are
We compute Aii" Av2 , Aii 3 and check orthogonality: nonzero, while ar + l, ... , a" are zero.

- .J61[3]
Au 1 = 3 ,
We can also check that the length of Au; is a; :
• The Singular Value Decomposition
Ju t a we expre sed the Gram-Schmidt process in term of a matrix decompo ition
(the QR factorization), we will now expres Fact 7 . 5.~ in t rm. of a matrix
c. The unit sphere in JR 3 consists of all vectors of the form
decomposition.
x= c1il 1 + c2 Ü2 + c3Ü3 , where cT + c~ + cj = l. Consider a linear transfonnation L(,r) = A.r from IR" to IR"', and choose an
orthonormal basis ii" ... , ü" as in Fact 7.5.2. Let r = rank (A). We k..11ow that the
The image of the unit sphere consists of the vectors vectors Aü 1
,Aur areorthogonal and nonzero, with IIAÜ; II = a; . We introduce
• • • ,

the unit vectors

L (x) = c1L Cii1) + c2 L Cii2),
- I - - I -
= - Av1 , , !Ir = -Aur.
where ci + ci : : : I (recall that L (ii3) = 0). 111
a1 ar
430 • Chap. 7 Coordinate Systems Sec. 7.5 Si ngul ar Va lues • 431
We can expand the sequence ü 1, .. • , ü,. to an orthonormaJ basis u1, .. .• Üm of IR"'. or, more succinctly,
Then we can write
AV = U'E .

Note that V is an orthogonal n x n matrix, U is an orthogonal m x m matrix, and

and "E is an m x n matri x whose first r diagonal entries are a ~> .. . , a,., and all other
entries are zero.
A v;= 0 for i = r +1 . . . , n. Mul ti plyi ng the equation A V = U "E witb V7 from the right, we find that
A = U"EV 7 .
We can express these equations in matrix fo rm a foll ow

Fact 7.5.5 Singular value decomposition (SVD)

Any m x n matrix A can be written as
Vr Vr+l v" A = U"EV 7 ,
where U is an orthogonal m x m matrix; V is an orthogonal n x n matrix; and
"E is an m x n matrix whose first r diagonal entries are tbe nonzero singular
values a 1, ••• , a,. of A , and aJI other entries are zero (r = rank(A)).
V Alternatively, this singular value decomposition can be written as
A =a1u- 1-uT1 + · · · +a,.u,.v,.
- - T
,
where the Ü; and the Ü; are tbe columns of U and V , respectively (Exer-
cise 29).
a,.u,. 0 0

A ingular value decomposition of a 2 x 2 mat:r:Lx A is presented in Figure 4.

~
a1
0
figure 4
u2 - A = E \I T

Aii l = lT IÜI
Ul u,. 0 0 a,. ul

0 0

al
VI = I'
orlhogonal
I~
e2 fnho~onal
lT2e l
0
Ul u,. Ur+ l Um a,.

0
L
0
el

-L = [ rrl
0
0 ]
cr2
lT 1e 1

u
432 • Chap. 7 Coordinate Systems
Sec. 7.5 Singular Values • 433
Consider a singular value decomposition
Here are two numerical examples.
A = U L:V r ,
EXAMPLE 4 ..,. Find an SVD for A = [ _
6 2-lJ. (Compare with Example 2 .)
7 6 where

Solution
- "J51 [_2] and- "J51 [l ] so hat
In Example 2 we fou nd u1 =
1
v2 = 2 , t
v,. and U= UJ Um

1 [- 21 2l].
V = "JS

The columns ii 1 and 1~ 2 of U are defined by

We know tbat
..
u =
I _ 1 [ 1] ,
cr Av 1 = J5 _
1
1
2 A v; = cr;Ü ; for i = 1, . . . , r

- = 1 - = "J51[2]
112 CT2 A U2 1 '
and
A i.i; =Ö fori=r + l , . .. , n.
and therefore
These equations tell us tbat
1 [- 2l 2]
u = "J5 l . ker(A) = span(i.ir+ 1, ... , Ün
Finally, and

o]=[1o0 o5 ].
cr2
im(A) = span(Ü1, .. . , Ür).
(Fill in tbe details.) We see that an SVD provides us with orthorrormal bases for
You can check that tbe kerne! and image of A .
Likewise, we bave
AT = v r; Tu T or AT U = vr; T.

EXAMPLE 5 .... Find an SVD for A = [ ~ ~ 6]. (Compare with Examp1e 3.)
Reading the last equation colunm by colunm, we find that
for i = 1, ... , r

Solution and
T- 0... for i = r + 1, . . . , m .
Using our work in Example 3, we find that AU;=
1/J6 l j../2 1;.J3] (Observe that the roles of the Ü; and the v; are reversed.)
V = 2/J6 0
[ Jj./6 - t;../2
- 1;.J3 ' As above we have
1/.J3

u = [ 1/.J~ - 1/../2 ] and

1/.J2 1;.J2 ' ker(AT) = span(Ür+ l, . .. , Üm).
and In Figure 5 we make an attempt to vi sualize these Observations. We represent
:E=[.J303 o1 o]0 . each of the kernels and images simply as a line.
Note that im(A) and ker(AT ) are orthogonal complements, as observed in
Checkthat A = U:EV r . Fact 4.4.1.
434 • Chap. 7 Coordinate Systems
Sec. 7.5 Singular Values • 435
6. Fin~ the singular values of A = [ ~ ~ J. Find a unit vector v
1 such th at

span (ii, +
AÜ; =a ; ll; if i s r II Au11l = a 1. Sketch the image of the unit circle.
1 •• • • • ii") spau (ii,...1• 1 • • •• • ii..,)
=ker(A) Aii; =Ö if i > r = ker(A7')
Find Sin gular value clecompositions for the rnatrice l[st.ed in Exercises 7 to 14.
Work with paper and pencil.

7.
[~ -~J 8.
[~ -q]p
span (ii 1 , •• • , ii,.)
= im(AT)
ATii ; = a;v; ifi :Sr
9.
[~ ~] 10.
[~ -7] 6
(see Ex.ample 4
Figure S ATii; =Öifi > r

We conclude th.is section with a brief discussion of one of the many applica-
11.
[~ ~] 12.
[! i] ( ee Example 5)

tions of the SVD-an application to data compression. We follow the exposition

of Gilbert Strang (Linear Algebra and Its Applications, 3d ed., Harcourt, 1986).
Suppose a satellite transmits a picture containing 1000 x I 000 pixels. If the
13.
[-~ ;J 14.
[~ ;J
color of each pixel is digitized, th.is information can be represeoted in a 1000 x 1000 15. Jf A is an invertible 2 x 2 matrix, what is the relationship between the singular
matrix A . How can we transmit the essential ioformation contained in thi.s picture values of A and A- 1? Justify your answer in terms of tbe image of the unit
without sending an 1,000,000 numbers? circle.
Suppose we know an SVD 16. If A is an invertible n x n matrix, what is the relationship between the singular
A=a 1u- 1v-T1 +--·+a,u,v,.
- -T values of A and A - 1?
17. Coosider an m x n matrix A with rank (A) = n and a singular value decom-
Even if the rank of the matrix A is !arge, most of the singular values will
position A = U:EVr. Show that the Ieast-squares solution of a linear system
typically be very small (relative to a 1). If we neglect those, we get. a good
approximation A ~ a,u au
1v[ + · · · + 5 5 v[, wbere s is much smaller than r. For
A-r
= b can be writteo as

example, if we cboose s = 10, we need to transmit only the 20 vectors a 1 ii 1, . . . ,

-
b·Ü 1-
-
b ·Ü 11 _
X" = - - U I +· · · + - -VII .
u
a10 10 and ti,, .. . , VJO in IR
1000 , which is 20,000 numbers.
0'1 0'11

18. Consider the 4 x 2 matrix

EXE R C I S ES
GOALS Find the singular values and a singular value decomposition of a matrix.
Interpret the singular values of a ' 2 x 2 matrix in terms of the imacre
circle.
0
of the unit A= w
l [I
1
-~
1
- 1
- 1
I
- 1
1
- 1
_:J[~ ~J [! -4]
- 1
1
0
0
3 .

1. Find the singular values of A = [ ~ _ ~ J. Use the result of Exercise 17 to find the lea t- ·quares olution of the linear
system
2. Let A be an orthogonal 2 x 2 matrix. Use the image of the unit circle to find
the singular values of A.
3. Let A be an orthogonal n x n matrix. Find the singular values of A
algebraically.

4. Find the singular values of A = [ ~ ~ J. Work with paper and pencil.

19. Consider an m x n matrix A of rank r and a singular value decompo ition
5. Find the singular values of A = [ : -; J. Explain your answer geo- A = U :E vr . Explain how you can expre ·s the least- quares Solution
sy. tem Ax = ;; a linear combination. of the columns v -~~ of V.
f a
1 , •• • •

metrically.
436 • hap. 7 oordinate Systems
Sec. 7.5 ingul ar Values • 43 7
20. a. Explain how any "quare matrix A can be written as 29. Show that an SVD
A = QS ,

can be written as
where Q i orthogo nal and S is symmetric posiöve emidefinite. Hint:
Write A = U'EV T = UV T V'EVr .
b . 1 it po ible to write = S 1 Q 1, w here Q1 is orthogonal and S 1 i sym-
30. Find a decomposition
metric positive semidefini re ?
21. Find a decompo iti on A = Q5 a discus ed in Exercise 20 for A = [ -~ ~ ] · - -T
A = a 1u1 v1 + a2u- 2v-T2
(Compare wi th Examples 2 and 4.)
22. Con ider an arb itrary ~ x 2 matrix A and an orthogonal 2 x 2 matrix 5. for A = [ _ ~ ~] (see Exerci e 29 and Example 2).
a. Explain in terms of the image of the unit circle w hy A and SA have the 31. Show that any matrix of rank r can be written as the su m of r matrice of
same ingul ar values. ran k I .
b. Explain aJgebraically why A and SA have tbe same singular values .
32. Consider an m x n matrix A an orth ogonal m x m matrix 5 and an orthogonal
1 1
23. Con ider a singular va lue decomposiöon n x n matrix R . Campare rhe Singular values of A and S A R.
33. True orfalse? If A is an n x n matrixl then th e singul ar values of A2 are the
squares of the ingul ar values of A. Explain.
of an m x 11 matrix A. Show that the colunms of U form an or1honorma l 34. For which sq uare matrices A is there a si ngular value decomposition A =
eigenbasi for A AT. Wbat are the a ociated eigenvalue ? What does your UL.VT with U =V ?
answer tell you about the relationship between the eigenva lues of A r A and 35. Consider a singular value decomposition A = ur.vr of an m x n matri x A
AAT? with rank(A) = n. Let ü1, ••• V11 be the col umns of V and ü 1• • •• ii 111 the
I I

24. If A is a symmetric 11 x n matrix. whal is the relation hip between the eigen-
vatue and tbe singul ar values of A?
25. Let A be a 2 x 2 matrix and ü a unit vecror in JR2 . Show that
columns of U. Without using the res ults in Chapter 4, compute (Ar A) - 1 ArÜ;.
Explain the re ult in terms of Ieast-square approximations.
36. Consider a singular value decomposition A = U'EVT of an m x 11 matrix A
with rank (A) = n. Let ~~ 1• . .. iim be the columns of U . Without u ing the
1

results in Chapter 4, compute A(ATA)- 1 ArÜ;. Explain your result in terms

of Fact 4.4.8.
where a 1 a 2 are the sing ular values of A. Illu trate thi inequality with a 37. If the ingular values of an n x n matrix A are all 1. is A nece arily orthog-
. ketch, and justi fy it algebraicall y. onal?
26. Let A be an m x n matrix and ü a vector in lR". Show that 38. Tru.e or false'? Sirnilar matrices have the same ingular value .

where a1 and a" are the largest and the sm allest si ngul ar va lues of A, respec-
tively. Campare with Exercise 25.
27. Let J.. be a real eigenvalue of an n x n matrix A. Show that

where a, and a" are the largest and the sm allest singul ar values of A, respec-
tively.
28. If A is an 11 x 11 matri x, what is the product of its singular values a 1 , • • • , an?
State the product in tenns of the determinant of A. For a 2 x 2 matrix Al
explain thi. resull in tenns of the image of the unit circle.
 Sec. 8.1 An lntroduction to Cont inuous Dynamical Systems • 439
At CS, by definition of continuous compounding, the balance x (t) grow at
an instan.taneous rate of 6% of the cunent balance:

dx
- = 6% of balance x( r)
dt

LINEAR SYSTEMS or

OF DIFFERENTIAL dx
-
dt
= 0.06x .

EQUATIONS Here, we use a differential equation to model a continuous linear dyoamical system
with one component. We will so lve the differential equation in two ways, by
separating variables and by making an educated guess.
Let us try to guess the solution. We think about an easier problern fir t. Do
we know a function x( t ) tbat is its own derivative: dxjdt = x? You may recall
from calculu that x(t ) = e1 i such a function (some people define x(r) = e1 by
thi s property). More generally, the function x(t) = Ce 1 is its own derivative, for
AN INTRODUCTION TO CONTINUOUS aoy constaot C. How can we modify x(r) = Ce 1 to get a function whose derivative
DYNAMICAL SYSTEMS is 0.06 times itself? By the chain rule, x(t) = Ce 0 ·061 will do:

There are two fundamentally different way to model the evolution of a dynamical
system over time: tbe discrete approach and the continu.ous approach. As a imple dx
dt
= !!.__
dt
(ce 0 061
· ) = 0.06Ce 0·061 = 0.06x(t )
example, consider a dynarnical ystem with onJy ooe component.

EXAMPLE 1 .... Suppo e you wi h to opeo a bank account in Switzerland and you shop araund Note that x(O) = Ce 0 = C; that is, C is the initial value, x 0 . We conclude that the
for tbe be t interest rate. You learn that Union Bank of Switzerland (UBS) pay balance after t year is
7 % interest, compounded annually. Its competitor, Credit Suis e (CS , offer 6%
interest compouoded continuously. Everything el e being equal, where hou ld x(l) = e0.06t xo.

you open the account?

Again, the baJance x(t) grows exponentially.
Solution Alternatively, we can solve the differential equation dx j dt = 0.06x by ep-
arating variables. Wri te
Let us think about the two banks: at UBS, the balance x(r) grows by 7% each
year if no deposits or withdrawals are made.
dx = 0.06dt
X

+ I inte rest I
and integrate both ide :
.j,
x( t + l) x(r) + 0.07x (t )
ln (x = 0 .06t + k,
x( r + l) l.07x(t)
This equation describes a discrete linear dynamical system with one component. for ome con tanr k. ow exponentiate:
The balance after r years i

x(r) = (J.07rxo
X = e ln (x) = e0.06t+k = e0.06t C,
The balance grows exponentially with time.
where C = ek.
438
440 • Chap. 8 Linear Systems of Diffe rential Equations Sec. 8.1 An lntroduct io n to Co nt inuo us Dynamical System s • 441
W lü ch bank offers the better dea l? We have to comparc the ex po n nti al or
functio n 1.OJI and e0·061 . Us ing a caJ ul ator, we o mpute

for some n x n. matri x A .

to see that UBS offe rs the better dea l. The extra interest from co ntinuo us com-
In tbe ~ontinuous approach, we mode l tbe gradu al change the sy tem un-
poundi ng doe" not mak up for th e o ne-poin t dj ffere nce in the nomin al intere. t
rate. .... dergoes as ttme goes by. Mathem ati ca ll y speaki ng, we model the (instanta-
ne~u s) .rates of change of the components of the state vector x(t ) , that is, the ir
denvattves
We can generali ze the work we have clone with the differenti al equation
dx jdt = 0 .06x.
dt

Fact 8.1.1 Consider the linear di fferential equaüon lf these rates depend linearly on x 1, x 2 , . .. , x," then we can write
dx clx 1
- = kx ~ = +
dt ' G JJX t +a 12 X 2

with given initial val ue x 0 (k is an arbitrary consrant). The so lution is dx2

1
~ = a 2 1X 1 + a 22 x 2 +
x t) = e k XQ .

Tbe quantity x will gro w or decay exponen ti all y (dependin g on the sign of
k). See Figure 1.

or, in matrix form,

Now consider a dynam.icaJ ystem with state vector x (t ) and component s
x 1(t ), ... . Xn (t ) . ln Chapter 6 we use the discrete approach to mode l thi s dynamical
system: we take a snap hot of the system at ümes t = 1, 2, 3, ... , and we describe
the Iransformation tbe system undergoes between these snap hots. If ,'t (t + I )
x
depends li nearly on (t ), we can write
where

I .x cr + 1) = Ax(r) 1
Gt:~

a," ]
Figure 1 (o) x{t) = ekl with positive k. Exponenliol growth. (b) x{ t) = ekr with negative k. Exponentiol decay.
A=
[""
021

a:,t
a 22

Gn 2
(1 2 11

Gn n
X

The derivative of the parameterized curve .x(l) is definecl component-wi se:

X clx 1
-
dt

:t 5
~
(a)
10 15
0.5

(b)
2
cLt
dt
dx 2
dt

dx"
dt
442 • Chap. 8 Linear Systems of Differential Equations
Sec. 8.1 An Introduction to Continuous Dynamical Systems • 443
We summarize these observ·nions:
In other words, to solve the system

d.r _
A li11ear dynamical system can be modeled by -=Ax
dt '
x (r + 1) = Bx(t) (di screte model)
x
for a given initial state 0 we have to find the trajectory in the x 1-x 2 plane that ta11
or at i o and whose velocity vector at each point i is the vector Ax. The existence
d.r A- (continuous mode l). and uniqueness of such a trajectory seems intuitively obvious. Our intu.ition can
- = X
dr be misleading in such matters, however, and it is comforting to know that we
A and B are 11 x 11 matrices, where 11 is the number of components of the can e tablish the existence and uniquene s of the trajectory later. See Fact 8.1 .2.
Fact 8.2.3, and Exercise 9.4.48.
systent.
We cao represent A-r graphically as a vector field in the x 1-x2 plane: at the
endpoint of each vector x we attach the vector Ax. To get a clear picture, we often
We will first think about the equation sketch merely a direction field for Ai, which means that we wi ll not necessarily
sketch the vectors Ax to scale (we care only about their direction).
d.r 4

- = Ax To find the trajectory ,r(t), you simply follow the vec tor field (or direction
dr field); that is you fo!Jow the anows of the field , starring at the point representing
from a graphical point of view when A is a 2 x 2 matrix . We are looking for the x
the initial state 0 . The trajectories are also called the fiow lines of the vector
parameterized curve field Ax.
To put it differently , imagine a traffic officer standing at each point of the
-=c) _
.~ 1 -
[x.tx2(t)
C r) J plane, showing us in which direction to go and how fast to move (in other
word , defining our veloci ty). As we follow his directions we trace out a
trajectory.
that represents the evolution of the system from a given initial value. Each xo.
point 00 the curve x(r) will represent the state of the system at a ce1tam moment
in time, a shown in Figure 2. .
It is naturaltothink of the trajectory ,~(t) in Figure 2 as the path of a movu:g
EXAMPLE 2 ..... Con ider the linear system d.r f dc = Ai where A = [ ~ ; JIn Figure 4, we
sketch a direcrion field for Ax. Draw rough trajectories for the three given initi al
particle in the x 1-x 2 plane. As you may have seen in a previ~us cour e, the vel~ct~ values.
vector dx f d t of this moving particle is tangent to the traJectory at each pmnt.
See Figure 3.

Figure 2 Figure 3
~
.T(O) = i o Trajectory
/

--1It is se nsible to attac h the veloci ty vector dx jdt at the en dpoint of Lhe staLe vector x(r). indicating
the path the panicle would take if it wcre to maintain the direclion it has at time t.
444 • Chap. 8 Linear Systems of Differential Equ ations Sec. 8.1 An Introd ucti n to Co ntinuous Dynami cal Systems • 445

Figure S Figure 7

Solution We have seen earlier that the eigenvalues of A = [! ; ] are 5 and -1.
Sketch the ftow lines for tbe three given points by following the arrows, as shown
in Figure 5. with corresponding eigenvectors [ ~] and [ _ ~ ] . These results agree with our
Thi picture does not teil the whole story about a trajectory x(t ). We don ' t graph.ical work in Figures 4 and 5. See Figu.re 7.
know the position x(t) of the moving particle at a specific time t. In other words, A in the case of a discrete dy namical system, we can sketch a phase portrait
we k.now roughly wbich path the particle takes, but we don ' t know how fast it for the system d,r jdt = Ai that shows some representative trajectories. See
moves along that path. ~ Figure 8.
In Su mmary: if the initia] tate ,rois an eigenvector, then the trajectory mo e
As we Look at Figure 5 our eye' s attention is attracted to two special line ,
aJong tl1e corresponding eigenspace, away from the origi n if tbe eigenvalue i
along which the vectors Ax point eilher radially away from t.he origin or directly
toward the origin. In either case, the vector Ax is parallel to .i :

Ai = J..x , Figure 8 x_

for some scalar ).. Tbis means that the nonzero vectors along these two special
lines are j ust the eigenvectors of A , and the special lines themselves are the
eigenspaces. See Figure 6.

Figure 6 (o) AX = ü,
for o positive J... (b) AX =
ü , for o negative A.

(a) (b)
446 • Chap. 8 Linear Systems of Differential Equations
Sec. 8.1 An Introduction to Continuous DynamicaJ Systems • 447
po 1u and toward the origin if the eig~nvalu e _is negativ .. lf the eigenvalue is
Now .let' s do a slightly barder example:
x
zero, then 0 is an equilibrium olution : x(t ) = xo. for all Limes ' ·
EXAMPLE 4 ..... Find all so lutions of the system
How can we solve the system dx jdr = A:i: ana lytically ? We starl with a
simp le case.. d,:r = [
dt
.1
- 1 4
2]-
X.

EXAMPLE 3 ..... Findall solutions of tbe y tem Solu tion

Above, we have seen that tbe eigenvalues and eigenvectors of A tell us a lot about
the behavior of tbe solutions of the system dx/dt = Ax. Tbe eigenvalues of A

Soluti on
are A. 1 = 2 and A.2 = 3 with corresponding eigenvectors ü1 =[ iJ
aod i:i2 =[ iJ.
The two differential equation This means that s- tAS = B , where S= [ i ~J aod B = [~ ~ J. the matrix
considered in Exarnple 3.
dxt = 2xt
dt We can write tbe system
dx2 dx _
dt -=Ax
dt
as
are unrelated or uncoupled· we can solve tbem separately u ing the formula tated
in Fact 8.1.1. cLt _
21
-=sss-
dt
1
x
X t (I) = e Xt (0),
31
X 2 (t) = e X2 (0), or

or s- 1 d,i: = Bs- 1.-r

dt
or (see Exercise 51)

Botb components of; t ) grow exponeotially, and the econd one wi ll grow ~aster .:!.._cs- t;) = Bcs- 1.r).
dt
than the first. In particular, if one of the components i initially 0, it remaw s 0
for aJl future times. In Figure 9, we s ketcb a rough pha e portrait for this Y tem. Let LI S i ntroduce the notation c(t) = s- t; (t); note that c(l) i the coordi nate vector
of; (t ) with respect to the eigenbasis Üt, ü2 • Theo the system takes the form
~
dc _
-dt = Be '
Figure 9 whi ch is just the equation we o lved in Example 3." We fo und that the olutioos
are of the form

c(t) = [ ee~;c~c2 J,
where c 1 and c2 are arbitrary con tants. Therefore, the olution of the original
sy tem
d,t -
- = A.r
dt
are
448 • Chap. 8 Linear Systems of Differential Equations
Sec. 8.1 An Introduction to Continuous D:rnamical Systems • 449
We can write tbis formula in more general term s as
This is a linear di stortion of the phase portrait we sketched in Figure 9.
~
x(r) = c 1e · v,
A lt -
+ c2e J.., r -
- v1 . ~ore precisely, the mat1ix S = [ 7 i] transform~ the phase portraits in Figure 9
Note that c 1 and c2 are the coordinates of .~ (0 with respect to tbe bas.is v,, v~ . mto the phase portrait sketched in Figure 10 (tran~fonning e into the eigenvector
ince [iJ and e into [ ~ ] ).
2
1

~
Our work in Examples 3 and 4 generalizes readily to any n x n matrix A
It is informative to consider a few special trajectories: if c , = l and c2 = 0, the that is diagonalizable over R i.e., for which there is an eigenbasis in ~Rn:
trajectory

x- (t) = e 2r [ 2
1
J
Fact 8.1.2 Consider the system dxjdt = Ax. Suppose there is a real eigenbasis ii " .. . ,
moves along the eigenspace E2 spanned by [ ~] , as expected. Likewise, if c , = 0
v" for A, with associated eigenvalues }. I , . .. ' A". Then the general Solution
of the system is
and c2 = 1, we have the trajectory

The c; are the Coordinates of x 0 with respect to the basis ü1, . _. , V

12
•

moving along the eigenspace E3.

lf c2 -=F 0, then the entries of c2 e 31 [ i] will become much !arger (in absolute We can think of the general solution as a linear combination of the solutions
e;..,, Ü; associated with tbe eigenvectors ii;. Note the similarity between this solution
value) than the entlies of c1 e21 [ ~] as t goes to infinity. The dominant tenn and the general solution of the discrete dynamica1 system ,i (t + 1) = Ax(t):

c2 e 3' [ ~], associated with the !arger eigenvalue, determines the behavior of the
system in the distant future. The state vector .t(t) is almost parallel to E3 for !arge
The terms )...~ are replaced by e;..,,. We have already observed this fact in a dynam-
r. For Iarge negative 1, on the otber band, tbe state vector is very small and almost
ical system with only one component (see Example 1).
parallel to E2.
. In Figure 10, we sketch a rough phase portrait for the system dxjdt =Ai.
EXAMPLE 5 ~ Consider a ystem d ,i 1dt = A.x , where A is diagonalizable over R When is
the zero state a stable equilibrium solution? Give your answer in terms of the
eigenvalues of A.
Figure 10 Xz

Solution
Note that lim eM = 0 if (and only if) ).. is negative. Therefore, we observe tability
,.....
if (and only if) all eigenvalues of A are negative. ~

Consider an invertible 2 x 2 matrix A with two distinct eige.nvalues AJ > )•2 ·

Then the phase portrait of d.xjdt =Ai Iooks qualitatively like one of the three
sketches in Figure 11. We observe stability only in Figure llc.
Note that the direction of the trajectories outside the eigenspaces always
approaches the direction of E;.. 1 as t goes to infinity. Compare with Figure 6.1.1 I.
450 • Chap. 8 Linear Systems of Differential Equations
Sec. 8.1 An Introduction to Contin uous Dynamical Systems • 451
dx
8. -,
Gt
= ../X, x(O) = 4.
dx
9. -
c.1t
= xk (with k =/:. 1), x(O) = I.
d.x 1
10. -d = - (- ), x(O) = 0.
{ COS X
dx . ?
11. -, = I +x-, x(O) = 0.
Gl

12. Find a differential eg uation of the form dx jdt = kx for which x(t) = 31 is a
solutioo.
13. In 1778, a wealtJ1y Pennsylvanian merchant named Jacob DeHaven lent
(c)
$450,000 to the Contineutal Congress to support the troops at Valley Forge.
Figure 11 The loan was never repaid. Mr. DeHaven 's descendants are taking the United
States Government to coutt to collect what they believe they are owed. Tbe
going interest rate at the time was 6%. How much were the DeHavens owed
in 1990
EXERCISES a. if interest is compounded yearly?
b. if interes t is compounded continuously?
GOALS Use the concept of a continuous dyna:mic.al system. Solve the differen-
(Adapted from The New York Tim es, May 27, 1990.)
Ax
tial equation dx jdt = kx. Solve the system d.t / dt = when Ais diagonalizable
14. The carbon in living matter contains a minute proportion of tlle radioactive
over R and sketch the phase portrait for 2 x 2 mat1ices A.
isotope C-14. This radiocarbon arises from cosrnic-ray bombardment in the
upper atmosphere and enters living systems by exchange processes. After
tlle death of an organism, exchange stops, and tlle carbon decays. Tberefore,
Solve the initial value problems posed in Exercises 1 to 5. Graph the solution. carbon dating enables us to calcuJate the time at which an organism died. Let
x(t) = proportion of the original C- 14 still present t years after death. By
dx
1. - = 5x with x(O) = 7. definition, x(O) = 1 = 100%. We are told tllat x(t) satisfies tlle differential
dt eguation
dx .
2. -
dt
= -0.7lx w1th x(O) = - e.
dx I
- = - - -x.
3. d p = 0.03P with P (O) = 7. dt 8270
dt
dy a. Find a formu la for x(t). Determine tlle half-l.ife of C- 14 (that is, the time
4. - = 0 .8t with y(O) = - 0.8.
dt it takes for half of tlle C- 14 to decay).
dy b. The Iee man. In 1991 , the body of a man was found in melting snow in
5. - = 0 .8y with y(O) = -0.8.
dt tlle Alps of Nortllern ItaJy. A well-known historian in Innsbruck, Austria,
deterrnined that the man bad lived in the Bronze Age , which starred about
2000 s.c. in this region . Examination of tissue samples performed inde-
Solve the nonlinear differential equations in Exercises 6 to ll using the metJ1od pendently at Zürich and Oxford reveaJed tllat 47% of the C- 14 present in
of separation of variables (p. 439): write the differential equation dxjdt = .f(x) the body at the time of his death had decayed. When did this man die ? ls
as dxj f(x) = dt and integrate both si des. the result of the carbon dating c.ompatible with the estimate of the Austrian
hi storian?
dx I 15. Justify the "rule of 69": if a quantity grows at an instantaneou rate of ko/o ,
6. - = - , x(O) = 1.
dt X then its doubling time is about 69 /k. Example: In 1995 the population of
dx ? . India grew at a rate of about 1.9%, witll a doubling time of about 69/1.9 ~
7. - = x-, x(O) = 1. Describe the behavior of your solution as t wcreases.
dt 36 years.
 Sec. 8. 1 An Introd uctio n to Continuous Dynami cal Sys tems • 453
452 • Ch ap. 8 Linear Syste ms of Differential Equations
24. Let A be an n x n matrix and k a sca lar. Consider the two y tems below:
Consider the system
dx
~ = [~ ~J.x.
(I) - = A-t
dl
dc
For the va1ues of ;, and A.2 given in Exercises 16 to 19, sketch the trajectories
(11) dl =CA+ k i")c
1
for all nine initial values hown below. Foreach of the points, trace out both the
Show that if .~(t) is a Solution of system (l), then c t ) = ek 1 x(t) is a solution
future and tbe pa t of the sy t m . of ystem (II).
25. Let A be an n x n matrix and k a scalar. Consider the two systems below:
cLx _
(I) - = Ax
dt

(II)
dc
-- = kAc
• • dt
Show that if x(t) is a Solution of y tem (I), then c(l ) = x (kt ) is a olution
----~--~---+-------- X I
of ystem (fl ). Campare the vector fields of tbe two sys tem .

• • In Exerci e 26 to 31, solve the ystem with the given initial value.

26. ~~ = [ ; ~ J.X with x (O) = [ ~ J.

cL.r = [ -4
27. dr _3 Jx with ,i(O) = [ O1 J.
2 3

16. A.t = 1, >-2 = - 1 28. ~~ = [: ~ J.r with x(O) = [: J.

17. AI = 1, A2 = 2
18. >. 1 = -1 , ).z = -2 29. :~ = U ~ J.r with .t"(O) = [ ~ J.
19. ),1 = 0, )..2 =1 d."V = [ I
30. dt 2 x J_ with x(O)
_ = [_ 21 ] .
20. Consider tbe system dx jdt = Ax with A = [ ~ - ~ J. Sketch a direction 2 4

fi eld for Ai. Based on your sketch, describe the trajectories geometrically. 31. dx = [7 ~ ~] ; with x(O) = [-~l ] ·
From your sketch, can you guess a fonn ula for the o lution wi.th xo = [ ~ ]? dt 3 2 2

Veri fy your guess by sub tituti ng in to tbe equation . Sketch ro ugh phase pmtraits fo r the dynarnicaJ systems gi ven in Exercises 32

21. Consider the system dx jdt = Ax with A = [ ~ ~ J. Sketch a direction field to 39.

of Ax. B ased o n your sketch, describe the trajectories geometricaUy . Can

x _ [ 3I
32. ddt - 2]
0
X-
.
you find the solutions analytically?
22. Consider a linear system dx /dt = Ax of arbitrar y size. Suppose it (t) and 33. ~;: = [ -~ -~ J.r.
as well? How do you know?
x
x 2 (t) are solutions of the system. Is the sum x(t) = x 1(t) + 2 (t) a solution

23. Consider a linear system dx jdt = Ax of arbi trary size. Suppose Xt (t ) is

34. ~;~ = [ : X. n
a solution of the system and k is an arbitrary constant. Is x(t) = kx t(r) a 35. :~ = [ ~ ; J.r.
solution as well? How do you know?
454 • Chap. 8 Linear Systems of Differential Equations
Sec. 8.1 An lntrodu ction to Continuous Dynamical Systems • 455
- + 1) = [00..92 0.22 ]-()
36. x(t 1. x t · c. What will happen in the long term ? Does tbe outcome depend on the
initial populations? If so, how?
_
37. x(t . =
+ 1) [ _ 1_
02
o.3
1. J-c)
7
x t · 43. Answer the questions posed in Exercise 42 for tbe system below:

-
38. x(t + 1) = [ -0.4
1.1
O.S x t ·
0.2] -() dx
-
dt
= Sx - y

39.
_
+ 1) =
[o.s - o.4 ] -c) dy
dt = -2x + 4y
X (t 0 _3 1.6 X I · - - .

40. Find a 2 x 2 matrix A such that the system dx jdt = Ax has

44. Answer tbe questions posed in Exercise 42 for tbe system below:
x(t -- [2e2' + 3e3']
3e2r + 4 e3r dx
- = x+ 4y
as one of its solutions. dt
Consider a noninvertible 2 x 2 matrix A with two distinct eigenvalues (note dy
41. -=2x- y
that one of the eioenvalues must be 0). Choose two eigenvectors ü, and dt
ÜJ with eioenvalue~ ), 1 = 0 and A.2 . Suppose A.2 is negative. Sketch a phase
p~rtrait fo; the system dx jdt = A.t , clearly indicating the shape and long-term 45. Two berds of vicious animals are figbting each other to the death. During the
behavior of the trajectories. fight, the populations x(t ) and y(t) of the two species can be modeled by the
system below: 1

dx
- 4y
dt
dy
- =- x
dt

a. Wbat is the significance of the constants -4 and -1 in these equations?

Which species has the more vicious (or more efficient) fighters?
b. Sketch a phase portrait for this system.
c. Who wins the fight (in the sense that some individuals ofthat species are
left while the other herd .is eradicated)? How does your answer depend on
tbe initial populations?
46. Repeat Exerc.ise 45 for the system
42. Consider the interaction of two species of animals in a habitat. We are tol d that
the change of the populations x( r ) and y(r) can be modeled by the equations dx
- py
dt
dx dy
- = 1.4x - 1.2y - =-qx
dt dt
d '
...!... = 0.8x - 1.4y where p and q are two positive constants. 2
dt

where time t is measured in years.

a. What kind of interaction do we observe (symbiosis, competition, predator-
prey)?
b. Sketch a phase porlnit for this system. From the nature of the problem,
we are interested only in the first qu adrant.
'This is the simples!. in a serie of c01nbm models developed by F. W. Lanchester duri ng World
War 1 (F. W. Lanchester. Aircraft in W'wfare, tile Daw11 of 1he Fourlir Arm. Tiptree, Constable and Co..
Ltd., 1916).
2The result is known as Lanchester's square luw.
456 • hap . 8 Linear tems of Differential Equations Sec. 8.1 An lntroduction to Continuous Dynamlcal Systems • 45 7
47. The intera tion of two popul ati on of anim als is modeled by the differential 52. Find all solutions of the sy. tem
equation

dx
dx
di =
[ ).,
0
1]-
)., X,
- =-x+k y
dr where ).. is an arbitrary constant. Hint: Exercises 21 and 24 are helpful.
dv Sketch a phase porh·ait. For which choices of 'A is the zero state a table
__:_ = kx - 4v eq uilibrium olution ?
dr ·
53. Solve the initial value problem
for ome po iti ve con tant k.
a. What kind of interaction do we ob. erve? What is the practica l ignifi cance
of the con tant k?
~;~ = [ ~ -! ]x with l
xo = [ b
Sk~tch the trajectory for the case when p is positive, negative or 0. In
b. Find the eigenvalue of the oeffic ient matrix of the sy tem . What can you
which cases does the trajectory approach the origin? Hint: Exercises 20, 24.
say about the sign of the eigenvalues? How doe your an wer depend
and 25 are helpful.
on the value of the constant k?
c. Foreach case you di cussed in part b, sketch a rough pbase portrait. What 54. Conside_r a cloor that open to on ly one side (as mo 't doors do). A spring
does each pha e portrait tell you about the fut ure of the two population ? mechant 111 closes the door automaticaUy. The tate of the door at a oiven
time r (measured in econds) is deterrnined by the angular displacemen7 a(t)
48. Repeat Exercise 47 for the sy tem (measured in radians) and the angular velocity r.v i) = da jdt . Note that a i
alway positive or zero (since the door opens to only one side • but r.v can be
dx positive or negative (depending on whether the door i opening or closing).
- = -x+ky
dr
dy
X -4 y
dt

where k i a po itive constant.

49. Hereis a continuou model of a per on ' s glucose regulatory syste m (compare When the door is moving freely (nobody i pushing or pulling), its movement.
with Exercise 6.1.36). Let g(r) and h (r ) be the excess glucose and in sulin is ubject to the foUowing differential equation :
concentrations in aper on's blood. We are told that
da
(l) (the definition of r.v)
dg dt
- g - 0.2h
dr dr.v ( -2a reflects the force of the spring,
- = - 2a - 3w
dh dr and -3r.v model friction)
- = 0.6g - 0.2h.
dt a. Sketch a phase pmtrait for this system.
b. Di cuss the movement of the door repre ented by the quali tativel y d ifferent
where timet is measured in hours. After a heavy holiday dinner, we measure trajectories. For which initial st.ates doe the door slam (i .e., reach a = 0
g(O) = 30 and h(O) = 0. Find closed formulas for g(l) and h (t ). Sketch the with velocity (tJ < 0)?
trajectory. 55. Answer the que tion posed in Exerci se 54 for the sy tem
50. Consider a linear system dx jdr = Ax, where A is a 2 x 2 matrix which is
da
diagonalizable over JR. Wben is the zero tate a stable eq uilibrium solution? (/)
dt -
Gi ve your answer in terrns of the determinant and the trace of A.
d (tJ
51. Let x(t ) be a differentiable cur-ve in JRn and S an n x n matrix. Show that = - pet - qr.v
dt
d _ dx
- (Sx) = S- . where p and q are po itive, and q 2 > 4p.
dt dt
458 • Chap. 8 Linear System of Differential Equations Sec. 8.2 The Co mplex Case: Euler's Formula • 459
We can write a complex-valued funct ion z(t ) in terms of it
2 THE COMPLEX CASE: EULER'S FORMULA nary parts: real and imagi -

Con ider a linear system z(t ) = x(t ) + i y (t )

dx _ iCon~ider the two ex~ple_s abo ve.) l f x(t) and y (t ) are differentiable real-valued
- = Ax
dt ' unctiOns, then the den vattve o f the comp lex-valued function z(r) is defined by
, bere the n x n matrix A i di agonalizable over C: ther i a complex eigenbasi dz dx dy
-= - + i-
jj " . . . • ü" fo r A. with a ociated complex eigenva lue A~o ... , A" . You may dt dt dt .
' onder ' hetber the formul a For examp le, if

then
with com plex c;) produce the general complex solution of the system, just as in
the real case (Fact 8.1.2). dz
Before we can make sen e out of the fommla above, we have to think about dt = I+ 2il .
the idea of a complex-valued fun ction and in partic ul ar about the exponential If
function e )J for complex A.
Z(l ) = COS(I ) + i sin (t),
then
+ Complex-Valued Functions
dz
A complex-valued ftmctio n z = f(t) is a function from IR to C (with domain lR dt = - sin (t) + i cos (t ) .
and codomain C ): the input t is real, and the output z i complex. Here are two
Pl e~se ve1ify that the basic rules of differential calculu (the sum, product,
examples:
and quot1 ent rules) apply to complex-valued functions. The chai n rule holds in
7 =L + i t 2 tbe fo llowi ng fom1: if z = f(l) i a differentiable complex-valued function and
Z = COS(f ) + i sin (t) t = g(s) i a differentiable function fro m lR to IR, tben
d- d z dt
For each r, the ou tpur ' can be represented a a point in the complex
plane. As we Iet t vary , we trace out a trajecto ry in the complex pla ne. ln ds dt ds ·
Figure l we sketch the trajectories of the two complex- valued fun ction defined T_he d_erivative d z / dt of a compl ex-valued function z (t) , for a given t, can be
above. vJsuahzed as a tangent vector to the trajectory at z(t), as shown in Fig ure 2.

Next let' s think about the complex-valued exponential function z = e).'

r.-e 1 (a) The lrajedory of where A is complex and t real. How should the function z = eJ..J be defin ed? We
Z=f + if 2.
1 = 1rll can get ome in piration from the real case: the exponential function x = ekr (fo r
(b) The trajectory of
~ =i real k) i the unique function uch that dxjdt = kx and x(O) = 1 (compare with
z = castn + isin(t). 1 = - I Fact 8. 1.1 ).

Figure 2
t =O I = 0
z=O z= l !!I:at l = .I
dt

I = 31Tf2
z= - i
(a) (b)
460 • hap. 8 Linear Sy tem s of Differential Equation Sec. 8.2 The Camp lex Case: Euler's Formula • 461
We an u e thi funda me nta l property of real xpone ntial fun ctio ns to defin e
z(l) = cos(t) + i sin(t)
the complex exponentiaJ function s:
I
Definition 8.2.1 Complex exponential functions
If ).. i a complex number, the n = e1'1 is the unique com ple -va lued fun ction
7

su h that
d~
- = )... - and - (0) = l.
dt

(The exi tence of uch a function, for any "J.. , will be established below; the Figure 4
proof of uniquene s is left as Exercise 34.)
It follow that the unique com plex-valued function z(r) wi.th
The unit circ/e, with parametri zation -:(1) = cos(t) + i sin (i), ati fi e these
and z(O) = o req u iremen ts:
dz .
dr = - Sll1 (t ) + i CO (1) = i z(t)
IS
and z(O) = 1. See Figure 4.
We have bown the following fundame ntal re ult:
fo r an arbitrary complex initial value zo.
Let us first con ider the impl e t ca e, - = e;', where )... = i . We are loolO ng
Fact 8.2.2 Euler's formula
for a complex-valued function z(t) ucb that d- jdt = i - and z(O) = l.
From a graphical point of view, we are looki ng for the trajectory z(L) in / r = COS(t ) + i in (t )
the complex plane tbat starts at z = 1 and whose tangent vector d z/ dt = i is
perpendicular to z at each poi.nt (see Example I of Seclion 6.4). In other ward ,
The case 1 = rc Ieads to the intriguing formula eirr = - 1; this ha been called
we are Iook.ing for the ftow line of the vector field in Figure 3 tm·ting at z = I. PRO JUV NTUTE 1957
the most beautiful formula in all of mathematics. 1
~ .
~
.;; z
Euler's formula can be u ed to write the polar form of a complex number
~ '~ ;

3 . ), ~ more succinclly:
. ~

Figure 3 • M

-. .< .,. z = r (cos<j> + i sin <j>) = re;q,

; . .·!, -~...
_
5 ~. . ' Now consider z = e)J , where ).. is an arbitrary compl ex number, )... = p + iq .
+5 EL\/ TlA By manipulating exponentials as if they were real , we find that

/ Figure S Euler's likeness end his

celebroted formulo ore shown on o
Swiss postage stomp.
e'J = e<p+iq)t = eP'eiqr = eP' ( co (qt)

We can va lidate lhi s result by checking th at the complex-valued function

z(t) = eP'( co (qt) +i
+i

in (qt))
in(qt ).

1 Benjami n Peirce ( 1809- 1880), n Hurvard mathcmatician. after observing tiM em = - I, u. cd to turn

to his studems and say, ·'Gentlemen, that is surcl y truc. it is absolu tcl y paradox i al, wc cn nnot under ta nd
it, and we don't know what it means, but we have pro ed it, und thercforc wc knO\ it must bc lhc tru th ."
Do you not now thin k that we understand not onl y that the formul a i' truc but al o what it m.:a ns?
462 • hap. 8 Linear Systems of Differential Equation s Sec. 8.2 The Co mplex Case: Euler's Fo rmula • 463
does indeed satisfy the definition of e).,1 , namel y, d-;, j d r = AZ and z(O) = 1: ~i genbas is. ii t, . . . , ii", with eigenvalues A1, ••. , An. Findall complex soluti ons
x(t ) of tht s system . By a complex solution we mean a functio n from IR to C"
d z =pe" 1 (co (qt )+ i in (qr))+ ePf ( - qsin (q t)+ iqcos(qt )) (that is, t is. 1: al and .r is in C"). In other words , the component funct.ions x 1 (t ),
dr ... , x"(t ) ol· x(t ) are cornplex-valued functions.
= (p + iq )e" 1
( co (qt ) + i sin (ql) ) = AZ .J A · you review our work in the last section , you will find tbat the approach
we took to the real case applies to the complex case as well, without rnod.ifi-
EXAMPLE 2 ~ Sketch the trajectory of the comple -valued fun ctio n z (t = e<O. t+ilt in the compl ex cations:
plane.

Solution Fact 8.2.3 Con ider a linear system

-;,(1) = eO.I I eil= eO I r ( COS (I ) +i in (r) ) dx _
- =Ax .
dt
The trajectory spiral outward a shown in Figure 6 since e 0· 1' grows expo-
nent.ially. <11111
Suppose there is a complex eigenba .is ii 1, •• • , v" for A with associated
complex eigenvalues A1, . .. , An. Then the general complex solutioo of the
EXAMPLE 3 ~ For which complex number ), is Lim eA1 = 0? sy tem is
1->

Solution where the c; are arbitrary cornp lex numbers.

Recall that
e~·' = e<P+iq)t = e"' ( cos(qt) + i sin (qt)) , We can checkthat the given curve .'i(t) sarisfies the equation dxjdt = A.i:
so that leAI 1 = e"1 . This quantiry approaches zero if (and only if) p is negative, we have
that is, if e"' decays exponentiall y.
We summarize: Lim e).,1 = 0 if (and only if) the real part of A is negative. <11111
1->

We are now ready to tackle the problern po ed at the beginning of th i (by Definition 8.2 .1 ), and
ection: consider a . y tem dx /dt = Ax , where the n x n matrix A ha a comple

Figure 6 because the Ü; are eigenvectors. The two answers match.

When i the zero state a stable equiiibrium solution for the system d.i jdt =
A.r? Con ideri ng Example 3 and the form of the solution given in Fact 8.2.3, we
can conc lude that this is the case if (and only if) the real part of all eigenvalue
are negative at least when A i diagonaiizable over C). The nondiagonalizable
case is left a Exercise 9.4.48.

Fact 8.2.4 For a system

c1.-r _
-=Ax
dr ·
the zero tate is an asymptotically stable eq uilibrium solut.ion if (and only if)
the real part of all eigenvalues of A are negative.
 Sec. 8.2 The Camplex Case: Euler's Fo rmu la • 465
464 • Chap. 8 Linear Systems of Differenti al Eq uation
~ he~~ A i_s a r~al 2x2 matrix with eigenvalues p±iq. Consider an eigenvector
EXAMPLE 4 .... Consider the ystem d-~ jd r = A.i=. where ~ is ~ (r al) 2 x 2 matrix. When i.s
v + 1w w1th e1genvalue p + iq. Tben
the zero tate a table equilibrium ·so lutJOn fo r th t system? Give your an wer in
terms of tr(A) and det(A ) . x(t) = e"l [w u ][ c?s(qt) - sin(q t) ] [ a ]
sm(qt) cos(qt) b

Solution where a and b are the coord.inates of .X0 with respect to the bas is w, v.
ility either if A has two negative eigenvalue or if A . has two
We o b erve Stab .
conj ugate eigenvalue p±iq, wber~ p is negative. In bot:h case , tr(A) l ' negatlVe
The trajectories are either ellipses (l.inearly d.istorted circles), .if p = 0, or
and det (A) is positive. Check that 111 all other cases tr(A) 2: 0 o r det(A) S 0. <1111 spirals, spiraling outward if p is positive andinward if p is negative.
Note the similarity of the fonnula in Fact 8.2.6 to the formu la

Fact 8.2.5 Consider the system

x(t) = r 1[ Ü; - si n(c'j>t) ] [
cos(c'j>t ) b
a]
dx _ in the case of the discrete system x(t + 1) = Ax(t) (Fact 6.5.3). The factor r'
- =Ax ,
dr is replaced by e" 1 , and 4> is replaced by q. This make good sense, because
where A is a real 2 x 2 matrix. Then the zero tate is an asymptotical ly stab le A.1 = r 1 ( cos(c'j>t) + i sin (c'j>t)) in the formula for the discrete system is replaced by
equi librium solut.ion if (and only if) tr(A) < 0 and det(A) > 0. e'·1 = e"' ( cos(q r) + i sin(qt)) in the continuous case (Fact 8.2.3).
EXAMPLE 5 ~ Salve the system

As a specia1 case of Fact 8.2.3, let's consider the system ddxt = [35 -2]_
_
3
x w1.th x_0 = [o]
1
.
d,r _
- = Ax.
dt
Solution
w here A j a real 2 x 2 matrix with eigenvalues A. 1,2 = p ± iq and corresponding
v
eigenvettors ii1.2 = ± iw. . The e igenvalues are A.1.2 = ±i, so that p = 0 and q = 1. This teiLs us that the
Tak:ing the same approach as in the di crete case (Fact 6 .5.3), we ~an wnte trajectory is an ellipse. To detennine the d.irection of the trajec tory (clockwise
tbe real SOlUtion x(t ) = C le ). 11 V1 + c2e).1 1 V2 (wbere C ! = h + ik and C2 = C!) as or counterclockwise) and its rough shape, we can draw the vector field Ai for a
1
x(t) = 2Re (c !e"' ÜI) few simple points .X, say x = ±e~. and sketch the flow line starring at [ ~ J. See

= 2Re(Ch +ik)e"1 (cos(qr) +i sin (qt))(ti + iw)) Figure 7.

= 2e"' (h cos(qr)v- h sin (q r)ÜJ - k cos(qt )w - k sin (qt)ti)

_ J [ - k cos(qt)- h sin (q t) J Figure 7
= 2 e"' [ w v - k sin (q t) + h cos(qr)

= 2e"'[w v][cos(qt) - sin (qt) J[ - k]·

sin (q r) cos(qt) h
Note that x(O) = - 2kw + 2hv. Let a = - 2k and b = 2h for si mpli c ity.

Fact 8.2.6 Cons ider a linear system

dx _
- = Ax ,
dr
466 • Chap. Linear tem of Di ffe rential Equation
Sec. 8.2 The Complex Case: Euler's Formula • 46 7
o w Iet u fin d a fom1Uia fo r the trajectory.
is represented by the point ( tr(A) , det(A)). Recall that the characteristic equation
i- 3 is
Ei = ker [ _ . 2 .., ] = span [ .-23 ]
5 I + J I -
2
>.. - tr(A)A. + det (A) = 0

[i ~ 3 J = ..._...,_.,
2
[ =~ J+ i [ n
'-,..-'
and the eigenvalues are

ii Ü;
AJ.2 = ~( tr(A) ± J (tr(A))2 - 4 det(A)).
T he linear tem ,r0 = aw + b- ha the oluti on a = 1 b = 0. T herefore,
Therefore, the eigenvalues of A are real if (and only if) the point (tr(A) , det(A))
.t(l = e P' [ ÜJ -] [ CO (q t )
in (qt )
- in q1 ) ] [
CO (qt ) b
a] is located below or on the parabola

-2] det (A) = ( tr~A) ) 2

= [~ -3
[ CO (t )
in(t)
- . in (t ) ] [ I ]
cos(t ) 0
in the tr(A) - det (A) plane. See Figure 9.
= [ ~ =~ J [ c~~i;~ ] = Co (t ) [ ~] +sin (t) [ =~ J Note that there are fi ve major cases, cotTe ponding to the regions in Figure 9,
and ome exceptional cases, coiTesponding to the dividing line .
- 2 in (t ) ] The ca e when
- [ CO (t) - 3 in (t) .
2

You can check that det(A) = ( - tr (A)

- )
2
c1.-r _
-
dt
= Ax and i (O) = [ ~]. is di cus ed in Exercises 35 and 37 .
What does the phase portrait Iook like when det(A) = 0 and rr(A) =!= 0?
In Figure 10 we take another Iook at the five major types of phase porrrai ts.
The trajectory is the ellip e shown in Figure 8. Both in the d.iscrete and in tbe continuous case, we sketcb the phase portraits
produced by variou eigenvalue . We include the ca e of an ellipse, since it is
Consider a 2 x 2 matrix A. The variou seenarios for the system dx / dt = A.r important in application .
can be conveniently represented in the tr(A) - det(A) p lane, where a 2 x 2 matrix A

Figure 8 31T
Figure 9
- det(A) = (
tr(A)
2 )2
r =-
2
468 • Ch ap. 8 Linea r Systems of Diffe rent ial Equati o ns
Sec. 8.2 Th e Camplex Case: Eul er's Fo rmula • 469
Figure 10 The major types of
Discrete Continuous E X. E R C I S E S
phase partraits.
GOALS Use tbe definition of the complex-valu ed expo nential function z = e'· 1•
Solve the system
cLx _
- = Ax
dt
for a 2 x 2 mattix A with complex eigenvalues p ± iq.
1. Find e2n i .
2. Find eC 112lrr i .
3. Write z = - I+ i in polar form as z =reit/> .
4. Sketch the trajectory of the complex-valued function
Z = e 3ir .

What .is the petiod?

5. Sketch the trajectory of the complex- valued function
z= e (-O. l -2i )r .

6. Find all complex solutions of the system

~: =[; -2]-
-3 X

in the form given in Fact 8.2.3. What solution do you get if you Iet c 1
C2 = I?
7. Derermine the stability of the system

"-1.2 = p ± iq AJ.2 = p ± iq
~~ = [ -~ -4 2]- X.

p2 + q 2 > 1 p>O 8. Consider a system

dx _
-=Ax
dt ,
where A is a symmettic matrix. When is the zero state a stable equilibrium
solution? Give your answer in terms of the definiteness of the matrix A .
"-1.2 = p ± iq "-1.2 = p ± iq 9. Consider a system
p2 + q2 < 1 p <O
d.r _
dt = Ax ,
where A is a 2 x 2 matrix with tr(A) < 0. We are told that A has no real
eigenvalues. What can you say about the stability of the system ?
10. Consider a quadratic form q (x ) = x · A.t of two variables, x 1 and x 2 • Consider
"-1.2 = p ± iq A. t ,2 = ± iq the following system of differential equations:
p2 + q2 = 1 dx, aq
=
dt ax ,
dx2 aq
dt ax2
470 • Chap. 8 Linear Systems of Differential Eq uations
Sec. 8.2 Th e Complex Case: Euler's Fo rmula • 471
or. more succinctly 16. Consider the system
dx
-
dt
= grad(q). d,t
dt =
[ 0
Cl.
1J-
b x,

a. Show that the ystem dx j d t = grad (q ) i linear by fi nding a matrix B (in where a and b are arb itrary constants. For which choices of a and b i the
terrn of the sym m tric matrix A) such tl1at grad(q) = Bx. zero state a stable equilibrium solution ?
b. When q is negative defi nite, draw a sketch howing po ible Ievel curves 17. Consider tl1e system
of q. On tl1e same sketch, draw a few trajectories of the system dxjdt =
grad(q). Wbat does your ketch suggest about tl1e stability of tl1e system
dx jd r = grad q)?
dx
dt =
[ - 1
k -k} J-X ,

c. Do the same a in part b for an indefinite quadratic form.
d. Explain tl1e re lationship between tbe definiteness of tbe fonn q and the
tability of tl1e y tem dx /dt = grad (q) .
11. Do parts a and d of Exercise 10 for a quadratic form of n variables.
12. Derermine tl1e stability of the sy tem
where k i an arbitrary constant. For which choices of k is the zero state a
stab le equilibrium soluti on?
18. Consider a diagonalizable 3 x 3 matrix A such that the zero state is a stable
equilibrium olution of the system dx j dt =Ai. What can you ay about the
determinant and the trace of A?
19. True or false? If the trace and the detenninant of a 3 x 3 matrix A are

~ ~] x.
1 both negative, then the origin is a table equ.ilibrium solution of the system
dx = [ 0 cLt j dt = Ax. Justify your answer.
dt - 1 - 1 -2 20. Consider a 2 x 2 matrix A with eigenvalues ±rr i. Let ü+ i iü be an eigenvector
of A witl1 eigenvalue rr i. Solve the initial value problern
13. l f tl1e ystem dx j dr = A.t is stable is dxjd t = A - 1; stable as well? How
can you tel!?
d.r _ _
- .- = Ax with xo = iü .
dt
14. Negativefeedback /oops. Suppose some quantities x 1(t), x2(t), ... , X11 (t) can
be modeled by differential equations of the fonn Draw tl1e olution in the figure below. Mark the vectors .t(O), ,r(4). x(l) , and
.r(2).
- bx11
üi

dx 11 21. Ngozi opens a bank account witl1 an initial balance of 1000 Nigerian naira.
X11-1 -k~~x" Let b(t) be the balance in the accmnH at timet ; we are told that b(O) = 1000.
dt
The bank is paying interest at a continuou rate of 5% per year. gozi make
where b is positive and the k; are posillve (the matrix of this system has deposits into the account at a continuous rate of s(l) (measured in naira per
negative numbers on the diago nal, 1's directly below the diagonal, and a year). We are told that s(O) = 1000, and s(t) is increasing at a continuou
negative nurnber in the top right corner). We say that tl1e quantities x 1, ... , xll rate of 7% per year. (Ngozi can save more as her income goes up over time.)
describe a (linear) negative feedback loop. a. Set up a linear sy tem of the form
a. Oe cribe the significance of tl1e entries in the system above, in practical db
tenns. - = ?b + ?s
dt
b. Is a negative feedback loop with two components (n = 2) necessarily
ds ?
stable? - = .b
dt
+ ?.s
c. I a negative feedback loop with three components necessarily stable?
15. Con ider a noninvertible 2 x 2 matrix A witl1 a positive trace. What does the (time i measured in years).
pha e portrait of the ystem d xjdt = Ax Iook Like? b. Find b(t ) and s(t) .
472. hap. 8 U near tems of Differen tial Equ at io ns Sec. 8.2 The Complex ase: Euler' FormuJa • 473
3
22. For each of the linear systems below, fi nd d1 matehing pha e portrait on the
3

following page. 2 2
a. .t(l + 1) = [ 22. 5 o~5J.~cr )
b. x- (l + .)
I = [ - -1.5 J-
- 1 x(r)
0.5
I 0 n 0
- I - I
c. d.t
dr =
[ 3J- 0 .\'
- 2.5 0.5 -2 -2

- 1 J-
- 3
~~ = [ -;·5 0.5
-3
d. X
-3 -2 - I 0 2 3 -3 -2 - I 0 2 3
3

e. ~;~ = [ -~ 0I J- X
2

Find all real olution of the y tems in Exerci e 23 to 26.

23. :~ = [~ -~ ] ; In 0 IV

- I - I
24. dx _ [
dt -
J- o
- 9 40
X
-2

25. d.r = [ 2 4 J-
dt -4 2 X -3
-3 -2 -I 0 2 3
- 3
-3 -2 - I 0 2 3

26. di=[ - 11 15 J- 3 3
dr -6 7 X

2 2
Solve the sy tems in Exercise 27 to 30. Give the solution in real form . Sketch
the solution.

27. :~ = [ -~ -2]
-1 X
with x(O) = [ -~ J. V 0 VI 0

28. dx = [
dl
o
-4 0 J-
I X with x (O) = [ 6]. - I - I

- 2
[-t 1] - l
-2
29. dx =
dr -2 l
X with i(O) = [~ - 3 - 3
-3 -2 - l 0 2 3 -3 - 2 - I 0 2 3
30. d.'i = [ 7
dr -4
10]-
-5 X with i(O) = [ 6]. 3

31. Coosider the mass-spring system sketched below.

VII vm 0

- I - I

- 2 -2

-3 -3
(continued on page 474) 2 3 -3 3
-3 -2 - I. 0
474 • Chap. 8 Linear Systems of Diffe rential Equat ions Sec. 8.2 The Camplex Case: Euler's Formula • 475
Let x(t ) be the deviation oftheblock fro m the equilibrium po ition at time 34. Let Z1(t ) and z2 (r ) be two complex-valued solutions of the ini tial value
1. Consider the veloci ty v(t ) = dx j dt of the block. There are two fo rces problem
acti no on the ma s: the pring force F5 , wlu h i - a umed to be proporti onal
dz
ro th~ di splacement x and the force F1 of friction, wh ich is assumed to be - = >..z with z(O) = 1
dt
proportional to the velocity:
(where >.. is a complex number). Suppose that z2 (r) =f. 0 for all t .
F = - px. F1 = - qv ,
a. Using the quotient rule (Exercise 33), show that the deri vative of
where p > 0 and q :::: 0 (q i 0 if the o cillation i fri ct ionl e ). There fore, Z l (t )
the total force acting on the mass is Z2(1)
F = F5 + FJ = -px- qv . is zero. Conclude that z 1(I ) = z2 (t ) .
b. Show that the initi al value problem
By Newton 's second law of motion we ha e
dv dz
F = ma =m - ,
-dt = >..z with z(O) =1
dt
where a repre ents acceleration and m the mass of the block. Combining the ha a unique complex-val ued olution z(t) . Hint: One solution is given in
the text.
last two equations we find that
35. Consider a real 2 x 2 matrix A with JA(>..) = >..2 • Recall that A 2 = 0 (see
dv Exercise 7.2.56). Show that the initial vaJ ue problern
m- = -px- qv
dt
dx _ _
or - = Ax with x (O) = .'to
dt
dv p q
-= - - X - - V. has the solution
dt m m
Let b = p j m. and c = q jm for simplicity. Then the dy nanlies of this mass- x(t) = ,ro + t Axo.
spri ng system are described by the system
Sketch the vector fi eld A-r and the trajectory i(t ) .
dx 36. U e Exerc ise 35 to so lve the initia l value problem
V
dr
(b > 0, c:::: 0).
dv
- =-bx - cv ~: = [ _! _!J-r with i(O) = [ ~] .
dt
Sketch a phase portrait for this system in each of the following case , and Sketch the vector fi eld and the trajectory .r(t ).
describe brieft y the significance of your traj ec tories in terms of the movement 37. Consider a real 2 x 2 matri x A with JA (A.) = (>.. - A.o) 2 . Solve the ini tial value
of the block. Comment on the stability in each case. problem
a. c = 0 (frictionless). Fi nd the period.
.h - 0
b. c? < 4b (underdamped). -d,'Y = Ax- W lt x( ) = x- o.
c. c2 > 4b (overdamped). dt

32. Prove the product rule for deri vati ve of complex-valued fu nctions. For which values of >..o is the zero state a stable equil ibri um soluüon of the
system? Hint: Use Exercise 8.1 .24 and Exercise 35 above.
33. a. Fora differenti able complex -valued function z (t ), fi nd the derivati ve of
38. U e Exercise 37 to solve the iniüal value problem

z(t ) ~;~= [ - 1 ~ J-r with .r (O) =[~J.

b. Prove the quotient rule for derivatives of complex-va lued functions.
In both parts of tlus exercise you may use the product rule (Exercise 32). x
Sketch the trajectory (t) .
476 • Chap. 8 Lin ea r System s of Differential Equ ations

39. So h re the system

d .r
dt u~ n,
Compare with Exerci e 8. 1.24. When is the zero s tate a stab le equili brium
so lution?
40. An eccentric mathemati ian i able to gai n autocratic power in a small A lpine
country . In her first decree, he anno unces the introduction of a new currency,
tbe Eul er wbi ch is meas ured in complex units . Banks are ordered to pay o nl y
im aginary interest on deposits .
a. lf you invest 1000 Euler at Si% in terest, compo unded annuall y, how much
money do you have after 1 year, after 2 year , after 1 years? Descri be
the effect of com pounding in thi case. Sketch a t:rajectory showing the
LINEAR SPACES
evoluti on of the bal ance in the complex plane .
b. Do part a in the case when the Si % interest i cpmpounded contin uously.
c. Suppose people ' s ocial standing i determined by the mod ulus of the bal-
an ce of their bank account. Under these circumsta nces, wou ld you choose
an account with annu al compound ing or with conti nu o us compoundi ng of AN INTRODUCTION TO LINEAR SPACES
interest?
(This p roblern is based o n an idea of Prof. D. Mu rnford , H arvard University .) In this chapter, we present the basic concepts of linear a lgebra in a m ore general
contex t.
Here is an introductory example, where we use many concepts of linear a loe·-
bra in the context of fun ction , rather than vectors in ~" . C onsi der the di ffe renG al
equation (OE)

cPx
- = -X or
dt 2
Going throu gh a Iist of " .imple" functi ons, we can gues rhe so lution

X 1(t) = COS(! ) and x 2 t ) = in (!).

We observe th at all " linear combinati on "

are olutions as weil (verify thi s). U ing techniques from C hapter 8, we an how
that these are in fact all the solutions of the OE (see Exercise 50). Thi mea n
th at the hmction s x 1 (t) = cos(t) and x2(1) = in (!) form a " ba i · of the " olution
space" V of the OE. The "dim nsion ' of V is 2 .
Now Iet C be the set o f aU smooth functi ons from ~ to (these are
the functions that we an differenti ale as many time, a we wa nt). Beca u e
x 1(t ) = cos(t ) and x 2 (t ) = sin (t) aTe smooth fun cti o ns and V i c lo ed under
linea r comb inati o n , V i a " ubspace" of C . Let u defi ne the Iran sform ati on

d 2x
T: C -+ C given by T (x ) = - J + x.
dr -
477
478 • Chap. 9 Linear Spaces Sec. 9. 1 An lntrod uction to Li near Spaces • 479
It follows from tbe basic rules fo r differentiatio n that We will now make theseinformal ideas more precise.

T (x + y) = T (x ) + T (y) Definition 9.1.1 Linear spaces

A (real ) linear space 1· 2 \1 is a set endowed with a rule for addition (if f and
and g are in \1 , then so is f + g) and a rule fo r scalar multiplication (if f is in
\1 and k in R then kf is in \/ ) such that these operations satisfy the eight
T (kx) = k T (x). ax ioms below3 (for all f, g, h in \1 and all c, k in IR):
1. (f + g) + h= f + (g +h ).
for all smooth functi o ns x and y, and all real scalars k .
We can summari ze the e pro pertie by saying that T is a '"linear tran Forma-
2. f +g = g + f.
tion·· from C to C . The " kerne!" of T is 3. There is a neutral element n in V such that f +n = f , for a!J f in \1.
This n is unique and denoted by 0.
ker T) = (x in C T (x ) = 0} 4. For each f in \1 there is a g in \1 such that f +g = 0. This g is unique
and denoted by (- f).
d2 r
= {x in C - ·2 +x = 0) 5. k(f + g) =kf+kg.
dt
= \1 . 6. (c +k )f = cf + kf.
7. c(kf) = (ck )f.
In Section 9.4 \.Ve will see that thinking of the soluti on space \1 as the kerne t of the 8. 1! = f.
linear tra.nsformation T helps u ana.lyze \1 (j ust as it was useful to interpret the
solution space of a linear system A.~ = Ö as the kerne! of the linear transformation Tlüs definition contains a 1ot of "fine print." In brief, a linear space is a et
T (.t ) = Ax). with two reasonably defined. operations, addition and scalar multiplication , that
Wbat are the '"eigenfuncti ons·· and ·'eigen values" of the linear transformation allow us to form Linear combinations of the element of this et.
T ? We are looking for nonzero mooth function x and sca lars A. such that Probably tlle most important exa.mples of linear spaces. besides !Rn , are
spaces offunctions, as in the introductory example.

T (x) = A.x or EXAMPLE 1 .... In IR", the prolotype linear space, the neutral element is the zero vector, 0. <11111

EXAMPLE 2 .... On page 477 we introduced the linear space C00 the set of all smooth function
In Section 9.4 we will sol ve this proble rn systematicall y. Here Iet us ju t give from IR to IR (functions we can differentiate a.s many time a we want) with the
some exa.mples: operations
(f + g)(t ) = f( t ) + g(t )
(kf)( t ) = kf (t )
T (e' ) = 2e' ,
for all t in IR.
Figure 1 illustrate the ruJe for scalar mu1tiplication of function from lR to
so that e' is an eigenfunction witl1 e igenvalue 2.
IR. Draw a sinülar sketch illustrating the rule for addition. <11111

T (t) = t , EXAMPLE 3 .... Another linear space is F (lR.lR"), the set of a.ll function s from IR to lR", that i ,
all parameterized curves in lR". The operations are defined as in Exa.mp le 2 . <11111

so that t is an eigenfunction with ei genvalue 1.

' The teml ve ror space is more collJmonly u·ed.
T( cos(l)) = 0, 2lf we work with complex scalars. thcn V is a co mplex linear pace. Our space · are real un le
srated otherwise.
JThese axioms were established by the halian muthe matician Giuseppe Peano ( 18 - 1932) in hi
so that cos(t ) is an ei genfunction with eigenvalue 0 . Calcolo Geomerrico of 1888. Pean called V a .. linear syste m:·
480 • Chap. 9 Linear Spaces
Sec. 9.1 An Introducti on to Linear Spaces • 481
for some Sealars c1, . . . , c11 • Since the basic notions of linear algebra (ini tiall y
introduced for lR") are defined in terms of linear combinations, we can now gen-
erali ze these notion to linear spaces without modification s. A hort version of the
rest of this section would say that the concept of subspace, linear independence,
basis, dimension, linear Iransformation kernel, image and eigenva lue · can be
defined for linear spaces in just tbe same way as for lR".
What fo ll ows is the lang version, with many exampl es.

Figure 1 Definition 9.1.2 Subspaces

A subset W of a linear space V is called a subspace of V if
EXAMPLE 4 ... Let x be an arbitrary et. Then V= F ( X , ~~~ , the et of all function from X to a. W contains the neutral element 0 of V .
lR" with the operations defined in Exampie 2, is a linear space. ~ b. W is closed under addition (if f and g are in W, then so is f + g ).
EXAMPLE 5 ... If addition and calar multipli cation are given as in Definition 1.3 .9 then ~"' x ", c. W is closed under scalar multiplication (that is, if f is in W and k is a
the set of all m x n matrices. i a linear pace. The neu tral element is the ze~ scalar. then kf is in W).
matrix. We can su mmarize parts b and c by saying that W is closed under linear
EXAMPLE 6 ... The complex numbers C formareallinear pace (and also a complex linear pace). combinations.
~

EXAMPLE 7 ... The et of all infinite . equences of real numbers is a linear space, where addition Note that a subspace W of a linear space V i a linear space in it own righ t.
and scalar multiplication are defined componentwise: (Why do the axioms hold for W?)

(x 0, X J . x 2... .) + (yo. ) 1, }'2 , ... ) = (xo + Yo, x1 + Yl · x2 + )12 , · · .) EXAMPLE 9 ... Here are two subspaces of C :
k (xo. x 1, x2, .. .) = (kxo, kx1, kx2, .. .) .
a. P, the set of all polynomial functions f(t) = ao + a1t + · · · + a"rn.
The neutral element i tbe sequence b. ? 11 , the et of all polynomial ftmctions of degree :::; n.

(0, 0, 0 .. . ·). EXAMPLE 10 ... The upper triangular matrice are a subspace of ~ 11 11
, the space of all n x n
EXAMPLE 8 ... The set of all geometric vectors in a (coordinate-free) plane is a linear space. The matrices. (Verify thi .) ~
rules for addition and scalar multiplication are illustrated by Figure 2. ~ I
EXAMPLE 11 ... The quadratic forms of n variables form a subspace of F(lR" , lR). lf q(.i) = .\;TA .r
Consider the elements .f1, h . . .. , f," and f of a lin ear space. We say that and p(x) = xTB,T, are two quadratic forms the n so is (q + p)(.i) = q(.i) + p(.r) =
f is a linear combination of the f; if ;T (A+ B)x . Think about scalar multiples. ~

f = cdt + c2h + · · · + c,.,J,, EXAMPLE 12 ... Consider an n x n matrix A. We claim that the set W of all so lution of the ystem
d.i/dt = A-t i a subspace of F(R ~"). We checkthat W i closed under calar
multiplication. lf x(1) is a solution and k is a calar, then
Figure 2 ü+w
-··
-.. : d d.-r _ < _)
- (kx) = k - = kAx = A kx ,
-·· -·· dl dt
so that k.'i(t) is a solution as ~eil. Checkthat W i closed under additi on and that
W contains the curve .r(t) = 0. ~
2
EXAMPLE 13 ... Consider tbe set W of all noninvertible 2 x 2 matrices in R -x . I W a ub pace
of ~2 x2?
482 • Chap. 9 Linear Spaces
Sec. 9.1 An Introduction to Li near Spaces • 483
Solution Solution
w contain the neutral element [ ~ ~] of JR 2 2
and is closed under scalar multi-
We can write any 2 x 2 matrix A = [ ~ ~] as
plication, but W is not closed under addition. As a counterexarnple, consider the
sum
[~ ~ ] =a[~ ~]+b[~ ~] + c [~ ~]+d[~ ~l
'-v--' '-v--' '--...--' '---v--'
E11 E1 2 E2 1 E22
in W in W not in W This shows that the matrices E 11, E 12, E 21, &z 2 span JR 2x 2. The four rnatrices
Therefore, W is not a subspace of JR x 2 2
. are also linearly independent: none of them is a linear combination of the otbers
since each has a 1 in a position where the three others have 0. Thi s shows tha;
Next we generauze the notions of linear independence and basi E11, E1 2, E21, E22 is a basis of JR 2 Y 2 ; tb at is, dirn (JR2x2) = 4. ~
EXAMPLE 15 ..... Find a basis of ? 3, the space of all polynomials of degree ::; 3.
Definition 9.1.3 Linear independence, span, basis
Solution
Consider the elements !I' .... _{,, of a unea:r space V.
We can write any polynornial f(t) of degree ::; 3 as
a. We say tbat the f; span V if every f in V can be expressed as a linear
combination of the f;. f(t) = a + +ct +
bt
2
dt 3 = a· 1
+ b. t + c. r 2 + d. r3 .
b. We say tbat the f; are linearlv independent if the equation This shows that the polynom:ials l t , t 2 , 13 span P3 . Are the four polynomials
also linearly independent? One way to find out is to consider a relation
CJ!J +c2h+···+c"J;, = 0
bas only the trivial solution
CQ · 1 + CJ · l + C2 · 12 + C) · t3 = Co+ c 1 t + c 2t 2 + c3 13 = 0.

lf some of the c; were nonzero. thi s polynomial could bave at mo. t rhree zero ;
Ct = C2 = · · · = C11 = 0,
therefore, all tbe c; must be zero. We conclude that I , 1 , r2 , 13 is a basis of p 3 o
that is, if none of the f; is a linear combination of the other fJ- 1 that dim ( P3) = 4. ~

c. We say that tbe f; are a basis of V if they are unearly independent and EXAMPLE 16 ..... Consider a system d,r jdt = A-i' where A is an 11 x n matrix. Suppose there is a
span V. This is the ca e if (and only if) every f in V can be written real eigenbasis ü,, Ü2, ... , Ün for A with associated eigenvalues A1, . .. , An. In
uniquely as a linear combination of the J; .2 Example 12 we have seen that the solutions of tbe sy rem form a sub pace W of
F(IR, lR"). Find a basis for W and thus determ.ine dim(W) .

Solution
Fact 9.1.4 Dimension By Fact 8.1.2 we can write the solutions of the syste m as
Jf a linear space V has a basis with n elernents, then all other bases of V ,((/) = c , e.A ,r Ü1 + · · · + c e.A,.r Ü11 ,
11
consist of n elements as weil. We then say that the dimen sion of V is n:
dim (V) = n. that is rhe curve x ,(t) = eAtli/ ,, ... , xf/(1) = e1'" 1
ÜJI pan w. Are the :\-;(1) linearly
independent? Consider a relation

We defer the proof of thi s fact to the next section. C ,c,>.,r-v I + · · · + c e>.,. r;;
II ""-
-0- .

EXAMPLE 14 ..... Find a basis of JR2 x 2 and tbus deterrnine dim (JR 2 x 2). Evaluating this equation at t = 0, we find that

CJ'ÜI + ·· · + C11 Ün = Ü.

Since the Ü; are linearly independent, we can concl ucle that c 1 = c2 = ... = c11 = 0;
1
2
See Definiti ons 3.2.3 and 3.2.4 and Fact 3.2.5. x;
that is. the (t are linearly independent. T herefore, the (t) are a ba is of W x;
See Defi niti on 3.2.3 and Fact 3.2.7. and dim (W) = n . ~
484 • Chap. 9 Linear Spaces
Sec. 9.1 An lntroduction to Linear Spaces • 485
EXAMPLE 17 ..... Find a basis of C a a real lineru· pace. EXAMPLE 19 ..... Fi 11 d a b·as 1·s of t1e
1
space
·
V of all po1ynom1al s f(t) in P2 suchthat f' (l ) = 0.

Solution
Solution
Any complex number - can be written uniq uely as
For this sp~ce, writing down a typical element of V requires some work. Consider
- = x ·l +y·i
a polynor~u.al f(~) = a + bt+ ct 2 in P2 , witb J'(t ) = b+2ct , o that .f' (l) = b+2c.
The co nd1t10n f (1) = 0 means that b + 2c = 0; that is b = -2c.
for two real numbers x and v. Therefore, 1, i is a basi of C, and dim (C) = 2. The
Thus a typical element of V ha the form
graphical representation of. the complex nurober · in the comp lex pla ne i ba ed
on this fact. ~
f( t ) = a - 2ct + ct 2 •
EXAMPLE 18 ..... Find a basis of the pace of all skew- ymmetri c 3 x 3 matrices A, and determine
the dimension of thi space. Recall that a matrix A is called skew-sy mmetri c if where a and c are arbitrary constants. Following step b m tbe rec1pe, we
AT = -A. Note that thi implie that the cliagonal elements of A are 0 ince write
a;; = - a;;.
f( t ) = a · 1 + c · (1 2 - 21 ).

Solution Be~ause the two functions 1 and t 2 - 2t are li nearly independent, they form a
b~ 1s of V . Check that the function satisfy the requirement f ' (1) = 0. See
A skew-symmetric 3 x 3 matrix can be written as
F1gure 3. ~

A = [-~-b a
0
b]
c , Not all linear spaces have a finite basis f 1, h . . . . , fn. Consider for exam-
-c 0 ple, the pace P of all polynomials. We will show that an arbitrary sequence J1 ,
h, ... J,, of polynornials does not span P , and therefore isn' t a basis of P. Let
where a, b, c are arbitrary constants. Writing A as a li near combination with the N be the maximum of the degrees of the polynomials f 1, h , . .. , fn · Then all
arbitrary constants a coefficients we find that linear combination of / 1, .. . , f" will be in PN (the space of the polynomial of
degree :::: N). A polynomial of higher degree, such as f(t) = tN +l, will not be
0 0]
[ -~
in the span of /1, ... , J,,. Therefore, the sequence f 1, j 2, . . . J,, is not a ba i
a
0 b]
c = a [ - 01 01 0]
0 +b [ 00 00 01] + c [ 00 0 1 . of P.
-b -c 0 0 0 0 - 1 0 0 0 - 1 0

These three matrices are clearly linearly independent: they are a basis of the space
of skew-syrnmetric 3 x 3 matrices. Thi space is therefore three-dimensional. ~ Figure 3
Based on our work in Examp1es 14 to 18 we present the following " recipe"
for finding a basis of a linear space V :

h r) = r2 - r = r(r - 2)
Finding a basis of a linear space V

a. Write down a typical element of V in terms of some arbitrary constants.

b. Using the arbitrary constants as coefficients, express your typical ele-
ment as a linear combination.
c. Ve1ify that the elements of V in this linear combination are liriearly
independent; then they form a basis of V.
486 • Chap. 9 Linear pace Sec. 9. 1 An I ntroducti on to Linear Space • 48 7
--------------------------------~ EXAMPLE 21 ..... Let C[O, I ] be the linear space of alt continuous functions from the clo ed interval
Definition 9.1.5 Infinite-dimensional linear spaces [0, l] to IR. We defi ne
A linear pace whi ch doe not have a fin ite ba is j ,, h , . .. , fn is call ed
infiniTe-dimensional. 1 I: C[O, l J ~ IR by / ( /) = 1 1
f(t ) d l .

As we have just seeo the pace P of alJ polyno mi als i infinite-dimensional. We _adopt the ~implified notation / Cf) = f01 f. To check that I is linear we app ly
bas1c rules of tntegration :
Next we generaJize the nolion of a linear transformation .

Linear transformations, kernel, image

' <f + g)= 1 'u+g) = 1' + 1
f I g = I (f ) + I (g)
Definition 9.1.6
Con ider two linear pace
linear transformation if
and W. A function T fro m V to W is called a / (kf) = 1' (kf) k1'! = k l (f)

What is the im age of ! ? Foreac h real number k, there i a function j (r) uch
T (f + g) = T (f) + T (g) that k = l (f): one possibl e choice i th e con tant functi on f (t ) = k. Therefore,
an d im (! )= ~ -
T (kf = k T (f).

fo r all f and g in V and all real sca lars k.

For a linear transformation T fro m V to W , we let EXAMP.LE 22 ..... Consider the functi on T: ~ 2 ~ C given by T [ ; J = x + i y. We leave it as an

ker(T) = (f in V : T (f) = 0} exerci e to check that T is li near. The tran fo rmati on T i clearl y invertible, with

and r - 1(x + i y) = [ ~ ] .
i m ( T ) = {T (f) : f in V}.

Note that ker(T) is a subspace of V and im(T) is a subspace of W .2 We can easily go back and forth betweeo [~ ] and x + iy :

T
EXAMPLE 20 ..... Let D : C00 ~ C00 be given by D (f) = f'.
What is the kerne! of D ? This kerne!
consists of all smooth functions f such that D (f) = f' = 0. A you may reca ll
from caJcul u , these are the con tant functions f (t = k. Therefore, the kerne! of
[~ ] y-1
X+ iy

D i one-dimension al (the fun ctio n f (t ) = 1 is a basis).

The in venible linear transformati on T in Exan1ple 22 make C into a carbon
What is the image of D ? The image consists of all smooth fun cti ons g
copy of JR 2 (as far a sum and rea l calar multiples are concerned). We ay th at
such that g = f' for . ome f in C 00 , that is, all smooth fu ncti ons g whi cb have
T is an isomotp hism and that the linear space ' 2 and C are i ·omorphi , whi ch
a smooth antideri vative f . The fundamental theorem of calculus teils us that aU
means in Greek that they have the ame tructure. Eac h carrie ome tru ctures not
smooth function s (in fact, all continuous functions) have an antiderivati ve. We
hared by the otber (for example, tbe dot prod uct in IR 2 and the comple product in
can conclude that
q , but a fa r as the two ba ic Operati ons of linear algebra are co ncerned (additi on
im(D) = C . and real calar multipli cation) th ey behave in the ame way.

1
More ad vanced tex!S inLroduce the concept of an infinite basis for uch spaces.
2 Definition 9.1.7 Isomorphisms
See Fac!S 3. 1.4 and 3. 1.6. To show that ke rne! and image are sub paces, yo u need the following
a uxili ary results. whic h we leave as Exercise 48 and 49 for tho e with an inte rest in ax iomatic : An invertibl e linear tran fo rmation i. call ed an isomorphism . We ay that the
a. l f Ov i the neutral el eme nt of a linear pace V, then Ov + Ov = Ov and kO v = Ov. fo r all
ca lars k. linear paces V and W are isomo1p hic if there i an isom rph i. m from V
b. Jf T is a linear tnmsformati on fro m V LO W, the n T (Ov) = Ow , where Ov and Ow are the to W.
neutral elements of V and W .
488 • hap. 9 Lin ea r Space
Sec. 9.1 An Tntrod uction to Linear paces • 489
Below, we tate om u fu l fact coocerning i orn rphi ms:
Be low, we give two mo re exa mples of isomorphi ms.
EXAMPLE 23 .... Show that the linear spaces lR"x"' and m.
IID mx11 are tsomorp
· h'
tc.
Fact 9.1.8 a. Jf T is an isomorph i. rn , then so is r- 1.
Solution
b. A linear tran fo rmatio n T from V to W i an i omorphism if (and only
if) ker(T) = (0 } and im(T) = W. We need to find ao isomorphi sm L fro m lR"x"' to JRm x n th t · ·
•
tr an s1orm . , a ts an m venible Li near
c. Con ider an i omorphi m T from V to W . Lf / 1, h, . . . J,, is a ba i of att on. Check that L (A) = AT does the j ob. <11111

V, tben T (f 1 ). T(h .. . . T(J,,) is a ba i of W. EXAMPLE 24 .... Let v-- F(R R) b h .

, e t e lmear space of all functions from JR to R We define
d. lf II and W are i o morph ic and dim ( V) = 11 then dim ( W) = 11.
T :V ~V by ( T f)( t ) = f(t- 1),
where we write T f instead of T (.f) to simplify the notation. Note th at T moves
Part d ~ hou ld come a no urprise, since isomorphi c paces have the sa me the graph of f to the right by one urtit. See Figure 4.
stru ture, as far as linear a lgebra i concerned. l s T Lin ear? We first have to check that ~

T U+g) =Tf+ T g
Proof T lli proof may be skipped in a fir t reading.
or
a. We mu st s how that r - 1 i linear. Con ider two e lemen ts f an d g of the
codo main of T (that i , the do ma in of r - 1). Then ( T U+ g))(l) = (T J + T g)(t) .
for all 1 in R Now
r -t Cf+ g) = r - 1(rr - 1Cf)+ r r - 1Cg))
= y - 1( r (r - 1Cf) + y - 1(g))) ( ince T is linear) (T U+ g ))(t) = U + g)(t - 1) = f( t - I)+ g(t - 1)
= r - t cn + r - t cg). and

In a similar way you can how that r- t (kf) = kr - t (f), fo r all f in the (T f + T g)(t ) = (T f)( t) + (T g)(t ) = f( t - l ) + g(t - 1):
codomain of T and all calar. k. the two re ult agree.
b. Suppose first that T i an i omorphi sm. To fi nd the kerne! of T , we have We leave it as an exercise to check that
to solve the equation T (f = 0. Applying r - t on both sides, we find that
T (kf) = k (T f).
! = 0 so that ker T ) = (0}, as claimed . Any g in W ca n be wri tten as
g = TT -t(g), so th at im (T) = W . (What doe thi Statement mean in tenn o f shifting graph ?)
Con versel y, s uppo e that ker(T) = {0} and im (T) = W. We have . . Note th at the transform ation T is in vertible: we can get f back fro m Tf by
to show that the equati o n T (J) = g has a unique so lu tio n f for any g shtfttng the graph of T f by one unit to the left (see Fig ure 5).
in W (by Defini tion 2.3. 1) . There is at least one uch solution f, in ce
(1' - 1 g)(t ) = g(t + I )
im (T) = W. Con ider two o luti ons, ft and !z: T (Jt) = T(h) = g. Then
0 = T Ut) - T (h) = T Ut - h), so th at ft - h i in the kerne] ofT ; that We conclude that T i an i ·omorphi m from V to V.
i , !t - h = 0 , and f t = h, a claimed.
c. We will show first that the T U;) pan W. For any g in W, we can write
Figure 4
T - (g) = c1f1 +· · ·+c"J,, because the J; span V. Appl yi ng T on both side
1

and u ing linearity, we find that g = c tT (f1) + .. . +c" T (f"), as c laimed.

To show the linear independence of the T (f;), consider a re lation
Ct T Cft) + · · · + c" T (j,,) = 0 or T (c if1 + · · · + c" j,,) = 0 . Since the kerne! of
T is 0 , we have c 1! 1 + · · · + c" j,, = 0. The n the c; are zero since the fi are
linearly independent.
d. Fe llows from part c.
• t - I
490 • Chap. 9 Li near Spaces
Sec. 9.1 An lntrod uction to Lin ear Spaces • 491
~he~~ C ~ s an arbitrary_constan_t. Thi~ shows that all scal ars ./,. are eigenvalue of
· e e1genspace E;, 1 one-dimenswnal, panned by e ).t . ~

EXAMPLE 26 .... Consider the linear Iransformation

Figure S r+ I
L. T!l) II X /1 ~ JD) II X /1 • T
!1'0. ----r Jl'l. g1ven by L (A) = A .

Find all eigenvalues and eigenspace of L. Is L di agonalizable?

ext we generali ze the not:ion of an eigen alue.

Solution
Definition 9.1.9 Eigenvalues, diagonalization I~ L (A ) = 'AA, then L (L (A)) = A = ), 2 A o that .1,. 2 = 1. The only possible
e1gen va lues a.re .1,. 1 = 1 and .1,. 2 = - I. Now
Co nsider a linear transform arion
T: V --+ V ,
E, = {A : Ar = A) = {symmetric matrices }
where V i a linear pace . A scalar 'A is cal led an eigenvalue of T if there is
a nonzero f in V uch rhat and

T (f) = ), j.
E_ , = {A : Ar = - A} = {skew-symrnetric matrices}.
If ). is an eigenvalu e of T , we define rhe eigenspace
EJ.. = {f in V: T (f) = 'Af}. We leave it as an exercise to show th at
Now suppose that V is finite-dimen sio nal. The n the transformation T is call ed
. rz (n + 1)
diagonalizable if there is a ba is f 1 , •• • , J,, of V such that T /;) is a scalar d1m(E 1) = l + 2 + ... + n = -----=-
multiple of f;, for i = 1, ... , n. Such a basi i called an eigenbasis for T. 2

and

EXAl\tPLE 25 .... Find all eigenvalues and eigenspaces of the linear transformation .
dim (L 1) =1+2+· ··+(n-l ) =
(n- l )n
,
2
dx
D: C 00 --+ C"" given by D (x) = - .
dt so that

Solution
We have to olve the differenti al equ ati o n We can find an eigenbasis for L by choosing a basis of each eigenspace; thu L
is diagonalizable. For example, in the case n = 2 we have the eigenba i
dx
D x) = - = 'Ax.
dt

By Fact 8. 1.1 the solutions of thi s OE fo r a fi xed )._ are the functio ns
~
[~ ~l
t /'
[~ n. [-~ 6]
t
basi. of E 1 ba is of E _ 1
492 • Chap. 9 Linear Spaces
Sec. 9.1 An Introducnon to Linear Spaces • 493
For the convenience of the reader, we include a list of notations introduced
in this section : Let V be the space of , 11 · fi · .
\VI . h f ' a In mte sequences of real numbers (see Example 7) .
llc o the Subsets of V given in Exercises 12 to 15 are subspaces of V ?
1 . The arithmet~ic seq uences, i.e., sequences of the form (a , a + k , a + 2k .
2
Notations a + 3k , · · .), for some constants a , k. ·
F (X , Y) the set of all functions from X to Y . 13
· The geometric sequences, i.e. , seq uences of tbe form (a , ar, ar 2, ar3, .. .). for
~m x n the linear space of all real m x n matrices. some constants a and r . ·

p the linear space of all polynormals with real coefficients. 14. The sequences (xo, x1 , .. .) which converge to 0 (that is, lim x = 0).
11
17-loCO
P" tbe linear space of all polynormals of degree :S n.
coo the linear space of all smooth functions from ~ to ~ (those
15. The "square-summable" sequences (x 0 , x 1, • •• ) , tbat is, those for which f x1
converges. i=O
functions that we can ditferentiate as many time as we want).
C[O, 1] the linear space of a.ll contin uou s functi.ons from the closed Find a basis for each of the spaces in Exercises 16 to 28, and determine its
interval [0, 1] to ~. dimension .
D the linear transformation from C 00 to C )O given by Df = .f' (the 16. ~3 x 2 .
derivative). 17, }Rtlll X T/ •

18. P".
19. The real linear space ([2.
EX ER C I S ES
20. The space of all matrices A in ~ 2 x 2 with tr(A ) = 0.
GOALS Apply basic notions of linear algebra (defined earlier for ~") to linear 21. The space of a.ll symmetric 2 x 2 matrices.
spaces.
22. Tbe space of all symmetric n x n matrices.
23. The space of all skew-symrnetric 2 x 2 matrices.
Which of the subsets of P3 given in Exercises 1 to 5 are subspaces of P3? Find a
basis for those that are subspaces. 24. The space of all skew-symmetric n x n matrices.
25. The space of all polynormals f (t) in P2 such that f(l) = 0.
1. {p(l): p(O) = 2)
26. The space ofall polynomials f(t) in P3 suchthat f(l) = 0 and J~ f(t)dt = 0.
2. {p(t): p(2) = 0} 1
3. {p(r): p' (l) = p(2)} (p' is the derivative) 27. The space of all 2 x 2 matrices A that commute with B = [ 1 0]
0 2 .
4. {p(t) : 11
p(t) dt = 0}
28. The space of all 2 x 2 matrices A that commute with B = [ 1 1
0 1 .
J
5. (p(t): p( - t) = - p(t) for all t}
29. In the linear space of infinite sequences, consider the subspace W of arithmetic
sequences, i.e., those sequences in which the difference of any two consecutive
Which of the subsets of ~ 3 x 3 given in Exercises 6 to 11 are subspaces of IR 3 x 3 ? entries is the same, for example,
6. The invertible 3 x 3 matrices. (4, 7, 10, 13, 16 . . . .) ,
7. The symrnetric 3 x 3 rnatrices. where the ctifference of two consecutive entries is 3 and
8. The skew-symmetric 3 x 3 matrices (recall that A is skew-symmetric if (10, 6. 2, -2. -6, ... ),
AT= - A).
9. The diagonal 3 x 3 matrices. where the ctifference of two consecutive entries is -4. Find a basis for w
and thu s determine the dimension of W.
10. The 3 x 3 matrices for which [ ~] is an eigenvector. 30. A function f(t) is called even if .f(-t) = f( t ), for all t in ~ . and odd if
f( - t) = - f(t) , for all t. Are the even functions a subspace of F(IR, ~), the
space of all functions from .IR to .IR? What about the odd functions? Justify
11. The 3 x 3 matrices in reduced row echelon fonn. your answers carefully.
494 • Chap. 9 Li near Spaces Sec. 9.1 An Introduction to Linear Spaces • 495
31. Find a basis of eacb of the fo llow in g linear paces, and thus determ ine their 46. Consider an n x n matrix A that is diagooalizab le over R Let W be the solu tion
dimension (see Exerci e 30). space of the system dx / dt = ki: ( ee Example 12). Is the transformation
a. [f in P_.: f i even }.
b. {f in P4 : f is odd}. T: W--+ lll" given by T(x(l)) = x(O)
32. Let L (iR". IT{111 ) be tbe et of aJI li near transfonnarions from iR" to ~~~~ . I · linear? Is it an isomorphism?
L(iR" , IT{111 ) a subspace of F (~", iR"'), the pace of all functions from iR" to 47. For the transfonn ation T defi ned in Example 24, veri fy tbat
iR"'? Ju tify your answer carefully.
T(kf) = k(Tj) ,

Decide which of the rransformatio n in Exerci e 33 to 41 are linear. For those for allreal scalars k and al l fu nctio ns f in F(JR, IR). Draw a sketch illustrati ng
th.i s fonn ula.
th at are linear, detennine whetber they are isomorphism .
48. S how that if 0 is the neutral element of a linear space V then 0 + 0 = 0 and
33. T: P3 ~ iR given by T(f ) = 1 3

-2
.f(t) dt .
kO = 0, for all scalars k.
49. Show that if T is a linear transformation from V to W, tben T (O v) = Ow,
34. T: JR2x 2 ~ JR2 2 given by T (A) = A + / 2. where Ov and Ow are the neutral elements of V and W, res pectively.
35. T : JR3 x 3 ~ lR given by T(A) = tr(A). SO. Find all so lutions of the differential equation
36. T : JR 2 x:>. ~ lR given by T (A) = det(A ) . d 2x

u J.
37. T: C ~ C given by T( z) = (3 + 4i)-. - +x = O.
dt 2
38. T: JR2x 2 ~ ]Jl2x 2 given by T (A) = s- 1AS, where S = ~ Hint : ln troduce tbe auxiü a.ry funcrion y = dx /dt . The DE above can be
converted into the system

39. T: JR2x2 ~ ]Jl2x 2 given by T (A) = [ ; ~ JA. dx

y
dt
40. T: P6 ~ P6 given by Tl.f) = !" +4f'. dy
- = -x
41. T : P2 ~ lll 2
given by T(J) = [ j<~L J.
dt
Apply tecbniques introduced in Chapter 8.
.
42. Do the positive senndefi . 2 x 2 matn.ces "1orm a
mte su b space of .JR2 x 2?.
51. Use the idea presented in Exercise 50 to fi nd all solutions of the DE
43. In the space F (R !Pl) of al1 func ti ons from iR to IR, consider the subset of al l
functions with period 3; th at is, those functions f(t) such th at f(t+3). = .f(t), d 2x dx
- + 3- +2x = 0.
for all t in lll. Do these functio ns fonn a subspace of F (IR, iR)? Just1fy your d t2 dt
answer carefull y. 52. U e the idea presented in Exercise 50 to fi nd all solutions of the OE
44. Con. ider the transform ation d 2x dx
- - 4- + 13x = 0.
T :IRmxn ~ F(lll" , IR/11) d t2 dt

given by Fi nd the (real) eigenvalues and eigenspaces of the li near transformati ons in Exer-
cises 53 to 60. Find an eigenbasi if po sible.
(T A)(v) = Aü.
53. L: !Pl2 x 2 ~ JR 2 x 2 given by L (A) = A +AT.
Show rhat T is linear. Wh at is the kerne! of T? Show that the image of 54. T: C 00 --+ C 00 given by T(f) = f + f' .
T i the space L (lll" , JR"') of all linear transformations from IR" to lR"' (see
55. T: C ~ C given by T( z) = z.
Exercise 32). Find the dimension of L(JR" , IRm) .
56. T: V ~ V given by T (xo , x ,, x 2, X3, . . . ) = (x,, x_, X3, .. . ), where V is the
45. If T is a linear transfonnation fro m V to W and L is a linear transformation space of infinite sequences of real numbers.
fro m W to U, is the composüe transfonn ation L o T fro m V to U · linear?
57. T:C 00 --+ C given by (Tf)(t) = f(- t ).
How can you te ll? If T and L are iso mo rphi sms, is L o T an isomorphism as
well ? 58. T: P2 --+ P2 given by (T f)(t) = t · f ' (t) .
496 • hap. 9 Linear Spaces
Sec. 9.2 oorcUnates in a Linear Space • 497
59. T : JR2 x2 ---+ JR 2 2
given by T ( A) = [! 2
] A. EXAMPLE 1 .... C 0 11 sider the linear space P2, wiili the basis
7 - 3 I + llt 2 , find UJa.
B consisting of l
, t , r2_1 For f (t ) =
60. T : JR.2 x2-+JR 2 •2 given by T (A) = S- 1 AS, whereS = [ ~ ~ ] ·
Solution
61. Find an igen ba i for the linear tran Formation T (A) = s- l AS from "JR.II X /1 to
The coordinate of f wi th respect to are 7, - 3, and 11 , so that

r
lR" x" , where S i an invertible diagonal matrix. B

62. Let :JR.+ be tl1e et of positive real numbers. On JR+ we defi ne the "exotic" 1
operati on
x EB y = xy (usuaJ multiplication) [fl, ~ ~n
and
k O x = xk. ~onsid_er a basis B o f a linear pace V, consisting of f 1 , • • • , f,, . Introducin o
coordm ates 111 ~ with respect to B allows us to transform V into "IR": we ca~
a. Show that JR+ wiili the e operati ons is a linear space; fi nd a basis of this define the coordmate transformation
space.
b. Show that T (x ) = ln (x) i a linear tran Formation from JR+ to :IR., where 1R I~ UJa
is endowed wiili ilie ordinary operations. Is T an i omorphi m?
from V to lR" . This coordinate transformation is in vertible: its in ver e is the
63. I it po ible to defi ne "exotic" 1 operation on !R( 2 o that dim (lR2 ) = I? tran Formation
64. Let X be the set of all citizen of Timbuktu. Can you define operarion on X

[ ~; l ~
that make X into a real linear space? Expl ai n.
c,f<+ + c.J,,

CII J
2 COORDINATES IN A LINEAR SPACE fro m IR" to V .
Tbe coordinate tran Formation is also linear, because [kf ]a = k(J] and
In thi s section we continue generalizing the basic concepts of li near algebra from 8
[/ + g] a = [/Ja + [g]a. We verify the first identity, leaving tbe econd as
IR(" to Linear paces.
Exercise 25. If I = c,l1 + · · · + c J," ilien kf =
11 kc 1 f 1 + ... + k c" J,, , o that

Definition 9.2.1 Coordinates

Consider a linear space V with a basis B con i ting of f 1, • • • , J,,. The n any [k!J a =
k c2
kc;
1

l = k [c'l
c2
: = k (J] 8 .
f in V can be written uniquel y as
f = ctf, + · · · + CnJ," We have shown ilie following result:
r k c" C11

for some scalars c1. c2 , . . . , c". The c; are called the coordinates of f with
re pect to B, and the vector Fact 9.2.2 Coordinate transformation
Consider a linear space V , wiili a basis B, consisting of j 1 , . . .• ln · Then the
coordinate tramformation

f ~ UJa
is called the coordinate vector of f, de noted by [!] 8 . from V to IR" is an isom01phism, i.e., an in vertible linear tran formation.

1
Exotic in the sen e of ·• trikingly out of the ord inary" (Webster) . 1
We re fer 10 the ba.s is I. 1• .. • • 111 a.s the swnda rd basis of P11 •
 ~"'""' - ···- ·-·-

498 • Chap. 9 Linear paces Sec. 9.2 Coordi nates in a Linear Space • 499
EXAMPLE 3 ..... The coordinate transformation for C with respect to the standard basis 1, i is
c ~ JR2

x+iy ~
[ xy ] .
EXAMPLE 4 ..... The coordinate transformation of iR 2 x 2 with respect to tbe standard basis

Figure 1 [b ~] ' [~ b]. [~ ~l [~ ~]

is
JR2 x 2 ]R4
Th i fact allow us ro gi e a conci e con ceptu al proof of Fact 9. 1.4 (rhis proof ~

wa deferred). Con ider a linear space V with two bases A (wi th n e lements) and
B (with m lement . We claim that 11 = m. To prov rhis fac t con ider the
coordinate transformations

and
T_", Cf) = [fh from V to iR"
EXAMPLE 5 ..... Let
[~ ~] ~

m
V be the solution space of the differential equation !" + f = 0. In rhe
introduction to Section 9. 1 we discussed the solutions, f(t) = c 1cos(t) + c2 sin(r).
from V to iR 111 • The coordinate Iransformation for V with respect to the basis cos(t), sin(r), is

See Figure I. V ~ JR2

Then T6 o T~- J i an isomorphism (an inve1tible linear transformation) from

lR" to lR111 (see Fact 9.1.8a and Exercise 9.1.45). The exi tence of such an invertible c 1 cos(t) + c2 in (t) ~
[ cc2' ] ·
Jjoear transformation (described by an invertible matrix) implies that n = m (by
Fact 2.3.3), as clairned.

Fact 9.2.2 teils us that by introducing a ba i in a linear space V we " make

EXAMPLE 6 ..... Derermine whether the matrices [ ~ ~ l [; ~ l [ i 1 ~~] are linearly inde-
V into a copy of iR"" (rhis is really just an extension of Descartes' concept of pendent. If not, find a nontrivial relation among them.
analytical geometry). This translation process makes computation easier because
we have powerful numerical tools in iR", based on matrix techniques. We do not
need a separate theory for finite-dimensional linear spaces, since each such space Solution
is isomorphic to some iR" (it has the same structure as lR"). For example, we can We translate the problem into iR4 , using the standard coordinate tran fonnation
say that n + 1 elements of an n-dimen ional linear space are linearly dependent, (Example 4). Then the question is: are the vectors
since the corresponding result holds for iR" (by Fact 3.3.4).
Here are some (rather harmless) examples of coordi.nate Iransformation .
2
EXAMPLE 2 ..... The coordinate transformation for P2 with respect to the Standardbasis 1, 1, t is
p2 ~ JR;3
linearly independent? We can use matrices to answer this q ue tion.

~~]
ao+ a,t+a, t' +--+ [ :; ]
rref
11
We u e double-headed arrows to emphasize the invertibi lity of the coordinate
12
transformation. ~
500 • hap. 9 Linear paces Sec. 9.2 Coordinates in a Linear Space • 501
Therefore. the tbree vector are li nearly de pendenl, with Written in coordinates, the transformation T maps

The matrix
m into
[b + 2c
2c
] = [ 0
0
1 2] [
0 2
~ J.
c

or

is called the matrix of the transformation T with respect to the bases A and B: it
EXAMPLE 7 ..... Consider a polynomial J(t = a0 + a 1t + · · · +_ 0 111 1111 and an n x n_ matrix A. de cribes the transformation T in coordinates.
We can evaluate f at A' that is, we can con 1der the n x 11 matnx f( A ) = T
J + a 1 A + ... + a 111 A 111 • For a given n x 11 matrix A , show that there is a nonzero
00 11
polynomial f (T of degree ~ n 2 uch tJ1at f( A ) = 0.

Solution
M
Since the linear space lR" x" J. n 2 -d1mensiOnal,
. .
the n 2 + 1 matnces
. 1"' A , A2 , . . . , A"2
2
are lioearly dependent , i.e., there i a nontrivial relation co /" + c,A + c2A + · · · +
111
2
c111 A"' =O(withm =11 ) amongthem. Thepolynomial f( t ) = co + c lt+ · · · + cm1
This diagram can be drawn more succinctly as follows:
ha the desired property. ~ T

+ The Matrix of a Linear Transformation

Next we examine how we can write a linear transfom1ation in coordinates. For
M
example con ider the linear transformation [T(f)] a
T : P2 -+ P1 given by T(J ) = J' + J".
More expli citly, we can write
Definition 9.2.3 The matrix of a linear transformation
T (a + bt + ct 2) = (b + 2 ct ) + 2c = (b + 2c) + 2ct Consider a linear transformation
or T: V-+ W.
T
a + bt + ct 2
(b + 2c) + 2ct. where dim (V) = n and di m (W) = m. Suppose we are. given a basis A of V
U ing the standard basis A = I , t , t 2 of P2 , we can tran sform P2 into lR .
3 and a bas i B of W:
T T
2
Likewise, we can use the standard basis B = l , t of P1 to transform P1 into 1R . V w
Let us apply these coordinate tran formations to the input and the output of the
Iransformation T above:
T
(b + 2c) + 2ct lR" [T(J)] a

1 Ts Then the matrix M of the linear tran fo m1ation Ta o T o

m.arrix ofT with re.spect ro A and B. Note that
r_."- 1 i ca lled the

b + 2c ] [T(J)] a = M[Jh ,
[ 2c
fo r all f in V.
502 • Chap. 9 Linear Spaces
T T
Sec. 9.2 Coordinates in a Linear Space • 503
\1 w f EXAMPLE 9.... For two given real nurnber d fi d .~..

1 TA

M
1 Ts '
T,.

M
P an q,

T( z) = (p
n ute matrix of the linear transformation
+ iq) z
IR" JRIII (f)A [T(f )]a from C to C with respect to the Standard basis B of C consisting of 1 and i .

Solution
Compare with Definition 7.1.3. . T

We can describe the matrix M column by colurnn compare w1th Fact 7.1.5) .
p + iq
Suppose the ba i A of V consi t of ! 1 , • •• , fi,. App lying t11 e formula
- q+ip --~
[T(J )]a = M[f ]A
Note that M is a rotation-di1 ation matrix.
to f = j; , we find that

[T /;) ] 8 = M[f;]A = M e; = i th column of M . Fact 9.2.5 The matrix M of the linear transformation

T(z) = (p + iq) z
from C to C with respect to the standard basis of <C is
Fact 9.2.4 Consider a linear transformation T from V to W . Suppose A i a basis of V,
consisting of f~> .. . , j," and B i a basis of W. Let M be the matrix of T
with re pect to A and B. Then M = [: -: l
ith column of M = [T(f; )]a.

EXAMPLE 10 .... Let V be tbe linear pace con isting of all function s of the form f (t) = a co (t ) +
b in (t ), iliat i the inusoidal funclions with period 2n. Find the matri x M of the
Fact 9.2.4 provide u with a mechanical proced ure for finding the matrix linear tran formation
of a linear transformation , as illu strated below.
(T.f)(t) = f(t - 8)
EXAMPLE 8 .... Use F act 9.2.4 to find the matrix of the linear transformation T(f ) = !' + !" from
fro m V to V with respect to the basis B con i ting of co (t) and sin(t ). ote that
P2 to P1 wiili respect to the tandard ba es.
T moves the graph of f to the right by 8 units (compare with Example 24 of
Section 9.1).

Solution
Solution
We apply the transfonnation T to ilie functions 1, r , and t 2 of the standard basis of
P2 . We then write the resulling function s in coordinate with respect to tbe standard Be prepared to u e tl1e addition theorems for ine and co ine.
basis of P1 • Next we combine the three resulting vectors in IR2 to construct the
matrix M of the transformation T. cos t)
T
--~ co (1 - 8) = cos(8) cos(t) + sin (o) in r )
[
CO (8)
in (8)
J)
T
M.
0 in(t) si n(l - o) = - in (o) CO (t ) + CO (o) si n(t - in (8) ]
[ CO (8
where

- in(8)
(o )
J'
2t + 2 - - -+ CO

a rotation matrix .
504 • hap. 9 Linear Spac · Sec. 9.2 oordinates in a Linea r Space • 505
proble m about a linear tran Formation T can oft n be done by ol ving lhe (we pick the pivot columns). Tran formin g these vectors back into R 2 x 2, the
corresponding problem for the matri x M of T with r pect to o n~e bases. Thi codomain of T , we find that
tech nique can be used to find the k rnel and image of T. to de termt~e whether T
is an isomorph.ism, to find the e igenvalues ofT , or to ol ve an equat10n T(J ) = g
for a given g .
12 = [1 0]
0 1 A= [ 1
3
2]
4

c V
T
w :::J im T)
is a ba ·i of the image of r.
ker(T) --7
t t Kerne!: To find the kerne! of M we have to solve the y tem
1
s; im M)
ker(M) ~II --7 ~111 :::J
+ 2xJ =Ü
x 2+ 5x 3 =0
I·
The vertical arrow · abo e repre ent coordinate tran fo rm ati o n .
The general olution is
EXAMPLE 11 ..... Con ider the linear transformation from P?. to IR 2 2
gi e n by

T (f) = .f( A) , where A= [

(compare with Example 7 . Find the matrix of T with re p ct to the standard

~ ~]
[ =~~~ ]-1.3
= XJ [=;]. 1

=~] . Tcan fomting back illto P

ba e . U e this matrix to find ba e for the kerne! and image of T .
A ba; of the kemel of M; [ 1, the domaill of
Solution
! · we find that f( t ) = - 2- 5t + 12 i a ba is of the kerne! ofT . Note that f (t)
Construct the matrix M of T column by column, u ing Fact 9.2.4: JS the characteristic polynomial of A (compare with Exerc i e 7.2.64).

ker(T) s; ? 2 ~ ~ 2 x 2 :::J im(T)

h
m t
ker(M)
t
s; ~ 3 ~ IR4
t
:::J
t
im (M )

A = u~] [il M = [~ 2
3
4
16l
15
22
EXAMPLE 12 ..... Let V be the linear pace consi ting of all functions of the form f t = a cos t
b sin (t). Con ider the linear transformation T from V to V given by
+

[l~J
T(.f ) = f "- 2/' - 3f.
{2 A= [ 157 2210]
2
a. ls T an isomorphism?
b. Find all olutions f in V of the differential equation
Let us find bases for the image and kerne! of M:
!"- 2 f'- 3f = cos (t).

[ ~
0
I
; ~~J
3
4
15
22
-
rre f
- -7
Solution
Find rhe matrix M ofT with respectto the ba i B consi ting of co (1) and in (r):

Image: A basis of the image of M is cos (t) - 4cos (t ) + 2 in(t ) ---7

si n(t) - 2cos(t)- 4 in (r) ---7

Note that /VI i. a rotation-dilation marrix.

506 • Chap. 9 Linear Spaces
Sec. 9.2 Coordinates in a Linear Space • 50 7
a. Since M is invertib le. T is an isomotphism.
Consider the matrix M of T . h .
b. We wtite the equation T (f) = co (t) in coordinate : matrix with ker(M) = {O ) ~It re~p~ct to. some bas t . Then M i a quare
an isornorphism. 'so tat M lS mverttb le by Fact 3. 1.7. Therefore, Ti
[T(f)]a = [cos (t)]a

or EX E R C I S E S
•
where .X = [f) s.
GOALS . Us e th e concept of coordmates.
.
formatwn. Find the matrix of a linear trans-
The solution is
_
x = M
_ 1 [ l] I [-4- 2 2][1]
0 = 20 -4 0
= [ - 0.2]
-0. 1 .
1. ~e the polynomiaJs f(t) = 7 + 3t +
ltnearly independent?
,2 g(t) = 9 + 9r + 4r 2, h (t = 3 + 2t + r2
2. Are the matrices
Transfonning th.is an wer back into V, we find that the uni.que olution in V
of the given differential eq uation is
J(t) = -0.2cos(r) - 0. 1 in(r) .
linearly independent?
Check th.is answer.
f E V
T
V 3 CO (r )
3. Do the p~lynot~aJs j(r) = 1 + 2t + 9r2 + r3, g(t) = 1 + 7 t + 71 3 h(t) _
l+B_t + r + 5t,k(t ) = 1+8t+4t-+8z 3 formabasi ofP ? ' -
3
t t t t 4. Con tder _the polynomials f(t) = r + 1 and g(t) = (r + 2)(t + k) where k
an ar~ttrary constant. For wh.ich cboices of tbe constant k are the three
~]
M 1
X E JR2 ll{2 3 [
polynomtals f(t), tf(l), g(t) a basis of p2 ?

The ba ic facts we have derived for linear transformation from lR" to

In Exercises 5 to 10 find the matrix of the given linear tran formation with respect
IR111 generalize easi1y to linear transformations between finite-dimensional Linear to the tandard bases.
spaces.
5. T:P2~lR 2 given by T(f)=[f,~ 1?)] ·
Fact 9.2.6 If T is a ünear transformation from V to W, where ker T and im(T) are 6. T : P2 ~ Pz given by (T f)(t) = f(t + l).
finite-dimensional linear spaces, then V is finite-dimensional, and
dim (ker(T))
Compare with Fact 3.3.9.
+ dim (im(T)) = dim(V).
7. T: P3 ~ JR- given by T(f) = [!'f(l) J.
f(r)dt
- I

Here i another useful result:

8. T: P2 ~ JR 3
given by T(f) = [f~~~~ ] .
f(2)
Fact 9.2.7 Consider a linear transformation T from V to W , where V and Ware finite- 9. T: JR1 2
~ IR given by T(A) = tr(A ).
dimensional linear spaces. Then T i an isomorphj m if (and only if) 10. F : JR 2 x - ~ IR 2 "' 2 given by F (A) = tA + iAT.
a. dim(V) = dim (W) and 11. For the tran formation T in Exercise 5. find bases of ker(T) and im (T).
b. ker(T) = {0}. 12. Ts the transformation T in Exercise 6 an isomorphi sm? Find the eigenvalues
and eigenspaces of T. Is T diagonali zable?
13. Find a basis of the kerne! of t11e linear transformation T in Exerci e 7.
Proof If T is an isomorphism, then dim (V) = dim (W) by Fact 9.l.8d, and ker(T) = {0}
by Fact 9.l.8b. Conversely, suppose that dim(V) = dim(W) and ker(T) = (0]. 14. For the transformation T in Exercise 8, find bases of ker(T) and im (T) . I T
an isomorphi m?
 Sec. 9.2 oord inates in a Lin ear Spa ce • 509
508 • Ch ap. 9 Linear Space
a. Find the matri x of the linea r tran Formation T:V -7 V given by
15. Oe rib the kerne! and image of the transfonnation F in Exerci e 10. Find
the e igenvalu es and eigen pace of F . I F diagonal izable? T (f) = f" + af' + bf
• ) ,
16. Let \1 be the li near space of al l quadratt forms q (x 1, x 2 = a.lj + /; -~'"1-r2 + c~-_?2
w_ith respect to_ the basis cos(t), in (t). Here a and b are arbitrary co n tant .
in two ariable . Concider the linear tran Formatio n
b. Fmd the functwn (s) f in V such that
ar of
T (J) = - ·- x_ - - x2 T (j) = f" + af' + bf =CO (l ).
OXI OX2
You_r sol ution will contain the arbitrary constant a and b. For which
fro m \1 to V. cho J c~s of ~ and b i there 11 0 such fun~ti on j? a n ph y ic ' the diffe rential
a. Find the matri x of T wi th re pect to the basis x~ , .r1x2, x~. equatt on f + af' + bf = cos(r) de cnbe a fo rced o c illator.)
b. Find bases of the kerne! and im age ofT. 22. Let V be the linear space of a ll functi ons of tbe form
c. Find the eigenval u and eigen pace ofT. I T diagonali zable?
f( r) = c 1cos(r) + c2 sin (r) + c3 r co (t) + c4 t in (t) .
17. Consi der the linear tra nsfonnation
Consider the linear transformati on T : V -7 V gi ven by
T (j) - [ f'o(3) 3f(l) ]
T: P2 -7 llll 2x '- gt. en by - f(S) .
IN.
T (J) = f" + f.
a. Find the matri x of T with re pect to the basis cos(t) , sin (t), t co (t ), t in (r).
a. Find the matri x of T with respect to the stan dard bases.
b. Find all olutions f in W of the differenti al equ ation
b. Find bases for tbe kernel and the image of T.
18. Con ider the linear transformation T (j) = J" + f = CO (t).

Graph your soluti on(s). (The di ffe rential equation f" + j = co s(l) de-
T : P2 -7 P2 given by T(f ) = f + af' + bj", cribes a forced undamped oscillator. In this example, we observe the
phenomenon of resonan.ce.)
where a and b are arbitrary constants.
23. Consider the linear transformation T: P11 -7 JR"+ 1 given by
a. Find the matrix of T with respect to the srand ard bas i of P2.
b. If g is an arbitrary polyno mial in P2 how many so lutio n f in P2 does f(no) ]
the differenti al equarion f(a,)
T (f) = . ,
f + af' + bf " = g [
J(~n)
have? Ju. ti fy your an wer. where the a; are distinct constants.
19. Consider a linear tran . Formati on T: V -7 V with ker(T ) = (0 }. If V is finite- a. Find the kerne! ofT. Hint: A nonzero polynom.i a1 of degree ~ 11 ha - at
dimensional , then T is an isomorphi sm, by Fact 9.2.7. Show that this is not most n zeros .
necessarily the ca e if V i infinite-dimensional: for \1 = P give an exampl e b. I T an i omorphi m?
of a linear transfonnation T : V -7 V with ker(T ) = (0} which is not an c. What does your answer in part b tell you about the po ibility o f fitrin g a
isomorphism (recall that P is the space of all polynomial s). polynomial of degree ~ n to 11 +I given points (ao, bo), (a 1. b 1) •.•.• (an . b11 )
20. Consider two finite-dimen ional linear spaces \1 and W. lf V and W are in the plane?
isomorphic, then they have the same dimension (by Fact 9. 1.8d). Conversely, 24. Consider the linear space V of all infinite eque nce of rea l number . We
if V and W have the same dimensio n, are they neces. arily isomorphi c? Ju stify define the subset W of L consi ting of all sequences (xo, x 1• x 2 , ... ) uch that
your an wer carefull y. X n+2 = Xn+l + 6 xn for all 11 ~ 0 .

21. Con ider the linear space V th at consists of all fun crions of the form
a. Show that W is a subspace of V .
b. Determine the dimension of W .
c. Does W contain any geometri c ·eque nce of the form ( I . c. c 2 • c· .
f( l ) = c 1 COS(I ) + C2 sin (t),
. .. ), for some constant c? Find all such . equ ences in W .
d. Can you find a basi of W con isting of geo metric sequence, ?
where c 1 and c2 are arbitrary constant .
510 • hap. 9 Linear Spaces
Sec. 9.3 Inner Product paces • 511
e. Con ider the equence in W w ho -e fir t two term are .ro = . 0. x 1 = I . 33. Let a , ... , a" be distinct rea l numbers.
Find x 2 , x 3 , x 4 . Fin d a clo ed fo rmula for the nth term x" of th1 s sequence. Show that there are "weigbts"
w1, . . . , w" such that
Hinr : Write thi
found in part d.
equence a a linear combination of lhe sequences you

25. Consider a basis B of a I in ear space V. Show that

f_
1

1
f(t) dt = t w;j(a;),

fo r all polynomials f(t ) in P"_ 1 • Hint: It suffi ces to prove the cl aim for a
(f + g]s = (f]s + [g]a, basis !1 , ... , j;, of P"_ 1. Exercise 32 is helpful.
fo r all fand g in V . 34. Find the weights w 1, w 2 , w 3 in Exerci e 33 for a 1 = - J, a2 = 0, a = 1
3
26. Con ider the linear tran formatio n I (f) = f
1
dx fTo m C[O, 1] to IR (see (compare with Simpson s rule in calculu ).
0 .f(x)
Example 2 1 of Section 9. 1). Show that the kerne) of I i infin ite-dimensional. 35. Consider a linear Lransformation
27. Let A be an m x n m atrix with rank (A) = n. Show that the linear transforma- T : V ---+ V,
tion L (x) = Ax from II(" to im(A) is an isomorphism. Show that the inverse
is L - 1 (}i) = (AT A) - 1 Ary. where \1 is a finite-dimensional linear space. How do you think the deten ni-
nant ofT is defined ? Explai n.
28. Let A be an 1n x n mat:Ii x. Show that the line~ tran fo rm ati on L (x) = _A-r fr~~
im (AT) to im(A) is an isomorphi m. Is the Im ear transformat1on F(y ) = A y
from im(A) to im(A T) nece aril y the inverse of L?
INNER PRODUCT SPACES
29. a. Let T be a linear tran for mation from V to \1. where V is a finite-
dimensional complex linear pace. Show that the transformation T ha
comp1ex eigenvalues. In the last two sections, we foc used on those concepts of linear algebra that
b. Consi der two matrice A and B in C" " such that AB = BA. Show that A can be defi ned in terms of linear combinati ons alone, i.e., in term of surns and
and B have a con.unon eigenvector in C". Hint: Let V be an eigenspace scal ar multiples. Other important concepts relating to vectors in IR" are defined in
for A . Show that Bx is in V fo r all .X in II. Thi implies that we can terms of the do t product: length , angles, and orthogonality (orthogonal projections,
define the linear transfom1 atio n T (x) = B X. fro m II to \1. orthonormal bases, orthogonal transformations).
It is sometimes usefu l to defi ne a product analogaus to the dot product
30. Let C be the set of all rotation-dilation matrices [ : -~ ] , a subspace of in linear spaces other than lR" . The e generalized dot products are cal led inner
products.
2 2 Thinkina of C and C as real linear spaces, defi ne an iso morpbi sm
"'
T: C -* C
Definition 9.3.1 Inner products
uch that T (zw) = T (z) T (w). The existence of such an i omorphism implies
An inner product in a linear pace V is a ru le tbat assign a real scalar
that C and C are isomorphic not just as linear space , but as fi elds.
(denoted by (f, g)) to any pair f, g of elements of V, such that the fo Uowing
31. In Exercises 4.3.34 and 6.4.37 we have presented two di fferent ways to think propertie hold for all f g, h in V, and all c in IR:
about the quatemion s. Mimic the approach of Exerci e 30 to show that the
two sets H and lHl carry the same structure as far as additi on and multiplication a. (f, g) = (g, f)
are concerned. b. (f + g , h) = (f. h) + (g, h)
32. Consider a basis .f1, ... , f" of P"_ 1 • Let a 1, . .. , a" be di tinct real numbers. c. (cf. g) = c(f. g)
Con ider the n x n matrix M whose i j th entry is jj (a;). Show Lb at the matrix d. (j, f) > 0 for all nonzero f in V.
M is invertible. Hint: If the vector
A linear space endowed with an inner product is cal led an inner p roduct
space.

The prototype of an inner product space is II(" with the dot product: (ü, w) =
ü . u) . Here are some other exampl s:
is the kernel of M , then the polynorn.i al f = c 1 f 1 +· · ·+c,,f" in P"_ J vani she EXAMPLE t ~ Con s ider the linear space C[a. bJ con i ting of all con.tinu ou fun ction who e
at a 1, • • • , a"; therefore, f = 0. domain i the closed intervaJ [a. b J, where a < b. See F1 gure 1.
512 • hap.9 LinearSpaces Sec. 9.3 Inner Prod uct pa ce • 513
. .Thi . approxi~ation shows th at the inn er product (f, g) = J:' .f(t g(t ) dt for
functions JS a contmuous version of the dot product: the more subd ivisio ns you
choose, the better the dot product on the right wilJ approx im ate the in ner product
(f ,g ). ~

EXAMPLE 2 .... Let e2 be the space of aLt "sq uare-summabl e" infinite sequence , i.e ., seq ue nces
X = (Xo , X 1 .x2 , ... • X 11 , • • •

Figure 1
uch that I: xl = xö + x~ + · · · converges. In this space we can defin e the inner
i=O
product

For fu nctions f and g in Cla , b], we define (x, y) = L X;y; = X o )'O + X I YI + · · ·

(J, g) = lb f( t )g(t ) dl.
i =O

(show that thi s se1ies converges) . T he verifi catio n of the axioms i

Ca mpare with Exercises 4. 1.1 8 and 9.1.1 5.
traightforward .
~
The veri fication of the fir t three ax iom fo r an inner prod uct i traightforward .
EXAMPLE 3 .... In IR"' x11 we can defi ne the inner product
For e ample,

(f, g ) = lb .f(t)g(r) dr = ib g (t).f(t ) d t = (g, f).

(A B ) = trace (ATB ).

We wi ll verify the fir t and the fourth ax iom .

T he verifi cation of the last axiom requires a bit of calcu lus. We leave i t as (A , ß ) = trace (Ar B ) = trace((Ar ß )T ) = trace (B r A) = (8 , A)
Exercise I .
RecaJJ that the Riem ann integral J:
.f(t)g (t ) dt i the Iimit of the R iemann To check that (A, A) > 0 for nonzero A write A in terms of its columns:
111
um L .f (tk) g (rd ßt , where the tk can be chosen as eq uall y paced poin ts i n tbe
i= l
interval [a , b]. See F igure 2 .
T hen
A= V) li2

(f, g) = 1
b
.f(t )g(t ) dt ~ b
111

.f(tk) g(tk) M =
f((12)
.{
.:
lJ J· [ : J) g(t
g (t2)
J)
ßt
([
J Um ) g(t", )

fo r !arge m.

(A, A ) = trace(ATA) = trace II

Figure 2
__.
---
/ j(l)

__. -T
VII

.,....--------- / g(t)

(l .
• 111 ~~ ~ 1 2
= trace
514 • Chap. 9 Lin ear Spaces
Sec. 9.3 In ner Product Spaces • 515
If A i no nzero, the n at lea t one of the Ü; i nonzero, and the sum !lli , J1 2 + Solution
II id~ + · · · + II Ü11 JI 2 is po itive, a de ired . ~
2
We can in troduce the ba ic conc pts of geometry for an inner product sp ace (f,. g) = fo
[ " Sln
. (t) cos(t) d t = [ J
1 sin2 (t) 12Tr =0
exactl y as we did in IR11 for the dot product. 2 0

EXAMPLE 6 .... Find the distance of f( t ) =t and g(t) = 1 in C[O, 1].

Definition 9.3.2 Norm, orthogonality
Solution
The norm of an eleme nt f of an inner product pace is
11111 = Ju. .n. dist(f, g) =
Two elements j. g of an inner product pace are ca lled orthogonal (or per-
pendicul ar) if
(f,g) =0. . T~e results and procedures discussed for the dot product generaUze to ar-
bJ trary mne~ product space . For example. the theorem of Pythagoras holds; the
~ram-~chnudt process can be used to construct an orthonormal ba is of a (finite-
We can defi.ne the distance of two elements of an inner product pace as the dunen JOnal) mner product space· and the Cauchy-Schwarz inequality teU us that
norm of their difference: l(f, g)l S ll fll llg ll for two elements f , g of an inner product space.
dist(f. g) = II! - gll + Orthogonal Projections
Consider the space C [a , b ]. with the inner product defi ned in Exa mple I .
In physics, the quanti ty 11 f 11 2 can often be interpreted as energy. For exa mpl ~, In an inner product space V , consider a subspace W with orthonormal basis g ,
1
it describes the acoustic eneray of a periodic sound wave f( t ) and the elast1c . . . , g"'. The orthogonal projection proj w.f of an element f of V onto W is
potential energy of a unifo rm s~ring with verti cal displa~ement f(x) (see Fi gure 3). defined as the unique element of W such that f - proi f is orthoaona1 to w
. J IV . e ·
The quantity 11!11 2 may a1 o measure thermal or electnc energy . As 111 the case of the dot product in !Rn, the Oitbogonal projection is given by the
formula below.
EXAMPLE 4 .... ln the inner product space C[O, 1] with (f,g) = j 01 f(t)g( t ) dt , find 11!11 fo r
f(t) = t 2. Fact 9.3.3 Orthogonal projection

Solution lf g , , .. . , 8m is an orthonormal basis of a ubspace W of an inner product

space V, then

11111 = Ju,n = jfo' l

4
dt = [f
for all f in V.

EXAMPLE 5 .... Show that f(t) = sin (1) and g(t) = cos(t ) are perpendi cular in the inner product
space C[O, 2JT] with (f, g) = J~rr f(l)g(l ) d t. (Verify this by checking that (.f - proj w.f, g;) = 0 for i = 1. . .. , m.)
We may think of proj IV f as the element of W closest to f. In other words,
if we choose another element h of W , then the di stance between f and Ir will
Figure 3 Displacement fix) exceed the di tance between f and proj w f.
As an example, consider a ubspace W of C[a . b], with the inn er product
introduced in Example 1. Then proj IV f is the function g in W that i clo est to
Vertical
f, in the sense that
di splacemen t at x A string attached
at (a, 0) and (b, 0)

a
I di t(.f, g) = II! - gll = 1" (f(t) - g(t) ) dt
2

X b
is least.
516 • Chap. 9 Li near Spaces
Sec. 9.3 Inner Product Spaces • 517
Solution

We need to find
. .
pro· f
~P1 ·
we fi
rst find an ortbonom1al basis of P for tbe
~;aen Smnher pdroduct, then we will use Fact 9.3. 3. ln general , we have t~ use the
B m- c hrru t process to find an rth ononna1. ba IS ·
°· .
of an mner product space
th at. IS ,
ec~u e t e two function s I, I in the Standard basis of P 1 ace already orthogonal.
,

Figure 4a Discrele Ieast-squares

(1 , I ) = r'
}_ I
f d1 = Ü.
condition: L:~ 1 (b, - g (a,)) 2 is
minimal. we merely need to divide each function by it norm:

The requirement that

11111 = II: 1 dt = -/2 and li l/l =jr
12
1- 1
t dt = Jf
. 3

An orthonormal basis of P1
be mimmal is a continu.ous Ieas t-squares condition , as opposed to the di crete
lea t-squares cooditions we discus ed in Section 4.4. We can use the di screte

~t.
Ieast-squares condition to fit a function g of a certain type to some data points 1
-1 and
(ab bk) , while the continuous least-squaces condition can be used to fit a function g .Ji
of a certain type to a given function f ("function of a certain type' are frequently
polynormals of a certain degree or trigonometric function s of a certain fom1 ). See ow
Figures 4a and 4b_
We can think of the continuous least- quares condition as a limüing case of
. 1 3
prü]pJ = 2 (l , f ) l + 2 (' f )l
a discrete Ieast-squares condition by writing
I
b m = - (e - e- 1) +3e- 11 (We orllit the Straightforward computations.)
la
2
m 4- L
k=l
2
(! (t )- g (t )) dr = !im "'(/ (tk) - g(td ) ßt .
See Figure 5.
2

EXAMPLE 7 ..... Find the linear function of the form g (c) = a + br

that be t approximates tbe
function f (l) = e 1 over the interval from - 1 to 1, in a continuous least- quares
sense.

Figure 4b (ontinuous Ieast-squares Figure S

condition: t !f!n - g( tJF dt is fl..l)
minimal.

fl..t) - g(f) projp,.f

.____""
'

- I :' I
a b
518 • Chap. 9 Linear Space Sec. 9.3 Inner Product Spaces • 519
What follow one of the major applicati on of thi theory . These equations tel l usthat the functions l , sin(t), cos (r), . .. , si n(nt ), cos(n.t) are
orthogonal to one another (and tberefore li.nearly iodependent).
+ Fourier Analysis' Another of Euler' s identities teils u that
In th pace C[ - TC, TC] we introduce an inner prod uct that i a light modi ficatio n
of th definition given earlier:
(/, g) =;l 1" - 71 l(t)g(t) dt
i: 2
sin (mt) dt = i: 2
cos (mt) dt =TC ,

for positive integer m. This means that the functions sin(t), co (t), ... , sio (nt ),
The factor 1/TC i introduced to facilitate the computation . Convince yourself co (nt ) ar~ of nonn I witb respect to the given inner product. This is why we
that this is indeed an inner product (compare with Exercise 7) . chose tbe Inner product as we did, with the factor .!. .
More generally, we can consider this inner product in the space of ~I piece- The norm of the fu nction l(t) = 1 i "
wise continuous fimctio ns defined in the interval [- TC, TC]. These are functw ns I (t)
that are continuous except forafinite number ofjump-discontinu.ilies, that is, point
c where the one- ided Jimit lim f(t) and lim... .f(t) both exist but are not equal.
r~ c - t---+ c
11111= ; 11"-;r ldt = ·h;
Also it is requi red that .f(c equal one of the two one-sided Iim its. See Fig~re ?·
Fora po itive integer n , consider the sub pace T,, of C[-TC,TC] which 1 therefore,
defined as the span of the functions I , sin(t), cos(t), sin(2t). cos(2t), . .. , sin(nt),
cos(nt). The space T" consists of all functions of the form f(t) 1
g(t) = lll(t)/1 = .Ji ;
f(t) = a + 1; 1 sin(t) + c 1 cos(t) + · · · + b" sin nt ) + Cn cos(nt) ,
is a fu.nction of nonn one.
called trigonometric polynomials of order ::: n .
From calculus you may recall the Euler identiries:

~~: in(pr) cos(mt ) dt = 0. for integers p, m , Fact 9.3.4 Let T,, be the space of alJ trigonometric polynornials of order ::: n , with the

I: sin(pt) sin(m.t) dt = 0, for disti nct iotegers p, m,

inner product

(J, g) =; 11" -71 .f(t)g(t) dt.

J::. cos(pf) cos(mt) dr = 0, for distin t integer p, m.
Then the function
I . .
.Ji' sm (t), cos(t), sm(2t). cos(2!), ... , i n(nt), cos(nt)
Figure 6 fW hos o
jump-discontinuity ot t = c. fo rm an orthonom1al basi of T,, .

For an arbitrary fu nction I in C[-TC JT], we can con ider

c
As we discussed above, J,, is the trigonometric polynomial in T,, that best approx-
imate .f. in the sen e that

1Named after the Fre nch mathematician Jean-Bapti ste-Joseph Fourier ( 1768- 1830). who developecl dist(f. J,,) < dist(f, g)
the subject in his Theorie analytiqrre de Ia clwleur ( 1822), where he investi gated the conduction of heat
in very thin s heet~ of melal. Baron Fourier was also an Egyptologist and govern ment admini strator; he
accompanied Napo leon on hi s ex pedition to Egypl in 1798. for al l other g in T".
520 • Chap. 9 Linear paces Sec. 9.3 Inner Product Spaces • 521
We can u e Fact 9 . .3 and 9.3.4 to find a formula for f,, = pro_ir,.J.
A, [ I Piano

35
F ac t 9.. Fourier coefficients • k
I 2 3 4 5 6
If f i a piecewise continuou functio11 defined 011 the i11te rv al [ - n. ;r ] , then
it best approximat ion /" in T" i
Violin
.f;,(r) = proj 7J(t) = ao ~ + b , sin(l) + c, co (l ) + ··· A, [ II
• k
+h1, si11 (n1) + c" cos(nf), I 2 3 4 5 6
Figure 7 Figure 8
where
bk = (f(r). in kt)) = -
l f ;r f(l) in (kt) dr.

ck = (f(r) os kr )) =
lf

~ L:
- rr

f( r) co (kr) dr.
frequencies that are integ~r mu.l tiples of the fundrunental frequency) give us the
other ter.ms of th~ harmoruc senes. The quality of a tone is in part determined by
the .relattve runphtudes o!
t?e harmonics. When you play concert A (440Hz) on

a0 = f(r), -
I ) = M
l f ;r .f(t) dt. a p1ano the first harmomc 1s much more prominent tban the higher ones, but the
same tone playe? on a vio~n . gives p~ornin~nce to higber harmonics (especially
( ..J2 ...;2n -n
the fifth): Se.e Ftgure 8. S~~ constderat:J.ons apply to wind instrument ; they
The bk> the ck. and a 0 are call ed the Fourier coefficients of the fu nction f. have a v1bratmg colunm of rur mstead of a vibrating string.
The function . The human ear carmot hear tones whose frequencies exceed 20,000 Hz. We
fn(t) = ao ~ + b 1sin r) + c 1cos(t) + · · · + h 11 sin(nr) + c" cos(nl) p1ck up only finitely many hru1nonics of a tone. What we hear is the projection
. ....;2 of f (t) onto a certain T,, .
called the nth-order Fourier approximation of f . EXAMPLE 8 ... Find tbe Fourier coefficients for the function f(t) = t on the interval -rr ::::: t ::::: n.
. 1 ["
bk = (f, s1n(kt)) = ; }_rr sin (kf)t dt
Note that the constant term , written somewhat awkwardly is

ao
1
M
...;2
1
=-
2n
f ;r
- ;r
f(f) dt , ~~ 1- U l cos(kt)t +~{ cos(kt) dt l Ontegration by parts)
2 .

I
which i the average value of the function f between 0 and 2n. It make sense - - if k is even
that the best way to approximate f(t) by a constant function is to take the average k
value of f(t). ~ if k is odd
The function bk sin (k r) + ck cos(kt) is calied the kth harmonic of f(t). Using k
elementary trigonometry, we can write the hat111onic altern atively as 2
-
k
bk sin (kr) + ck cos(kr) = Ak si11 (k( t - 8k)), AU ckand ao are zero, since the integrands are odd functions .

J
where Ak = b~ + c~ is the amplitude of the harmo11ic and 8k is the phase shif1.
The first few Fourier polynornials are:
f 1= 2sin(t)
Consider the sound generated by a vibrating string, suc h a in a piano or 011
a violin . Let f(t) be the air pressure at your eardrum a a fu11ction of timet (the h = 2 sin(t) - sin(21)
function f(t) is meas ured as a deviation from the normal atmospheric pressure). !3 = 2sin(t)- sin(2t) + ~ si n(3 t)
In thi s case, the harmonics have a simple physical interpretation: lbey correspond
to the various si11usoidal modes at which the string can vibrate. See Figure 7. ? 1
14 = 2 sin(t) - si n(2t) +~ in(3t)-
The fundamental frequency (corresponding to the vibration shown at the 2 sin (4t).
bottarn in Figure 7) gives us the.first harmonic of .f(t) , while the overtones (w ith See Figure 9.
 Sec. 9.3 Inner Prod uct paces • 523
522 • Ch ap. 9 Li near pace
Co mbinin oo the last t wo " boxe d , eq uatton
. , we get the fo llowing identity:

Fact 9.3.6 2 + b2I + Cl2 + · · · + b~? + C2 + · · · = 2

00 ll f !1
11

The .in fi nite _ seri~s of the quares of the Fo urier coefficient of a piecew i e
contmuous l un ctiOn f converges to 11 ! 1 2.

For the function f( t ) studied in Example 8, thi mean that

4 + -4
4 + -4 + .. . + ?4 + . . . = -I
9 n- JT
!"
- rr
(2 dt = -2 JT 2
3
or
Figure 9
I I l 1 rr 2

How do the en·ors llf - fnll and II! - f u+ t ll of the 11. th and the (n + l ) th
L ?n- =1 + -4 + -9 + -16 +· · · = 6- '
11 =1

Fourier approximation compare? We hope that f u+l will be a better approximati on an equ atio n cl i covered by Euler.
than f 11 , or at lea t no wor e: Fact.9.3.6 h.as a ph.ysical interpretati o n w hen 11! 11 2 represent e nergy. F or
II J - fn + I II S II f - J,, II exampl e, 1f f (x) IS the di splacement of a vibrating tring then b~ + c~ repre ents
~he energy of the kth h ~m o ni c, and Fact 9.3.6 teil s us that the total energy 1 1.fll 2
Thi s is indeed the ca e, by definitio n: f 11 is a po lynomial in T,,+ 1, ince T,, is 1s the um of the energ1es of the harmonics.
cont ained in T,,+ l, and . There is an interesting appli cation of Fourier analysis in quantum mechan-
II f - fn+ t ll s II f - g II. tcs. In the . 1920s quantum mecbanics was presented in two quite distinct forrn s:
Werner Het enberg ' s matri x mechanics and Erwin Schrödinoer' wave mech anic .
for all g in Tu+ l • in particular for g = f,,. In other words a n goe to infi nity,
Sch~·ödinger (1887-1 9? 1) later .sh?wed that the two theori~s are mathe matically
the error II! - f,,ll becomes maller and smaller (or at lea t not !arger). U in g equt valent: they use tsomorphic tnner product spaces. Hei enbero work w ith
somewhat advanced calculus, one can how that thi error ap proache zero: the space f2 introduced in Example 2, while Schrödinger works wftb a fun ction
space. related to ~[ -JT , rr]. The isomorphism from Schröd.inger' pa e to 2 is e
established by tak mg Fourier coeffic ients (see Exerci e 13).
What does this teU us about lim 111;,11? By the theore m of Pyth ago ras, we
"""""'
have

2 EX E R C I S E S
As n goes to infinity, rhe fi rst summand , II f - f 11 11 , approaches 0 , so th at
GOALS Use the idea o f an inner product, and apply the ba ic result derived
lim
n~ oo
11 !" 11 = 11!11 . earli er for the dot product in !R" to inner product spaces.

We have an expansio n of f in terms of an orthonormal basis:

1. In C [a. b ], define t.he product
11

1
f,, = a 0- - + b 1 sin (t) + Ct co (t ) + · · · + bu in (n t) + Cn cos(n t ), (f, g ) = 1b f( t )g(t ) dt .
-/2
where the b b the ck. and a0 are the Fourier coefficie nts. We ca n ex pre. s II !"II in Show that this product sati sfie the property
terms of these Fourier coeffi cients, usin g the theorem of Pythago ras : (J, f) > 0

II J,,ll 2 = a~ + bT+ CT+ · · · + b~ + c;, I· for all nonzero f .

524 • Chap. 9 Linear Spaces
Sec. 9.3 Inner Prod uct Space • 525
2. Does the equation True or false? If f · · .
f f h . IS a contmuous even functiO n and g is a con tinuo u odd
(.f, g + h) = (.f. g ) + (f, h ) unc wn , t e n fa nd gare orthogona l in C[- l 1] E I .
10. c · . , . . xp a m .
hold for all elements f, g , h of an inner product pace? Explain. onsJc1er the space p2 with inner product
3. Consider a matrix S in ~" x n . ln ~~~, de-11 ne tbe product
(.r , ji) = (Sx )TS.y .
u.g) = ~ 1: f (t )g(l ) dt .

a. For which choice of S is this an inner product? Find an orthonormal basi of the space of all function s ·,n
f (t ) = 1. Pz orthogonal to
b. For which cboices of S is (x, ji) = .t · y (the dot product)?
11. !he angle between two nonzero element
4. In ~m x u , consider tbe inner product JS defined a v and w of an inner prod uc t space
(A , B ) = trace(AT B )
4-(v, w) = arcco (v, w)
defined in Example 3. llvll llwll .
a . Find a formula for this inner product in ffi."' x 1 = ffi."' . In the space C[- rr , rr] with inner product
b. Find a formula for thi inner product in ~ 1 x u (i.e., the space of row veclor
with n component ).
5. Is ((A. B )) = trace(ABT ) an inner product in ~m " ?
(f, g) = ;1 1"-;r f(t)g( t ) dt.

(The notation (( A , B)) is chosen to di stinguish this product from the one con- fi~d the angle between f(t ) = cos(r) and g(t) = cos(t + 8), where 0 < 8 < rr
sidered in Example 3 and Exercise 4.) ~tnt: Use the formu la cos(t + 8) = co (1) cos(8) - sin (t si n(8). - - .
6 . a. Consider an m x n mat.rLx P and an n x m matrix Q. Show that 12. Fmd all Fourier coefficient of the ab olute value function

trace( PQ ) = trace(QP) . f(t) = lt l.

b. Campare the two inner product in ~m x u below: 13. For ~ function f in C[ -rr , rr_] (with .the inner product defined on p age 518
coos1der the sequeoce of all 1t Founer coefficients,
(A, B ) = trace(A T B
(ao. b, , c, , b2 , c2, .. . , b" , c", .. .).
and
this infinite sequence in f2 ? lf so, what is the relationship betwee n
((A , B)) = trace (A BT )
ll.f II (the norm taken in C[ - rr. rr])
(see Example 3 and Exercises 4 and 5 ).
and
7. Consider an inner product (u , w} in a space V , and a scalar k. For which
choices of k is (the norm taken in 2 )?

((u, w}} = k(v, w) (The inner product space e2 was introduced in Examp]e 2.)
14. Which of the following i an inner product in P 2 ? Explain .
an inner product?
a. (f, g) = f(l)g(l) + f 2)g(2)
8. Consider an inner product (u, w} in a space V. Let w be a fixed element of
b. ((f,g)) = f(l)g(l) + f(2)g(2) + f(3)g(3)
V. ls the following transfonnation linear?
15. For which choices of the constants a, b, c, and d is the followin cr a n inner
T: V -r IR given by T(v ) = (u , w) product in JR 2 ? ""

Wbat i its image? Give a geometric interpretation of its kerne!.

9. Recall that a function f(t) from ~ to ~ is called
(L;~] ' [ ~~ ]) = ax1 Y1 + bXIJ'2 + CX2 )11 + dx2y2
even if f( -1) = f(t), for all t, 16. a . Find an orthonormal ba is of the space P1 with inne r product

and

odd if f( - t) = - f(t) , for all t.

(f,g) = 1' f( t g(t ) dt .

(con tinued)
526 • Ch ap. 9 Li nea r Spaces Sec. 9.3 Inner Product Spaces • 52 7
b. Find the linear polynomi al g r) = a+br that be t approx.im ates the function For three polynornials f , g, f1 we are g1
·ven tbe fo Uowtng
. .
mner prod ucts:
I (t) = 1'2 in the intervaJ [0. 1] in the (conti nuous) least-. quares en e.
(, ) I g h
Draw a sketch.
17. Consi der a linear space V . For which linear transfo rrnations I 4 o 8
g 0 1 3
T : V~ IR"
h 8 3 50
1-
(v. w) = T (v) · T (w) For example, (f, f) = 4 and (g, h) = (h, g) = 3.
a. Find (f, g + h).
dot product b. Find \lg + hll-
an inner product in \1? c. Find proj Eh where E = span f g). Express your solution a linear
combinations of f and g.
18. Consider an orthonormal ba is B of the inner p roduct space V. For an element
I of V, what is the relationship between II !II and II [.f]e II (the norm in lR" d. Fin? an o_rthononnal ba i of span(f, g , h ). Express the functions in your
basts as linear combinations of f, g, ancl h .
defined by the dot product)?
25. Find the norrn 11-t ll of
19. For which 11 x 11 mauices A is
(li . ÜJ) = VT AÜJ
x = (I , ~ ~· ...,~ · .. .) in e2 .
an inner product in lR"? Hint: Show first that A must be symmetric. Then
give your answer in terms of the definiteness of A . (e2 is defi ned in Example 2).
20. Consider the inner product 26. Find the Fourier coefficient of the piecewise continuou function
( V,~ W- ) = -T
V [5 2]-
2 8
W
f (I) = { -I !f l < 0
1 lf t ~ 0.
2
in ~ (see Exercise 19).
a. Find all vectors in JR2 perpendicul ar to [ ~] (with respect to th i inner
Sketch the graphs of the fir t fe w Fourier polynornials.
27. Find the Fourier coefficient of the piecewise continuou function
product). Draw a sketch.
v
b. Sketch all vectors in JR2 with lliill = 1 (with respect to thi inner product). f(t) = { 0 ~f t < 0
21. Cousider a positive defi nite quadrati c form q (,t) in !Rn. Doe the formu la . [ Lf t ~ Ü.

(ü, w ) = q (ü + w) - q ü)- q (w ) 28. Apply Fact 9.3.6 to your an wer in Exerci e 26.
define an inner product in lR11 ? How can you tell ? 29. Apply Fact 9.3.6 to your an wer in Exerci e 27.
22. If .f(l ) is a continuou function, what is the relationship between 30. Consider an ellip e E in JR 2 whose center is the origin. Show that there is an
inner product (· , ·) in IR2 suchthat E con i ts of all vectors with !lx ll = 1, x
where the norrn is taken with re pect to the inner product (·, . ) .
and
31. Gaussian Integ ration: In an introductory calculus cour e you may bave een
approximation fonnulas for integrals of the form
Hint: U e the Cauchy-Schwarz inequality.
b
23. In the pace P1 of the polynomi al of clegree ::: 1, we define the inner procluct ll

(f, g) = ~ (!(ü)g(O) + j(l )g (l)). 1a j(r) dt ~ .{; w;J(a;) ,

where the a; are equally spaced points in the interval (a , b), and the w; are
Find an orthononnal basis for this inner product space.
certain "weights" (Riemann sum , trapezoidal sums, Simpson 's rule). Gaus
24. Consider the linear space P of all polynomials, with inner product has shown that with the same computational effort we can get better approx-

(f, g ) = 1 1
f (r)g (r) dt.
imalions if we drop the requirement that the a; be equ ally paced. BelO\.v we
discuss bis approach.
528 • hap. 9 Linear Spaces
Sec. 9.4 Linear Differential Operator • 529
Con ider tlle pace P" with the inner product
Verify that a linear differential operator is indeed a ünear tran Formation
(exercise).
{f, o) = [ I j(l g(l ) d t .
Examples of linear differential operator are
L
et Iio,I f . · · · ·Jr" be an orthonormal ba is of thi·d spacetl , with degree(ji,. ) =
t ddb- I D (f) = f ',
k (to· con truct u 11 a ba 1·s~ . appl y Gram-Schn11 t ro 1e a n ar d· 1 h ,
t , ... , r") . lt can be shown rhat f 11 has n di srinct root. a 1, a2 . . .. , a" 111 t e
T (J) = !"- S f' + 6f. and
interval ( - 1, 1). We can find . weight " wl , W2 , .... w" uch th at L(f) = f"' - 6 f" + SJ,

(*) [
I

1
j(t ) dr = 8 "
w;j(a 1),
of first, second, and third order, respecti vel y.
tf T i an nth-order linear differential operator and g is a smooth function
then the equation
for a11 polynomial of degree less than n (see Exerc i ~ 9.2.33). In fact, much
more is true: The formula (* holds for all pol ynol11J a1s f (t ) of degree less T(J ) =g
or
than 2n. · b' b k
You arenot asked to prove t11e a ertion above for ar Jtrary n , ut wor
out the case n = 2: find 0 1, 0 2 and w 1, w 2 , and how that the formu la ! (") + an - 1f tll - 1) + ... + a l ! ' + aof = g
1
is called an nth-order linear differential equation (OE). The OE is called homo-
[ j(1) d t = w 1j(a 1) + w2f(a_) geneous if g = 0, andinhomogeneaus otherwi e. Examples of linear OE' s are
1
holds for all cubic polynorruals. !" - f'- 6f = 0 (second order, homogeneous)
and

f '(t ) - Sf (t ) = sin(t) (first order, inhomogeneou ).

LINEAR DIFFERENTIAL OPERATORS
Note that solving a homogeneaus OE T (J) = 0 amount to finding the kerne! ofT .
In this final section, we will study an important class of linear tran sformations We wiU first thin.k about the relationship between the solutions of the OE'
from c to c . Here C 00 denotes the linear space of complex-valued smooth T (J) = 0 and T(J ) = g.
functions (from JR to q , wbich we consider as a linear space over More generally, consider a linear transformation T from V to W where V
and W are arbitrary linear paces. What is the relationship between the kerne! of
T and the solutions f of the equation T(J ) = g, provided that this equation has
Definition 9.4.1 Linear differential operators
olution at all (compare with Exerci e 1.3.48)?
A Iransformation Here is a simple example:

~ C00
~ J.:r
T:C
EXAJ\tfPLE 1 .... Consider the linear transformation T (.r ) = [ ~ ; from 3 to JR2 . Oe cribe
of the fonn
rhe relationship between the kernel of T and the solutions of the linear ystem
T(f) = j 111 ) + a11 _ J! (II - IJ + · · · + aJ!'
is called an 11th-order linear differential operator. 1•2 Here
+ aof
f (k) denotes the kth
T (x ) = [ ~
1 J
both algebraically and geometrically.

derivative of J , and the ak are complex scalars.

Solution
Using Gauss-Jordan elirrunation, we find that rhe kerne! ofT consi ts of aJI vector
of the form
--~More precisely, this is a linear differential operator wirh consranr coefjiciems. More advanced texLS
consider the case when the ak are funclions. .. . . . . . of
2The term "operator'' is often u ed for a Iransformation whose domam and codomam consJst
function s.
 Sec. 9.4 Linear Differen tial Operat ors • 531
530 • Chap. 9 Lin ear Spaces . Note tha.t T (f) ~ T(cJ/, + · · · + c11 f 11 ) + T (fr,) = 0 + g = g , so that f is
mdeed a solutLon. Venfy that aU solutions are of tb.is form .
-olulions of

T(i )= Ln What i ~ the s!gnificance of Fact 9.4.2 for linear differenti al equati ons? At
the end of thts sectLOn we will demoostrate the following fundamental re ult:

Fact 9.4.3 The kerne] of an nth-order linear differenti al operator is n-dimen ional.
kernet ofT

. Fa~t 9.4.2 now provides us with the following strategy for solving Linear
dtfferent.J al eq uations:
Figure 1

.F act 9.4.4 To solve an nth-order linear OE

with basis T(f) = g
we have to fi nd
a. a basi ! 1 ... , fn of kernel (T), and

The solution set of the system T (x)

- = [ 6] ·12 con
1
t
of all vectors of the b. a particular solution /p of the OE.
Then the solutions f are of the form
form
f = cJ!J + · · · + Cnf,, + JP,
6-2x-3x3] [-2]+ .[-3]+
2

[ X2
X3
= X2 l
0
X3 0
1
a particular olution
where the c; are arbitrary constant .

a vector in the
kerne! of T of the sy. tem EXAMPLE 2 .... Find all olution of the OE
T(x) = [ ~~} J"( c) + f( c) = e'.

We are told that / p(t) = ~e' is a particular olution (verify tb.is).

The kerne) of T and the solution set of T (x) = [ 1 ~] fo rm two parallel
...
planes in JR3 , as shown in Figure 1. Solution
Consider the linear differential operator T f = f '' + f. A ba is of the kerne! of T
These observations generalize as follows : is f 1 (t ) = cos(r) and h(t ) = sin(r) (compare with Exercise 9.1.50).
1
Therefore, the olutions f of the OE ! " + f = e are of the form

Consider a linear transformation T from V to W , where V and W are arbitrary . I

f(t) = c, cos(r) + c2 m(t) + 2e 1

linear spaces. Suppose we have a basis f ,, h, .. . f " o~ the kernel of T ·

Fact 9.4.2 ,

Consider an equation T (f) = g with a particular solutLOn fp · Then the

where c 1 and c2 are arbitrary constant .
solutions f of the equation T (f) = g are of the form
f = e lf, + c2f2 + · · · + Cnf,, + fp , We now present an approach that allows us to find olutions to homogeneaus
where the c; are arbitrary constants. linear OE' more systematically .
532 • Chap. 9 Linear Spa s
Sec. 9.4 Linear Differential Operators • 533
• The Eigenfunction Approach 4
to Solving Linear DE's EXAMPLE ... Find all exponentia1 function e;. 1 in the kernet of the linear differential operator
T (.f) = .f" + J' - 6f.
Definition 9.4.5 Eigentunetions Solution
Con ider a lin ear different ial operator T from C to C . A s mooth functi on
f is called an eigenfunction ofT if T (f) ~ f..J f?r ome .compl ex . calar f..; The characteri stic polynomial is Pr (J..) = ).. 2 +)... - 6 = (.A + 3) ).. _ 2), witb root
this scaJar)... is caJJed the eigen alue assoc1ated w1th the eJgenfunctJon f. 2 and -~. Therefore, the functions e21 and e- 31 are in the kerne] ofT. We can
check th1s:

E~IPLE 3 ... Findall eigenfunction and eigenvalue of the operator D (J) = f' .
and
Solution
We have to solve the differential equation
S_in~e rno t po lynomials of degree n have 11 distinct complex root , we can find n
F= f..J. d1 tinct exponential function e/. 11 • • • , eA. 1 in the kerne] of most nth-order linear
We know that for a ~iven )... the solution are all exponential functions of the form d.1 fferentia~ Operators. Note that these functions are linearly independent (they are
f(t) = C eM. This ~ean that all comp.lex nu~bers ar~ eige~walues of D, and tl~ e1genfunctwns of D witb di tinct eigenvalues· the proof of Fact 6.3.5 applies).
Now we can use Fact 9.4.3.
eigenspace a ociated with the eigenvalue ).. 1s one-d1menswnal , panned by e<lllll
(cornpare with Example 9.l.25 .
It follows that the exponential functions are eigenfunctions for all linear Fact 9.4.8 Con~ider an nth-order linear differential operator T who e cbaracteristic poly-
differential operators: if nonual Pr (J..) has 11 disünct roots .A 1, ••• , An. Then the exponentia1 f unctions

T(f ) = J <n) + a"_tf (n- l) + · · · + a1J' + aof,

then form a basis of the kernet of T , that is, a basis of the solution pace of tbe
homogeneaus DE
11
T(e · ) = (f.." + Dn- IA."- 1+ · · · + a1f.. + ao)e 1 J.

T(f) = 0.
This ob ervation motivates the following definition:

See Exerci e 38 for the case of an n th-order linear differential operator whose
Definition 9.4.6 Characteristic polynomial characteristic polynomial has less than n distinct roots.
Consider the linear differential operator EXAMPLE 5 ... Find all solution f of the differential equation
T(J ) = /">+ a" _ J/ <n- l) + · · · + a1 ! ' + aof. ! " + 2!' - 3f = 0.
The characteristic polynomial of T i defined as
Solution
Pr(f..) = A + On- IA
11 11
- l + ... +DIA+ ao.
Tbe characteri stic polynomial of the operator T(J) = !" + -!'- 3f i Pr ,\) =
2
)..+ 2f..31- 3 = (f.. + 3)(A - 1), with root I and - 3. The exponentia l fun tion e1
Fact 9.4.7 If T is a linear differential operator, then e}.J is an eigenfunction of T, with and e- form a ba is of the olution space, i.e .. the solution are of the form
associated eigenvalue Pr (J..) for all A.:
T(e'J ) = p 7 ('J..)eÄ1

In particular, if p 7 ()..) = 0, then e/.1 is in the kernet of T. EXAMPLE 6 ... Find all solution f of the differential eq uation
!" - 6J' + 13! = 0.
534 • Ch ap. 9 Linear Spaces Sec. 9.4 Lin ear Differenti al Operator • 535
Solution f( t) =eP'(cl cos(qt) + c2 sin(ql))
The characteri ti polynomial i Pr (A.) = A. 2
- 6A. + I 3, with complex roots 3 ± 2i.
The exponential functions
et3+ 2ilt = e31 ( cos (2t + i sin (2t))
and
e<3 - 2i l t = e31 ( cos (2t - i sin (2r))

form a basis of the so lutio n space. We may wish to find a basis of the soluüon Figure 2
pace consisting of real-valued functio ns. The foll owing observation is helpful : if
f(t) = g r) + i h(r ) is a olution of the DE T(f) = 0, tben Tf = Tg + i Th = 0,
o th at g and h are solutions as weil. We can appl y this remark to the real and
the imaginary parrs of the solu tion e <3+ 2i) t : the functions What .about . nonhomogeneou. differenti al equations? Let us di scuss an ex-
ample that ts part.Icularly important in applications.
e 31 co (2t ) and e 31 sin(2t)
EXAMPLE 7 .... Consider the differential equation
are a basis of the solution space (they are clearly linearly independent), and the
general solutio n is ! " (!) + f ' (t) - 6 f( t ) = 8cos(2t )

a. Let. V be the linear space consisting of all functions of tbe fom1 ci cos(2t ) +
c2 Sin (2r) ..Show that the linear differential operator T (f) = j" +!' _ 6!
defines an Isomorphism from V to V .
b. Part a. implies. that ~he DE T (f) = 8 co (2t) has a unique particular solution
Consider a di fferen tial equation
.ft,(t ) 10 V. Fmd thi s solution.
Fact 9.4.9
c. Find all solution s of the DE T (f) = 8 cos(2t).
T (f) = !" + af' + bf = 0,
where the coefficient a and b are real. Suppose the zero of Pr (A.) are p ± iq ,
Solution
wi th q =!=- 0. Then the solution of tbe given DE are
a. Consider the matrix A of T with respect to the basi o (2t ) . -in(2t). A
f(t) = eP
1
(c i cos(qt) + c2 sin (qt ),
traightforward computation shows that
where c 1 and c2 are arbitrary constants.
Tbe special case when a = 0 and b > 0 is important in many applica- A= [ - 10 2]
tion s. Then p = 0 and q = -Jb, so th at the solutions of the DE -2 -10 '

!" + bj = 0 a rotation-dilarion matri x. Since A. is invertible, T defines an isomorphism

from V to V .
are
b. If we work in coordinates with respect to tbe ba is cos(2t). sin (_r) the DE
f(t) = Ci cos(.Jbr) + c2 sin (.Jbt ).
T (f) = 8 cos(2t) takes the form A.x = [ ~ l with the solution

ote that the fun ction

1
f( t ) = eP (ci cos(q t) + c2 sin(qt ))
x = A- 1 [ 8 ]= - '
0 104
[-lo -2] [8] [-10/13]
2 -10 0
=
2/ 1 .
The part.icular solution in V is
is the produ ct of an ex ponential and a inusoidal function . Tbe case wben p is
negative comes up frequ ently in physics, when we model a damped oscillator. lO 2
See Figure 2. J~(t ) = -
8 cos (2t) + T3 in(2r).
536 • Chap. 9 Linear Spaces
Sec. 9.4 Linear Differential Opera tors • 53 7
A more Straightforward way to find f 1)(t) is to set f,, (t) ~ P cos 2i) + can be written as
Q sin(2t) and substirute this trial solution into the OE to determme P and Q.
c. In Example 4 we have seen that the functions / 1(t ) = e~r and f2 (!) = e- 31 T = D
2
+D - 6.
fom 1 a basis of the kerne] ofT. By Fact 9.4.4, the so luti ons of the DE are Treating T formally as a polynomial in D , we can wri.te
of the fom1
f (t) = c1/1 (I)+ c2!2Ct) + f~, (t ) T = (D + 3) o (D - 2).

10 2 . 2) We can verif-y that thi s formula gives us a decomposition of the operator T :

= c l e 2t + c2 e-3r - 13 cos(2t ) + ]3 sm ( t .
(CD +3) o (D -2))! = (D +3)(f' - 2f) = f" -2j' +3!' - 6f = (D 2 + D- 6)f.
Let us summarize the methods developed in Example 7:
This works because D is linear: we have D(f' - 2f) = f"- 2f' .
Fact 9.4.10 Consider the linear differential equation The fundamental theorem of algebra (Fact 6.4.2) now teils us the following:
J"(t) + af' (t) + bf(t) = C cos(wt) ,
Fact 9.4.11
where a. b, C, and w are real numbers. Suppose that a # 0 or b # w 2 . This An nth-order linear differential operator T can be expressedas the composite
of n first-order linear differential operators:
DE has a particular solution of the form
T =D" + a"_ 1D" - 1 + · . . + a 1 D +ao
f p(t ) = P cos(wt ) + Q sin (wt).
= (D- A.1)(D- A.2) ... (D- ).") ,
Now use Facts 9.4.4 and 9.4.8 to find all solutions f of the OE. where the A.; are complex nurnbers.

What goes wrong when a =0 and b = w 2?

. We can therefore hope to understand alllinear differential operators by study-
mg first-order operators.
+ The Operator Approach to Solving Linear DE's
EXAMPLE 8 ~ Find the kerne! of the operator T =D - a where a is a complex number. Do not
We will now present an alternative, deeper approacb to DE's, which _al~ows us to use Fact 9.4.3.
solve any linear OE (at least if we can find the zeros of the charactenstJc p~lyno­
mial). This approach will Iead us to a better understanding of the kerne! and 1mage Solution
of a linear differential operator; in particular, it will enable us to prove Fact 9.4.3.
Let us first introduce a more succinct notation for linear differential operators. We have to solve the homogeneaus differential equation T (f) = 0 or j'(t ) -
Recall the notation Df = f' for the derivative operator. We Iet af(t) = 0 or f'(t) = af(t). By definition of an exponential function, the solutions
are the functions of the form f(t) = Cear, where Cis an arbitrary constant. (See
Dm = D o D o ... o D Definition 8.2.1). ~
m times
that is, Fact 9.4.12 The kerne! of the operator

T= D-a
Then the operator is one-dimensional, spanned by
T(f ) = j <nl + a" _IJ<n- l) + · · · +aJ/' + aof f(t) = ea'.
can be written more succinctly as

T = D 11 + a.n - 1D" -
1
+ ·· · + a. 1 D + ao, Next we think about the nonhomogeneaus equation

the characteristic polynomial PrO.) "evaluated at D". (D - a)f=g

For example, the operator or
T (f) = f" + 6 j' - 6 f'(t) - af(t) = g(t) ,
538. hap. 9 Linear Spaces
Sec. 9.4 Linear Differentia l Operato rs • 539
where g(r) is a smooth fu nct10n.· 1~ w ill- atturn out to
· be u eful to multipl y both
We can break thi s DE down in to n fi rst-order DE 's :
ide of t.hi s equation wi th the fu nctwn e
D->.,. D-).,. _1
e-fll j'(r) - ae- fll f r) = e- nt g(r)
f ~ Jn - i ~ fll -2
. the left- h an d J.de of thi equation as the deri vaLive of the fun ction
We reco 0omze .
e-m j(r), o that we can wnte We can successive ly solve the first-order DE's
(e - at f(l)) ' = e - at g(t . (D - A. ,) j, = g
O\ we can integrate: ( D -A.2)h = Ii

(D - AII - I) f n- l = f n- 2
(D - A" ) j = f n- l·
and
j(r) = e"' I e - a'g(t ) dt , In particul ar. the DE T (f ) = g does have solutions f .

where Je-a t g t ) dt denote the inde~ nite i?tegral , that i ' rhe fa mil y of all aJ1li-
deri ari ve of the fu nction e - at g(r), lllvolvwg a param eter C. Fact 9.4.14
The image of aU linear differential Operators (fro m C to c ) is c::c , that
is, any li near DE T(f ) = g ha solutions j.
Fact 9.4.13 Con ider the differential equation
f' (r) - af (t ) = g (r),
EXAMPLE 10 .... Fi nd all soJutions of the DE
where g(t ) is a smooth funcüon and a a constant.
Then
j (r ) = e111 I e -at g (t ) dt .
2
Note that Pr (A.) = A. - 2)...
T (f) =

+1=
f"- 2j' + j = 0.

()... - I )2 has onlv one root I o that we cannot

use Fact 9.4.8. • ' '

Fact 9.4. 13 shows that the differential equaüon (D - a)f = g has olutions
f , for any smooth function g ; thi means that Solution
im(D- a) = C00 .
We break the DE down into two first-order DE's, as discussed above.
EXAMPLE 9 .... Find the solutions f of the DE
D- 1 D-l
! ' - aJ = ceat , f ....----....... .f, ....----....... 0
where c is an arbitrary constant.
The DE ( D - l )j, = 0 has the generaJ o lution .f1 (1) = c 1e' , where c is an
1
arbitrary con. tant.
Solution
Then the DE (D - l ).f = .f1 = c ,e' ha the generaJ solu tion f (t ) = e1 (c t +c_),
1
U ing Fact 9.4.13 we find that where c_ i another arbitrary constant ( ee Example 9).

j(t ) = e01 I e- at ce 0 1 d t = ea1 I c d t = e"' (c t + C),

1 1
The functions e and t e form a basis of the solutio n space (i.e., of the kerne!
of T ). Note the kerne! is two-dimen ional. since we pick up an arbi trary constant
each time we solve a first-orcler DE. ..
where C is another arbitrary constant.
Now consider an nth-order DE T (f) = g, where Now we can explain why the kerne! of an n tb-order linear di fferential opera-
tor T is n-dimensional. Roughly speaking, thi i true because the general o luti on
T = D 11 + a11 _ 1 D" - 1 + · · · + a, D + ao , of the DE T(f) = 0 contain n arbitrary con tants (we pick up one each time we
= ( D - A.,)( D - A.2 ) . . . ( D - A11 - I)(D - A.n) . solve a fir t-01·der linear DE).
540 • Chap. 9 Linear Spaces Sec. 9.4 Linear Differential Opera tor • 541
Hereisamore formal proof. We will argue by indu tion o n n . Fact 9.4. 12 d 2x dx
take care of t11e ca e 11 = 1. We can write an nth-order linear d iffere ntial operator
7. J"(!) + J'( t )- l 2f(t) = 0. 8. - + 3- - 1Ox = 0
dt 2 d1 ·
a T = L o (D - A. 11 ), where L i of order n - I : 9. J"(t)-9f( t ) = 0. 10. J"(t ) + f(t) = 0.
d 2x dx
f .......--......,,.......--......o
D -/.. 0 L
11. -dt 2 - ?-
- dt
+2'· - 0
" - . 12. J"(t ) - 4f' (t) + 13f (t ) = 0.
Arguing by induction. we can assum that the kerne! of L is (n - I ) -dimensiona l, 13. f"(t) + 2J'(t) + f(t) = 0. 14. f"(t) + 3f'(t) = 0.
with ba is h 1.h 2 • •• • , h 11 _ 1• Then the ol uti ons f ofthe DE 15 . .f"(l) = 0.
16. J"(t) + 4j'(t ) + 13.f(t) = COS(l ).
17. f "(t) + 2f' (t) + f(t) = sin (t). 18. f"( t ) + 3f' (t ) + 2f(t) =
( D - A11 )j = c J11 + · · · + Cn- lhn - 1 cos(t .
d 2x
are 19. df2 + 2x = COS(l). 20. f"'(l) - 3 f"( t ) + 2f'(t ) = 0.
j(1) = e;.., , J e-J..·'(c 1h, 1) + ··· +c"_,flu - IC!)) dt 21. .f"' (t )+2J"(t )- f'( t ) -2f (t ) = 0. 22. ! 111
(1) - j"(1) -4 j'(t ) +4j(r) = 0.

Let H; 1) be an antiderivative of e_;..,., h ;(t ), for i = 1, ... , 11 - I. Let F;(t) = Salve the initial va lue problems in Exercise 23 to 29.
e>.. , H;(t); note that (D- A.11 ) F; = h ; by constr uction. Then the so.l utions f(t)
above can be written a 23 . .f' (t )- Sj"(1) = 0, j(O) = 3.
dx
f(t) = e }."r(c, H, (t ) + · · · + c"_, H 11 - 1(t ) + C) 24. -d
t
+ 3.x = 7, x(O) = 0.
= c 1 F1 (t) + · · · + C 1 F,,_, (/) + Ce'-•'.
11 _ 25. f'(t) + 2j(t) = 0, f( l) =
I.
26. J"(l)- 9f(t ) = 0, .f(O) = 0, f'(O) = 1.
where C i an arbitrary constant.
1l1i how that ker(T) i spanned by the n function F, (1), .... Fn-1 (t), e/.." 1• 27. j"(1)+9j(t) =0. f(0) =0, f(!}) =l.
We claim that these functions are linearly independent. Consider a relation 28. J"(t) + J'(t) - 12f(1) = 0, f(O) = f'(O) = 0.
29. f"(l) + 4.f (t ) = sin(1), f(O) = f'(O) = 0.
Ci F1 (t) + ·· ·+ C 11 - t F11 - i (t) + Ce/.." 1
= 0.
30. The temperature of a hot cup of coffee can be modeled by the DE
Applying the operator D - A11 on both side , we find that
T ' (t) = -k(T(l)- A).
CJh l (! ) + · · · + Cu -l h u- 1(1) = 0.
a. What i the ignificance of the constants k and A?
We conclude that the c; must be zero, si nce the function s h; (t ) are linearly inde- b. Solve the DE for T(t) , in term of k, A, and the initial temperature To.
pendent; then C = 0 a weil. Hint: There i a constant particular solution .
We have hown lhat the functions F1(!), ... , F11-1 (t ), e /.." 1 form a ba i of 31. The peed v(1) of a falling object can omelimec be modeled by
ker(T), so that ker(T) is n-dimensional, as claimed.
dv
m- =mg - kv
dt
or
EXERCISES dv k
- + - v=g.
GOAL Solve linear differential equation . dt m
where 111 i tl1e mas · of the body, g the g ravitational when acceleration. and
Find all real solutions of the differential equation in Exercise I to 22. k a con tant related to the air res istance. Solve thi DE when v(O) = 0.
De cribe the long term behavior of (!). Ske tch a graph.
dx
1. f'(t)- S f( t ) = 0. 2 . - + 3x = 7. 32. Consider the balance B(l) of a bank account, with initial balance B(O) = Bo.
dt We are withdrawing money at a continuou rate r (in DM/year). Th intere t
dx rate is k (%/year), compounded continuously. Set up a differenrial equation
3. f '(l) + 2f(t) = e31. 4. - - 2.x = cos(3t).
dt for B (t ) and solve it in term of Bo, r and k . 'Nhat will happ n in the lon a
5. J'(t) - f(t) = t. 6. f'( t ) - 2f(t) = e21 . run ? Describe all pos ible sc~::narios. Sketch a graph for B(t in each ca e.
542 • Chap. 9 Linear Spaces Sec. 9.4 Linear Differen tial Operators • 543
33. Con ider a pendulum of lengtl1 L . Let x t ) be the angle tl1e pendul.um .make , 35. The di splacement x(t ) of a cettain oscillator can be modeled by the DE
with tbe vertical (measured in radi ans). For mall angle , the mot10n ts weil
approximated by the DE d 2x dx
d 2x g -1 2 + 3 -d + 2x =0.
- --- r c. I t
dt- - L-
a. Find all solutions of tlli s DE.
(wbere g is the acceleration due to gravi.ty, g ~ 9.81 m/sec ). Ho"':' .long does
2
b. Find the Soluti on with initial values x(O) = I , x' (0) = 0. Graph the
the pendulum have to be o tbat it swings from one extreme po 1t1on to the solution.
oilier in exactly one second? c. Find the solution with initial values x (O) = l , x ' (O) = - 3. Graph the
solution.
d. Describe tlle qualitative difference of the solutions in parts b and parts c, in
tenns of the motion of the oscil lator. How many times will the oscillator
go through tlle equilibrium state x = 0 in each case?
36. The displacement x(t) of a certain oscillator can be modeled by the DE

N ote: x(l) i negati ve
when the pendul um is
on the left.
d 2x dx
- +2- + lülx = 0.
dt 2 dt
The two exLreme posi tion Find all olutions of this DE, and graph a typical solution. How many times
of the pendu lum.
will the oscillator go tlrrough ilie equilibrium state x = 0?
37. The di placernent x ( t ) of a certain oscillator can be modeled by ilie DE

Historical nore: The result of this exerci e was considered a a po · ible d 2x dx

- ? +6- + 9x=0.
definition of ilie meter. The Frencb comm.ittee reforming the measures in the dt- dt
1790's finally adopted another definition: a meter i · the lO,OOO,OOOili part of Find the solution x (r ) for the in.itial values x (O) = 0, x '(O) = 1. Sketch the
ilie distance from ilie Norili Pole to tbe Equator, measured along the meridian graph of tbe solution . How many time will the oscillator go through the
though Paris. equilibrium state x = 0 in this ca e?
34. Consider a wooden block in the shape of a cube, with edge 10 cm. The 38. a. lf p (t ) is a polynom.ial and ). a scalar, show that
density of ilie wood is 0.8 gj cm 3 . The block is submersed in water; a guiding
mechanism guarantees iliat tbe top and ilie bottom surfaces of the block are ( D- A.)(p(t)el 1 ) = p ' (t ) e)J .
parallel to ilie surface of ilie water at all times. Let x (r) be tbe depth ~f the b. If p(r) is a polynom.ial of degree les than m , what is
block in the waterat time t. A ume that x is between 0 and 10 at all umes.
( D - A.t' (p (t )e'1 ) ?

c. Find a basis of tlle kerne! of the linear differential operator ( D - A.) 111 •
d. If A. 1, . .• , A., are distinct calars and rn 1• • • • , m, are positive integers, find
a basis of the kerne] of the linear differential operator
(D- A. 1) 111 ' ••• (D-A.,)'"' .
a. Two forces are acting on the block: it weight and the buoyancy (the 39. Find all olutions of tlle linear DE
weight of the displaced water). Recall iliat the den sity of water is
1 g/cm 3 . Find formulas for these two forces. /
111
(!) + 3/" (1) + 3f'(t ) + f (t) = 0.
b. Set up a differential equation for x(t ). Find the olution, assum.ing that (Hint: U e Exercise 38.)
the block is initially completely submersed (x( O) = 10) and at rest.
40. Find all olution of the linear DE
c. How does the period of the oscillation change if you chaoge the dimen ions
of the block (consider a !arger or smaller cube)? What if the wood has a d 3x d2x dx
- + -2- - - x = O.
different density, or if the initial state is different? What if you conduct dr dr dt
the experiment on the moon? (Hint: U e Exercise 38.)
544 • hap. 9 Linear Space
Sec. 9.4 Linear Differential Operators • 545
41. If T i an nth-order linear differ nüal operator and ). i an arbitrary sca lar,
i ). nec ssarily an eigenva lue of T ? [f so, what i lhe dimen sion of the
b. ~h~w that. the zero state is a table eq uilibriurn soluti on of thi system if
an only If) the real pmt of all the ). 1. is neoati ve
e igen pace associated wirh ). ? 48. Consicler an n x n
ma t n· x A Wit
. h m d1.stmct
. • "' •
e1genvalue ;,_ 1 ).
42. Let C be the s pace of all real-valued smooth functi ons. a . S how that the inüial value problern ' • .. ' m·
a. Consider the lin ar differential operator T = D 2 from C to cco. Find all
(real) eigenvalue ofT. Foreach eigenva lue . find a ba is of the associated dx
e igenspace.
dl =Ai with x(O) = ; 0

b. Let P be the ub pa e f C con i ting of all periodic funcüons f( t ha a unique solution ,r(t).
with period one (that i . .f(t + 1) = f(t), for a ll r). Con ider the linear b. Show that the zero tate i a table equilibrium so lution of the ystern
differential operator L = D2 fro m P to P. Findall (r a l) e igenvalues and
eigenfunctions of L.
dx
-
dt= A .i
43. The displacement of a certain forced oscillator can be modeled by the OE
if and only if the real part of all the A· is negative. Hint: Exercise 7.3 .37
d 2x dx and Exercise 47 above are helpful. '
dt 2 + 5 dc + 6x = cos(t).

a. Find all solution of thi OE.

b. Oescri be the long-terrn beha ior of this o cillator.
44. The di placement of a certai n forced oscil lator can be modeled by the OE
d 2x dx
- . , + 4 - + 5x = cos(3t).
dr- dt
a. Find all solutions of this OE.
b. Describe the long-terrn behavior of tbis oscillator.
45. Use Fact 9.4.13 to solve the initial value problern

c.dtx.r- -- [o1 21 Jx- with .T(Ü) = [ _ IJ.

1
Hint: Find first x 2 (t) and then x 1(t).

!n ul
46. Use Fact 9.4.13 to olve the initial value problern

~~ ~ [ g with X(0) ~
Hint: Findfirst x 3 (t), then x 2 (r), and then x 1 (t).
47. Consider the initial value problem
dx
- = Ai with x(O) = io ,
dt
where A is an upper triangular n x n matrix with m distinct diagonal entries
AJ. . .. , ),m· See the examp les in Exercises 45 and 46.
a. Show that this problem has a unique sol ution x(t), whose cornponents
x;(t) are of the form

x;(t) = PJ(t)e>" r + · ·-+ p111 (c)e>.",r,

for some polynomials PJ (t ).

Hint: Findfirs t x"(t ), then x" _ 1 (t ) and so on.
 App. A Vector • 547
b. Scalar multiplication
The product of a · calar k and a vector ü i defined componentwi e as
weil :

kÜ=k v,v2 J
= kv2 J
rkvl
r~II k~/1
EXAMPLE 1 ....

VECTORS
EXAMPLE 2 ....
Here we will provide a conci e ununary of basic facts on vectors. In Section 1.2,

vectors are detined as matrice with only one column : ü = [ ~ J. The scalars v;

v"
are called the components of the vecto r. 1 The set of all vectors with n components The negative or opposite of a vector ü in ~~~ is defin ed as
is denoted by ~" . -Ü=(- l )Ü.
You may be accustomed to a different notation for vectors. Writin g the
components in a column is the most convenient notation for linear algebra. The dijf~rence ü - wof two vectors ü and ÜJ in ~~~ is defined component-
Wlse. Alternat1vely, we can express tbe di ffere nce of two vector a
ü- w= u+ c- w) .
+ Vector Algebra The vector in IR" that consi ts of n zero i cal led the zero vector in IR.":

Definition A.1 Vector addition

a. The sum of two vectors ü and win ~~~ is defined "componentwise":

Fact A.2 Rules of vector algebra

The followin g formulas hold for all vectors ü v, w 10 " and fo r all calars
_
v+w=
_
v'. J+ [w
V2
.'J [v,+. w
W2
,J V2 + W2 c and k:
[ ~~~ ~II V" ~ Wn
1. (Ü + ü) + w= ü + (Ü + tv) (additi on i associative)
2. ü + w= ÜJ + ii (addi tion i commutative)
3. ii + Ö= ii
1
In vector and matrix algebra. the term "scalar" i synonymaus wi th (real) number. 4. ~or ea~ h ü in IR" there is a unique ;r in IR" such that ü + i: = 0, namely,
546 x = - v.
548 • App. A Vectors

+ iü) = k Ü + kiil
5. k (Ü
App. A Vectors • 549
6. c + k )v = cü + kv
7. c(kv) = (ck)ü
8. 1- = -

The e rule follow from the corre ponding rule fo r calars (commutativity,
Figure 3 The components of o vedor
associativity, ilistributivity); for example, in standard represenlation are the
coordinates of its endpoint.

+ Geometrical Representation of Vectors

The stan.dard representation of a vector R

- [Äl]
x=
-'2
Figure 4 (a) xis a vedor
on the line l.
in the Cartesian coorctinate plane i as an arrow (a ilirected lined segment) con- x
(b) is a vector
necting the origin to the point (x 1 • x 2 ), a shown in Figure I. in the region K. (a)
Occasionally it is helpful to translate (or shift) the vector in the plane (pre- (b)

erving its direction and length), o that it will connect some point (a 1, a 2) to the
point (a r + x 1, a2 + x2) . See Figure 2.
We need not clearly distinouish betw . .
identify them as long as we consi' tentJ ' use e~: a vector and tts end_point; we can
In this text, we consider the standard representation of vectors, unl e we For example, we will talk abou/ "the vec:~~dard represe~tatwn of vector .
explicitJy state that the vector ha been tran lated . mean the vectors whose endpoints are o th r s ot~ a !Jne L when we really
Likewise we can talk about "the t ~ e 1 ~e L (m Standard representation).
' vec or tn a reo 100 R" · th 1 .
Adding vector in JR2 can be representedo b . tn efp ane. See Frgure 4 .
A vector in 1R2 (in standard representation) is uniquely determined by its bown in Figure 5. Y means 0 a paraiJelograrn, a
endpoint. Conversely, with each point in the pl ane we can associate its position
If k is a positive scalar, then kv is obtained b , . -
vector, which connects the origin to the given point. See Figure 3. factor of k leaving its düection unchanged If k . ) stre_tchmg the vec~or v by a
15
reserved. See Figure 6. · negatJve, then the direction is
Figure 1 Figure 2

Figure S Figure 6
tran lated i

./
'
----- --- --- --- -- -----' translated ü
V

translated ÜJ
(- t) jj
550 • App. A Vectors App . A Vectors • 551
Definition A.3 We say that two vectors v a.nd win IR" areparallel if one of them is a scala.r
multiple of the other.
(0, O,x3)
.. - - - - - - - - - - ;71
, :: ______
-- '
---.. . :
:(x 1, x2, x3):
EXAMPLE 3 .... The vectors : '

are parallel, since

UJ and

UJ o:
/---i---7------l~
- - (0,-"2. 0)
Xz

Figure 7 Figure 8

+ Dot Product, Length, Orthogonality

EXAMPLE 4 .... The vectors
Definition A.4 Consider two vector ü and w
with components u1, ll2 , . . . , v,. and w 1•
and
w2, . . . , w" , respectivel y. Here and v w
may be column or row vectors,
and they need not be of the sarne type (these conventions are convenient in
linear algebra). The dot product of and w is defined as v
are parallel, since

Note that the dot product of two vectors is a scalar.

EXAMPLES ...

Let us briefly recalJ Cartesian Coordinates in space: lf we choose an ongm UJ ·[=lJ =I 3+2 (-1)+1(-1)=0
0 and three mutually perpendicular coordinate axes through 0, we can describe
any point in space by a triple of numbers, (x 1, x 2 , x3). See Figure 7.
The standard representation of the vector
EXAMPLE6 ...

x= [;~ ]
X3

is the arrow connecting tbe origin to the point (x 1 , x2 , x3 ), as shown in Figure 8.

552 • App. A Vectors
App. A Vectors • 553
Fact A.S Rules for dot products
The fo ll ow ing equations hold for all column or row vectors ii, v, Ü; with n
components. and for a ll sca lars k:
translated y
w
a. v. = w · v
b. (ü + ü . w= ii · w+ v· w
c. (kü) . w= k(v. w)
d. ü. ü > 0 for all non zero v. Figure 10 0

The erification of the e rule is traightforw ard. Let u justify ru le d : ince

ü is nonzero, at least one of the components v; is nonzero, o that vt is po. itive . Solution
Then
llx 11 = ~= J49 + 1 + 49 + 1 = 1o
- -
V· V = v 21+ ~+
2
·· ·+V;
2+
···+ V 11
2

Definition A.7
positive as well. A vect~r _ü in II{" is cal led a unit vector if llü II = 1; tbat is, tbe lengtb of the
Let us think about the length of a vecto r. The length of a vector vector u ts 1.

Consider two perpendicular vectors x and ji in JR2, as shown in Figure 10.

By the theorem of Pytbagoras,
jxT
i n JR:! is + x~ by the Pythagorean theorem. See Fi gure 9.
Trus length i often denoted by llxll . Note that we have 11-i + .Y II2 = llxll 2 + II.YII 2
or
- - [Xt] [Xt] =
X· X = X~ · X
2
?
Xj" + x 2= ~~ - ~~2
2 X ;
c; + ji) . <x + y) = .:r . .x + .Y . .Y.
therefore, By Fact A.5,
Figure 9

l lxll=~- ; . x + 2(,r . ji) + ; . 5i = ; .; + .Y . .Y

Verify that this formul a holds for vectors in JR 3 as weil.
or
x
We can use this formu la to define the length of a vector in lR": x · ji = 0.
~ou can read ~ese equ~tions ba~kward t? show that ,t . .Y = 0 if and onl y jf ;t= and
Definition A.6 The length (or norm) llxll of a vector x in lR" is y are perpend1cular. This reasomng appbes to vectors in JR3 as wtf~.
We can use this cbaracterization to define perpe nd icul ar vectors in II{" :
IJ xll = ~= jxT + xi + · · · + x;.
Definition A.8 Two vectors v and w in II{" are called perpendicular (or orthogonal) if
ü . w= 0.
EXA MPLE 7 ..... Find llx II for

+ Cross Product
Here we present tbe cross product for vectors in JR 3 only; for a genera lization to
IR" see Exercises 5.2.30 and 5.3.17.
554 • pp. A Vectors
ANSWERS TO ODD-NUMBERED EXERCISES
In C hapter 5 we di cu tJ1e cro product in th cont , t of li near algebra .

Definition A.9 The cro

5.[;~] = [-; ]
CHAPTER l
1. 1 l. (x.y) = (- 1, 1)
3. o so luti on X3 - l
X4 I
5. (x, y) = (0, 0)

7.[~j] ~ [-2~ ]
7. No so lu ti ons
Unlike the dot product , t he cro Prod uct v x ÜJ i a vector in !R
3
.
9. (x, y, z) = (1, 1- 2t , t) , where 1 is arbitrary
I I . (x,y) = (4, I)
EXAMPLE 8 ....
13. No s lut ion
15. (x, y, z) = (0, 0, 0) Xi 1- s -2r
17. (x, y) = (- Sa + 2b , 3a - b) X2 ,.
19. a. Product are competing. X) -t+ s +I
b. P1 = 26, P_ = 46
9.
X4
= l - 2s +2
21. a = 400, b = 300 Xj s
Fact A.lO Geometrie interpretation of the cross product X6
3 23. a. (x ,y) = (1, 2t ); b. (x ,y) = (l, - 31);
Let v x wbe the cross product of two vectors v and ÜJ in ffi- . Then
c. (x , y) = (0, 0)
a . ü x ÜJ is orthogonal to both ü and
b. T he lenath of the vector x v w
w.
is numerically equal to the area of the
25 . a. If k = 7
b. lf k = 7, there are infinitely many solutions .
I I. r;:J r-;:+4 J
,t 4
=
- 2
parallel;gram defined by ü and ÜJ (see Figure lla). c. lf k = 7, the solution are
(x , y, z) = ( I - 1. 2r - 3, r). 13. o olutions
c. 1f ü and ware not parallel , then the vectors
IS. m ~ m
27. 7 children (3 boys and 4 girl
Ü. W. ÜX W
29. f(f) =I - Sr + 3r 2

l -~::~:::;0l
form a tight-handed sy tem ( ee Figure 11 b). 3 I. If a - 2b +c= 0
33. a. The intercepts of the line x + y = I are
( I 0) and (0, I ). The in tercepts of the line
x + ~ y = r are (t , 0) and (0, 2). The lines 17. [ ; : =[
Figure 11 (o) !IV X wll is X4 - 459/434
numericolly equollo lhe shaded oreo. intersect if r =f. 2.
b t 21 - 2 X5 699/434
(b) Arighl-honded syslem. .X =- i - 2')1 = ~

35. Tl; ere are_~~~~~ ~~r~ ect an wers. Example:

y - 3- = - 2

m~ [~~: ~ I:]
__,
,'
' ii X w
--- ,' 2 1. 4 rypes
: 1.2 I 25. Ye ; perform the operations backward .

3 m[4_2: _3:J
'
'
' 27. No; you cannot make the last co lu mn zero by
~
w
elementary row operation .
U)

(b) 29. a = 2, b =c = d = I
(a) A-1
A-2 • Answers to Odd-Numbered Exercises Answers to Odd- umbered Exercises • A-3

: u; !~]
4 19. Projecti o n o nto lhe eI axis; not in vertible.
31. f(t) = l -51 + 4t 2 + 3t 3 - 21 13.
[ 2u~2u1u2
-I 2u 1u2 J

[-Il
3 2 1. Rota tio n throug h an ang le of 90o in the clockw ise 2u~- I
33 . f(t) = - 5 + 13r- 10t 2 + 3c
dJrectl o n; in vert ible.
= 2u ; uj
~ittruy
15. a;; = 2u7 - I, and a;j when i ;;j:. j
23. C loc kwise rotati on through an angle of 900
J
: ;:e~ ~T:::e ~b'"M'
35 wh"e t ;, fo ll owed by a di lati on by a factor of 2; in ve,rtible. 17. Projection onto the line spanned by [:
33. Ax = X.
~["tT r] a dilation by a factor of 2
25. Dilati o n by a factor of 2
.

~
27. Re Aecti o n in the e-1 axis
o: ,A 19
37. [ : :] [:]

39. a. Nei!.her the manufacruring nor the energy

secror makes demands on agriculture. 39. [~0 ~0 ~1 : ]
29. Re tl ccti on in the ori g in

J l. ::~,::'Fr ~~J ,., . tepre<e"ted by the

2 1.
0

0
0

[~ -~ ~]
0
0

1
*
u~ n
b. X ] ~ !8.67, X1 ~ 22.60 , X3 ~ 3.63
41. m 1 = 1 2m2 41. One solution 33.1 [: -;J
un
43. a ::::: 12.17, b::::: -1.15 , c ~ 0.18. Tbe Iongest 43. No solutions 23.
day is about 13.3 bours. 47. a. ; = 0 is a solution.
35.
~ -~ J;you
b. By part a and Fact 1.3.3 _ _ _
1.3 1. a. No solutions b. One solution 25. [ hear back.
c. lnfinitely many solutions c. A(x 1 + x2) = Ax1 +_ A x~ = 0 + 0 = 0 37. T (x) = T (v) + k(T(w)- T (v) ) i on the line
d. A(kx) = k (Ax ) = kO = 0 27 . Yes ; use Fact 2.2. I.

uJ n n
segment.
3. rank is l
49. a. Infinitely many solution or none 29. Ye
39. True

-~ ~ ~]
5. a. X +Y[ = [ 1 b. One solution or none
c. No solutions 41 . Y = Ci .X I + c2x2; the graph is a plane through the 31. A = [
b. X = 3, y = 2 origin in JR3.
d. Infinitely many solutions 0 0 0
7. One solution 43. a. T is linear, re presented by the matrix 33. Use a parallelogram
51. lf n. = r and s = p

9 [~~nm~m
[2 3 4].

: ::, m~ -~ mm
35. Yes; mirnie Example 2.2.2.
37. Write ÜJ = C J Vj + c2 ü2 ; then
+2 b. If v = [ ~~'VJJ , then the matrix of T is T ( w) = C JT(ÜI ) + c2 T (~ )
II . Undefined
39. A= [l -,ab - b2 J
13. [ ~;] 57. [:1] = [~] + [~] [v1 Vz v3]. a- 1 +ab

[~~ ] · [~~ ]
41. refQx = -refp .t
15. 70 C. Y = C ]X J +c2X2 +c3X3 = COSCt sinet J, a clockwise rotation througlJ
m
17. Undefined 43. .
[ - SIO
C3 X3 Ct COS Ct

ny:::'nr
CHAPTER 2 45 . False t he angle et.
19. 2.1 I . Not linear 2.2 2 x
45. Write = C J VJ + c2 ii2 and u e Fact 2.2. 1 to
L ;n R defined by U,e ,eotot< compute T (x) and L (x).
3. Not linear

21. ['i~J
123
5.A= [ 1 ~ ~ -:~ ]
7. T is linear; A = [vl Vz v"]
3. The parallelegra m in JR 3 defined by the vectors
T( e-I) and T (e2)
47. Write T(.'r)
of a, b, c, d .
= [~ !J .1:. Expres f(t) in term

49. The image i an eJijp e with s mimaj or axes

9. Not invertible
J
~ !]
5. About 2.5 radians ±5el and semim.inor axes ±_e2.
~ ~j~
23. [ ;
I I. Tbe inverse is [ _ -
15. A is inv.ertible if a ;;j:. 0 or b # 0. In this case, 7. ~ [: i] 51. The curve C is the image of the unit circle under
the Iransformation with matrix [ 1 2 w w J.
25. The system Ax = c has infinitely many solutions
1
~
A - = a2 1/ [ -~ !] . .. 9. A shear parallel to the e2 axis
2.3 I. [ -~ -~ J
or none.
27. a. True b. False
17. Reftection in the origin; this transformat1on
own inverse.
IS 1ts
II . ~[
25
7
24
24 ]
-7 3. [ -t ~ J
 Answers to Odd-Numbered Exercises • A-5
A -4 • Answers to Odd-Numbered Exerci es 45 . A = BS - 1
79. g(f(x)) = x , for all x.

n
3. U ndefined
5. Not invertible 47. A =! [-~ I ~ ] f(g(x)) = { x if x _i s e~en
X + I If X IS odd
7 . Not inverti ble The functions f and g are not invertible.
5. [ :
9. Not invertible
~ ~ ~] ~ ~]
[-~ ~ ~]
81. False. Consider A = B = [

u1-n
matrix of T: [ -
7.
0 - 1 0
49.

[ ~ ~ ~]
l1 - 6 -2 -4
mat ri x of L :

1- 2
~ 9. [ ~ ~J 0 0 1 CHAPTER 3
3.1 l. ker(A) = {0}

l
13. :0 _: 51. Yes; yes; each ele mentary row operation can be
I I. [ I 0]
-2 "undone" by an elementary row operation.
13. [II)
15. h: Fact 2.4.9 applies to square matrice o nl y.

l
-~ -~ =~
53. a. Use Exercise 52; Iet S = E 1 E 2 . . . Ep

15. - )
1
- 5
2
9
-3
17. False
19. Fa1se
b. s= [ -~ n[! n -~ n = [

~ ~ l [~ ~ l [~ ~ l [! ~ l [~ ~ J
21. True
17. Not invertible 55
XJ = 3)' 1 - 2.5)'2 + 0.5 )'3 23 . True . [
whe re k is nonzero aod c is arbitrary. Cases 3
19. X1 = -3 )' t + 4)'2 - )'3 25. True
and 4 represent shears, and case 5 is a reflection.

lilil
X3 = Yl - 1.5)'2 + 0.5 Y3
2 1. Not inverti ble 27 57. Ye ; use Fact 2.4.4. -2 -3 0
1 0 0
23. lnverti ble
0 -2 0
25. lnvertible
29. For example, B = [ -~ ~ J 13.
0 -I 0

l
27. Not invertible 0 1 0
29 . For all k except k = I and k = 2
3 L [ -2: ~ ~ -2: ~ ~ foc rubitrary aod '
0 0

[~l [~]
3 1. l t's never invertible I . [ A(m) A1 ]
6 1. a. Wn te A = A .~~. ,
2 3 nJj 15.
33. I:f a 2 + b = I
35. a. Invertible if a, d, f are all nonzero
33. No = [L(m) 0] = [u(m) 17. All of IR2
b. lovenible if all diagonal entries are nonzero 35. a. = x 0
b . .X = ih L L3 L4 ' U 0
19. The Jine spanned by [ _ ~]
c. Yes; use Fact 2.3.5.
c. rank(A) =
rank(B) =m Use Fact 2.4. 12.
c. Solve the equation All of ~3
d. Invertible if all diagonal e ntries are nonzero d. By Example 1.3.3a
37. Al l diagonal 2 x 2 matrices - [ A(n--Il Ü]-[L' 0] [U' y]
21.
23 . kerne! is {0}, image is all of JR 2

l
1 A- - - 0 1
37. (cA)- 1 = iA - W k X I

~ ~
25 . Same as Exercise 23
39. M is invertible; if m ij =
k (where i f= j) , then 39. T he matrices [ for an arbitrary k for x, y and 1.
27. j(x) = x 3 - x
the ijth entry of M - 1 is - k; all other entri es are
:ii:~~ c~~i:~ ]
4 1. a. T he matrices Da Dß and DßDa both represent 65. A is invertible if both A 11 and A22 are invertible.
the same. the coun terclockwi se rotatio n through the In this case, 29. f [: ] = [
4 1. The tran fonnatio ns in part a, c, d are inverti ble,
angle a + ß. A - l= [A;II 0 1] cos(rp)
0 A-

31:o:r~r,~rroal
while the projection in part b is not. 22
b. Da Dß = DßDa = 1
43. Yes; x= B - 1(A - 1
y cos(a + ß) - sin(a + ß) J 67. (i th row of A B ) = (i th row of A)B oootdloote

123 [ sin(a + ß) cos(a + ß) 69. rank(A) = rank (A 11 ) + rank(A23)

m~
3 c. }3 = 64 (seconds)
45. a. 33 = 27 b. n
47. f(x) = x 1 is not inven ible, but the equation
cos(2n /3)
43. A = [ sin (2n /3)
- . in(2n /3)
cos(2n /3)
J_ -
71. Only A = !"

f(x) =
0 has the unique solution x 0. = 73. (ijth entry of AB ) = L~= l a ;k b kj :::: 33 T x + 2y + 3l
1 [ ~ -~] s L~= l
~ ~J
b kj ::::; sr
2.4 I. [
 Answe rs to Odd-Numbered Exercises • A-7

fil
A-6 • Answe rs to Odd- umbered Exerci se
5 1. a. Consider a relation c 1ii + ... + . _ _

~]
. . . - - . I - CpVp+d l w l +
l I. Independent -!=. dq Wq = 0. Then c 1v 1 + ... + c - _
35. ker(T) is the plane with normal vector - d I W I - . .. - d - . -
13. Dependent ., .
P Vp -
·r~ Wq IS 0, because thi s vector
ii; im T) = IR. 21 b'5 is ofkemel [
15. Dependent IS. both In V and In W. The claim follows.
37. im(A) = pan(e1, e2) : ker(A) =
2 17. Independent b. ~~ om part a we know that the vector
. pan (e1): im(A2) = pan(e 1); ker(A ) =
3 V I , ... ,Üp , w l , · · ··wq are linearly

l-ll UHil
pan (e1. e 2 : A3 = 0 3
ker(A ) = IR and 19. Dependent
_I. Dependent Independent. Consider a vector in V + Wx
im (A3 ) = {Ö} of v + w we can wn.1e
-2 -3 -4 -5 -By definition
_ ·
bMis of imoge
39. a. ker(B) i contained in ker(A 8 ). but they need x = v + ÜJ fo r a ü in V and a ÜJ ii' W Th -
I 0 0 0 · 1· • . ev
not be equal. 0 IS. a mear combi nati on of the u,, -. and w - 1
.s a
I 0

r;r n1-[-!J.[-~ J
b. im(AB) i contai ned in im(A), but they need 25. 0 23
I 0 linear com bin ation of the w-J·. Tb"IS ShOW
0 0
not be equal. th at the vectors ii 1• · · · · iiP•wl,
- ... , wq
- pan
0 0 0
41. a. im (A) i the line panned by [ ! l and ker(A)

-~ J. 27
Ul m 3.3 I. [ -n
V + W.

:1~l
i the perpendicul ar line, spanned by [

b. A2 = A : if ii is in im (A), then Aii = ii.

c. [!]"gonol proj<etion ooto the line sponned by 29. [!l [~ J 3 [ -n basis of image:

43. Suppose A is an 111 x 11 matrix of rank r . Let B

,.m m 5
[~Jfn
[

[~lm·[nm [!J
be the matrix you get when you omit the first r
1 0 0
rows and the firs r n colunms of rrefl.A : / 111 ] .
0 -2 -3 25
(What can you do when r = 111 ?)

mm·m
7. 0 I 0
45. There are n - r nonleading variables, which 33 .
0 0 -4
can be cho. en freely. The general vecror in the
0 0 I 27. They do.

-nr~J
kerne! can be wrinen as a linear combinaüon of 35. Consider a relation I - 1 1 -I
n - r vector , with the nonleading variables as c1ii1 + · · · + c;Ü; + · · · + c",ii", = Ö where 1 0 0 0
coefficients. c; =/: 0 but Cj = 0 for all j > i . Solve 9. 0 I 0 0 29. [
47. i.m (T ) = L 1 and ker(T) = L2 fo r ii;. 0 0 I 0

l~ -r -~l
51. ker(AB) = (0} 37 . The vectors T (Ü 1), .. . , T (Ü 111 ) arenot neces ari ly 0 0 0
independent.
[~]
0 I
31 h
0 I 39. The vectors Üt, ... , ü", v
are linearly I I.
I I 0 independent.

m·m
33. The dimension of a hyperplane in lR" i 11 _ I.
53. a. I 0 0 0 4 1. The colu mns of B are li nearly independent, while 13 35. The dimension is 11 - 1.
0 I 0 0 the columns of A are dependent.

l~ ~ ~ ~ ~l
0 0 1 0 43. The vectors are linearly independent. 15. 5
0 0

[lll~J
0 37. A =
b. ker(H ) = pan(v1, "iil , Ü3, Ü4) , by parl a, and 45. Yes 0 0 0 0 0

[~ !]
0 0 0 0 0 0
im (M) = span(ii 1, ii2, v3, Ü4) , by Fact 3.1.3. 17
I
Thus ker(H ) = im (M). H (Mx) = Ö, since 47 . 39. ker(C) is at least I-dimensional, and ker(C) i
0

Jl[-;]
Mx is in im (M) = ker(H ). contai ned in ker(A)

m~ ~ [~ll
0
3.2 I. Not a subspace 4 1. A basis of V i also a basi, of w. by Fact 3.3.4c.
3. W is a subspace
7. Yes
49. L ~ im ke< [
0 - l
- I
J 19.
43. dim( V + W) = dim(V ) + di m W), by
E, ercise 3.2.5 1
9. Dependenl
A-8 • Answers to Odd- umbered Exercises
Answe rs to Odd - u mbered Excrcises • A-9
~j~ J
45. The fir t p co lumns of rref(A) contai n leading l 's 2 1. For exa mple: b = d = e = g = 0,
becau e the Ü; are linearly independent. ow I -)3 _ .fj 15. [ [3] multiplying the i lh row of A with -I whe never
a = 2· c = T • f - - 2 - 2/ 3 i negati ve.
a ;;
apply Fact 3.3.7 .
25. a. llkiil l = j (kü). (kv = ) k2 ( Ü . v) =
[:~; _:~; J[~
49 . [ 0 I 0 2 0 ) . [ 0 0 I 3 0 ) . 43 . Wri te the QR facto rizatio n of A in partitioned
#~ = lkl ll iill form as
[o o o o 1]
ll~ll ii ll = 1 .~ 1 lliill =
17 . 3n
51. a. A and E have the ame row space, s ince b. By part a, ll 1. A = [ A, A2] = [Q, Q2] [ ~~ ~~ J Then
eleme ntary row operatio n leave the row space 2/ 3 - 1; .JT8J
3 3 A, = Q , R, is the QR fac torizatio n of A 1•
unchanged. 19. 2/ 3 - 1/ .J IS [ ]
b. rank (A) = di m(rowspace(A) , by part a and
Exerci e 50. UJ [
1/ 3 4/ .J I8 O .JT8
4.3
45. Ye
I. (AÜ ) · ÜJ = (AÜ)TÜJ = üT A TÜJ = Ü . ( A T ÜJ ) .

n-; -nn~ -:n

27
55 . Suppo e rank(A) = 111 . The ubmatrix of A 21 3 x ( L (-) L (-) ) _ L (Ü) · L ( w )
consi ti ng of the m pivot column of A is
invenible, since the pi vot columns are linearly
29. By Pyrhagoras, llxll = .)49 +9+4 + I+ I= 8 t · <+ v ·_ ~ - arcco IJL (v) ll II L w) ll =

arccos llvll .llwll = 4(ii, w) (The eq uation

3 1. p ~ llx11-. Eq uality ho lds if (and only if) x is a
[i~ ~;;;gl [~ ~~] = ü · ÜJ
independent. L Ü) · L (ÜJ) is ho wn in Exerci e 2
linear combination of the vec tors V;.
Conversely. if A has an inve rtible m x 111 23. 5. Yes, by Fact 4.3.4a.
R(x) v, . x)v, + 2cü2 . •r)v2- .r
ubmatri x, then the column of tl1at ubmatri x
pan JRm, so im (A) = Rm and rank(A) = m .
33. = 2
35. No; if ü is a unit vector in L , the n
l/ 2 1/ 10 J 7. Ye . ince A AT =/ 11

[Y~~;~:~ Jl[~ :~1

57. Let m be the mallest number such that x · proj Lx = x · (Ü · x)Ü= (Ü · .r ) 2 ~ 0 9. The fir t co lu mn i a uni t vector· we can write
A 111 = 0. B y Exerci e 56 tbere are m linearl y . - [ COS (ip) J
~~~]
25. Jt a 1 = . in (ip) fo r some ip . The secend
· independent vectors in Rn: therefore, m ~ 11 , and
4.2 I. [ 3/ 5 4/ 15 colu mn is a unü ve tor orthogonal to ü1~ there

J J and [ - COin (ip) J.

-2/ 3

=: u~ -!]
61. a. 3, 4, or 5 b. 0, 1, or 2 are two choice : [ - in (ip)
63. a. rank (A B) ~ rank A) CO (ip) (ip)
3. [ 4/ 5 ] [ 3/ 5] [ cos(ip) - in(ip) J and
3~5 -4~5
b. rank (A B) ~ rank( B ) Solution:
• _7 { Sin (ip) COS (ip

CHAYfER 4
4.1 ].
3.
JT76
J54
~) ~ 0 . 12 (radians)
5
. nn ks[ =iJ 29.
3 1.
r J. r
-!j~
e,. e2, e3
;jn
CO (<P)
[ s in (<P)
in (ip) J
_ cos(IP) , fo r arbitrary ip .

II. For example, T (x ) = 1[ -~ -; ~] x

5. arcco (
7. obtuse
9. acute
Jn)
7. [~j~ ] · [ -~j~ ] , [ -~~~ ]
l/3 2/3 2/ 3 33.}, Ul}, [-IJ 13. No, by Fact 4.3 .2

= [~
2 2 I

~ J and B = [ ~ ~ J
:~ ~ii:~]
15. No : con ider A
lf (as n -+ oo)
35. [~j~ ] · [ ~j~J
I I. arcco ( -+
9 17. Ye. since (A- 1/ . = (AT )- 1 = A- 1

: t~Irenl-~J
· [ ] . [
1/ 2 1/ 10 2/ 3 - 2/3 19. (ij th entry of A) = v;v1
2 1. A ll e ntrie of A are .!.

II. [4~5 J•[~~;:~] I[: -:j[34] II

37. 'L J - J 0 5 23. A represent the refl ec ti on in the line spanned by

ii (compar 1 ith E ampl e 2), and 8 reprc ents

[-i].[-f]
I I
3/5 4/ 15 the rcfl ection in the plane " irh normal vcc tor ii .

· _I
m_ [~ ] _I_ [ -~]
[ :~] . [=:i~] . [-:~]
25. di m (kcr(A)) = 11 - rnnk(A) (by Facr 3.3.- ) and
17
39 _ I_ [ :J dirn (kcr(A T) ) = 111 - rank (A T) = 111 - rank(/\ )
3 • JJ - J · .J42 1
19. a. Onhogonal projection onto L j_ 13. (by Fac t 4.3.9c). Th refore. the dimensions of the
1/2 1/2 - 1/2 4 1. Q i. d iago nal wilh q;; = I if a; ; > 0 and rwo kerne! are equal if (and onl y it) 111 = 11. that
b. Reftection in L .l c. Reflection in L
q;; = - I if a ;; < 0. You can gct R from A by is, if A is a square matri x.
A-10 • nswers to Odd-. umbered Exerci es

AT = QRTQR=RTQ T QR = R T R 1 . b. L +( L .r)) = proj v_r , "here

Answers to Odd- umbered Exerci es • A-ll
~[-
_7_
Least-squares. o luti o n [ co ] 1.6 16]
29 . By Exercise 4. _.45, we can wri te AT = QL V = (ker(A)) ..L = im (AT) cf 0.664 ·
9. - 72
whe re Q i orthogonal and L is lower rriangular. c. L ( L + (y)) = projiV\·, where U e approx imatio n log(z) = - 1.6 16 + II. 8
Then A = (QL T = LT Q T doe the job. W = im (A) = (ke r(A T)) .L . 0.664 log(g) 13. 8
d. im ( L + ) =[i1rA~]) and ke r( L + ) = ker (A r) b. Exponentiate the eq uation in part a. z = 15 . Ana logaus to Example 4
3 1. a. Im = QfQI = STQJQ2S= STS . o tllat S is IOiog(;:l = IQ- 1.616+0.664 1og(g) ~ 0.0242. 80.664
orthogonal.
b. R2 R! 1 is both orthogona l (by parl a) and
e. L + (y) = ~ ~ ~· 17. a. det[
1 1
= a 1 - a0
ao a 1
J
CHAPTER 5 b. Use Laplace Expan ion d wn the last column
upper triangular. v ith po iti e diago nal entrie .
15. Let 8 = (AT A) - 1 AT . 5.1 I. -2
B Exercise ~oa we ha e R2 R! 1 = Im so tJ1at to . ee 1hat /(1) is a polynomial of degree .:::; n .
3. 6 The coefficient k of t " is [1 (a; - aj ).
R2 = R 1 and Q 1 = Q2. a claimed. 17. Yes; note that ker(A = ker AT A).
5. -24 n - 1?; 1> j
3 . u· A = L DU. then A T = U T D L T i the L DU ow det(A) = f(a") = k(a"- ao)(a11 -
factorization of A T_ Since A = AT the two
fa torizations are idenrica.l. o that U = L T. a
19. [ : ] 7.
9.
0
0
a1) · · · (a" - Gn-J) =
claimed. n?: i> J
n
(Cl; - aj) , a

claimed. 2 1. ; • = [ -~ J. llh - Ax *ll = 42 II.0

n
II

4.4 1. im(A ) = pan [ ~] and ker(A T ) = spa.n [ -; l 23. [ ~]

13. 36
15. 24
19.
i= l
Cl; . [1 (a;
i>j
and Exerci e 17)
- Clj) u e linearity in the column

3. The vecrors form a ba is of :?n.

I - 3t] , f or ar b'1trary t
17. - 1
19. a ps - aqr
21. [;~ ] = [:~ ] and [;~ ] = [:~ ] are olution.
- v.L = (ker(A)) .L = im(AT) , where 25 .
[ 1 The eq ua tion is of rhe form px 1 + qx2 +b = 0.
21 . invert ible
A= [ I
I
2
I I
5
I]
4.
,n 23. in venible if a, b, c are aJJ nonzero 23.
lhat i , it defines a line.
±I

[iHH
27. [
25. invert ible

B~is
25 . det(A TA) = (det (A))2 > 0
~ 2l
·* ~ 27 . 3, 8
~ ~J
29 ..x,
·* = x2
of vL 3 1. 3 + 1.51
29 . I 27. a. [ - b. Tbey are 0
31. - I ' 0, 2 c. det(A) = det(A T) = det( -A) =
33 . approximately 1.5 + 0 . 1 sin(l ) - 1.41 cos(r ) 33. det(A) = 0
7. im (A) = (ker(A)).L (- I)" det (A) = - det(A) , so det(A) = 0
co + 35c 1= log(3 5)
9. co + 46c 1= log(46)
35. det(A) = I· there are 50· 99 in ver ions.
29. ATA = r ~ÜII2 Ü·ÜJ2 J
im(A T) = (ker(A J)..L
37. a. Try to olve the . y tem +
co 59 c,= 1og
(77) 37. det(M ) = det(A) det(C) w
llw 11 · . o
v·
det(Ar A) = llii ll 11 w ll2 - (ü · w) 2 ::: o. by the
2

[
39. Let n;; be the firs t diagonal entry that does not
co + 69c 1=log ( l 33)
beleng to the pattern. The partern mu t conrai n a Cauchy-Schwarz inequality. Eq uality hold onl y
S: parallelto ker(A) Least- quares so luti on [ ~n ~ ~:~: ~ J. Use numbe r in lhe ith row to the right of a;; and also i f ii and ÜJ are parallel.
a number in the ith column below a;; . 3 l. Expand down the first col umn :
approx imatio n log(d) = 0.915 + O.O l7t.
41. det(A) =
2x - x2
= - x(x
-x 3 -1 ) 2 .
The matrix f(x) = - ..r det(A4 1) + constan t, o
b. Exponentiate the eq uatio n in part a:
d = IQiogd = !00.9 15+0.0171 ~ 8.221 . 1QO.OI71 ~ A is in ve rtible except when x = 0 or x = I. f'(x) = - det(A-1 1) = - 24.
43. - k 33 . T is linear.
8.22 1 . 1.041
45 . k 35. det(Q, ) = det(Q2) = I and
llxoII < ll.t II for all other vectors x in S. c. Predicts 259 displays for the A320; there
det(Q") = 2det(Q11 _t) - det(Q" _ 2 ) . so
are much fewer since the A320 is highl y 47 . det(- A) = (- l )l'det(A)
I I. b. L (L + C})) = y det(Q" ) = I for alln .
computerized. 49. det(A - I) = det~A )
c. L +(L (x)) = proj vx. where 37. det(P 1) = I and det(P") = det(P11 _ 1) , by
V= (ker(A)) .L = im (AT) 39. a. Try to solve the system 51. det(A ) = I if 11 i. even and det(A) =- I if 11 i
ex pan ion dow n the fir t co lumn , o det ( P11 ) = 1
co + log(600, OOO) c 1= log(250) odd (the re are n 2 inversions for all n.
d. im ( L + ) = im(AT and ker(L + ) = {Ö} co + log(200 , 000)c 1 = log(60) 5.2 I. - 3
39. a. Note thar det(A)det(A - 1) =I , and both

oL+GJ~ [~ !]> co + log(60 , 000)c 1 = log(25)

co + log( lO , OOO) c 1 = log( 12)
co + log(2, 500)c 1 = log(5)
3. 9
5. 24
7. I
fac tors are integers.
b. Use 1he fo rmu la fo r rhe inver e of a 2
m<llri x (Fact 2.3.6).
A-12 • An wers to Odd- umbe red Exercises
nswers to Odd- ·umbered Exercises • A -13
~ -~ -~] ; A- 1 u, = e; becau. e Se; = ii;
S- I -
J
23. n. -
5.3 I. 50
3. 13
25. adj(A) = [
- 2 0 I
b. s- 'AS = s- 1 A(ü 1 ... ii" ] = 35. x(t) =2 1
[: J+ 6 1
[ - : We need a matrix
s- I [ AI jj I .. · A." v" ] = [ AI e1... /.. e" ] =
7. 110
9. Geometrically: ll lidl is the ba and llii2 ll sin a
the height of Lhe parallelogram defi ned by li,
det~A) •dj(A) ~ - •dj(A) ~ ~ [ - ! -~] [
At · ·
0
0

A"
11

] , the diagonal matri x with

A wi th eigenvectors [: J,[-:J. with
a. soc1ated e1genval ues 2 and 6, re pectively. Let

and - 2· 27. x = - 2 ° ? > 0: )' = 2 - bb? < O;x decrea es

a + b- a + -
diagonal entries A. 1, . . . , A." .
A [: -: J= [ ~ -~ Jand . olve for A. We
Algebrnical/y: In Exerci e 5._.29 we leamed that a b increa es. 25. 4 2
2 2
det(Ar A = ll lit Wllii2ll - (ii , · -2) = fi nd A = [- -
4] .
29. dx 1 = - D - 1R2( l - Rt ( I - a) 2de2, 2
2 2 2
lliitll 2 ll ii2 ll - llii tl lli tl l co a = dy 1 = o - 1(1 - a) R_(R t ( I - a) + a)de_ > 0, 37. a. A represents a reflection in a line followed b
Jlli 111 2 llli2 112 sin2 a , so that I det(A) I = dp = o-I R t R2d e1 > 0 . a di lation by a factor of ._/32 + 42 = 5. The

13.
) ctet (AT A) = ll litll ll ii tl l sina.
II . 1det(A) 1 = 12, the expansion factor of T on tbe
parallelogram defined by ii , a11d ii2.
hO
31. ITTTT

27 . 39. Jf A i an eigenvalue of
r
eigenvalues are therefore 5 and - 5.
b. So lving the lineru· systems Ax = 5x

[";"f[ :~ we '"d the e;ge" b' ;,

s- 1AS. wi th
15. V(v 1, ~ .. .. , vk) and
j det (AT A are zero. One of the V; isa linear CHAPTER 6 c rrespondi ng eigenvector ü. then
combination of ü1, .. . , ii;- t. Then proj v,_1 = Ü; v; 6.1 l. Yes ; the eigenvaJ ue i A3 s - 'ASv=)-.
0
ru1d - ; - proj .,_
" f- 1
v;
= Ö. This show that 3. Yes; the eigenvalue i A + 2
ASü = S),Ü = ASÜ,
\l(ü 1. . .. , vk) = 0, by Definition 5.3.6. 5. Yes and A. is an eigenvalue of A (Sii is an
Fact 4.4.3 implies that det(AT A) = 0. 7. ker(U11 - A) i= lÖ} becau e (U" - A)v = Ö. The eigenvector). Likewise. if is an eigenvector ofw
17. a. V(v 1 , v2 , Ü3, ü, x ii2 x Ü3) = matrix Al" - A is not invertibl e. w
A, then s- I is aJl eigenvector of s - 1AS witb

~ ~J
V (v 1 , ii2, ü3)1 Jli, x ii2 x Ü3ll becau e the same eigenvalue.
v1 x ü2 x ii3 is orthogonal to ii t , ii2, and V:,. 9. [
29.
4 1. TIT
vcü,. ii2. li), v, x
~]
b. ~ x v3) = 6.2 I. I , 3
I det[Ü t x ~ x v3 v, Ü1 li) ]J = l l. [ 2a + :b - 2d 3. I , 3
llvt x Ü2 x Ü3 ll 2, by defin ition of tbe cross
product. 13. All vectors of the form [
31
5t
J,
where t "I= 0 ( olve
5. none
7. I. I , l
c. V(v 1, ii2, Ü3) = lllit x Ü2 x ii31l. by parts a t.he linear system A.'t = 4x). 9. I , 2. 2
and b. 15. The nonzero vectors in L are the eigenvectors II. - 1
19. det [ü 1 ii2 Ü3] = ii 1 · Cii2 x Ü3) is positive if with eigenvalue 1, and the nonzero vectors in LJ. 13. I
(and only if) ii 1 and ~ x Ü3 enclose aJ1 acute have eigenvalue -I. Construcl an eigenbasis by 15. Eigenvalues A.u = l ± .Jk. Two di tinct real
angle. picking one of each. 3 1. eigenval ues if k is po iti e: none. if k i negati e.
21. a. reverse b. preserves c. reverses 17. No eigenvectors and eigenvalues (com pare with 17. A r pre ent a reflection-dilari n. ' ith a dilation
Example 3). fa tor of .Ja 2 + b- . The eigenvalue. are
det [ ~ -~] 19. The nonzero vectors in L are the eigenvectors ±Ja2 + b2 .
23 . .x, = - 1_ .
- 17 ' with eigenvalue I, and the nonzero vectors in 19. True (the di criminant t.r(A) ~ - 4 det(A ) i
det [ -~ -~ ] the plane L J. have eigenvalue 0. Construct an po. itive)
eigenbasis by picking one nonzero vector in L 2 1. det(A) i the product of thc cigenva lue .

-~ ~ J
and two linearly independenl vectors in L J. . and tr(A) is thei r . um. To see thi s,
det [ (Compare with Example 2.) write JA (A) = (),-A l)··· (A. - A") =
6
X2 = de{ -~ -~ ] = T7 21. All nonzero vectors in JR 3 are eigenvectors wi th 33 . c(t) = 300( 1. 1) 1 - 200(0.9) 1 A." _ (A l+ ... +A")Att - 1 + .. . + - I)"A.t ·. ·A"
eigenvalue 5. Any ba i. of JR3 i an eigenbasi . r(t) = 900( J.I Y- 100(0.9) 1 and compare with the for mul a in Fact 6.2.5.
A-14 • Answer to Odd- umbered Exerci e
Answers to Odd-Numbered Exercise • A-15
_3. Js(A.) = det(A. /" - B ) = det(H" - s- 1
S) then IA. I :::: I . e need not be an eigen ector of A;
U] U]
. [0.O. 9 0O..9] ~ (1 + 'f) [:] + Hl' [ _:] +
2 1· We want A = ancl

n l
n
= det(S- I ()../" - A)S)

-r ~]
con 1cler A = 1 1 ' X(t )
= (cter(s) r
1
ctet(M"- A)cter(S) A[ = 2[ = [: that is,
=JA().)
A and B ha e the same eigenva lue , with the
33.: :~ [~( ~]b).. - Cl
A [ ~ ~] = [ ~ : ] . The un ique so luti on i H )' '* [= Cad w;, ;r heohoo""
ame algebraic multiplicitie .

~ ] = [ ~ ] and A [ _ :] =
Tr - 5 17
A = u=~ J CQ < I.

, u]
~ li -~ ~ -!] ~:~ ~:n
5 A[ (a- c) [ _ : ] .
23. Th e onl y eigenvalue of A is 1 with 4 1. a. A = [ b=
ote that Ia - cl < I

[ ~ ~J
Phase ponrail when a > c: 35 A E1 = span e1). There is n eigenbasis. A
represent a shear parallel to the x 1-axi . b. 8 =

line spanned by [ ~] 37. We ca n wri te fA (A.) = (A. - A.o) 2 g (A.) . 25. Tbe geometri c mu ltipl icity is always 1.
c. The eigenval ues of A are 0.5 and -0. 1, those
of 8. 0.5, - 0. 1, I . If ü i an eigenvector
By th procluct ru le,
~]
2
f~ A.) = 2(A. - A.o) (A.) + (A. - A.o)-g'(A.), o that 27 . }A(A.) = A. -5A. + 6 = (A.-2)(A. - 3) , othat
of A. then [ i an eigenvector of B.
J;
m;"
the eigen va lues are 2, 3.
1 (A.o) = 0.

6.3 I. eigenbas is: [ ~ ]. [ ~]. with e igenvalues 7 , 9. 29. 8oth multiplicities are n- r.
F"rthe<more, [(I -:)-'b ] ~ 0

line spanned b [ _: ] 3. eigenba is: [ -~ ] . [ : l with eigen al ues 4 , 9.

3 1. They are the ame.

33 . a. AÜ·ÜJ = ( AÜ)TüJ = ÜT ATÜ; = ÜT AÜJ = Ü·AÜJ

e1genvector of B with eigenvalue I .

d. Willapproach (h- A) - 1
b = [~ ] for any
5. No real eigenva lue . b. S·uppo e Aü = A.v ancl Aw = fl-W. Then
27. a. AÜ · ,-i; = A.(Ü · w) ancl Ü · A'w = /1-(Ü . w) . initial va lue.
7. eigenbasis: e .e2, e3, wi th eigenvalues I , 2, 3.

mmf n
1

~~~]. Then .'i(t +

By part a, A.(v · w) = 11-(Ü · w) , so that

9. e;geob~;, (A. - /1-)(Ü · w) = 0. Since A. =/= 11- it follow

w
that Ü · = 0, as claimed.
43. Let x(t) = [
w(t
I ) = A.Y(t)

with e igenvalues I , I . 0.

il Hl Ul
35. a. E1 = V ancl Eo = v.t, . o that the geometric
mult:ipliciry of ] · is 111 and that of 0 is 11 - 111.
The alge braic multiplici rie are the ame ( ee
! ! ~ J.
where A = [
0 !
Eigenbasis for

u] ·[_n~ [~n
.!.
II . e;geoba ;, [

e;geoval o~
Exerci e 3 1 .
wi th eigenvalues 3, 0, 0. b. E1 = V and E- 1 = v.t, o that the

Ul hH-a
A wiili I,
mul tip licity of I is m and that of -I i n - m .

U~ l
13
e;geob.,;,
wirb eigenvalues 0. I , - I . o7. E;geoba. ;. focA m. [-n[-n t.o Xo ~ •, ~ ! [}~ [ _n +~ [-H
[!l[-n
b. A' = [ A'e 1 j
[~1] +(~) I]
Ne1 ] approaches
dgeov~tOß
with eigenva lues 1.2, -0.8. - 0.4;
15. .Xo = SOv 1+ 50ü 2 + 501 . so.r(t)=!
r+ l [
-~ fort >O.
' 7. :anbaL [; :] with eigenvalue 0, l. o eigenbasis.
j (I) = 450( 1.2)' + I 00( - 0.8)' +50( -0.4)' The proportion in the long run i I :2: I .
11!(1) = 300(1.2)' - 100(- 0.8)' - 100(-0.4) '
29. Ae e,
= e
so that is an eigenvector with 17 . eigenbasis: e2 , e4 , e1, el' e3- a t ) = I 00( 1.2)' + 50( - 0.8)' + I 00( - 0.4)'
45 . I ( rref(A) i likely to be the matri wirb all
associated eigenvalue I. with eigenvalues I , I , 0, 0. I 's direc tl y above the main diagonal ancl 0 s
3 J. A and AT have the same eigenvalues, by 19. The geornetric multiplici ty of the eigenvalue I is T he populatio n approach the proport ion 9:6:2. everywhere eise)
Exercise 22. Si nce the row ums of AT are I , we
can use the resu lts of Exerc ise 29 and 30: J is
an eigenvalue of A; if A. i an eigenvalue of A ,
I if a =/= 0 and c =/= 0; 3 if a = b = c = 0; ancl 2
otherwi se. Therefore, there is an e igenbas is on ly
if A = h
39 a h ~ [: ~ l] 6.4 I. JfS(co

3.
<-i + i sin(-i )
cose~k) + i sine~k). for k = 0, .. . , 11 - I
A-16 • An wer to Odd- um be red Ex e rci es An swe rs to Odd-N um bered Exerci s • A -17
CHAPTER 7
19 · ·-~·(1 ) =
~] , [~0 0~ ~]
5. If z = r(co~ (<f>) + i j n 4> ) . then JT3r [ - . in (</>1) ] . wherc

w = ::/f(co ( 4> +} :rrk) + i si n(<l> +} ;rk)), fo r

os(</>1)
4> = a rctan (~ ) : . pi rals o ut wa rd.
7.1 I. [-~ ] 6
D=
- J
k =0 .... . n - 1.
7. Cl ockw ise rotatio n through an angle f ~ 2 1. ;i·(l) = v
J,;';r [
1/
5 sin ((/11) ] 1
co. (</>1) + 3 sin (/11) , w lere
3. [-~ -~ ] 13. s = [: I J.D = [ p +0 iq
followed by a d ilation by a fac tor of Ji. -1

9. pira L out ward ince lz l > I .

4> = ar wn(*) ; pira ls outward. 5. a. [ ~ ~] b. [ 0.9
0.3
0.3 ]
O. J 15. s= [
8 0
]
I I. f (A.) = (A.- I )(A. - I - 2i)(A.- I + 2i) -: - (l)' [ 5 ·in ((/11 ] ' whe re - 4 I
:2 3. .\ (t )- 2 CO (</> ! ) + 3 in((/11)
13. Q is a field. 7 [ - 149 - 23 1]
15. The binary dig it fo rm a fie ld . 4> = arc tan (i) ; piral. in ward. . 99 154 19. A1 = 21 [ l - t
-1
1
I + 1
J
17. H i not a fie ld (mulri plicati on i
noncommu tati ve)
25. not table
27 . . table
9.
[~ ~] 2 1. Ye

19. a. tr(A) = 111 . det(A) = 0 29 . may or may not be stable; cons ider A = ± ~ /2 ~
b. tr(B) = 2m - 11. der 8 ) = (- n n-m compare 3 1. a. Follow from w + :: = w + z and wz = 11. [_lJ 23. Ye. ; both are sim il ar to [ :]

with Exer i e 6.3.35)

2 1. 2 ± 3i
w:;
ijth entry of
A8 = L,a;~: bkj = '[) ;k Vkj = L,a;k bkj 13.
[=:]
25.
1
0; con ider A = h and 8 = [ ~ : J
23 . l, -~ ± 1i = ijth e ntry of A B)
b. Take the conj ugate of both ide of the
2
29. [ / J
25 .
27 .
_9.
±I . ± i
- I. - 1. 3
tr(A) = ) 1 + ),2 + ), 3 = 0 and
det(A) = >. 1>. 2 ), 3 = bcd > 0. Therefore. there are
equa1ion
A(ü + iw ) = <P + i q) ü + i w) .
JS.
[-n 3
1. rn
one positive and two negative eigenvalue ; the
and u e part a: 17.
[~n [ I~ J
[-; ~ ]
33 . I
po iti ve o ne i largest in ab olute value. A(ü- i ÜJ) = (p- i q)(Ü- iw)
3 1. ~ A i a regular tran ition matri x (compare with
Exerci e 30), so that lim (Ys AY e xi t and ha
33. The matrix repre. ent a rotati on-di lati on with a 19.
35. 21 -l [ - t
2- I
J
dilation facto r of J0.99 2 + 0 .0 12 < 1. Trajectory

n -~]
r-
identical co lumns (see Exercise 30 ). T herefore, 0
spiral ioward. 37. Not imi lar, becausc A2 = 0 and 8 2 :f= O.
the column of A 1 are nearl y ide ntical for Iarge t . 2 1. 0
35. a. Choose an eigenba i Ü1, ... , ü" and write 39. Ye
33 . c. Hinr : Let )q. ),2, .. .. )s be the eigenvalues, 0
xo= c1Ü1 + · · · + c"ü" . 4 1. Ye
wi th Ä 1 > p,jJ, for j = - · . . . , 5. Let
ü1. ü2 , . . . , v5 be corre. ponding eigenvectors. Then 23.
[ 00.36
.48
0.48
0 .64
08 ]
- 0 .6 43. Yes
Write e;= C l V l + ... + csüs. Then - 0 .8 0.6 0 45 . Yes
ith column of A' = A e;
1
and 27. b ;j = Cln + l - i.n+ l - j
47. No
= c1 A.'1Ü1 + · · · + c - }.~Üs
is nearly parallel to - 1 fo r Iarge t . llx (r)l l ~ lcd lllid l + ·· · +Je" I llii"ll = M 29. Both [ ü+ iw ü -i w ] and [~ -i]
1
are 49 . . o; con ider A = [ ~ ~]
6.5 I .. table (use thetriangle incqua lit.y ll ii +wl l ~ l l ,~ l l + ll wl l.
3. not stable
inve rt ible.
~ ~ J and 8 = [ ~ ~ J
:J[n
and ob. erve that JA.'; I ~ I) 7.2 I . S = 12 5 1. A = [
5. not , table
7 . not stable
9. not table
b. The trajectoryx(l ) = [~
not bounded . Th is cloes not contradi ct part a,
= [ ~] is
3. s= [ ; : J.
5. not diagonalizab le over IR.
D= [ ~ _ n 53. [ ~ C~l ]
I I. For Jkl < I

n
·ince there is no eigenbasis for the matri x 55. Ye (by Exa mple 6)
13. For all k
15. ever table [~ :l 7 s ~ n ~ !J. v ~ [~ ~ 61 .
65.
ote tha t
JA (A) = 0
.f.1(),) = fa (A.)

17. x (t ) = [ - . in(<f> r ] . where 4> = arctan(13) ; a

cos(</>1 ) 39. [ ~] is a stable cqu ilibrium.
9. not diagonalizab lc 67. True ( A has three di sti nct eigenva lues)
circle.
A-18 • n wer to Odd- umb r d Exercises nswer to Odd- um bered Exercises • A -19
3 1. True I I. The . ame (the eigenva lucs of A and A - I have
27. True
thc sa mc signs).
13. a;; = q(e;) > 0.
29. False
[An, ).. 1J
5. a = arccos(- f,). Hint: lf Üo, ... , Ü11 are 15. Ellipse; principal axes spanned by [2]
1
an d
3 1. The closed interval
33 . 8 =_I_ [ 6 2]
. uch vectors. Iet A = [ iio . . . v Js
~ ] ; equation 7cT + 2ci = 1
11 ] . Then - 3 4
the n ninvertibl e matrix AT A ha 1's on Lhe [ -
diagonal aod co (a) everyw here el e. ow u e 35. B = ~ [ ~~ ~n
Exercise 17. 17. Hyperbola; principal axes spanned b [ 2 ]
II
-I] 1 and· Y 37. L = _I_ [
.../2
4 0]3
-1
39. a. ijth entTy of \A B\ = L a;kbkj [ 2
, cquat1on 4cj - c2 = 1.
. ? ?

k= l
II
19. A paiclof Ii o.";; pd odpal axos spaoood by [ - n 39. L =[ -~ ~ ~]
~ L la;k\lbkj \ 4 3 I
k= l

= ijth enLry of\A\\ B\

and [
2
] ; eq uation Sc~
= 1. 43. For 0 < a < a.rccos (--~ -)
II- I
b. By inducüon on r, using part a: ote th at we can write x2I + 4 x I x 2 + 4.\.2 _
7.5 I. a,
= 2, a2 = l
. -

[~- ~0
2
\N i = \N- 1A\ ~ \A' - 1 \\ A\ ~ \A\' - 1\A\ = ( x I + 2x2) 2 = I , o that x 1 + 2x = ± 1
2
3. All ingu lar value are 1 ( ince AT A = In)
I I. Same S as in 9, D = 001] \A\ 1
5. a 1 = a2 = J p- + q2

n[~ ~ J
_ ). a. The fir t i an ellip oid. the econd a

~ ~ J[~
hyperboloid of one heet and lhe th ird a
13. Ye (reflection in E 1 4 1. Let).. be tbe max_imum of all \r;; \, for i = I , . .. n.
hyperboloid of two sheet (see any text in 7. [-
Note that ).. < I. Then \R\ ~ J...(/11 + U),
15. Ye (can use the same orthonorma l eigenba is)
17. Let A be the n x 11 mauix who e entries are l.
The eigenvaJues of A are 0 (w ith multiplicity
where U is upper triangu lar with u ;; = 0 and
u;j = \rij \j).. if j > i. Not e that U" = 0 (see
Exercise 38a). Now IR'\ ~ IR\1 ~ J.. ' (/11 + U) ~
1
multivariab le calculus). On ly the ell ipsoid is
bounded, and the first two surfaces are
connected.
9
. Js u-n [~ ~ Js [-~ n J
11- I ) and 11. 0\ B = qA + p- q )l n, so
that the eigenvalues of B are p - q with
multiplicity 11 - 1) and q11 + p- q. Therefore,
A1 111 Un + u + ... + uu-l . From calculus we
know that I im ) 1 t" = 0.
,.....
b. The matrix A of thi qu adratic form ha
po iti e eigenva lues J.. 1 ~ o. -6 . ..\2 ~ 4 _44 _ and
),3 = I ' with corresponding unit eigenvectors
II [! ~ n[~ nr~ ~]
3
det(B) = p - q)n-! qn + p - q ).
~ 3s] 15 [- ~ ~]
_I . -l8 = 6 · 4 · - (note that A ha 6 unit eigenvectors) 7.4 I. [ -~.5 -~.5] ii, ~ [ - 0.47
~:~~]. v2 ~ [~:;~].
0.78
ü ~ [ -~::~]
0.41
3
13. /_ [

15. Singular value of A- ' are the rec iprocaJ of

23. The only possible eigen alue are I and - 1
tho e of A.

[~ LH -2]
(becau e A is orthogonal). and the eigenspaces Since all eigen value are positive, the . urface
Et and E- t are orthogonal complements 3 i an ellip oid. The points farth e t from the 21. [ 0.8 0.6] [ 9
- 0.6 0.8 -2 6
(becau e A is symmetric). A repre ents the orig in are
=I, . . . ,r
reftection in a ub pace of iR". - 3. AAT-u,. -_ { aFü
?
- ; for i
I 1 0 0 0 5. indefinite 0 fo r i =r+ 1• • . . , 117
1 ü,
± 11 ~ ± [ 0._6
1. 15 ] .
0 I I 0 The nonzero eigen alue of AT A and T are
0 - 0.63
s= h
7. indefinite vAt
Lhe ame.
25. 0 0 0 0 ./2
0 0 I - I 0 and those closest are 25. Choo e vectors ü1 and Ü?- a in Fact . . W n.te
• 7 .52
9. a. A 2 is symmetric
- 1 0 0 0 b. A 2 = - AT Ais negative semidefinite, so that ft = Ci VJ + c2 2·
27. If 11 i even, we have the eigenbas is its eigenvalues are ~ 0. ± 11
1
u2 ~ ± [ 0.26
0.1 5 ] .
Note thal
e,- eu, e2 - en-1· . .. , en/2 - en/2+1. e, + vA2 0.37
c. The eigenvalues of A are imaginary (that is, of
en, e2 + en- l, .... eu/2 + e"/2+1' with associated the form bi , fo r a real b) . The zero matrix
eigenvalues 0 (n /2 ti me and 2 (11 j2 ti mes). is the only skew-sym meLric matri x that is 23. Ye.; A = ~(M +MT) ow

29. Yes diagonalizable over lR. 2s . q(ii) = v·J...v = J....

A-20 • Answers to Odd- umbered Exercises
Answers to Odd-Numbered Exercises • A-21
19. 2 2
so that . = span [ - 1] and E1.4 = span [ -3 ] . .L.00 ks
39. E 1 (/)

ro ugbly li ke the phase portrait in Exerci se 35.

[]AÜI I2 = cf ii Aii l I I ~ + c~ I ] Aii2le 41.
? ? ~ ? __.. trajectory I
= cjuj + c:zui
? ?) ?
:S (Cl -f- C2 Uj

EA,
- trajectory 2

We conclude that IIAüll :S

o-2 :s JI AÜ I) is analogous.
o-1. The proof of
- [l ']-
2 1. x(t) =
0 1
xo
EA.,

-trajectory 3

27. Apply Exercise 26 to a unit eigenvector v witJ1 23 . x(t ) is a so lution. door s Iams 1·r w(O)
a(O) < A.2 .

-~ ] + 0.4e- l [ ~ J
associated eigenvalue ),. 8.2 I. 1
61
27. x(l) = 0.2e- [
33. False; consider A = [ ~ ~] . 3. ,Jie3rr if4

T
35. (A A)- 1 A Üi
T l
_l_ Vi
= l!i for i = l , ... , r
29.x(t) = 2 [ 2]+ e [I]
_
1
s1
2 43 . a. Competition b. Y 5. e-0 11 ( cos(2t) - i sin(2t)); spiraJs inward, in the
clockwise direction '

37. Yes, s ince A A 7

=
0
!11
for i = r + I , . . . , 111
31 <(,) ~ ,, [ -n 7. not stable
9. stable

ll. a. B = 2A
33.
X d. The zero s tate is a stable equilibrium of the
c. Species 1 "wins" if _ng~ < 2. system ~; = grad(q) if (and only if) q is
CHAPTER 8 negative definite (then, the eigenvalues of .4
8.1 I. x(t) = 7e 51 45 . b. y and B are alJ negative).
3. P (t) = 7e 0·031 13. The eigenvaJues of A- I are the reciprocals of the
5. y(t) = - 0.8e0·81 eigenvalues of A ; the real parts have the same
sign.
7. x(t) = 1 ~ 1 , has a vertical asymptote at r = I.
15.
9. x(t) =(Cl - k)t. + I ) 1/ (i - k) X

II. x(t) = tan (t) c. Species I "wins" if y(O) < l

· x(O) 2
13. a. about 104 billjon dollars 47 . a. syrnbiosis
35.
b. about 150 billion dollars
b. the eigenvalues are ~ ( - 5 ± .J9 + 4k2). There
15. The solution of the equation ekT! LOO = 2 is are two negati ve eigenvalues if k < 2; if k > 2 17. If lk I < 1
T
= lOO kn(2) ~ 6f there is a negati ve and a positive eigenvalue.
19. False; consider A with eigenvalues I, 2, - 4.
17. 49. g(t) = 45e- 0 ·81 - l5e- 0.4t and
h (t) = - 45e - 0·81 + 45e - 0 41 21. a.
~~
d
= O.Sb + s I
I af = 0.07s
53. x(t) = e P1
[cos(qt) ] , a spiral if p =/= 0 and a
sm (q t) b. b(t) = 50.000e 0·071 - 49,000e 0·51
circle if p = 0. Approaches the origin if p is s(t) = I , 000e0·071

37. Eu = span [ ~ Jand E1.6 = span [ ~

rough ly like the phase portrait in Figure '10.
l Looks
negative.
55. Eigenvalues AJ.2 = H-
eigenvalues are negative
q ± J q 2 - 4p) ; both
?3
~ ·
-: ) _ [ cos(3t)
'' U - sin(3t)
- sin (3t)
cos(3t)
are arbitrary constants
J[b J' where
a
a' b
A-22 • n wers to Odd- ·umbered Exercise
Answers to Odd- umbered Exe rcise • A-23
[ ~0
uj j j
45 . Yes , yes.
_s. EigenvaJue 2 + 4i wirh corre pond ing
39. The _ystem dc = = =
eigenvector [ _ ] . ~ e Fact 8.2. 6, ' ith dt 49. Ow = T (Ov) - T (Ov)
T (Ov ) + T (Ov) - T (Ov)
T (Ov + Ov) - T (Ov)
T (Ov ) =
p = 2. q = 4 . w= [ ~ J. V= [ _ n. 5 1. Convert the OE into the system
d.x
17'

X(l ) = e2
1

27. Eigen alue -I + 2i with

[ ~ - n[~~~~;; -c~:~::) ] [~ ] where k 1 , k2 • h ar arbitrary con tants. The
Tt =
dv

[~ ] =
y
cit = - 2x -3 y

Cte - r [
.
, Wllh so lut ions

-~] + c2e-
21
[ -~ J
The
b. Basi. of ker(T) : 5 - 61 + 12

Basis of im(T ): [
0
0 - 1 ' 0 -5
3
] [
19. T (J) = gf , where g is a polynomial of
1 3
]

corresponding eigenvector [ ~]. olut ion of the gi en y tem are .~ ( 1 ) = e )..1 r), c so lu l.l ons of the given OE are
degree :::: I, for exa mple, g(l ) = 1.
by Exerci se 8. 1.24. The zer state is a stable 21 [b- 1 J
[-:]=
(l
· a. -a b- I
-~(r) = c~ng;~ c:~~:~ ] = c ,e- r + c2e - 2r _
equili bri um soluti n if and onl y if) the real part x(l )
e- r [ of A. i negative.
53 . E igen paces: E2 = sy mmeti c matrices,
b. f (t ) = (b - l ~ 2 +a 2 ((b - l ) cos(r) +a in(!)) .
2
e- r [ cos (2 l ) + sin ( l ) ] . Spi ral in ward, in the No soluti on if a = 0 and b =I .
in (2r - cos (21)
counterclockwi e direction.
CHAPTER 9
9.1 I. Not a ubs pace.
3. Sub pacewith ba ·i · J-1 , 2 - t 2 .5 - t 3 .
[!~~ rTfiTT~ m
irn E~rr; 23. a. ker(T) = [OJ b. Yes
c. There i (j us t one) such polynomiaJ.
29.
E[ ig~nvaJ] ue_i with c[ ~rre t]on[d~:g(~;g]envector 5. Subspace with ba is 1, 1"
55 . E1genspaces: E, = rea l num ber 9.3 3. a. Jf S is invertibl e. b. If S is orthogonal.
E _, = imagin ary num ber . Eigenbasis I, i. 5. Yes.
I + i · x (l ) = I I in r) = 7. Subspac .
57. E igen pace : E, = even function. , E_ 1 = odd 7. For positi ve k.
1 9. Subspace.
sin ) ] . A n ellipse wi th clockwi e fun cti on . 9. True.
[ in (l ) +CO (I I I. Not a ubspace.
orientation. 13. ot a subspace. 59. E_ , = span ([ _ : ~ ] , [~ _:]). Es= II. The angle is 8.
13. The two norms are equal, by Fac t 9.3.6.
35. - trajectory .r (t) 15. Subspace.
17 . Matri ce with one entry equa l to I and all other span (U ~ l [~ ~ ]) . The e four matrices 15. If the matrix [ ~ ~ J is sy mmetric and positi ve
/ ker A) '9 :n::.nJ' Un~n~r ; mn .
fo rm an eigenbas is.
6 1. The standarcl basis of !Rn x n (the matrices with one
definite. This mean that b = c, a > 0, and
ad- bc > 0.

//
/
21 tr
~::e~ ~].[~ ~].[~ a entry equal to I and all other entries equal to 0).
63. Ye . LeL f be an invertible function fro m JR 2 to
R and defi ne
17. lfk er(T) = {0} .
19. A must be positive defi rri te (compare ' ith
Exercise 15).
/
/ 23 . : ::" tY~
Dimen ion: I.
a 9.2
ü EB w = J Cv) + f(w )
A. O v = A..f(ü) .
I. Ye
3. Ye
21. Ye .
23. I. 2r- 1.
25. j I + ! + ~ + . .. = :16·
25. Basi s: I - r, I - 2
1 . Dimension: 2. ao h-
k = 0 for all k.

[~ ~]
27. =

~ ~ ] , [ ~ ~] .
Note that Ax is parallel to ker(A) for all .X 5. ·v 2
27. Basis: [ Dimension: 2. .
because A 2 = 0. 2

dx
Check anal yticall y that .T(1) = io + 1 Axo
o I ves dx
di = A x:
_
-
29. Basi :
2.
I. I , I . .. .), (0, I, 2 . 3 , . . .) . 01mens10n:

2 4
3 1. a. Basis: I , t , 1 . Dimension: 3.
7.

9. [I
u~ 0 0
2/ 3
I)
~J
bk = 7(i[ if k is odd
{
l
0

29. I: 7'Y = -u-

k odd k-
7T 2
0
if k i even.

- = Ax0
dl b. Basis: 1, 13 . Dimension: 2. II. ba i of ke r(T ) : I - 21 + t 2 . Ba i of
Ai = Axo + 1 A2.io = Axo .J
37 . U e Exerci e 35 to solve the system
33. Linear; not an i omorphi sm.
35 . Linear; not an isomorphism. im (T): Ul [~ J 9.4 I . Ce 51
3. ~ e 31 + ce--1 (use Fa t 9.4. 1-')
5. - I - r + Ce1
37. Isomorphi sm. 13. t - r 3 , 1 +2r - 3r 2.
~~ = (A - Ao /2)c, then apply Exercise 8. 1.24.
39. Linear; not an isomorphi sm. 7. c ,e-·lr + c2e3r
15. ker( F) = Eo = skew-symmetric matrices.
4 1. Linear ; not an isomorphi sm . =
im(F ) = E 1 symmetric matrices. F is 9. c 1e + c2e - 31
31

43. Subspace. diagonaJizable. II. e (c1 cos(r) + c2 sin(l))

1
A-24 • n ' er to Odd- umbered E. r i es

13 . e- 1 c 1 + c2 1) (compare ' i th - xamp le 10). . 5. a. c 1e- 1 + c1 e -~ 1 b. 2e- 1 - e- 21

SUB]ECT INDEX
c. - e - t + _ e -2t
1- . C( + C2 l
d. l n pm1 th e o. cillmor gocs through Lhe
17. e- 1 (C t + '2 1) - ~ CO ( 1)
equ il i bri um state once; in part b it never
19. cos(1) + c 1 cos(J2t ) + c~ . i n( v'2t ) reac he it.
2 1. Ci + C2e- + C3e - lt
1 1
37. x(l) = re- 31 A
T . 3e 51 39. - I (c l + C2 i + C3 f 2 ) of linear differential operator 532
25 . e - 21 +~ 4 1. A. i an eigenva lue ' i th dim (EJJ = 11 , becau e EJ. Affin e . ys tem, 337 of imil ar matrices, 393
_7. - in 1) i. the kem el of the 11th- rder l inear di fferential Algebraic multipli city of an eigen va lue 3 12 Chole ky factorization, 423
29. 1i n r) - t sin(2r ) operator T (x ) - A.x.
.n. !'o
and geo metri c rnultipliciry, 325, 400 Ci rculant rnatri x, 355
1~
o (1)+ in (l )+c , e- 21 +c2 e- 31 and co mp lex eigenvalue , 349 Cla ica l adjoin t 285
31. u r )= qf- CI - e-kt fm) Algorithm, 23 Closed-forrnul a solution
,.....lim v(r) = '.!.!..Kk
'· = tenninal velocity . 45. e, [ l = 121] Angle, 186 for discrete dynamica l svstem 30 1
fo r in verse, 285 - '
and orthogonal tra nsformations, 208
Argument (o f a co mplex number), 343 for Ieas t-squares approximat ion, 223
of a produ ct, 345 for linear system, 283
Associati ve law for matri x multiplicati on 104 Codomain (of a function), 55
A ymptoti ca lly table equilibrium, 358 ' Coefficient marrix
(see table equilibri um) of a linear y tem, 14
Augmented rn atri x, 14 of a linear transformaüon, 53
Colu mn of a matrix 12
Colu mn pace of a matrix, 132
B Column vector, 13
Cornmuting matri ces, 102
Basi , 15 I , 482
Complement , 174
and coord inates, 377
Comp lex eigenvalues. 348
and dimen ion 162 482
and determinant, 350
and un iqu e representation, 154
find ing ba i of a li near space, 484 and rotation-di lati on matrice , 359, 36 1, 380. 465
and stable equitibrium, 359
of an i mage, 167
and trac . 350
of a kernet, 164
of ~~~ . 169 Camplex num bers, 34 1
standard , 163 and rotati on-dilati ons, 345. 503, 5 10
in polar fo rm, 344
Binm·y di gits, 143 , 35 1
Complex-valued functions. 458
Bounded trajecto ry, 367
deri va ti ve of, 459
ex pon nti aL 460
c Component of a veclor. 13
Compo ite fun cti ons, 97
Carbon dat ing 45 1 Cornputati onal co mplexity, 94, 122
Cauchy-Schwarz inequ ality, 186, 51 5 Consistent linear system, 33
and angles, 187 Continuous lea t-squ ares cond iti on. 2 6 5 16
and triangle inequ ality, 192 Continuous linear dynami cal syst m, 442
Center of mass, 3 1 stability of. 463
Characteristi c polynomi al, 311 with cornpl ex eigenbasis. 463
and algebraic multiplicity 312 with rea l eigenbasis, 449
and it deri va ti ve, 320 Coordinate ·, 377 496
1-1 .

1-2 • ubject Index
Coordinate tran formation. 497
Coordinat.e vector, 377 , 496
Correlation (coefficient , 188, 189, 190
Cramer' Rule, 283 , 28-
Cro product
in JR3 , 65 , 554
in IR" , 269
Cubic equation (Ca.rclano's formula) , 320, 356
Cubic spbnes, 27
D Distributive Laws, 106
Data compre.ssion, 433
Data fitting, 225
multi ariate, 228
De Moivre' s forrnula 346
Deterrninant, 243
a.nd characteri tic polynorn:ial 311
a.nd complex eigenvalues, 350
a.nd Cra.mer' s rule, 283
and elementa.ry row operations, 256
and invertibibty, 257
a.nd Laplace expansion. 262
a.nd QR factorization , 277
a a.rea, 274
a.s expansion factor , 279
a volume, 276
i linear in row and column , 252, 253
of inverse, 259
of orthogonal matrix, 273
of permutation matrix, 248
of product. 258, 272
of rotation matrix, 273
of sirn:ila.r matrices. 393
of 3 x 3 matrix. 240, 276
of tra.nspose, 251
of triangular matrix, 246
of 2 x 2 matrix, 239, 274
Vandermonde, 266
Diagonabzahle matrices, 388, 389
ancl eigenbases, 388
and power 391
orthogonall y, 402, 408
imultaneously, 400
Diagonal matrix, 13
Subj ect Index • I -3
and Lrace, 350
Diagonal of a matrix, 13 Flo w line, 443
co mpl ex, 349, 350, 36 1
Dilati on 7 1 Fo uri~r anal y is, 236, 51 8, 520
geo metri c rnulti plic ity of, 325
Dimension. 162, 48 Functlon 55
or: orthogonal matri x, 300
and isomorphi m, 488 Fundamenta l theorem of algebra, 347
ot r? tati on-dil ati on matri x, 359
of imag , 167, 506
of w ntlar matrices, 393
of kerne! , 165 506
of orthogonal complement, 220
of ymmetric matri x, 406 G
of tri angul ar matri x, 309
Direction field , 443 Gaussian integration, 527
. power method fo r findin g, 353
Discrete linear dynamical sy tem, 301 Gauss-Jordan elimination 22
E1 genvectors 298
and complex eigenva lue. , 359 36 1, 363
and linear independence 329 and determinant, 256, 257
and table quibbrium. 358, 359 and inverse, 91
dominant, 448 '
Di tance, 51 4 of ymmetri c matrix, 403 flow chart for, 24
Elernentary matri x, 115 Geometrie m~ltiplicity of an eigenva]ue, 325
Doma.in of a function. 55 and a~gebrcuc multiplicity, 325, 400
Elementary row operati.ons, 2 1
Dominant eigenvector 448 and e1genba es 330
and determin ant 256
Dot product 28, 2 11 , 55 1 Golden section, 340
and e lementary matrices l J 5
and matrix product, 103, 2 11 Gram-Schmidt process, 201 , 515
Ellip e, 85
a.nd prod uct Ax, 41 and determinant, 273
a image of the unü circle, 85, 408, 427
rules for, 552 and orthogonal diagonalization, 408
as lev.el curve of qu adratic fo rm 418, 420
Dyn amical system and QR factorization, 202
(see co ntinuous, di crete bnear dynarnical a traJectory, 363 370, 46 5
Equilibrium (state), 358, 368
system)
Error (in Ieast-squares Solution), 221 H
Error-correcting codes, 144
Euler identities, 51 8 519 Ha.rmonics, 520
E Eu ler' · forrnu1a , 461 Hilbert space e2 , 195, 513
Euler' theorem , 335 , 386 and qua.ntum mechanics, 523
Eigenbasi , 326, 490
a.nd continuous dynamical ystem , 449 , 463 Exotic ?perations (in a linear space), 496 Hornogeneou linear system 48
Expcu1SIOn factor, 279 Hyperbola, 420
and di agon alization, 388
Exponenti al function , 440 Hyperplane, 173
and di crete dynamical y tem , 30 I
and di tinct eigen value , 329 complex-va1ued , 460
and geometric multiplicity , 330 I
Eigenfunction , 532
Eigenspaces, 321 , 490
F Identity matrix /11 , 57
and geometric multiplicity, 325 Identity tra.nsformation, 57
Factorization Image of a function , 128
and principal axes, 420 '
Cholesky , 423 Image of a li near Iransformation ' .1-32'486
Eigenvalue(s), 298, 490
algebraic multiplicity of, 312
LDC , 2 18, 424 and rank, 167
and characteristic polynomia.l, 308 LDU , 116, 218 i a subspace, 133, 146
L{r , 423 ort~ogonal complement of, 2 19
and determinant, 350
and positive (semi)definite matrices, 417 LU , 116, 250 wntten as a kerne!, 143
and QR factorization, 316
QR , 202, 217, 277, 316, 424 Image (of a ubset), 67
and shear-dilations, 395
sos - 1, 39 .1 of the unit circle, 85. 408 427
SDS T, 402 of the unit square. 68
and shears, 393
UL.VT , 43 1 lmag.i i~ary part of a complex number, 342
and singular values, 425
Field, 348, 351 Imphcit function theorem, 272 .
and stable equilibrium, 359, 463
I -4 • Subject Index
Inconsistent linear sy tem, 6, 21, 33, 42 Leading one, 19
and lea t- quares , 221 Leading variable 20
Indefinite matrix, 416 Lea t- quares o lution , 23, 195 222
Infi nite-dimensional linear space, 486 and normal equation, 223
minimal 231
Inner product (space). 511
Input-output analysis, 7, 29, 96, 120, 386 Left inver e, 159
Intermediate value tbeorem, 84 Length of a vector, 177, 552.
Inter ection of subspaces, 156 and orthogona l transformatJOn . 207
dimension of, 174 Linear combiJ1ation, 38, 479
lnver e (of a matrix), 88, 91 and span, 131
and Cramer' s rule, 285 Linear dependence 151
determinant of, 259 Linear differential eq uation, 529
of an orthogonal matrix, 212 homogeneaus 529
of a product. 105 order of, 529
of a 2 x 2 matrix, 61. 91 solution of, 531
of a transpo e, 213 solvi ng a first order, 440, 538
Inversion (in a matrix), 243 solving a second order 534, 535
Invertible function. 86 Lmear differenria l operator, 528
lnvertible matrix, 88, 89 cbaracteristic polynomial of, 532
and kernel. 138 eigenfunctions of, 532
and deternunant, 257 kerne) of, 531
Isomorphism, 487 image of, 539
and dimension, 488
Linear independence
in lR", 151
in a linear space, 482
J and dimension, 164
Jordan normal form, 393 and kemel, 154
and relations, 152
of eigenvectors 329
K and determinant, 258
of orthonorma1 vector 179
Linearity of the determinant, 252, 253
Kemel, 134 486
Linear relation , 152
and invertibility, 138
Linear space(s), 479
and Linear independence, 154
basis of, 482
and rank, 165
dimension of, 482
dimension of, 165, 167, 506
finding basis of, 484
i a subspace, 138, 146, 486
mfinite-dimensional, 486
k-parallelepiped, 276
isomorphic, 487
k-volume 276
Linear system
clo ed-formula solution for, 283
L consistent, 33
homogeneous, 48
Lap lace expansion, 262 inconsistent, 6, 33
LDLT factorization , 218, 424 Ieast-squares solutions of, 222
LDU factorization , 116, 218 matrix form of, 39
minimal so lution of, 231
nurnber of so lutions of, 33
of differential equations: see continuous
linear dynamical system
unique so lu tion of 36
vector form of, 37
with fewer equations than unknowns 35
Linear tran sforrnation , 56, 66, 486
image of, 132, 486
kerne! of, 134, 486
matrix of, 59, 378, 501 , 502
Lower triangular matrix , 13
LU factorizat ion, 116 250
and pri ncipal submatrices, 117
M
Main diagonal of a matrix, 13
Mass- pring ystem, 472
Matrix, 12
(for a composite entry such as ' zero matrix"
see zero)
Minimal so lution of a linear , ystem, 23J
Minor of a matrix , 260
Modulus (of a complex number), 343
of a product 345
Momentum, 31
Multiplication (of matrices), 101
column by co lumn , 102
entry by entry, 103
is associative, I 04
i. noncommutative, 102
of partitioned matrice , 107
N
Negative feedback loop, 470
Neutral eJement, 479
Nilpotent matrix , 176, 412
Norm
of a vector, 177
in an inner product space, 5 14
Normal equation, 223
Subject Index • I -5
0
Orthogonal complemen t.., 179
dimension of 220
is a subspace, 180
of an image, 219
' Orthogonally diagonalizable matrix. 402, 408
Orthogonal matrix, 207, 212
determinant of, 273
eigenvalues of, 300
has orthonormal columns. 209
in verse of, 212
transpo e of, 212
011hogona1 projection, 76, 181 , 515
and Gram-Schmidt proces , 201
and reflect:ion, 76
as close t vector, 221
matrix of, 214, 224
Orthogonal transformat:ion , 207
and Ot1honorma1 bases, 209
preserve angles, 208
preserves dot product 2 J 5
Orthogonal vectors. 177, 514, 553
and Pythagorean tbeorem, 185
Orthonorma1 bases, 180, 183
and Gram-Schmidt process, 201
and orthogonal transformations, 209
and symmetric matrice , 402
Ot1hononnal vectors. 178
are linearly mdependent, 179
0 cillator. 509
p
Parallelepiped, 275
Parallel vecrors, 550
Paramet:rization (of a curve), 129
Partitioned matrice , I07
Pattem (in a matri ), 242
Permutation matrix. 94
determin anr of, A8
Perpendicular vector . 177, 5 14, 553
Pha ·e portrair.
of discret y tem, 296, 30 I
of continuous system, 445 450
summary. 468
I -6 • ubj ct Index
ubj ect In dex • 1-7
o f irni lar matrice , 393 Span
Pi ecewi e o ntin uou function. -1
of tra n pose, _1 3 in IR!", 13 1, 132 in vertible, 93
Pi ot column . J 67
Real part of a complex num be r. _42 in a li near space, 482 Tri angu li zable matrix, 41 2
Pi o tin g 23
Redu ced row echelon form (rref) , 19 Spec tra l theo re m 402 Tri gonometri e po lyno mi als, 5 18
Po lar form (of a complex number), 344
and derermin ant. 257 Spirals, 346, 360, 363, 465
and po' e r . 346
and product . 345 and in er e. 89, 9 1
and rank. 34
Square matr ix, 13
Square- umm a ble eq uences, 1. 94, 5 13
u
v ith Eul er· fo mm la, 461
Po iti e mi definite matri x. ~ 1 6 Refte ti o n. 77 , 112 Stab le eq uil ibrium, 358
Unit vector 177, 553
and i!!en aJue . 4 17 Regular tran irion matrix. 3 1 . 352 of di . cre te y tem, 359
Upper tri ang ul ar matri x, 13
and prin ipaJ ubmatri e . -+ 17 Relation among ecto1 ), 152 of co nti nu o u ystem, 463
Pos itively a ri enred basi . _ Re onan ce, 509 Sta ndard ba i , J 63, 497, 499
Po' e r merhod fo r fi nding eigen a 1ue. . 3 - 3 Ri mann integral, 236, 51 2 Stare vector, 29 J V
Principal axe . 4 0 Rotati on matrix, 5 7, 70 81 , 112, 273 Sub pace
Principal ubm atrice , 11 7 Rotati on-dil ation marr ix. 72 of IR", 146 Vandermonde deterrninant, 266
and po iti ve de finüe m atrice . 4 17 and co mp lex eigenva lue , 359, "' 61 , 380, 465 of a linear pace, 48 1 Vector( ), 13
Produ ct A.r. 38 and co mplex number . 345 . 503, 5 10 Sum of ub pace , 160 add iti on, 546
and dor product, 41 Row of a matri x, 12 dim ension o f J 74 column 13
and matri · mul tiplication, 10 Ro' pace, 175 Sum of matri es, 42 coordin ate, 377, 496
Proj ecti on, 1 RO\: vector, 13 Symmetrie matrice , 2 1 J cro produ ct of, 65, 269 554
(see al o orthogonal projectio n) Rul e of 69, 45 1 are orthogo na lly d iagona1izab le, 402 dot product of, 551
P eudo-in ver e, 230, 231 have rea l eige nvalu e , 406 geometri c representation of, 548
Pyth agorean rheorem, 185, 5 15, 553 length of 177, 552
s T orthogonal, J 77. 553
parall el. 550
Q Sarru · rule, 240 Tetrahedro n. 11 4 286, 385 po iti on. 548
doe not generalize, 242 row, 13
Theorem of Pythagora . 185, 5 l5. 553
QR facrorization. 202 Scalar. 15 rule of ector algebra, 547
Trace, 3 10
and Chole ky factorization, 4_4 Scalar mul tiple of matrice . 42 calar multi plica ti on, 547
and cbaracteri ti c po lynornial , 3 J 1
and dererrni nam. _77 econd deri ari e te t. 424 tandard repre entati on of 548
and compl ex eigenvalue , 350
and eig n a1ue , 316 Separation of variable , 439 unit, 177. - 53
and inner product, 513
i unique, 217 Shear . 73 velocity 442
of imj]ar matrice , 393
Quadrati fo nns . 4 15. 4 16 and e ige nvalues, 393 v . po in , 549
Traj ecto ry , 295, 442
indefinite, 4 16 Shear-di lation , 395 zero 547
bounded, 367
negati ve ( emi definite. 4 16 S imil ar matri ces, 392 Vector fi eld 443
po iti ve e mi definite, 4 16 Transpo ·e (of a matri x , 2 11
and c haracteri tic po lynomial , 393 Vector form of a linear y tem, 37
principal axe for , 4 _0 and determ in ant, 25 1
Sim ultaneou ly diagonaJi zabl e matrice , 400 Vector pace, 479
Qu aremion . 2 1 . 369, 5 10 andin ve r , 2 13
Singul ar alue decom po ition S D ), 4 3 1 ee linear pace)
fo m1 a ke\ fi e1d. 355 and rank. 2 13
Si ngul ar alue , 425 Velocity vector. 44_
of an orthogo nal matri x, 2 12
and ellip e , 427 of a produ ct, 2 J 3
R a nd rank, 429 Tri ang le inequ ality, 192 z
Skew field , 355 Triang ular matrices, J 3
Rank (of a matrix), 34, 167 Ske w-symmetric matrices, 2 16, 42 1 and det:ermin ant , 246 Zero matrix , 13
and im age, 167 dete rrninant of, 268 and e igenva lues, 309 Zero vector, 547
a nd kerne !, 165 Smooth fu nction, 477
and ro w pace, 175 Space, 146
and in g ul ar va lues, 429 of fun clions, 479
 NAME INDEX Peirce, Benjamin, 46 1 ame Index • 1-9
Pi azzi, G iu s ppe, 22 Sch:varz, Hennann Amandu s, 186
Pyth ago ras, 184 Sek1 Kowa, 243
Strassen, Volker, 122
Sylves ter, James Jo. eph, 12
A H R
bei, iel Henrik. 3 l 6 Hami lton, Willi am Rowan, 2 18, 35 5 Ri emann , Bernh ard, 236 V
AI -K110wariz mi Mohammed, I . 23 Hei enberg, We rner, 195, 523
Archi mede-. 34 1 Hilben. Dav id. l 95 Vandermonde ' Alexand Ie-Theophile,
. , . 244. 266
rgand. Jean Roben , 342 s
J Sarrus, Pi er-r Fn~deric, 240 w
c Jacobi . Carl Gu tav Jacob. 244
Schläfli , Ludwig, 163
Schmidt, Erhardt, 20 1 Weierstra , Kar!, 272
Jordan , Camille, 393 Wes e1 Caspar, 342
Cardano, Gerolan1o. 3 16. 320 Sc hrödinger, Erw in , 523
Jordan, Wilhelm. 22 Weyl. Hermann , 163
Cauchy. Au gustin-Loui , 186, 244
Cramer. Gabriel. _83
K
D Kepl er, Joh annes. 238
Kronecker. Leopold. 244
D'Alembert, Jean Le Rond, 163
D Moivre. Abraham , 346
De cartes. Rene, 498 L
De Wirt , Johan. 4 16
Lagrange, Joseph-Loui , 369
Laplace. Pierre Simon . 259
E Leo nardo da Vinci , 193
Leonardo of Pi a, 339
Einstein, Albert, 356 Leon tief, Wa sily, 7, 386
Euler. Leonhard. l I, 335,342, 3 6, 46 1, 518, 523

M
F
Mas-Co lell Andreu , 195
Fibonacci , 339 Mumfo rd, David, 476
Fourier, Jo e ph 236, 5 18
0
G Olbers, Wi lhelm , 23
Galoi, , Evari te, 3 16
Gau . Carl Fried1ich , 22, 62, 342, 346, 527 p
G ram , Jörgen 20 I
Gra mann , Hermann Günther, 163 Peano, Giu eppe, 479

1-8
 IS BN

9 780131 90729lf