1

Applied Linear Algebra

by S. K. Cho and P. D. Ion
Panglossian Publisher, 4776 Carter Place, Ypsilanti, Michigan 48197
1- 855- 224- 0357

This book is linear algebra applied to engineering and physics problems. Since tensors, exterior
forms as well as differential and integral equations are increasingly employed in modern physical
problems, and they invariably come down to linear algebraic ones at the stage of numerical com-
putation(there is little exception to this), we regard them as part of linear algebra. There is also a
matter of physical dimension that cannot be avoided. While mathematics is founded on numbers,
and the notion of physical dimension may be a nemesis to mathematicians, it is natural to physicists
and engineers. Therefore, we are mindful of them in this book.

There are two special topics of interest, which are based on our discoveries. One is a certain
algebraic structure, the other is a reinterpretation of the traditional “Fourier series expansion” as a
system of linear algebraic equations; namely as a map, not as an expansion.

I found that as a fundamental rule, a system of equations alone cannot exist, but only with its
dual system, and they come entangled , forming an irreducible whole. Following consequences occur
from this entanglement:

• The matrix in general does not map directly from the domain to its codomain(which is pos-
sible only for a specialized case in which an exacting condition is met), but instead via the
fundamental route , or “a taxi-driver’s” route,

• furthermore, there is no room for the eigenequation, but only for a pseudo eigenequation: the
familiar eigenequation for a matrix is fundamentally subsumed to the pseudo eigenequation.

Every m × n-dimensional ill-posed system of equations, Ax = b, induces its n × m-dimensional dual

system of equations A† b̃ = x̃, which are entangled in the sense that one cannot exist without the
other. These entangled systems are tied together by a linear functional and a dual linear functional,
and lead to a pseudo and a dual pseudo eigenequations. The pseudo eigenvectors {ui } of A and the
dual pseudo eigenvectors {ṽi } of A† are, simultaneously, the eigenvectors and dual eigenvectors of
the following selfadjoint matrices A† A and AA† , which are equivalent:

A† Aui = νi ui ,
) )
Aui = λi ṽi ,
⇐⇒ i, j ≤ r < (n, m).
A† ṽi = µ̃i ui AA† ṽi = νi ṽi .

Here r denotes the rank of A(hence of A† , A† A and AA† ); i.e., the number of non-zero eigenvalues
of the selfadjoint matrices. These pseudo and dual eigenequations show that the given system and
its dual system cannot exist without the other; namely, they are united in an irreducible single
structure. This irreducibility is fundamental in the sense that it holds in every system of linear
algebraic equations. David Hilbert was the first who detected an iceberg piece of this structure
peaking out in his algebraic alternative, and Cornelius Lanczos explored it further, finding more bits
and pieces, but was unable to uncover the whole structure. The equivalence mentioned above can
be seen by multiplying A† to the pseudo eigenequation, and A to the dual pseudo eigenequation,
and also A and A† to the first and the second of the selfadjoint eigenequations to return back to the
pseudo and the dual pseudo eigenequations.
2

The pseudo eigenvectors {ui } and dual pseudo eigenenvectors {ṽi } form respectively an r × n-
and r × m-dimensional semi-unitary matrices Ur and Ṽr as

U † Ur = In , Ur Ur† = Ir ,

Ṽr† Ṽr = Im , Ṽr Ṽr† = Ir .

Now, the pseudo and dual pseudo eigenequations in matrix form are

AUr = Ṽr Λr ,

A† Ṽr = Ur M̃r ,

where Λr and M̃r are r-dimensional diagonal matrices with λii and µ̃ii on the main diagonals(the
double-indices on them are because they are elements of matrices). Mulitplying them with Ṽr† and
Ur† and using the semi-unitarity, there result

Ṽr† AUr = Λr ,

Ur† A† Ṽr = M̃r .

The rectangular matrices A and A† are diagonalized in the r-dimensional subspaces. We call these
fundamental spectral decompositions of A and A† because they hold for every matrix, ill-posed as
well as well-posed(the full-ranked case). This means that the matrix A and its dual A† are activated
from the beginning only in the r-dimensional subspaces(the n-dimensional spaces in the full-ranked
cases). Backtracking these decompositions leads to natural inverse of A (dual natural inverse of A† ),
the terminology coined by Cornelius Lanczos, a Hungarian mathematician(cf. Linear Differential
Operators, C. Lanczos, Dover).

Multiplying the pseuo eigenequation with Ur† and the dual pseudo eigenequation with Ṽr† from
the right, we obtain the fundamental decomposition of A and A† as

A = Vr† Λr Ur† ,
)

A† = Ur M̃r Ṽr† .
The route of the fundamental decomposition (the dual fundamental decomposition) is not a direct
route from the domain to the codomain(the dual domain to the dual codomain) as traditionally
interpreted, but roundabout “taxi-driver’s ” routes as the fundamental rule.

In the n-dimensional domain V of A resides an r-dimensional pseudo eigenspace Γr of A, and

likewise in the m-dimensional codomain W of A† an r-dimensional dual pseudo eigenspace Φ̃r .
0
Then, the n- and m-dimensional vectors x ∈ V and b ∈ W are related to x0r ∈ Γr and b̃ ∈ Φ̃r as
0 0
x = Ur xr , b = Ṽr br . Substituting these into the equation Ax = b, we obtain the solution we sought:

b0i
x0i = , i = 1, 2, · · · , r.
λi
This corresponds to the least squares solution, but is an r-dimensional vector solution. Outside the
matrix Λr , the double indexing on λii in Λr is dropped. The solution can be expressed in V and W̃
spaces if we so wish.
 3

The Fourier series expansion of continuous or even piece-wise continuous functions has been with
us since 1807. But I discovered that it cannot be a series expansion, but instead a system of an
infinite-dimensional linear algebraic equations in which the “Fourier coefficients” are components of
an unknown vector in the domain space and the function to be expanded in series is a denumerably
continuous pre-assigned non-homogeneous vector in the codomain space, and the Fourier matrix
maps the domain to its codomain in one-to-one. The scalar kernels of the Fourier matrix are made
of biorthonormal sinusoidal functions. That is, this system of equations is a mapping between a pair
of dual vector spaces, not an expansion in a single space.

Let us consider an n-dimensional equation as an example. To begin with, the finite intervals in the
domain and the codomain are made of denumerable rational numbers. In contrast, the real numbers
are not denumerable as in the case of the Fourier integral transformation. Thus, the Fourier’s works
are composed of two different kinds of maps defined on two different sets of numbers: One on the
finite interval of denumerable rational numbers for the system of linear algebraic map for continuous
functions, the other on an infinite interval of non-denumerable real numbers for the integral map for
square-integral functions.

First, a given continuous function over a finite interval [0, l]. The unknown vector c(k) is de-
fined over a denumerable finite interval [k1 , kn ], km = 2πm/l as {cj (kj )} in the domain space. The
problem is to find this unknown vector in the domain space, hence an inverse problem or an infinite-
dimensional algebraic equations. Let the variable x in the codomain be a distance vector(with a
physical dimension of length). Then, the variable k in the domain must represent a wavenumber
vector(with a physical dimension of inverse length) if xk is to be used as the dimensionless indepen-
dent variable of sinusoidal functions in the matrix as Fourier proposed. In other words, xk must be
a linear functional.

At each point xi in the codomain, the ith equation is given by

n
1 X
fi (xi ) = √ αi,j (xi |kj )cj (kj , αi,j = ei(xi kj ) .
l j=1
Here (αi,j (xi |kj )) is the n × n-dimensional Fourier matrix , which maps the domain in one-to-one
into the codomain on the strength of the biorthonormality of the scalar kernels of the Fourier matrix
with those of its adjoint Fourier matrix, which will be defined shortly. This biorthonormality was
proved by Lejeun Dirichlet, a German mathematician. The matrix form of this algebraic equation
is

eix1 k1 eix1 k2 eix1 kn

    
f1 (x1 ) ··· c1 (k1 )
 f2 (x2 )  1  eix2 k1 eix2 k2 ··· eix2 kn   c2 (k2 ) 
= √  .
    
 .. ..  ..
 .  l .  . 
fn (xn ) eixn k1 eixn k2 ··· eixn kn cn (kn )
The so-called Fourier coefficients in the expansion theory is the solution of the above equation (the
vector in the column matrix on the right side), which takes the form
Z l
cj (kj ) = dxf (x)e−ikj |xi , j = 1, 2, · · · , n
0

where the denumerably continuous points on which f (x) is defined is expressed at denumerably
continuous points, and e−i(kj xi are the elements of the adjoint Fourier matrix , which is obtained
by interchanging the rows and columns of the Fourier matrix and complex-conjugationog the en-
tries. f (x) under the integral sign is a denumerably continuous function given by an n-dimensional
4

algebraic equations in which the Fourier matrix maps the unknown vector {cj (kj ))} to it. The
biorthonormal relationship of the entries of the Fourier matrix and its adjoint Forier matrix guar-
antees the one-to-one relation between c(k) and f (x). We now see that Fourier proposed, without
realizing it, a system of linear algebraic equations using biorthonormal sinusoidal functions, and
offered its solution at the same time! The objection which Fourier’s contemporary mathematicias
raised in 1807 melts away as the mapping from the domain space to its codomain space.

Incidentally, the entries of the Foruier matrix, which is unique in the realm of the mathematical
physics, are made of infinite numbers of eigensolutions of the Sturm-Liouville homogeneous bound-
ary value problem or an interior resonance problem. It is astounding that we are surrounded by
macroscopic natural phenomena as well as man-made information technology which can be explained
in one fell sweep by the Fourier’s idea conceived more than two hundred years ago of constructive-
destructive interference of pure sinusoidal waves which even extends deeply into the mysterious
microscopic quantum world where particles are waves, thereby bringing in the unavoidable notions
of the Heisenberg’s uncertainty principle and the Schrödinger’s entanglement.