Linear Algebra

LINEAR ALGEBRA FOR

TEST AND ANALYSIS
IMAC - XIX
Daniel C. Kammer
Department of Engineering Physics
University of Wisconsin

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MOTIVATION
The use of matrix and vector algebra is an absolute requirement
for the efficient manipulation of the large sets of data that are
fundamental to applications in structural dynamics, both test
and analysis.
Primary problems to be solved:

[ A]{x} = {b}

[ M ]{x} + [C]{x} + [ K ]{x} = {F(t )}

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

LECTURE AGENDA
Common Nomenclature and Definitions
Solution of Determined Sets of Equations
Solution of Overdetermined Sets of Equations
Solution of UnderDetermined Sets of Equations
Example Applications

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

NOMENCLATURE
[ A]
{ x}

matrix

vector

n
m

number of rows (equations)

number of columns (unknowns)

matrix transpose

matrix Hermitian transpose

matrix inverse

matrix generalized inverse

complex conjugate

[ A]T
[ A]H
[ A]1
[ A]+
*

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MATRIX EQUATIONS,
THREE CASES CAN OCCUR
[ A]nm {x}m1 = {b}n1
1. Underdetermined: n < m

Optimization
FEM updating
Projection of data onto subspaces

2. Determined: n = m

Analytical structural dynamics using

finite element models

3. Overdetermined: n > m

Time and frequency domain parameter

estimation
Least squares applications
Sensor placement algorithms
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

BASIC DEFINITIONS
Matrix:

A matrix is an array of numbers. Entries can be

referred to by their row and column location.

a11
a
21
[ A] =
a31
a
41

Vector:

a12
a22
a32
a42

a13
a23

a33
a43

A vector is a special case of a matrix with either one

row or one column.

b1
b
2
{b} =
b3
b4

Column subscript is dropped

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

CAN TRANSFORM A SET OF

ALGEBRAIC EQUATIONS TO A SINGLE
MATRIX EQUATION
3 x1 2 x3 = 2
2 x1 + 5 x 2 4 x3 = 1
7 x1 3 x3 = 3
4 x1 + 6 x 2 + 2 x3 = 5
Equivalent to:

3
2

7
4

0 2
2
x1

5 4 1
x2 =
0 3 3
x3

5
6 2

No Exact
Solution

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

RULES FOR MATRIX OPERATIONS

ka11
k[ A] = ka21

ka31
Multiplication of a matrix by a matrix:
Multiplication by a scalar:

ka12
ka22
ka32

ka13
ka23

ka33

[ A][ B] [ B][ A]

NOT cummutative in general

([ A][ B])[C] = [ A]([ B][C])

Associative

([ A] + [ B])([C] + [ D]) = [ A][C] + [ A][ D] + [ B][C] + [ B][ D]

Matrix cancellation:

[ A][ B] = [0]

Implies one
of following

[ A] = [0]
[ B] = [0]
[ A] and [ B] singular

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MATRIX MULTIPLICATION
[ A][ B] = [C]
a11
a
21
a31

a12
a22
a31

a13
a23
a31

b11
a14
b21

a24
b31
a31
b41

b12
c11

b22
= c21
b32
c31

b42

c21
c22

c32

c22 = a21b12 + a22 b22 + a23 b32 + a24 b42

cij = aik bkj
k

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SPECIAL MATRICES
Identity Matrix:
1s on the diagonal, zeros everywhere else.

1 0 0
[ I ] = 0 1 0
0 0 1
Zero Matrix:
Zeros in all locations.

0 0 0
[0] = 0 0 0
0 0 0

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MATRIX TRANSPOSES
Transpose:

Interchange rows and columns.

1 2 3
[ A] = 4 5 6
7 8 9
Hermitian Transpose:

1 4 7
[ A]T = 2 5 8
3 6 9

Interchange rows and columns and then take

complex conjugate of each element.

2
3i
1+ i
5
6
[ A] = 4 4i

8 + 8i 9
7

7
1 i 4 + 4i
5
8 8i
[ A]H = 2

6
9
3i

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SPECIAL MATRIX FORMS

Symmetric:

[ A] = [ A]T

1 2 3
[ A] = 2 4 5
3 5 6
Hermitian:

[ A] = [ A]H

4 + 3i 5i
1
2
2 + i
[ A] = 4 3i

2i
0
5i
All diagonal terms real

Skew Symmetric: [ A] = [ A]

0 2 3
[ A] = 2 0 5
3 5 0
Skew Hermitian:

All diagonal
terms 0

[ A] = [ A]H

4 + 4i
7
i
(8 8i )
[ A] = ( 4 4i ) 5i

7
8 + 8i
0

All diagonal terms imaginary or 0

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SPECIAL MATRIX FORMS

Orthogonal

[ A][ A]T = [ A]T [ A] = [ I ]

Unitary

[ A][ A]H = [ A]H [ A] = [ I ]

Idempotent

[ A]m = [ A]

Nilpotent

[ A]k = [0]

any integer m

for some integer k

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SPECIAL MATRIX FORMS

Diagonal:

Triangular:

1 0 0
[ A] = 0 2 0
0 0 3

1 4 5
[ A] = 0 2 6
0 0 3

Upper
triangular

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

TOEPLITZ MATRIX
All elements on any superdiagonal and subdiagonal are equal.

t1
t
2
[T ] = t3
t
4
t5

t6
t1
t2
t3
t4

t7
t6
t1
t2
t3

t8
t7
t6
t1
t2

t9
t8

t7
t6

t1

Does not have to be square.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

TOEPLITZ MATRIX EXAMPLE

Discrete Time
Invariant SISO System:

Combine to produce
Convolution equation

{x(k + 1)} = [ A]{x(k )} + [ B]u(k )

y( k ) = [C ]{x (k )} + Du( k )

y( k ) = H ( k )u( k i )
i=0

Can write convolution equation in matrix form.

H0
0

0
0

H1
H0
0
0
0

H2
H1
H0
0
0

H3
H2
H1
H0
0

H4 u4 y4
H3 u3 y3

H2 u2 = y2
H1 u1 y1

H0 u0 y0

For MIMO systems, Hi

are matrices, producing
a block Toeplitz matrix

Time domain identification techniques

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

HANKEL MATRIX
All elements on any superdiagonal and subdiagonal perpendicular
to the main diagonal are equal.

h1
h
2
[ H ] = h3
h
4
h5

h2
h3
h4
h5
h6

h3
h4
h5
h6
h7

h4
h5
h6
h7
h8

h5
h6

h7
h8

h9

Does not have to be square.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

HANKEL MATRIX EXAMPLE

Modal parameter estimation using Eigensystem Realization Algorithm (ERA)
For SISO system, form Hankel matrix:
hi is ith Markov parameter
derived from measured FRF
or free-decay data
MIMO results in a block
Hankel matrix

h1
h
2
[ H (0)] = h3
h
4
M

h2
h3
h4
h5
M

h3
h4
h5
h6
M

h4
h5
h6
h7
M

L
L

L
L

A singular value decomposition (SVD) is performed that then leads

to estimates of the modal parameters.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

VANDERMONDE MATRIX
First column is 1s with successive columns being the second column
with its elements raised to increasing integer powers.

1
[V ] = 1

1
1

v1
v2
v3

v12
v22
v32

v4
v5

v42
v52

v13

v23
v33

v43
v53

Nonsingular iff
All vi distinct

Occurs in curve fitting and some frequency domain

parameter estimation methods.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

VANDERMONDE MATRIX EXAMPLE

{y1

Fit a polynomial to a set of data points

either test or analytical.
Find an expression for y as a
function of variable x

y2

y2 = a0 + a1 x 2 + a2 x 22 + a3 x 23
y3 = a0 + a1 x3 + a2 x32 + a3 x33
y4 = a0 + a1 x 4 + a2 x 42 + a3 x 43
y5 = a0 + a1 x5 + a2 x52 + a3 x53

y4

y5 }

y = p( x ) = a0 + a1 x + a2 x 2 + a3 x 3
yi = p( xi )

Generate matrix equation using x-y pairs.

y1 = a0 + a1 x1 + a2 x12 + a3 x13

y3

1
1

1
1

x1

x12

x2
x3
x4
x5

x 22
x32
x 42
x52

x13
y1
a
3 0
x 2 y2

a
3 1
x3 = y3
a
3 2
y
x4
4

3
3
y5
x5

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

RESPONSE SURFACES CAN BE SYNTHESIZED

TO ACT AS SURROGATES FOR COMPLEX
NUMERICAL SIMULATIONS

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SOME MATRIX MEASURES

a b
= ad cb
[ A] =
c d

Determinant:

a11
[ A] = a21
a31

a12
a22
a32

a13
a22
a23 = a11
a32
a33

a23
a21
a12
a32
a31

Exists only for square matrices.

[ A][ B] = [ A] [ B]

[ A] = [ A]T

a23
a21
+ a13
a33
a31

a22
a32

Etc.

Determinant is zero for singular matrices.

[ A]* = [ A]H

k[ A] = k n [ A]

Trace:
Sum of diagonal elements
for a square matrix.

tr([ A]) = aii

i

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

VECTOR SPACE
The set of all n dimensional vectors make up an n dimensional vector
space, Rn.
n vectors {e}i in Rn are said to be linearly independent if the equation

0 = a1 {e}1 + a2 {e}2 + a3 {e}3 L an {e}n

only has a solution in which the constants ai are all zero. The n vectors {e}i
are said to span the vector space.
Any n dimensional vector {x} can be expressed uniquely as a linear
combination of the n linearly independent vectors {e}i :

{x} = b1 {e}1 + b2 {e}2 + b3 {e}3 L bn {e}n

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

VECTOR SPACE THEORY APPLIED TO

MATRICES

[ A]nm {x}m1 = {b}n1

Rank of a matrix is the number of linear

independent columns or rows.

rk ([ A]) = r

rk([A]) = m

[A] is full column rank.

rk([A]) = r < m,n

[A] is rank deficient. If [A] square, it is

singular and [A]-1 does not exist.

Column Space of [A] is the vector space spanned by its columns.

R([ A])

Row Space of [A] is the vector space spanned by its rows.

Null Space of [A] is the set of vectors {x} such that:

N ([ A])

[ A]{x} = {0}

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

EXAMPLE
1
3
[ A] =
8
1

0
7

3 13
4 6

2
1

0.12
0.35
R([ A]) =
0.92
0.12

0.37
0.18

0.55
0.73

rk ([ A]) = 2

[ A]T [ A] = 0
Determinant of any 2-by-2 submatrix
is nonzero.

0.82
N ([ A]) = 0.41

0.41
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

FOUR SUBSETS OF MATRIX ALGEBRA

[ A]nm {x}m1 = {b}n1
R

[A] has rank r

[ A]

Rn

Dim(m-r)

R([A])
N([A]T)

N([A])

Dim(n-r)

Dim(r)
R([A]T)

Dim(r)

[ A]T
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SPECTRAL DECOMPOSITION
In most cases, a square n-by-n matrix [A] can be decomposed into a product
of three matrices
[ A] = [][][]1
where [] is diagonal with entries called eigenvalues and [] is the
modal matrix containing columns {}i called eigenvectors.
This implies that [A] is diagonalizable.

[]1[ A][] = []

Eigenvalues and eigenvectors satisfy

the eigen-problem:

([ A] i [ I ]){}i = {0}

Eigenvalues satisfy the characteristic

equation:

([ A] i [ I ]) = 0

Note: Eigenvectors {}i are in null

space of matrix ([A] - [ ])

[ A] = i
i =1

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SPECTRAL DECOMPOSITION
Matrix [A] is diagonalizable iff it possesses n linearly independent
eigenvectors.
If [A] has distinct eigenvalues, it is diagonalizable.
1

a12 11 12 1 0
11
12 a11

a

=0

21

22 21
22 21
22
2

1 2 3
[ A] = 4 5 6
7 8 9

0.41
0.23 0.79
[] = 0.53 0.09 0.82
0.82 0.61 0.41

0
0
16.12
1.12 0
[] = 0

0
0
0

Note: Real nonsymmetric matrices can have complex eigenvalues and

eigenvectors. If so, they occur in complex conjugate pairs.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SPECIAL CASE
REAL SYMMETRIC MATRICES
Always are diagonalizable.

Always possess real eigenvalues and eigenvectors.

Modal matrix is orthogonal.

[]T = []1

[]T [ A][] = []

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

STRUCTURAL DYNAMICS
[ M ]{x} + [ K ]{x} = {0}

it
Assume: {x} = {}e

[M] real symmetric positive definite. [K] real symmetric positive semidefinite.
Multiplying through by [M]-1 and
substituting leads to eigenvalue problem:

([ M ]

[ K ] i [ I ]){}i = {0}

Modal matrix simultaneously

diagonalizes [M] and [K].

[]T [ M ][] = [ I ]

[]T [ K ][] = []

Y
Z

Decoupled system in modal

coordinates.

{q} + []{q} = []T {F}

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SINGULAR VALUE DECOMPOSITION

Any n x m matrix [A] of rank r can be
decomposed into a product of three matrices:

[U ](n n)
[ S](n m)
[V ]( m m)

[ A] = [U ][ S][V ]H

[U ]H [U ] = [ I ]
[V ] H [V ] = [ I ]

[U] and [V]

are Unitary
Matrices can be partitioned as:

[ A] = [[U1 ]

or

[U1 ](n r ) [U2 ](n (n r ))

[V1 ](m r ) [V2 ](m (m r ))

H
[ ] [0] [V1 ]
[U2 ]] 0 0 H
[ ] [ ] [V2 ]

[ A] = [U1 ][ ][V1 ]

[ A] = {u1 }i i {v1 }i

i =1

{u1}i is ith
column of [U1]

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SINGULAR VALUE DECOMPOSITION

1 0 0 L
0
0 L
2

[ ](r r ) =
0 0 O 0
M
M
M r

i2 , {u1 }i

{u2 }i
i2 , {v1 }i

{v2 }i

Singular values are

real and satisfy:

1 2 L r > 0

- are eigenvalues and eigenvectors of [A][A]T

- are eigenvectors of [A][A]T with zero eigenvalues.
- are eigenvalues and eigenvectors of [A]T[A]
- are eigenvectors of [A]T [A] with zero eigenvalues.

[U1] spans the column space of [A].

[U2] spans the column null space of [A].

[V1]T spans the row space of [A].

[V1]T spans the row null space of [A].

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SVD CAN BE USED TO DETERMINE

RANK OF A MATRIX
2

10
0

10
-2

10
-4

10
SINGULAR VALUE

It is obvious to the
casual observer that
matrix [A] has
a rank of 11

-6

10
-8

10
-10

10
-12

10
-14

10
-16

10

8
10
12
14
SINGULAR VALUE NUMBER

16

18

20

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

UNFORTUNATELY, IT IS NOT ALWAYS

SO EASY
1

10

10
SINGULAR VALUE

Singular values for a

block Hankel matrix
640 rows and 1380
columns.

-1

10

Whats the rank?

-2

10

20

40

60
80
100
120
SINGULAR VALUE NUMBER

140

160

180

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MAXIMUM SINGULAR VALUE OF FRF

MATRIX VS. FREQUENCY

Gives measure of
participation of
modes in system
response.
Use to rank dynamic
importance of modes.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SOLUTION OF DETERMINED
EQUATIONS
Have as many independent
equations as unknowns.

[A] is square, full rank,

and invertible.

{b} is always in range

space of [A].

[ A]nn {x}n1 = {b}n1

{x} = [ A]1 {b}

A unique solution always

exists.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MATRIX INVERSE
The inverse of a nonsingular matrix [A]
is a matrix [A]-1 that when multiplied by
[A] is the identity matrix.
Properties:

Application:

([ A][ B]) 1 = [ B]1[ A]1

([ A] )
([ A] )

[ A][ A]1 = [ A]1[ A] = [ I ]

T 1

= ([ A]

H 1

= ([ A]1

)
)

Solve for static

deflection

[ K ]{x} = {F}

1 T
H

1
k
1
=
[ A]

(k[ A]) 1 = [ A]1

[ A]1

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

A METHOD TO COMPUTE INVERSE

[ A]1 =

[ Adjo int([ A])]

[ A]

Adjoint of [A] is a matrix with elements

equal to the cofactors of [A] transposed.

Let [Mij] be the submatrix of [A]

obtained by deleting the ith row
and jth column. The determinant
of [Mij] is called a minor of [A].

a b
[ A] = d e
g h

c
f

There are better methods!

[ ]

Cofactor([ A])ij = cij = ( 1)i + j Mij

e f
+ h i

d f
1
1

[ A] =
[ A] g i
d e
+
g h

h
a
+
g
a

c
i
c
i
b
h

b
+
e
a

d
a
+
d

c
f

c
f
b

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

LU DECOMPOSITION
Any nonsingular matrix square matrix [A] can be factored into the
product of two matrices.

[ A] = [ L][U ]
[L] is a lower triangular matrix and [U] is an upper triangular matrix.

6 2 4 4
3
3 6 1

[ A] =
21 8
12 8
6 0 10 7

2 0 0
1 1 0
[ L] =
4 2 1
2 1 2

0
0

0
2

3 1 2 2
0 2 4
1

[U ] =
0 0 5 2
0 0 0
4

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

LU DECOMPOSITION
WHATS IT GOOD FOR?
Inverse:
Determinant:
Solving determined set
of equations.

Can be written as two

sets of equations.

[ A]1 = ([ L][U ]) 1 = [U ]1[ L]1

[ A] = [ L] [U ]

[ A]{x} = {b}

[ L][U ]{x} = {b}

[ L]{z} = {b}

Solve using
forward substitution

[U ]{x} = {z}

Solve using
backward substitution
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

BACKWARD SUBSTITUTION
3 1 2 2 x1 1
0 2 4
1 x 2 5
=

0 0 5 2 x3 2
0 0 0
4 x 4 6

[U ]{x} = {z}

Last equation
implies:

x4 =

6
4

Substitute into
third equation, etc.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

TEST-ANALYSIS-MODEL DEVELOPMENT
USING STATIC REDUCTION
[ M ]{x} + [C]{x} + [ K ]{x} = {F(t )}

Reduce mass matrix to test sensor locations for test-analysis

correlation and still maintain accurate modal properties.

Objective:

Partition static equation

into sensor dof (a-set) and
dof to be reduced out (o-set).
Solve first equation
for {xo}.

{xo } = [ Koo ]

n dof in FEM

[ K oo ]
K
[ ao ]

[ Koa ] {xo } = {0}

[ Kaa ] {xa } {F}

1
x o [ K oo ] [ K oa ]
{ x} = =
{x a } = [T ]{x a }
x
[I]
a

[ Koa ]{xa }

[T] is a transformation from {xa} to {x}.

[ M ]TAM = [T ]T [ M ][T ]
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

STATIC REDUCTION OFTEN GIVES

GOOD RESULTS
300 Degrees of Freedom
90 Shell and Bar Elements
17 Modes with Frequencies below 70.0 Hz

Want TAM to be able to Predict

all 17 Modes Under 70.0 Hz.
- denotes master

dof location

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

20 DOF STATIC TAM CAN ACCURATELY

REPRESENT GPSC FEM MODAL PARAMETERS
IN 0.0 - 70.0 Hz. FREQUENCY RANGE

Absolute Frequency Error (%)

2.0

1.5

1.0

0.5

0.0
1

10 11 12 13 14 15 16 17

FEM Mode Number

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

X-33 REPRESENTS A MORE

CHALLENGING PROBLEM

100,000 Dof
833 modes
below 55.0 Hz.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

17 TARGET MODES BELOW 25 Hz.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

X-33 FEM/TAM FREQUENCY ERROR

6

1200 dof static TAMs

accurately predict
most of the target modes
below 25.0 Hz., but
two are not predicted
and none of the 10
target modes above 25.0
Hz. are predicted accurately.

4
TAM5
3

MSFC

84

83

81

75

71

68

46

23

22

21

16

15

12

11

Frequency Error - %

Target Mode Number

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SOLUTION OF UNDERDETERMINED
EQUATIONS
Have fewer independent
equations than unknowns.
[A] is rectangular with more
columns than rows and is
assumed to be full row rank.
A solution always exists,
but there are infinitely many.

Dim(N([A])) = m - r = m - n

[ A]nm {x}m1 = {b}n1

n<m

{b} is always in column

space of [A].
Suppose {x}1 is a solution,
then if {x}n is any vector in
N([A]),
{x}s = {x}1 + {x}n
is also a solution.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

BRUTE FORCE SOLUTION PROCEDURE

FOR UNDERDETERMINED SYSTEMS
[ A]nm {x}m1 = {b}n1
The original matrix
Equation becomes:

n<m

Define a new
vector:

[A] is full row rank, therefore

[A][A]T is nonsingular.

[ A][ A]T {z} = {b}

Solve for {z}:

{x} = [ A]T {z}

[ A]T {z} = [ A]T ([ A][ A]T ) {b} = {x}m

1

{z} = ([ A][ A]T ) {b}

1

[ A]+ = [ A]T ([ A][ A]T ) Right Generalized

1

Inverse of [A]

Out of infinity, which

solution is {x}m?
Substitute:

For arbitrary solution {x}s:

{x}m = [ A] ([ A][ A]
T

T 1

{b} = [A]{x}s

[ A]{x}s = [ P]{x}s

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

PROJECTORS
{x}m = [ P]{x}s

[ P]mm = [ A] ([ A][ A]
T

T 1

[ A]

Is the orthogonal projection of general solution {x}s

onto the row space of [A] or column space of [A]T.
{x}m is the minimum norm (length) solution.
[P] is an orthogonal projector.

N([P])

{x}s

[P] is Idempotent:

[ P][ P] = [ P]

tr([ P]) = rk ([ P]) = r = n

r eigenvalues of magnitude 1.0

R( [A]T )

{x}m

m-r eigenvalues of magnitude 0.0

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

APPLICATION TO TIME DOMAIN

SYSTEM IDENTIFICATION

Identify MIR space station

modal parameters using
response from excitation due
to Space Shuttle docking.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

IDENTIFY SYSTEM MARKOV

PARAMETERS USING ONE DATA SET

Measure input and output and form

underdetermined matrix equation.

102 015

[YS ][ H ] = [YD ]

101 022

101 016
101 014

Solution process projects [H] onto

Row space of data set.

10 100 9

1 070 12

10 6010
104 014

105 011
X

118009

Z
Y

ys 0
y
s1
ys 2
M

ysnt 1

0
ys 0
ys1
M
ysnt 2

L
0
ys 0
M
L

L
0
H 0 yd 0

L
0
H1 yd1

=
L
M
M M

L
M
H N ydn 1
L ysnt NR R t

[Ys] has 650 rows

and 1950 columns
in application.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

PULSE RESPONSE AT 104014z

PROJECTED ONTO STS-81 DATA
0.2

0.15

0.1

0.05

____ FEM
0

____ Docking Data

-0.05

-0.1

-0.15

-0.2
0

5
Seconds

10

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SOLUTION OF OVERDETERMINED
EQUATIONS
Have more independent
equations than unknowns.
[A] is rectangular with more
rows than columns and is
assumed to be full column rank.

[ A]nm {x}m1 = {b}n1

n>m

{b} may or may not be

in column space of [A].

If {b} is in R([A]):

A unique solution exists.

If {b} is not in R([A]):

No exact solution exists. But

can find an approximate one.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

BRUTE FORCE SOLUTION PROCEDURE

FOR OVERDETERMINED SYSTEMS
[ A]nm {x}m1 = {b}n1

([ A] [ A]){x} = [ A]
T

{b} = {b }

{x} = ([ A] [ A]) [ A] {b}

T

Plug this solution into

original equation:

[ P ]

Premultiply by [A]T

n>m

([A]T[A]) is nonsingular, therefore

equation can be solved using previously
discussed techniques.

[ A] = ([ A] [ A]) [ A]
+

Left Generalized
Inverse of [A]

[ A]{x} = [ A]([ A] [ A]) [ A]T {b} = [ P ]{b} = {b }

is an orthogonal
projector onto R([A])

{b} is the orthogonal projection

of {b} onto R([A])

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

BRUTE FORCE SOLUTION PROCEDURE

FOR OVERDETERMINED SYSTEMS
Case I:

{b} R([ A])

{x} = ([ A] [ A]) [ A]T {b}

T

[ ]

Unique solution
to [A]{x}={b}

P {b} = {b}

Case II: {b} R([ A])

{x } = ([ A]T [ A]) [ A]T {b}

1

[ ]

{}

P {b} = b

{x } Minimizes Euclidean
norm (length) of error
vector {e}.

{e} = {b} [ A]{x}

Unique solution
to [A]{x}= b

{}

Least-Squares
Solution

Works well in the presence of noise.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

CURVE FITTING

M2

YSM
YBMA

M1
Data

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

PREVIOUS BRUTE FORCE TECHNIQUE

FOR LEAST-SQUARES USES THE
NORMAL FORM
([ A] [ A]){x} = [ A]
T

{b}

Direct inversion of ([A]T [A]) is costly and inaccurate:

o(n 3 )

LU decomposition of ([A]T [A]) and backward/forward

substitution is faster but can still be inaccurate:

o(n 2 )

In general, its best not to use the Normal Form of the equations to
obtain the Least-Squares solution.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SOLUTION OF LEAST-SQUARES
PROBLEM USING QR DECOMPOSITION
Operates directly on matrix
equation for general full
column rank matrix:
[A] can be uniquely
factored in the form:

[Q][ R]{x} = {b}

Partition:

[ A]nm {x}m1 = {b}n1

[Q]T [Q] = [Q][Q]T = [ I ]

[ A]nm = [Q]nn [ R]nm

Premultiply
by [Q]T

n>m

[R] upper
triangular

[U ]
[ R] =
[0]

[Q]T [Q][ R]{x} = [ R]{x} = [Q]T {b}

[Q1 ]T
[U ]
[0] {x} = Q T {b}

[ 2 ]

Preferred Technique for L.S.

2
o
n
(
)
Fast and Accurate

[U ]{x} = [Q1 ] {b}

T

Solve by
backsubstitution.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

EXAMPLE - POLYNOMIAL CURVE

FITTING (JUNKINS)
Matrix has 20 rows

{t1

t
t
M
t m2

10

n 1
1
n 1
2

L t x1 y1

L t x 2 y2
=
O M M M

L t mn1 x n ym

t2 L tm } = {0 1 2 L m 1}

10
-2

10

Direct
Inversion

-4

10
Fractional Error

1 t1

1 t2
M M

1 t m

2
1
2
2

-6

LU

10
-8

10

QR

-10

10
-12

10
-14

{y1

y2 L ym } = {1 1 1 L 1}

10
-16

10

5
6
7
8
Number of Columns in 20 Row Data Matrix

D. C. Kammer
University of Wisconsin - Engineering Physics

10

Linear Algebra

MOORE-PENROSE GENERALIZED INVERSE

For EVERY matrix [A]nxm of rank r, an mxn matrix [A]+ is called a
Generalized Inverse if it satisfies:

[ A][ A]+ [ A] = [ A]

Infinitely Many
generalized inverses

[ A]+ [ A][ A]+ = [ A]+

If in addition, [A]+ satisfies:

[ A] [ A] = ([ A] [ A])
+

[ A][ A] = ([ A][ A]
+

Then, inverse is UNIQUE and called the

Moore-Penrose Generalized Inverse.

+ H

[ A]
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

M-P INVERSE MOST ACCURATELY

CALCULATED USING SVD
[ A]nm {x}m1 = {b}n1

SVD

[ A] = [U1 ][ ][V1 ]

[A] has rank r

Moore-Penrose
Inverse given by:

[ A] = [[U1 ]

H
[ ] [0] [V1 ]
[U2 ]] 0 0 H
[ ] [ ] [V2 ]

[ A] = [V1 ][ ]1[U1 ]

0 L
11 0
0 1 0 L
1
2

[ ] (r r ) =
0
0 O 0

M
M
M

SVD is costly but very

Accurate and Stable.
Trick is to determine correct
value of r.
How small is small, when it
comes to singular values?

MATLAB truncates singular values at:

tol = max (n, m) 1 2.22 10 16

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MOORE-PENROSE GENERALIZED INVERSE

CAN SOLVE MATRIX EQUATIONS FOR
ANY CASE
[ A]nm {x}m1 = {b}n1

{x } = [ A] {b}

The following
equivalencies
can be derived.

Case I: Minimum norm: n < m and r = rank([A]) = n.

[ A] = [ A] ([ A][ A]

T 1

Case II: Least squares: n > m and r = rank([A]) = m.

[ A] = ([ A]T [ A]) [ A]T

1

Case III: General or rank deficient case: r = rank([A]) min( n, m).

[ A] = [V1 ][ ]1[U1 ]

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

RANK DEFICIENT EXAMPLE

1
0
[ A] =
0
1

2 3 0 0
0 4 5 6

0 0 0 7
2 7 5 6

1
1

{x} = 1
1

6
15

{b} =
7
21

0.3209
0.6419

{x } = [ A] {b} = 1.4651
0.6279

1.0000

U =
1.3032e-01
5.9082e-01
3.3749e-01
7.2114e-01

-4.3184e-01
6.8625e-02
8.2273e-01
-3.6321e-01

6.8059e-01
-5.5937e-01
4.5740e-01
1.2122e-01

-5.7735e-01
-5.7735e-01
-3.5691e-18
5.7735e-01

1.4570e+01
0
0
0

0
5.8276e+00
0
0

0
0
2.9611e+00
0

0
0
0
3.7220e-16

0
0
0
0

5.8442e-02
1.1688e-01
5.3552e-01
4.5024e-01
7.0244e-01

-1.3643e-01
-2.7286e-01
-6.1149e-01
-2.5275e-01
6.8494e-01

2.7078e-01
5.4156e-01
2.2047e-01
-7.3983e-01
1.9349e-01

-6.9696e-01
-3.7060e-01
4.7939e-01
-3.8351e-01
-1.8459e-16

6.4722e-01
-6.9373e-01
2.4674e-01
-1.9740e-01
-1.1620e-16

Minimum
norm

{x} = 2.236

{x } = 2.014

S =

tol = 1.62e-14

V =

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MATRIX CONDITIONING
[ A]nm {x}m1 = {b}n1

Condition Number:

Matrix [A] may be considered

full rank, but it can still be
Ill-conditioned.
Ill-conditioning of [A] implies that
small changes in the elements of [A]
and {b} can cause very large changes
in the computed solution {x}.
Relative error in {x} can be times
the relative error in the data.

If

([ A]) =

max
min

1
approaches the computers
([ A])

floating point precision, the matrix

is ill-conditioned.

1.0 ([ A])
Good
Bad
Regularization can help.
Look for solution of well-posed
problem in neighborhood of
ill-posed problem.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

GENERAL PURPOSE SPACECRAFT

ESTIMATE LOADS AT INTERFACE WITH LAUNCH VEHICLE
USING MEASURED RESPONSE AND MARKOV PARAMETERS
6 Inputs - 6 Outputs
Spacecraft - 0.0-300.0 Hz.
Inputs - 0.0-150.0 Hz.

Inverse problem is ill-posed

leading to an ill-conditioned
matrix convolution equation.

[ H ]{U} = {Y }

45(z)

44(xyz)

49(z)

[H] has more rows than columns

and quickly becomes ill-conditioned
as more data is used.

48(x)

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

COMPUTE LEAST SQUARES SOLUTION

Ill-conditioning of
data matrix [H] results
in unstable computation
of input forces.

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

COMPUTATION CAN BE REGULARIZED

[ H ]{U} = {Y }

Matrix Convolution
Equation

[H] =

[h0 ]
[h1 ]
[h2 ]

[h0 ]
[h1 ]

[h ] [h ]
nt 1

nt 2

L
0
O
L
L

L
L
0
O

[h1 ]

{Y } =

{U} =

[h0 ]
0
M
M
0

Replace ill-posed problem with closely

related well-posed problem.
Regularized solution:

{{y(0)}
{{u(0)}

{y(1)}

{u(1)}

}
{u(n 1)} }

{y(nt 1)}

L
L

([ H ] [ H ] + [ I ]){U} = [ H ]
T

{Y }

{U} = ([ H ]T [ H ] + [ I ]) [ H ]T {Y }
1

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

REGULARIZED FORCE COMPUTATION

Exact

Predicted

= 0.00005

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

MIR SYSTEM IDENTIFICATION

Priroda

Kvant-1
Core

Spektr
Soyuz
Krystall
Progress

Identify MIR Markov

parameters from response
due to Shuttle docking.

Kvant-2

Shuttle

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

ACCELERATION MEASURED ON
KRISTALL - STS-81
0.015

Acceleration - g

0.01

Response data
filtered to 5.0 Hz.
by NASA.

0.005

-0.005

-0.01

10

15
Seconds

20

25

30

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SOLVE FOR MARKOV PARAMETERS

[ H ] [Yr ] = [Ye ]
[ yr 0 ]

0
[H] 0

M
0

[ yr 1 ]
[ yr 0 ]
0
M
L

[H] has 1,950 rows and 3,300 columns

L
M
M
[ yr 0 ]

L
L
L
L

[ y ]
[y ]
rnt 1

rnt 2

[y

M
M

rnt N R

Compute Moore-Penrose
Inverse of [Yr] using SVD.
Due to ill-conditioning, SVD
algorithm FAILS TO CONVERGE.

= [ ye 0 ]

[ ye1 ]

[ y ]]
ent 1

Regularize data:
Adjust SVD singular
value tolerance.
Add a little artificial noise.
D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

ADD 1.0% RMS NOISE, COMPUTE

PSEUDO-INVERSE
0.25

0.2

0.15

Compute Markov
parameters and
filter to 5.0 Hz.

0.1

0.05

____ FEM

-0.05

____ Docking Data

-0.1

-0.15

-0.2
0

5
Seconds

10

D. C. Kammer
University of Wisconsin - Engineering Physics

Linear Algebra

SUMMARY
We have only scratched the surface of what there is to know
about matrix algebra.
As an experimental or analytical structural dynamicist, you
CANNOT do your work without using matrix analysis.
There is a veritable galaxy of neat applications, many of which
have not been thought of yet.

D. C. Kammer
University of Wisconsin - Engineering Physics