CHAPTER I

VECTOR SPACES
Department of Foundation Year,
Institute of Technology of Cambodia

20142015

VECTOR SPACE

ITC

1 / 32

Contents

Definitions

Spanning Sets and Linear Independence

Bases and Dimension

Change of Basis

Inner Product Spaces

Gram-Schmidt Orthogonalisation

VECTOR SPACE

ITC

1 / 32

Contents

Definitions

Spanning Sets and Linear Independence

Bases and Dimension

Change of Basis

Inner Product Spaces

Gram-Schmidt Orthogonalisation

VECTOR SPACE

ITC

2 / 32

Vector Spaces
Let K denote the set of real numbers or the set of complex numbers.

Definition 1
Let V be a nonempty set (whose elements are called vectors) on which
are defined an addition operation (+) and a scalar multiplication
operation (.) with scalars in K. We call V a vector space over K,
provided the following ten conditions are satisfied.
1
2
3

4
5

u, v V : u + v V
u, v V : u + v = v + u
u, v, w V : (u + v) + w =
u + (v + w)
0 V, v V : v + 0 = 0
v V, v V :
v + (v) = 0
k K, v V : kv V
VECTOR SPACE

k K, u, v V :
k(u + v) = ku + kv

k1 , k2 K, v V :
(k1 + k2 )v = k1 v + k2 v

k1 , k2 K, v V :
(k1 k2 )v = k1 (k2 v)

10

ITC

vV :

1v = v.
3 / 32

Vector Spaces

In this whole chapter, V will be denoted as a vector space over K.

Otherwise, it must be mentioned.

Theorem 2
Let V be a vector space over K. We have
1

The zero vector is unique.

0v = 0 for all v V .

k0 = 0 for all scalars k K.

The additive inverse of each element of V is unique.

For v V :

kv = 0 k = 0 or v = 0.

VECTOR SPACE

v = (1)v.

ITC

4 / 32

Subpaces

Definition 3
Let =
6 S V . If S is a vector space under the same operations of
addition and scalar multiplication as used in V , then we say that S is a
subspace of V .

Theorem 4
Let S V . Then S is a subspace of V iff
1

S 6= ,

S is closed under addition and scalar multiplication.

VECTOR SPACE

ITC

5 / 32

Subpaces

Theorem 5
Let =
6 S V . Then S is a subspace of V iff:
1

0 S,

u, v S, k K :

u + kv S.

Theorem 6
If S1 , S2 , . . . , Sp are subspaces of V , then S1 S2 Sp is also a
subspace of V .

VECTOR SPACE

ITC

6 / 32

Contents

Definitions

Spanning Sets and Linear Independence

Bases and Dimension

Change of Basis

Inner Product Spaces

Gram-Schmidt Orthogonalisation

VECTOR SPACE

ITC

7 / 32

Subpaces
Definition 7
Let v1 , v2 , . . . , vp V , and k1 , k2 , . . . , kp K.
1

The expression of the form k1 v1 + k2 v2 + + kp vp is called a

linear combination of v1 , v2 , . . . , vp .

The set of all linear combinations of v1 , v2 , . . . , vp is called the

span of v1 , v2 , . . . , vp , and denoted by Span{v1 , v2 , . . . , vp }. That
is,
Span{v1 , v2 , . . . , vp } = {k1 v1 + + kp vp : ki K, i = 1, 2, . . . , p}

A subspace S of V is said to be spanned by v1 , v2 , . . . , vp if

S = Span{v1 , v2 , . . . , vp }.
In this case, {v1 , v2 , . . . , vp } is called a spanning set for S.
VECTOR SPACE

ITC

8 / 32

Subpaces
Definition 8
The vectors v1 , v2 , . . . , vp V are said to be linearly independent if
the equation
k1 v1 + k2 v2 + + kp vp = 0
has a unique solution k1 = = kp = 0. Otherwise, v1 , v2 , . . . , vp are
said to be linearly dependent.

Theorem 9
Let v1 , v2 , . . . , vp V . Then
1

Span{v1 , v2 , . . . , vp } is a subspace of V and the smallest subspace

containing {v1 , v2 , . . . , vp }.

A vector v Span{v1 , v2 , . . . , vp } can be written uniquely as a

linear combination of v1 , v2 , . . . , vp if and only if v1 , v2 , . . . , vp are
linearly independent.
VECTOR SPACE

ITC

9 / 32

Subpaces

Theorem 10
The set {v1 , v2 , . . . , vp } V is linearly dependent if and only if at least
one of the vectors of the set can be expressed as a linearly combination
of the others.

VECTOR SPACE

ITC

10 / 32

Subpaces
Definition 11
Let A = (aij ) be an m n matrix with entries in K. The row space of
A denoted r(A) is a subspace of Kn spanned by row vectors of A and
the column space of A denoted c(A) is a subspace of Km spanned by
column vectors of A.

Theorem 12
Two matrices A and B are row-equivalent matrices, then r(A) = r(B).

Theorem 13
The set of nonzero row vectors in any row-echelon form of a matrix is
linearly independent.

VECTOR SPACE

ITC

11 / 32

Contents

Definitions

Spanning Sets and Linear Independence

Bases and Dimension

Change of Basis

Inner Product Spaces

Gram-Schmidt Orthogonalisation

VECTOR SPACE

ITC

12 / 32

Bases and Dimension

Definition 14
A set {v1 , v2 , . . . , vn } V is called a basis for V if:
1

v1 , v2 , . . . , vn are linearly independent,

Span{v1 , v2 , . . . , vn } = V .

Theorem 15
If Span{v1 , v2 , . . . , vn } = V , then any collection of m vectors in V ,
where m > n, is linearly dependent.

Theorem 16
If {v1 , v2 , . . . , vn } and {u1 , u2 , . . . , um } are both bases for V , then
m = n.

VECTOR SPACE

ITC

13 / 32

Bases and Dimension

Definition 17
The dimension of a vector space V is the number of elements in the
basis. That is, if B = {v1 , v2 , . . . , vn } is a basis for V , then dimension
of V equals to n. We write,
dimV = n.
V is said to be finite-dimensional if there is a finite set of vectors
that spans V ; otherwise, we say that V is infinite-dimensional.

VECTOR SPACE

ITC

14 / 32

Bases and Dimension

Theorem 18
Let B = {v1 , v2 , . . . , vn } V , and dimV = n. The following statements
are equivalent.
1

B is a basis for V .

B is linearly independent.

B spans V .

Theorem 19
If S is a subspace of V and if dimV = n, then dim S n.

VECTOR SPACE

ITC

15 / 32

Contents

Definitions

Spanning Sets and Linear Independence

Bases and Dimension

Change of Basis

Inner Product Spaces

Gram-Schmidt Orthogonalisation

VECTOR SPACE

ITC

16 / 32

Bases and Dimension

Theorem 20
Let B = {v1 , v2 , . . . , vn } be a basis for V . Then every vector v V can
be written uniquely as a linear combination of v1 , v2 , . . . , vn . That is,
! c1 , c2 , . . . , cn , such that
v = c1 v1 + c2 v2 + + cn vn .

VECTOR SPACE

ITC

(1)

17 / 32

Bases and Dimension

Definition 21
The unique n-tuple (c1 , c2 , . . . , cn ) Kn defined in (1) is called the
coordonates of v relative to the ordered basis
B = {v1 , v2 , . . . , vn }. We denote

c1
c2

[v]B = .
.
.
cn

VECTOR SPACE

ITC

18 / 32

Bases and Dimension

Theorem 22
Let B be an ordered basis for a finite dimensional vector space V . Let
u, v V and c K. Then,
[u + cv]B = [u]B + c[v]B .

Definition 23
Let B = {v1 , v2 , . . . , vn } and C = {w1 , w2 , . . . , wn } be two ordered
bases for a vector space V . We call the n n matrix defined by
PCB = ([v1 ]C [vn ]C )
the change-of-basis matrix or transition matrix from ordered
basis B to ordered basis C.
VECTOR SPACE

ITC

19 / 32

Bases and Dimension

Theorem 24
Let A, B and C be three ordered bases for a finite dimensional vector
space V and v V . Then
[v]C = PCB [v]B ,

PCB PBC = I,

and

PCA = PCB PBA .

Theorem 25
The set of nonzero row vectors in any row-echelon form of an m n
matrix A is a basis for the row space r(A).

VECTOR SPACE

ITC

20 / 32

Bases and Dimension

Definition 26
Let S1 , S2 , . . . , Sp and S be subspaces of V .
1

The sum of S1 , S2 , . . . , Sp is defined by

S1 + + Sp = {s1 + + sp : si Si , i = 1, 2, . . . , p}

S is the direct sum of S1 , S2 , . . . , Sp denoted by

S = S1 S2 Sp
if for each s S can be written uniquely as
s = s1 + s2 + + sp
where si Si , i = 1, 2, . . . , p.
VECTOR SPACE

ITC

21 / 32

Bases and Dimension

Theorem 27
Let S1 , S2 , . . . , Sp be subspaces of V . Then S1 + S2 + + Sp is also a
subspace of V .

Theorem 28
Let S1 , S2 be two subspaces of a finite dimensional vector space V .
Then
dim(S1 + S2 ) = dimS1 + dimS2 dim(S1 S2 )

VECTOR SPACE

ITC

22 / 32

Bases and Dimension

Theorem 29
Let S1 , S2 , . . . , Sp be subspaces of V . The following properties are
equivalent.
1

The sum of S1 , S2 , . . . , Sp is a direct sum

Each s S can be written uniquely as s = s1 + s2 + + sp , where

si Si , i = 1, 2, . . . , p.

For all i {1, 2, . . . , p}, Si

Sj = {0}
1jp
j6=i

!
4

For all i {1, 2, . . . , p}, Si

Sj

= {0}

1ji1
5

For all (s1 , s2 , . . . , sp ) (S1 {0}) (Sp {0}), s1 , s2 , . . . , sp

are linearly independent (Suppose that the subspaces S1 , S2 , . . . , Sp
are all different from {0}).
VECTOR SPACE

ITC

23 / 32

Bases and Dimension

Theorem 30
Let V be a finite-dimensional vector space of n dimensions
over K and

S1 , . . . , Sp be subspaces of V . Let dj = dim Sj and bj1 , . . . , bjdj be a
basis for Sj . Then the following statements are equivalent.
1
2
3

S1 + + Sp = S1 Sp .


b11 , . . . , b1d1 , . . . , bp1 , . . . , bpdp is a basis for S1 + + Sp .
dim (S1 + + Sp ) = dim (S1 ) + + dim (Sp )

VECTOR SPACE

ITC

24 / 32

Contents

Definitions

Spanning Sets and Linear Independence

Bases and Dimension

Change of Basis

Inner Product Spaces

Gram-Schmidt Orthogonalisation

VECTOR SPACE

ITC

25 / 32

Inner Product Spaces

Definition 31
A map h., .i : V V K is called an inner product in V , if it
satisfies the following properties. For all vectors u, v, w V and for all
scalars k K,
1

hu, ui 0, and

hu, vi = hv, ui.

hku, vi = k hu, vi .

hu + v, wi = hu, wi + hv, wi .

VECTOR SPACE

hu, ui = 0 u = 0.

ITC

26 / 32

Inner Product Spaces

Definition 32
Any vector space V that is equipped with an inner product hu, vi
is called an inner product space.
The norm of u is defined in term of the inner product by
p
kuk = hu, ui.

The distance between u and v is defined to be the number

d(u, v) = ku vk.

VECTOR SPACE

ITC

27 / 32

Inner Product Spaces

Definition 33
Let V be an inner product space over K.
1

Given two vectors u, v 6= 0. The scalar projection of u onto v is

and the vector projection of u onto v is
defined by k = hu,vi
kvk


1
defined by p = k kvk
v = hu,vi
v.
kvk2

Two vectors u, v in V is said to be orthogonal if hu, vi = 0.

A set of vectors {v1 , v2 , . . . , vp } in V is called an orthogonal set

if hvi , vj i = 0 for i 6= j.

A vector v V is called a unit vector if kvk = 1.

An orthogonal set of unit vectors is called an orthonormal set of

vectors.

Let dimV = n. A basis {v1 , v2 , . . . , vn } is called an orthogonal

basis if it is an orthogonal set. It is called an orthonormal basis
if it is an orthonormal set.
VECTOR SPACE

ITC

28 / 32

Inner Product Spaces

Theorem 34
Let V be an inner product space over K. Let u, v V and k K.
Then,
1

kkvk = |k|kvk

ku + vk kuk + kvk

| hu, vi | kukkvk

ku + vk2 + ku vk2 = 2 kuk2 + kvk2

VECTOR SPACE

ITC

29 / 32

Contents

Definitions

Spanning Sets and Linear Independence

Bases and Dimension

Change of Basis

Inner Product Spaces

Gram-Schmidt Orthogonalisation

VECTOR SPACE

ITC

30 / 32

Gram-Schmidt Orthogonalisation
Theorem 35
If {v1 , v2 , . . . , vp } of nonzero vectors in an inner product space V
is an orthogonal set then it is linearly independent.
If {v1 , v2 , . . . , vn } is an orthogonal basis for a finite-dimensional
inner product space V , then for every vector v V ,
v=

hv, v2 i
hv, vn i
hv, v1 i
v1 +
v2 + +
vn .
2
2
kv1 k
kv2 k
kvn k2

If {v1 , v2 , . . . , vn } is an orthonormal basis for a finite-dimensional

inner product space V , then for every vector v V ,
v = hv, v1 i v1 + hv, v2 i v2 + + hv, vn i vn

VECTOR SPACE

ITC

31 / 32

Gram-Schmidt Orthogonalisation

Theorem 36 (Gram-Schmidt Orthogonalisation Process)

Let {x1 , x2 , . . . , xp } be an independent set of vectors in a
finite-dimensional inner product space V . Then an orthogonal basis for
a subspace of V spanned by these vectors is {v1 , v2 , . . . , vp } where
v1 = x1 , vi = xi

i1
X
hxi , vj i
j=1

kvj k2

vj ,

i = 2, 3, . . . , p.

VECTOR SPACE

ITC

32 / 32