Chapter 3

Vector Spaces

3.1

Definition and properties

In this chapter we present the formal definition of a vector space. The most elementary vector spaces are the well-known Euclidean spaces Rn , n = 1, 2, . . . , which illustrate clearly the concept. Let us consider R2 first. Nonzero vectors in R2 can be
represented geometrically by directed segments, taking into account that those with
the same length and direction are considered to be equal.
Thus, given a nonzero vector in R2
v = (v1 , v2 ), it can be represented in
v

the plane either by the directed segment from (0, 0) to (v1 , v2 ), or by the
directed segment from point (x, y) to
v

(x + v1 , y + v2 ), for any x, y R.

This representation helps us visualize

how the operations of addition:
1

v1 u1 v1 + u1

+
=

v2
u2
v2 + u2
and scalar multiplication

v1 v1

v2
v2
work in R2 , as shown below.

Multiplication of a vector by a scalar

Addition of two vectors

2 x (2,1)
2

(2,1)+(1,1)

(2,1)

(1,1)

x (2,1)

(2,1)

In general, addition and multiplication by a scalar in Rn are defined by

v1

v2

+

..
.

vn

u1

v + u1
1

u2
v2 + u2
=

..
..
.
.

un
vn + un

and
2

v1

v1

v2
v2
=
..
.
. ..

vn
vn

where the scalar is any real number.

We can also view Rn as the set of all n 1 matrices with real entries: the addition and scalar multiplication of vectors in Rn is just the usual addition and scalar
multiplication of matrices. More generally, we can represent by Rmn the set of all
m n matrices with real entries. The operations of addition and scalar multiplication in
Rmn , m, n N, obey certain algebraic rules, that are used to define the general concept
of a vector space.
Vector Space Axioms
Consider a set of scalars, which can be either R or C. Let V be a set on which the
operations of addition and scalar multiplication are defined and verify the closure
property. The set V together with these operations is said to be a vector space if the
following properties are satisfied:
1. u + v = v + u, for any u and v in V;
2. (u + v) + w = u + (v + w), for any u, v and w in V;
3. there exists an element 0 in V such that u + 0 = u, for each u in V;
4. for each u in V, there exists an element u in V such that u + (u) = 0
5. (u + v) = u + v for each scalar and any u and v in V;
6. ( + )u = u + u, for any scalars and and for any u in V;
3

7. ()u = (u), for any scalars and and for any u in V;

8. 1 u = u, for all u in V.
The elements of a vector space V defined as before are called vectors. Notice that
the elements of V can be numbers, n-tuples of numbers, matrices, polynomials or
even functions, and that the definition of the operations of addition and multiplication
by scalars is what will make us call them vectors.
Example: If we consider C as the set of scalars, then R2 , with the ordinary addition
and multiplication by scalars, is NOT a vector space, as it does not verify the closure
property for the multiplication:
i (1, 1) = (i, i)
/ R2 .

The following theorem states three more fundamental properties of vector spaces.
Theorem: If V is a vector space and u is any element of V, then
i) 0 u = 0.
ii) u + v = 0 implies that v = u (i.e., the additive inverse of u is unique).
iii) (1) u = u.

3.2

Subspaces

Given a vector space V, it is often possible to form another vector space by taking a
subset S of V, using the operations of V. Since V is a vector space, the operations of
addition and scalar multiplication always produce another vector in V. For a new
4

subset S of V to be a vector space, the set S must be closed under the operations of
addition and scalar multiplication, i.e., the sum of two elements of S must always be
an element of S and the product of a scalar and an element of S must always be an
element of S.
Example: The subset of R2 given by
S = {(x, 1), x R},
does not verify the closure properties:
(1, 1) + (3, 1) = (4, 2)
/ S,
and
3 (1, 1) = (3, 3)
/ S.

Example: The subset of R2 given by
S = {(x1 , x2 ), x2 = 2x1 }
verifies the closure properties:
(x, 2x) + (y, 2y) = (x + y, 2x + 2y) = (x + y, 2(x + y)) S
and
(x, 2x) = (x, 2x) = (x, 2(x)) S
It is easily seen that S with the operations from R2 is itself a vector space.

If S is a nonempty subset of a vector space V and S satisfies the closure properties,

then S is said to be a subspace of V. In other words, S is a subspace of V if it is closed
under the operations of V. Notice that this definition implies that 0 S.
5

3.3

Linear dependence and linear independence

Let v1 , v2 , . . . , vn be vectors in a vector space V. Recall that a sum of the form

1 v1 + 2 v2 + + n vn ,
where 1 , 2 , . . . , n are scalars, is called a linear combination of v1 , v2 , . . . , vn . The
set of all linear combinations of v1 , v2 , . . . , vn is called the span of v1 , v2 , . . . , vn , and
we will denote it by Span{v1 , . . . , vn }.
Example: Consider vectors (1, 1, 1) and (1, 1, 0) in R3 . The span of these vectors is
given by the set of all linear combinations
{(1, 1, 1)+(1, 1, 0) : , R} = {(+, +, ) : , R} = {( , , ) : , R}.
Now consider vectors (1, 1, 2), (1, 1, 1) and (1, 1, 0). The span of this collection
of vectors can be described as
{(1, 1, 2) + (1, 1, 1) + (1, 1, 0) : , , R} =
= {( + + , + + , 2 + ) : , , R}.
However, the span can be also described as
{( , , ) : , R}.
Thus:

Span{(1, 1, 1), (1, 1, 0)} = Span{(1, 1, 2), (1, 1, 1), (1, 1, 0)}.

Theorem: If v1 , . . . , vn are elements of a vector space V, then Span{v1 , . . . , vn } is a

subspace of V.
Proof: Let v = 1 v1 + + n vn be an arbitrary element of Span{v1 , . . . , vn }, and
any scalar. Since
v = (1 v1 + + n vn ) = (1 )v1 + + (n )vn ,
6

if follows that v Span{v1 , . . . , vn }.

Now, consider v, w Span{v1 , . . . , vn }, v = 1 v1 + + n vn , and w = 1 v1 +
+ n vn . Since
v + w = (1 + 1 )v1 + + (n + n )vn Span{v1 , . . . , vn }
we have proven that Span{v1 , . . . , vn } is a subspace of V.

The previous theorem motivates the following definition: the set {v1 , . . . , vn } is
a spanning set for V if every vector in V can be written as a linear combination of
v1 , . . . , vn .
Example: Consider the following vectors in R3 : v1 = (1, 1, 2), v2 = (2, 3, 1) and
v3 = (1, 3, 8). Let S be the subspace of R3 spanned by v1 , v2 , v3 . Actually, any linear
combination of v1 , v2 and v3 can be reduced to a linear combination of v1 and v2 :
1 v1 + 2 v2 + 3 v3 = 1 v1 + 2 v2 + 3 (3v1 + 2v2 ) = (1 + 33 )v1 + (2 + 23 )v2 .
Thus S = Span{v1 , v2 , v3 } = Span{v1 , v2 }.

This example illustrates the following propositions:

If v1 , . . . , vn span a vector space V and one of these vectors can be written as a
linear combination of the other n 1 vectors, then those n 1 vectors span V.
Proof: Suppose vn can be written as a linear combination of v1 , ..., vn1 :
vn = 1 v1 + + n1 vn1
Let v V. Since v1 , . . . , vn span V we can write
v = 1 v1 + + n1 vn1 + n vn =
7

= 1 v1 + + n1 vn1 + n (1 v1 + + n1 vn1 ) =
= (1 + n 1 )v1 + + (n1 + n n1 )vn1 ,
and any vector v V can be written as a linear combination of v1 , . . . , vn1 , i.e.,
V = Span{v1 , . . . , vn1 }.

Given n vectors v1 , . . . , vn , it is possible to write one of the vectors as a linear

combination of the other n 1 if and only if there exist scalars c1 , . . . , cn not all
zero such that
c1 v1 + + cn vn = 0.
Proof: Suppose one of these vectors, say vn , can be written as a linear combination of the others:
vn = 1 v1 + + n1 vn1 .
Subtracting vn from both sides of this equation, we get
1 v1 + + n1 vn1 vn = 0.
If we set c1 = 1 , . . . , cn1 = n1 , cn = 1, it follows that
c1 v1 + + cn vn = 0.
Conversely, if
c1 v1 + + cn vn = 0,
and at least one of the ci s, say cn is nonzero, then
vn =

cn1
c1
v1 + +
vn1 .
cn
cn

8

These properties motivate the following definition: the vectors v1 , . . . , vn in a vector space V are said to be linearly independent if
c1 v1 + + cn vn = 0
implies that all the scalars c1 , . . . , cn must equal 0. Conversely, the vectors v1 , . . . , vn
in a vector space V are said to be linearly dependent if there exists scalars c1 , . . . , cn
not all zero such that
c1 v1 + + cn vn = 0.
The following theorem is given without proof.
Theorem: Let v1 , . . . , vn be vectors in a vector space V. A vector v in Span{v1 , . . . , vn }
can be written uniquely as a linear combination of v1 , . . . , vn if and only if v1 , . . . , vn
are linearly independent.

3.4

Basis and dimension

The vectors v1 , . . . , vn form a basis for a vector space V if and only if

1) v1 , . . . , vn are linearly independent, and
2) v1 , . . . , vn span V.
Example: The standard basis of R2 is B0 = {e1 = (1, 0), e2 = (0, 1)}. Let us verify that
B0 is effectively a basis.
Clearly, e1 and e2 are linearly independent, since 1 e1 + 2 e2 = 0 implies that 1 =
2 = 0.
In addition, every vector v = (v1 , v2 ) in R2 can be represented as a linear combination
of these two vectors as v = v1 e1 + e2 v2 , and thus R2 = Span{e1 , e2 }.
Therefore, {e1 = (1, 0), e2 = (0, 1)} is a basis for R2 .
9

Theorem: If {v1 , . . . , vn } is a spanning set for a vector space V, then any collection of
m vectors in V, where m > n, is linearly dependent.
As a consequence, if {v1 , . . . , vn } and {u1 , . . . , um } are both bases for a vector space
V, then n = m. In view of this result, we can now refer to the number of elements in
any basis for a given vector space V as the dimension of the vector space. If there is a
finite set of vectors that spans V, we say that V is finite-dimensional. Otherwise, V is
said to be infinite-dimensional.

Theorem: If V is a vector space of dimension n > 0:

Any set of n linearly independent vectors spans V.
Any n vectors that span V are linearly independent.

Example: The 3-dimensional space R3 is spanned by the linearly independent vectors

e1 = (1, 0, 0), e2 = (0, 1, 0) and e3 = (0, 0, 1), which are the most widely used basis of
R3 . The set {(1, 1, 1), (1, 1, 0), (1, 0, 0)} is also a basis of R3 . To show this, consider the
linear combination
1 (1, 1, 1) + 2 (1, 1, 0) + 3 (1, 0, 0) = 0

1 + 2 + 3 = 0

1 = 2 = 3 = 0
1 + 2 = 0

1 = 0
and thus the three vectors are linearly independent; therefore they form a basis for
R3 .


10

3.4.1

Change of basis

Let V be a vector space of dimension n, and let B = [v1 , . . . , vn ] be an ordered basis for
V. If v V, v can be written in the form
v = c1 v1 + + cn vn ,
where c1 , . . . , vn are scalars. Thus, we can associate with each vector v a unique vector
c Rn given by c = (c1 , . . . , cn ). This vector is called the coordinate vector of v with
respect to the basis B, and the scalars ci are referred to as coordinates of v with respect
to B.
Many applied problems can be simplified by changing from one coordinate system
to another. Changing coordinate systems in a vector space is essentially the same as
changing from one basis to another. Once we have decided to work with a new basis,
we have the problem of finding the coordinates with respect to that basis. To illustrate
this, consider the following example:
Example: Suppose that we want to use the following basis in R2 , B1 = {u1 = (3, 2), u2 =
(1, 1)}, instead of the standard basis B0 = {e1 , e2 }, and that later we want to switch
back to the original coordinate system. Thus, for a given vector v, with coordinates
(a1 , a2 ) with respect to the basis B0 , we want to compute the coordinates (b1 , b2 ) with
respect to the basis B1 . We have that
v = b1 u1 + b2 u2 = b1 (3e1 + 2e2 ) + b2 (e1 + e2 ) = (3b1 + b2 )e1 + (2b1 + b2 )e2 =
= a1 e 1 + a2 e 2 .
And we have to solve the system
3b1 + b2 = a1
2b1 + b2 = a2
11

that can be re-written in matrix notation as

3 1 b1 a1
b1 3

=

=
2 1
b2
a2
b2
2

1
1

a1

.
a2

If we know the coordinates (b1 , b2 ) of a vector with respect to {u1 , u2 } and want to find
the coordinates (a1 , a2 ) with respect to {e1 , e2 } we only have to substitute (b1 , b2 ) in
the first system. Conversely, if we know the coordinates (a1 , a2 ) and want to compute
(b1 , b2 ), we should use the second system.

3 1
The matrix
is called the transition matrix from {u1 , u2 } to {e1 , e2 }, and as
2 1
it can be seen, its columns correspond to the coordinates of vectors in basis {u1 , u2 }
with respect to basis {e1 , e2 }.

This situation can be easily generalized to higher dimensions. If V is a n-dimensional

vector space, it is possible to change from one basis F1 = {u1 , . . . , un } to another basis F2 = {v1 , . . . , vn } by means of an n n transition matrix, which is, necessarily,
non-singular, and can be defined by the linear system

u1 = s11 v1 + + sn1 vn
..
.

un = s1n v1 + + snn vn
that expresses each basis vector uj as a linear combination of the vi s. Then, the
coordinates (b1 , . . . , bn ) of a given vector with respect to the basis F2 can be computed by multiplying the coefficient matrix of the previous system by the coordinates
(a1 , . . . , an ) of the vector with respect to the basis F1 . We will normally denote such
matrix by TF1 F2 . Conversely, to obtain (a1 , . . . , an ), we have to multiply the inverse
of the transition matrix, (TF1 F2 )1 , by (b1 , . . . , bn ).
12

3.5

Orthogonality

Consider a vector v in R2 . If we use the geometrical representation of v as a directed

segment, we can use Pythagoras theorem to define the length of v as

||v|| =

q
v21 + v22

where v1 and v2 are the coordinates of v with respect to the standard basis B0 =
{e1 , e2 }. A vector of length 1 is called a unit vector.
Let u, v be two vectors in R2 , and let denote the angle between them; by this, we
shall always mean the angle [0, ]. If < , the vectors u, v and w = v u form a
triangle, where w is the side opposite to the angle .
Length of a vector

Angle between two vectors

v=(v1,v2)

w
v

v21 + v22

v2

v1

We say that two nonzero vectors u and v are orthogonal (or perpendicular, or

normal) if the angle between them is = . If two unit vectors are orthogonal, they
2
are said to be orthonormal.
These definitions are easily generalized to Rn .
13

3.5.1

Inner product

Given two vectors u, v in the vector space Rn , that form an angle , we define their
dot or inner product as the number
u v = ||u|| ||v|| cos .
When the vector coordinates (u1 , . . . , un ) and (v1 , . . . , vn ) are given with respect to
the standard basis B0 = {e1 , . . . , en }, then the inner product is computed as

v1
.
.
u v = u1 v1 + + un vn = (u1 , . . . , un )
. .

vn
When the vector coordinates are given with respect to a different basis, and the transition matrix is given by T , then the dot product is computed as

v1
.
.
u v = (u1 , . . . , un )T t T
. .

vn
Example: Consider vectors u and v in R2 , with coordinates with respect to the standard basis B0 (1, 2) and (3, 5). Their inner product is given by u v = 3 + 10 = 13.
If we refer these vectors to the basis B1 = [(1, 1), (2, 0)], then the transition matrix

from B0 to B1 is
1 2
T =
,
1 0
the coordinates of u and v with respect to B1 are (2, 23 ) and (5, 4) respectively, and
the inner product is computed by



3 1 1 1 2 5
u v = 2,

= 13.
2
2 0
1 0
4
14


Theorem: The vectors u and v in a vector space Rn are orthogonal if and only if their
inner product is zero.
Theorem: The inner product satisfies the following properties, for all u, v, w Rn
and for all R:
1. u v = v u,
2. (u v) = (u) v = u (v),
3. (u + v) w = u w + v w,
4. u u = ||u||2 0, and u u = 0 if and only if u = 0.

The inner product can be generalized to vector spaces other than Rn as follows.
Let V be a vector space. An inner product on V is a mapping from pairs of vectors
(u, v) to R, the result of which is a real number denoted by u v, which satisfies the
following properties:
1. u v = v u, for all u, v V,
2. (u v) = (u) v = u (v), for all u, v V and for any scalar ( R or C),
3. (u + v) w = u w + v w, for all u, v, w V,
4. u u 0, for all u V, and u u = 0 if and only if u = 0.
In general, if V is a vector space in which a dot product has been defined, we refer
to it as an inner product space, and we say that the length or norm ||u|| of a vector
u V is the real number:

||u|| =

15

u u,

and if a vector has norm 1, then it is said to be a unit vector. The notion of angle
between two vectors u and v in a given inner product space V is generalized by
cos =

3.5.2

uv
.
||u|| ||v||

Orthogonal projection

Consider two vectors u and v, geometrically represented as in the following figure.

The orthogonal projection of vector v on vector u is the
blue segment (which is perpendicular to the red dotted
segment), and we will represent it by Pu (v). Using the

notion of inner product, the length of this projection can

be easily computed as
||Pu (v)|| =

uv
.
||u||

Since the projection has the same direction as u, we can write

Pu (v) =

uv u
uv
=
u.
||u|| ||u||
uu

If u is a unit vector, then we write simply

Pu (v) = (u v) u.

In general, let V be any inner product space, and let u V be any unit vector. Let
Pu : V V be a mapping defined by Pu (v) = (u v) v, for all v V. The vector Pu (v)
is called the orthogonal projection of v on u.

16