CSE211 Lecture Notes – 2004/2005-II

CSE 211: Data Structures

Lecture Notes II
by
Ender Özcan, Şebnem Baydere

Sparse Vectors and Matrices

In the previous lecture we have seen the implementation of ordered lists with one-dimensional arrays
and the concept of sequential mapping. Later, we have seen some commonly used features for data
structures; pointers and structures. In this lecture firstly, we will discuss the representation of higher
degree arrays and sparse matrices. After then, we will see what algorithm analysis means and how we
can estimate the time required for an algorithm. We will also talk about the techniques that can be
used to reduce the running time of an algorithm.

2.1 Representation of Higher Degree Arrays (2D , 3D)

We have seen the implementation of one dimensional arrays and representation of ordered lists. How
can we represent matrices? How higher degree arrays are implemented in the memory.
Ex: int a[4]; // assume &a[0] is (3F00)H
a[0]=0;
a[1]=1; 3F00
a[2]=2;
a[3]=3; 3F04
3F08
3F0C
3F10

We define R x C matrix M as a rectangular array of elements having R rows and C columns and then
refer to any particular element in the matrix by using two subscripts r and c in an expression M[r,c].
As in one-dimensional case we have an abstraction. In all programming languages we can declare
two dimensions and store and retrieve elements using indexes. But if we want to reach the elements
using pointers how we are going to know the storage structure of the two-dimensional array in
memory?

Compilers effect the implementation of arrays. Recall that memory is linear with the addresses of the
storage units running in a single increasing sequence. So a two-dimensional structure has to be
mapped onto a structure with one dimension. Most languages implement the abstraction matrix in
row-major where the two-dimensional array is stored row by row. In some languages (such as
Fortran) it is stored column-major.
Assume a row-major implementation of an array where each element occupies n bytes of
memory:
Array mRxC is mapped to m’1xL

1
CSE211 Lecture Notes – 2004/2005-II

m(1,1) …………………… m(1,C)

m(2,1) …………………… m(2,C)
. . .
. . .
. m(i,j) .
. . .
m(R,1) …………………… m(R,C)

m[i,j] maps to m’[k], m’ array has L=R*C elements

address of ( m’[k] )= start location + (i-1)*C*n + (j-1)*n

m(1,1) m(1,2) … m(2,1) m(2,2) … m(3,1)

Row 1 Row 2 Row 3 …

Ex: a[3,2];
1 2 3 4 5 6 k
1 2
1,1 1,2 2,1 2,2 3,1 3,2 i,j
1 1,1 1,2
Row1 Row2
2 2,1 2,2
3 3,1 3,2
Abstract
view

For higher dimensions, the implementation is a generalization of two-dimensional case. For the three
dimensional case, third dimension is conventionally called a plane. The new subscript is added
before the one for a row.

M[plane][row][column]

2
CSE211 Lecture Notes – 2004/2005-II

Abstract view

Plane 1 Plane 2 Plane 3

Row1 Row 2 Row 1 Rrow 2

Row Major Implementation

As in 2-D case this abstract structure is also mapped onto one dimensional linear structure.

Assuming n=1 and start location=1; m[p,i,j] = m’[ (p-1)*R*C+(i-1)*C + j-1 +1]
Ex: a[2,3,2]; 1 2
2,1,1 2,1,2
2,2,1 2,2,2 2
1 2 3 4 5 6 2,3,1 2,3,2
1,1,1 1,1,2 1,2,1 1,2,2 1,3,1 1,3,2 1 1,1,1 1,1,2
2 1,2,1 1,2,2 1
Row1 Row2 Row3
3 1,3,1 1,3,2
Plane1 Plane2

Abstract view

2.2 Sparse Matrices

Here we will consider how to handle vectors and matrices with special property that many of their
elements are "zero". This is an abstract term for whatever value is appropriate. Example: for strings
it is Null, for integers it is numeric 0, for structures it is empty record.

Structures with high proportion of zeroes are common in real world. Sometimes they are too large to
fit in main memory. We have to find a way to store them efficiently. There are two cases:

1) zero elements are concentrated in a predictable part of the structure.

2) Structures in which nonzero elements are randomly distributed.

IDEA: Find a way to store only nonzero elements and keep their actual position by a storage
mapping function.

Lower Triangular Matrices

A common structure in many computational problems representing physical systems is a square
matrix. All the elements above the diagonal are zero. We can reduce memory requirements by 50% as
the nonzero element locations are predictable.

3
CSE211 Lecture Notes – 2004/2005-II

We can obtain a storage structure for a lower triangular matrix by remembering how 2-D arrays are
mapped onto linear storage. We can devise a method to reach an element of the matrix.

m(1,1) …………………… 0 0 0 0
m(2,1) …………………… 0 0 0 0
. . .
. . 0 . m[i,j] = 0, where j>i
. m(i,j) .
. none zero . .
m(R,1) …………………… m(R,R) total number of elements = R2

m(1,1) m(2,1) m(2,2) m(R,1) m(R,R)

R
total number of elements =  x = R(R+1)/2
x 1

m(i,j) maps to m’( 1 + (i (i-1)/2) + (j-1)) for i≤j

Hence,

m(i,j) = m’( i (i-1)/2 + j ), if i≤j

0, if i>j

Symmetric Matrices
To see how this works, consider that for c>r we are in the upper region and we can deduce the value
by looking its diagonal.

m(i,j) = m(j,i)
We use the similar mapping function in the case of lower triangular matrix and store only the lower
triangle of the symmetric matrix

m(i,j) = m’( 1 + (i (i-1)/2) + (j-1)), if i≥j => m’( i (i-1)/2 + j ), if i≥j

m’( 1 + (j (j-1)/2) + (j-1)), if i<j m’( j (j-1)/2 + i ), if i<j

Band Matrices
Nonzero elements are concentrated on the
diagonal and within a fixed distance above w
and below it.
w

Sparse Vectors and Matrices

Zeroes are distributed unpredictably but uniformly throughout the structure.

4
CSE211 Lecture Notes – 2004/2005-II

Sparse Vectors: Say 1000 integers, 70-80% is zero

Problem: Save space, use other means instead of storing 1000 elements as an array.
1. Each nonzero element of the vector is a record containing index and value
2. Store the vector as an array of these records
S[0 0 0 0 0 0 43 0 0 0 0 0 -134 0 0 0 0 0 0 0 3 ….]

S’= ( (7,43), (13,-134), (21,3) )

S’(1) S’(2) S’(3)
Problem: This is static, consider the case when the number of nonzero elements changes as the
program runs.
1 Use an array of records where each record represents an index-value pair.
2 Assuming we can guess the maximum number of nonzero elements, so we need that information
3 We need to know the number of current zero elements

7 43 1 current nonzero = 3, max. nonzero = 6

13 –134 2
21 3 3
4

5
6

Since we assumed that the values frequently change, we need to consider the performance of
operations:
1. Element changing from zero to nonzero (adding a new item)
- if value is zero delete the item from the table (if it is in the table)
- if value is nonzero search the table for the desired index: if exists change the value, else add
(update the table)
2. Element changing from nonzero to zero (delete an item)
3. Retrieve the value of an element with a given index
- Search the table for an element with the desired index,
if successful return the value, else return 0.
4. Print entire sparse vector in order by index

Example: Implement polynomial as a data structure, allowing operations add, subtract, multiply
and divide on them. Sol.: Use a sequence of Exponent-Coefficient pair, corresponding to index-value
pair to represent a polynomial, assuming integer-valued coefficients. Assuming max. nonzero = 3

P(x)=8x10-2x5+3 Q(x)=2x5-8x10 R(x)= P(x)+Q(x)

0 3
10 8 10 -8
current nonzero=1
current nonzero = 2
5 -2 5 2

0 3

5
CSE211 Lecture Notes – 2004/2005-II

current nonzero = 3