Algorithms:

COMP3121/3821/9101/9801
Aleks Ignjatovic
School of Computer Science and Engineering
University of New South Wales
COMP3121/3821/9101/9801 1 / 18
What is this course about?
Simple answer:
Problem solving
Our goal:
To learn the principles and techniques needed to solve vast array of
unfamiliar problems that arise in a rapidly changing eld
Course content:
a survey of algorithms design techniques
applied abstract thinking (is this an oxymoron??)
algorithm design SKILL: how to design new algorithms for any
problem that may arise
COMP3121/3821/9101/9801 2 / 18
Warming up problems
Problem 1: Two thieves have robbed a warehouse and have to split a pile
of items without price tags on them. How do they do this in a way that
ensures that each thief believes that he has got at least one half of the
value of the whole pile?
The solution: One of the two thieves splits the pile in two parts, so that
he believes that both parts are of equal value. The other thief then
chooses the part that he believes is at least one half.
Problem 1A: Three thieves have robbed a warehouse and have to split a
pile of items without price tags on them. How do they do this in a way
that ensures that each thief believes that he has got at least one third of
the value of the whole pile?
COMP3121/3821/9101/9801 3 / 18
Warming up problems
Problem 2: Tom and his wife Mary went to a party where nine more
couples were present. Not every one knew every everyone else, so people
who did not know each other introduced themselves and shook hands.
People that knew each other from before did not shake hands. Later that
evening Tom got bored, so he walked around and asked all other guests
(including his wife) how many hands they had shaken that evening, and
got 19 dierent answers. How many hands did Mary shake?
COMP3121/3821/9101/9801 4 / 18
Warming up problems
Problem 3: We are given 27 coins of the same denomination; we know
that one of them is counterfeit and that it is lighter than the others. Find
the counterfeit coin by weighing coins on a pan balance only three times.
COMP3121/3821/9101/9801 5 / 18
Warming up problems
Problem 4: We have nine coins and three of them are heavier than the
remaining six. Can you nd the heavier coins by weighing coins on a pan
balance only four times?
COMP3121/3821/9101/9801 6 / 18
Warming up problems
Problem 5: Consider a block of 7 X 7 houses:
The inhabitant of each house thinks that all four houses around him (to
the left, right, top and bottom) are nicer than his house and would like to
move to any of the four. Can you move the inhabitants around to make
them happier?
COMP3121/3821/9101/9801 7 / 18
Warming up problems
Problem 6: In Elbonia all cities have a circular one-way highway around
the city; see they map. All streets in the cities are one-way, and they all
start and end on the circular highway (see the map).
A block is a part of the city that is not intersected by any street. Design
an algorithm that, given a map of a city, nds a block that can be
circumnavigated while respecting all one-way signs. For example, the
green block has such property, but the red one does not.
COMP3121/3821/9101/9801 8 / 18
Basics revisited: how do we add two numbers?
C C C C C carry
X X X X X first integer
+ X X X X X second integer
-----------
X X X X X X result
adding 3 bits can be done in constant time;
the whole algorithm runs in linear time i.e., O(n) many steps.
can we do it faster than in linear time?
no, because we have to read every bit of the input
no asymptotically faster algorithm
COMP3121/3821/9101/9801 9 / 18
Basics revisited: how do we multiply two numbers?
X X X X <- first input integer
* X X X X <- second input integer
-------
X X X X \
X X X X \ O(n^2) intermediate operations:
X X X X / O(n^2) elementary multiplications
X X X X / + O(n^2) elementary additions
---------------
X X X X X X X X <- result of length 2n
We assume that two Xs can be multiplied in O(1) time.
Each X could be a bit or in some arbitrary base
Can we do it in LINEAR time, like addition?
No one knows! Even simple problems can turn out to be dicult!
COMP3121/3821/9101/9801 10 / 18
Can we do multiplication faster than O(n
2
)?
Let us try a divide-and-conquer algorithm:
take our two input numbers A and B, and split them into two halves:
A = A
1
2
n
2
+A
0
XX . . . X
. .
XX . . . X
. .
B = B
1
2
n
2
+B
0
n
2
n
2
A
0
, B
0
- the least signicant bits; A
1
, B
1
the most signicant bits.
AB can now be calculated as follows:
AB = A
1
B
1
2
n
+ (A
1
B
0
+B
1
A
0
)2
n
2
+A
0
B
0
(1)
How many steps does this take?
Each multiplication of two n digit numbers is replaced by four
multiplications of n/2 digit numbers: A
1
B
1
, A
1
B
0
, B
1
A
0
, A
0
B
0
, plus we
have a linear overhead to shift and add.
T(n) = 4T
_
n
2
_
+kn (2)
COMP3121/3821/9101/9801 11 / 18
Claim: if T(n) satises
T(n) = 4T
_
n
2
_
+kn (3)
then
T(n) = n
2
(k + 1) nk
Proof: by fast induction. Assume it is true for n/2:
T
_
n
2
_
=
_
n
2
_
2
(k + 1)
n
2
k
and prove that it is also true for n:
T (n) = 4T
_
n
2
_
+kn = 4
_
_
n
2
_
2
(k + 1)
n
2
k
_
+kn
= n
2
(k + 1) 2n k +kn = n
2
(k + 1) n k
COMP3121/3821/9101/9801 12 / 18
Thus, if T(n) satises
T(n) = 4T
_
n
2
_
+kn
then
T(n) = n
2
(k + 1) nk = O(n
2
)
i.e., we have not gained anything!
COMP3121/3821/9101/9801 13 / 18
The Karatsuba trick
Let us try a BETTER divide-and-conquer algorithm:
take again our two input numbers A and B, and split them into two halves:
A = A
1
2
n
2
+A
0
XX . . . X
. .
XX . . . X
. .
B = B
1
2
n
2
+B
0
n
2
n
2
A
0
, B
0
- the least signicant bits; A
1
, B
1
the most signicant bits.
AB can now be calculated as follows:
AB = A
1
B
1
2
n
+ (A
1
B
0
+B
1
A
0
)2
n
2
+A
0
B
0
= A
1
B
1
2
n
+ ((A
1
+A
0
)(B
1
+B
0
) A
1
B
1
A
0
B
0
)2
n
2
+A
0
B
0
How many steps does this take?
T(n) = 3T
_
n
2
_
+kn
COMP3121/3821/9101/9801 14 / 18
The Karatsuba trick
Since
T(n) = 3T

n
2

+ kn
implies
T

n
2

= 3T

n
2
2

+ k
n
2
and
T

n
2
2

= 3T

n
2
3

+ k
n
2
2
and
. . .
we get
T(n) = 3T

n
2

+ kn = 3

3T

n
2
2

+ k
n
2

+ kn = 3
2
T

n
2
2

+ k
3n
2
+ kn
= 3
2

3T

n
2
3

+ k
n
2
2

+ k
3n
2
+ kn = 3
3
T

n
2
3

+ k
3
2
n
2
2
+ k
3n
2
+ kn
= 3
3
T

n
2
3

+ kn

3
2
2
2
+
3
2
+ 1

Continuing in this way ...

COMP3121/3821/9101/9801 15 / 18
The Karatsuba trick
T(n) = 3T

n
2

+ kn = 3

3T

n
2
2

+ k
n
2

+ kn = 3
2
T

n
2
2

+ k
3n
2
+ kn
= 3
2

3T

n
2
3

+ k
n
2
2

+ k
3n
2
+ kn = 3
3
T

n
2
3

+ k
3
2
n
2
2
+ k
3n
2
+ kn
= 3
3
T

n
2
3

+ kn

3
2
2
2
+
3
2
+ 1

=
= 3
3

3T

n
2
4

+ k
n
2
3

+ kn

3
2
2
2
+
3
2
+ 1

=
= 3
4
T

n
2
4

+ kn

3
3
2
3
+
3
2
2
2
+
3
2
+ 1

=
. . .
= 3
log
2
n
T

n
2
log
2
n

+ kn

3
2

log
2
n1
+ . . . +
3
2
2
2
+
3
2
+ 1

3
log
2
n
T(1) + kn

3
2

log
2
n
1
3
2
1
= 3
log
2
n
T(1) + 2kn

3
2

log
2
n
1

COMP3121/3821/9101/9801 16 / 18
The Karatsuba trick
So we got
T(n) 3
log2n
T(1) + 2kn
_
_
3
2
_
log2n
1
_
We now use a
log2b
= b
log2a
to get
T(n) n
log23
T(1) + 2kn
_
n
log2
3
2
1
_
= n
log23
T(1) + 2kn
_
n
log231
1
_
= n
log23
T(1) + 2kn
log23
2kn
= O(n
log23
) = O(n
1.58...
) << n
2
COMP3121/3821/9101/9801 17 / 18
The Karatsuba trick
1
Can we multiply large integers faster than O
_
n
log23
_
??
2
Can we avoid having to compute messy things like:
T(n) = 3T

n
2

+ kn = 3

3T

n
2
2

+ k
n
2

+ kn = 3
2
T

n
2
2

+ k
3n
2
+ kn
= 3
2

3T

n
2
3

+ k
n
2
2

+ k
3n
2
+ kn = 3
3
T

n
2
3

+ k
3
2
n
2
2
+ k
3n
2
+ kn
= 3
3
T

n
2
3

+ kn

3
2
2
2
+
3
2
+ 1

=
= 3
3

3T

n
2
4

+ k
n
2
3

+ kn

3
2
2
2
+
3
2
+ 1

=
= 3
4
T

n
2
4

+ kn

3
3
2
3
+
3
2
2
2
+
3
2
+ 1

=
. . .
= 3
log
2
n
T

n
2
log
2
n

+ kn

3
2

log
2
n1
+ . . . +
3
2
2
2
+
3
2
+ 1

3
log
2
n
T(1) + kn

3
2

log
2
n
1
3
2
1
= 3
log
2
n
T(1) + 2kn

3
2

log
2
n
1

COMP3121/3821/9101/9801 18 / 18