15-451 Homework 1

Rukhsar Neyaz
January 26, 2014

Problem 1.

(a) Solving the recurrence by unfolding :

T (n)

2T (n 1) + 2n

T (n)

2n + 2.2n1 + 2.2.2n2 + ... + 2n1 2n(n1)

T (n)

2i .2n1
i=0
n

T (n)

2n

=
i=0

T (n)

= n.2n

T (n)

(n.2n )

(b) Solving this by recursion tree method :

4
5

T (n) = 4T

level
0
1

size
n
n
5

work/node

n
5

log4
log5

log4
log5

(n)

2
.
.
.

n
52

n
52

i
.
.
.

n
5i

n
5i

.
.
.
.
.
.

# of nodes

log4
log5

42

.
.
.

.
.
.

log4
log5

total work in level

.
.
.
.
.
.
.
.
.
.
.
.

4i

.
.
.

4i .

.
.
.

log5 n

T (n)

log4

+ n log5

log4

n log5
i=0

log4
log5

n
5i

.
.
.

log4
log5

log4

log5 n

= n log5

log4

5 log5

i=0
log4

log5 n

1i

n log5

i=0

log4
log5

(n

(log5 + 1)

log4
log5

(c) Solving this by recursion tree method :

level size work/node # of nodes

.
.
.

total work in level

.
.
.
.
.
.
.
.
.
.
.
.

.
.
.

.
.
.

nlogn

n
3

n
n
3 log 3

2
.
.
.
i
.
.
.

n
32

n
n
32 log 32

92

n
3i

n
n
3i log 3i

9i

.
.
.

.
.
.

.
.
.

.
.
.

log3 n

T (n)

3i .n.log

=
i=0

log3 n

3i log

= n

log5 n)

i=0

n
n
9i . 3i .log 3i

n
3i
n
3i

log3 n

3i log3 n 3i i

= n
i=0

n1
2

3n 3nlogn 3
+
+
4
2log3
4
2
2
1 2
3n
3n logn 3n
=
(n log3 n nlog3 n) +

2
4
2log3
4
(n2 )
= n log3 n.

(d) Solving this by recursion tree method:

level

size

work/node

# of nodes

n1/4

n1/4

n3/4

n1/4

3
.
.
.

n1/4

i
.
.
.

n1/4

n1/4

n3/4 (n1/4 )3/4

n1/4

n3/4+3/4 (n1/4 )3/4

.
.
.
.
.
.

total work in level

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.

.
.
.

n1/4

.
.
.

1
j=1 4j

n1/4 .n

.
.
.

1
j=1 4j

.
.
.

log(logn)

T (n)

n1/4 .n(14

i=0
log(logn)

n
i=0

n(log(logn) + 1)

(n.log(logn))

Problem 2.

(a)

For Karastubas algorithm, we multiply two numbers by writing down the

number in binary form and dividing the bits into two parts. Let X and Y be
the given two numbers for multiplication. We divide X into A and B and divide
Y into C and D.
X.Y = (2n 2n/2 ).AC + 2n/2 (A + B).(C + D) + (1 2n/2 ).BD

Let n be the number of bits in each number : X and Y. After dividing the
bits equally, A,B,C and D have each n/2 number of bits.
We have six additions above. For each addition we will look at the maximum
number of bits required.
For (2n 2n/2 ) the maximum number of bits required will be n as we have
n
2 .
For (A+B) the maximum number of bits required will be n/2 as both are
n/2 bits. Similar will be the case for (C+D) which would then require n/2 bits.
For (1-2n/2 ) the maximum number of bits required would be n/2 as we have

2n/2 .

This all will add upto 5n/2 bits.

Now for the large additions, we rst multiply two n/2 bits numbers which
would give us a n bits number. Later we also shift these numbers by n which
3

makes it 2n bits. Thus, we require two 2n bits large additions = 4n bits. We

recursively call this function thrice when the multiplication function is used.
Here the base case would be constant.
T (n)

T (1)

5n
n
+ 4n + 3T ( )
2
2
13
n
=
+ 3T ( )
n
2
= constant
=

Solving the recurrence by recursion tree method:

level size work/node # of nodes total work in level
.
.
13n
0
n
1
.
2
.
.
n
13 n
1
3
.
2
2
2
.
.
n
13 n
2
2
3
.
22
2
22
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
n
13 n
n
i
i 13
i
3
3 . 2 . 2i
2i
2
2i
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
T (n)

=
=

13n
2
13n
2

log2 n

3
2

i=0
log2 2n

(b) In the standard divide-and-conquer algorithm for matrix multiplication, we

divide each matrix into four sub-matrices. If n is the size of the initial matrix
then the sib-matrices are of size n . Let X and Y be the given input matrices.
2
We divide the matrices into four matrices.
A B E F
AE+BG AF+BH
X.Y=
.
=
C D G H
CE+DG CF+DH
Above we have eight multiplications and we recursively keep dividing the
matrices until we have one integer and then we multiply the integer. Thus the
base case would be 1 multiplication and in other cases we would just recursively
keep dividing.
T (1)

n
4

T (n)

=
=

n
8.T ( ) = 8log2 n
2

For the addition, we are adding two matrices of size each n thus having
2
2
integers. We perform four such additions. Thus we have 4 n additions.
2
4

And we have eight recursive calls. At the base case as we perform only one
multiplication and zero addition thus it would be zero.
T (1)

T (n)

n2 + 8T

n
2

n
2

log2 n

8
22

n2

n
2

+ 8T

i=0

log2 n

n2

2i

i=0

n (2n 1)

(c) In Strassens algorithm, we use seven dierent matrices which are calculated
by some multiplication and addition after dividing the input matrices of size n
into matrices of size n/4. We always add only matrices of size n/4 in all the
2
steps thus resulting in matrices of size also n/4. Thus this would require n
4
integer addition for every pair of matrix addition. There are 18 such additions
and we call this function seven times recursively to multiply the matrices. Thus
the recurrence would be:
T (1)

= constant
n2
n
T (n) = 18
+ 7T ( )
4
2
=
=
=
=

18n2
4

log2 n

i=0

7
22

9 2 1 7log2 2n
n
4
2
3
n2
3 log2 2n
(7
4n2 )
2
3 log2 2n
.7
6n2
2

Problem 4.

(a) Extraction sort is similar to deterministic quicksort in terms that we select

the rst element of the input array to be the pivot and then we divide the array
into two parts, the elements which are larger than the pivot and others which
are not. Even in quicksort we divide the array into two parts. Extraction sort
5

is not asymtotically faster than quick sort in the worst case as both give bounds
of O(n2 ) in the worst case.
Let A = [5,4,3,2,1]. Below is a chart designing the state of the list,the sublist
and the sorted list after each iteration.
A
sublist
sorted
[5,4,3,2,1]
[]
[]
[4,3,2,1]
[5]
[]
[4,3,2,1]
[]
[5]
[3,2,1]
[4]
[5]
[3,2,1]
[]
[4,5]
[2,1]
[3]
[4,5]
[2,1]
[]
[3,4,5]
[1]
[2]
[3,4,5]
[1]
[]
[2,3,4,5]
[]
[1]
[2,3,4,5]
[]
[]
[1,2,3,4,5]
Here, we recurse through the whole list every term we recurse. Only one
element gets removed from the list. Thus we have n comparisons in every n
recursive calls. Thus this will be O(n2 ).
Let T(n) be the number of comparisons on an input of size n in the worst
case.
=

2n + T (n 1)

T (n)

O(n2 )

(b)
We let here X(n) be the required number of comparisons on an random input.
We need to nd the expected number of such comparisons T(n) = E[X(n)]. In
the algorithm, the while loop is entered only if the size of A is larger than zero,
thus for the base case the expected number will be zero as we do not enter the
loop. Thus, T(0) = 0.
The number of comparisons here completely depends on how many elements
are there in the list A. Thus, it depends on how many elements remain after
each iteration of the while loop. Let Y be the random variable representing the
number of elements remaining in the list A after each such iteration. Now since
there are n elements in A initially, Y can only range from 0 to (n-1). Then we
would calculate T(n) as:
n1

T (n)

P (i elements remaining)E[required comparisons|i elements remaining]

i=0
n1

P (R = i)E[Xn |R = i]

=
i=0

Now here we have at each step, i elements remaining in the list A. Thus,
E[X(n)|R=i] = 2n + E[X(i)] = 2n + T(i). Substituting in the above equations,
n1

T (n)

P (R = i)(2n + T (i))
i=0
n1

n1

2n.P (R = i) +
i=0

P (R = i)T (i)
i=0

n1

2n +

P (R = i)T (i)
i=0

If we use the above recurrence to nd T(n-1), we can then use it for removing
the summation above.
n2

T (n 1)

2(n 1) +

P (R = i)T (i)
i=0

n2

T (n 1) 2(n 1)

P (R = i)T (i)

=
i=0

We can just make the summation in our main recurrence go till (n-2) by
taking out the (n-1)th term. Thus,
n2

T (n)

2n + P (R = (n 1))T (n 1) +

P (R = i)T (i)
i=0

2n + P (R = (n 1))T (n 1) + T (n 1) 2(n 1)

2 + T (n 1) + P (R = (n 1))T (n 1)

In the above recurrence, R=(n-1) implies that we have n-1 left elements left
in the list after an iteration, thus the rst element was the smallest element in
1
the list. The probability of it happening is n . Thus,
=

2 + T (n 1) +

T (n)

2+ 1+

1
n

1
T (n 1)
n

T (n 1)

(c) As extraction sort takes out sorted sublists out from the input and then keep
merging into the nal result, thus when we have lists which are almost sorted,
these can be used to sort those lists as it would take out the sorted sublists and
the cost of sorting would be less.
7

Problem 3.

(b) Let the bottom card of the deck at the starting of the algorithm be B. Let
X(i) be a random variable representing the number of steps taken for B to reach
the ith position counting from the top of the deck. As B would be the last card
to reach the top of the deck , the number of steps needed to bring B at the top
would be (the total number of steps in the algorithm - 1). Thus, E[X(n)] will
be the required expected number of steps before the algorithm terminates.
We have two cases here which contributes to the E[X(n)]. One case is when
B moves up in any iteration or other case is when it stays at the same position.
E[X(n)]

= P (B moves up).E[X(n)|B moves up] + P (B remains at n)E[X(n)|B remains at n]

If we are at any position i, then there would be n-(i-1) below B at which the
current card removed should be placed for B to move up. Thus the P(B moves
up) at any position i would depend on where the card removed is placed and
hence would be n(i1) . Thus, B would remain at the same position if we place
n
the removed card in any (i-1) positions above B. Hence,
E[X(i)] =

i1
n (i 1)
E[X(i)|B moves up] +
E[X(i)|B remains at i]
n
n

Now for the recursive calls, we notice that when B moves up, we count the
current step and then recursively call the same function to nd the number of
steps in the next recursive call. Similarly, when B remains at i, we count the
current step but call the same recursive call again as B does not move up.
E[X(i)]

n.E[X(i)]
E[X(i)]

n (i 1)
i1
[1 + E[X(i 1)]] +
[1 + E[X(i)]]
n
n
i1
n (i 1)
E[X(i 1)] +
E[X(i)]
= 1+
n
n
= n + n (i 1)E[X(i 1)] + (i 1)E[X(i)]
n
=
+ E[X(i 1)]
ni+1
=

Base case would be with only one card and that card itself would be bottom
card. We would just require one step to place the card again at the same position
to complete the algorithm. E[X(1)] = 1. Using unfolding method we would get
the following summation:
n

E[X(n)] =
i=1

n
ni+1