Recurrences

When an algorithm contains a recursive call to itself, its running time can often be
described by a recurrence.
A recurrence is an equation or inequality that describes a function in terms of its
value on smaller inputs.
For example, the running time T(n) of the merge-sort algorithm can be expressed
by the recurrence
T(n) =
_
(1) if n is small,
2T(
n
2
) +(n) otherwise.
For recurrences that describe the running time of an algorithm, we assume (but
usually dont state) that T(n) is constant for small values of n.
COMP3600/6466: Lecture 5 2012 1
Methods for solving recurrences
Substitution method: Guess a bound and use mathematical induction to prove
that the guess is correct.
Recursion-tree method: Convert the recurrence into a tree, use this tree to
rewrite the function as a sum, then use techniques of bounding summations to
solve the recurrence.
Apply a standard result such as the master theorem.
COMP3600/6466: Lecture 5 2012 2
Substitution method
The substitution method consists of two steps:
Step 1. Guess the form of the solution.
Step 2. Use mathematical induction to show that the guess is correct.
It can be used to obtain either upper or lower bounds on a recurrence.
A good guess is vital when applying this method. If the initial guess is wrong, the
guess needs to be adjusted later.
COMP3600/6466: Lecture 5 2012 3
Substitution method
Example: Prove that T(n) = 2T(
n
2
) +n = O(nlgn).
Proof: We show that there exist c >0, n
0
>0 constants such that T(n) c nlgn for
all n n
0
. Note that the same constant c has to work for all n n
0
. We assume that
T(1) = k for some positive constant k.
Base case: n = 1 does not work: T(1) = k>c 1 lg1 = 0 for all positive c.
Try n = 2: We get T(2) = 2T(1) +2 = 2k +2 c 2lg2 = 2c, as long as c k +1.
Inductive step: We assume that for the constant c k +1 that we use in the
base case, and for all n

such that 2 n

<n, we have T(n

) c n

lgn

(induction
hypothesis), and show that the same statement holds for n, that is, T(n) c nlgn,
where c is the same constant that is used in the base case and in the hypothesis.
COMP3600/6466: Lecture 5 2012 4
Substitution method
Lets do it: We have
T(n) = 2T
__
n
2
__
+n
2 c
_
n
2
_
lg(
_
n
2
_
) +n
cnlg
n
2
+n = cnlgncnlg2+n = cnlgncn+n
cnlgn,
as long as c 1, and if the hypothesis can be used to bound T
__
n
2
__
, that is, if
2
_
n
2
_
<n, so n 4.
Since the inductive step has to work for all n >2, the case of n = 3 has to be treated
seperately: T(3) = 2T
__
3
2
__
+3 = 2T(1) +3 = 2k +3 c 3lg3, as long as c
2k+3
3lg3
.
We have proved that there is some large positive constant c such that
T(n) c nlgn for all n 2. Namely, any c max{k +1, 1,
2k+3
3lg3
} works.
COMP3600/6466: Lecture 5 2012 5
Substitution method
Example: Give an upper bound for T(n) = T(
n
2
) +T(
n
2
) +5.
Proof: We use mathematical induction, starting with the guess that the answer
might be T(n) = O(n).
Inductive step: We assume that for some constant c >0 and all n

such that
n
0
n

<n, we have T(n

) cn

(induction hypothesis).
For n, substitution gives
T(n) = T
__
n
2
__
+T
__
n
2
__
+5
c
_
n
2
_
+c
_
n
2
_
+5
= cn+5
> cn.
In such cases, we use induction to prove a slightly stronger result (if we believe that
our guess is correct).
COMP3600/6466: Lecture 5 2012 6
New hypothesis: Assume that for some constant c >0 and all n

such that
n
0
n

<n, we have T(n

) cn

b where b >0 is a constant.

Substituting this assumption gives
T(n) = T
__
n
2
__
+T
__
n
2
__
+5
c
_
n
2
_
b+c
_
n
2
_
b+5
= cn2b+5
cnb
for any c >0 as long as b 5, and if the hypothesis can be used for
n
2
and
n
2

(check this after establishing the base case).

Base case: Lets assume that T(1) = k where k >0 is a constant. For n = 1 we have
T(1) = k c 1b, as long as c k +b. So n
0
= 1 above, and we check and note
that the inductive step works for all n >1.
Summary: Let b 5 be a constant. Choose a constant c such that c k +b. Then,
we have T(n) cnb for n 1, which implies T(n) cn for all n 1. Thus,
T(n) = O(n).
Be careful!
Example: Find an asymptotic upper bound for T(n) = 2T(
n
2
) +n.
Fallacious proof: Suppose we guess T(n) = O(n), so in the inductive step we
assume that there is some constant c such that T(n

) cn

for any n
0
n

<n.
Substituting this assumption gives
T(n) = 2T
__
n
2
__
+n
2c
_
n
2
_
+n
cn+n = (c +1)n
= O(n). (WRONG!)
Note the difference between the main statement and the statement of the inductive
step! In the inductive step, we assume a statement for values smaller than n with a
xed c, and show that the same statement holds for n with the same c.
COMP3600/6466: Lecture 5 2012 7
How to make a good guess?
Bad news:
There is no general way to guess the correct solution to a recurrence.
Good news:
Experience helps.
If a recurrence is similar to one you have seen before, then guess a similar
solution.
Exercise: Show that T(n) = 2T(
_
n
2
_
+17) +n = O(nlgn).
Recursion trees can be used to make a good guess.
COMP3600/6466: Lecture 5 2012 8
Recursion-tree method
This method, also called the iteration method, can be used to guess or nd the
solution to a recurrence.
We tolerate some sloppiness, since we verify our guess later by the substitution
method.
We ignore oors and ceilings as they are usually insubstantial in solving recurrences.
The idea of the method is to expand (iterate) the recurrence and express it as a
summation of terms.
COMP3600/6466: Lecture 5 2012 9
Recursion-tree method
Consider the recurrence T(n) = 3T
__
n
4
__
+cn
2
.
Simplication: We assume that n is a power of 4.
(Cormen, p89)
COMP3600/6466: Lecture 5 2012 10
(Cormen, p89)
COMP3600/6466: Lecture 5 2012 11
T(n) as a sum
At depth i the subproblem size is
n
4
i
. We stop building the tree when we reach
subproblem size 1, so when
n
4
i
= 1. This gives i = log
4
n. Thus, the depth of the tree
is log
4
n (and the number of levels is log
4
n+1).
The cost at depth i is
_
3
16
_
i
cn
2
, except for the last level, whose cost is its number of
nodes times T(1), that is, 3
log
4
n
T(1) = n
log
4
3
T(1) = (n
log
4
3
).
These give
T(n) =
log
4
n1

i=0
_
3
16
_
i
cn
2
+(n
log
4
3
) = cn
2
log
4
n1

i=0
_
3
16
_
i
+(n
log
4
3
).
Note that
log
4
n1
i=0
_
3
16
_
i
= (largest term) = (1) since this is a decreasing geometric
sum. Finally, observe that n
2
grows faster than n
log
4
3
as its exponent is larger. Thus,
T(n) = (n
2
).
COMP3600/6466: Lecture 5 2012 12
A trickier recurrence
T(n) = T
_
n
3
_
+T
_
2n
3
_
+cn (Cormen, p91)
COMP3600/6466: Lecture 5 2012 13
A trickier recurrence
Why is the depth of this tree log3
2
n?
Do all paths from the root to the tips have the same length?
What is the number of nodes at the last level?
To answer these questions is not so easy but remember that all we need is a good
guess, which we can check using the substitution method.
Exercise: Guess an asymptotic upper bound on T(n) and prove by induction that it
is correct.
COMP3600/6466: Lecture 5 2012 14
Important things to remember
When using asymptotic notation, we take the fastest growing additive term and
ignore its constant coefcient.
Recall the difference between the dentions of O, , and o, : there exists some
positive constant versus for any positive constant.
Recall the hierarchy of growth rates. To compare the order of growth of functions,
use basic operations, take logarithms, or exponentiate.
In inductive proofs for statements like f (n) = O(g(n)), we have to use the same
constant c throughout the proof.
COMP3600/6466: Lecture 5 2012 15
Mathematical induction for recurrences
When we ignore ceilings and oors in recurrences, we often write expressions like
T
_
n
a
_
, where a is usually a positive integer. In such cases, we either prove the
statement only for values of n that are exact powers of a, or we prove the statement
for a larger set, for example for any rational number n.
Dont forget that an inductive proof is not complete without a base case. However,
we always assume that the running-time function T(n) is constant for small values of
n, and we use this fact in the base case.
COMP3600/6466: Lecture 5 2012 16