Algorithms….

Recurrences
Dr. ISSAM ALHADID
Modification date 25/2/2019
Algorithm Analysis

 Running Time: O(n4)

Algorithm Analysis

???

???

 Running Time: ???

 Running time is the Exact # of times sum++ is executed
..
 Inner loop : 1 + 2 + 3+ …. + n 2
 (Paper soln.)
 Algorithm Analysis –
Nested Loops ( While )

Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8

The time required for this algorithm is proportional to n2
Recall – nested loops time complexity = N * M
Algorithm Analysis
Inner loop : 1 + 2 + 3+ …. + n 2
Recursive Functions
 long power(long x, long n)
 if (n == 0)

◦ return 1;
 else
◦ return x * power(x, n-1);

How many times is this executed?

Recursive Functions
 T(n) = Time required to solve a problem of
size n
 Recurrence relations are used to determine

the running time of recursive programs –

recurrence relations themselves are recursive.

 T(0) = time to solve problem of size 0 – Base

Case
 T(n) = time to solve problem of size n –

Recursive Case
Recursive Functions
 long power(long x, long n)
 if (n == 0)

◦ return 1;
 else
◦ return x * power(x, n-1);

 T(0) = c1 for some constant c1

 T(n) = c2 + T(n − 1) for some constant c2

 Soln. ????
Solving Recurrences
 when an algorithm contains a recursive call to
itself, its running time can often be described
by a recurrence.
 A recurrence is an equation or inequality

(variance ) that describes a function in terms

of its value on smaller inputs.
 Solving Recurrences
 For example, we saw that the worst-case
running time T (n) of the MERGE-SORT
procedure could be described by the
recurrence

Where the solution was claimed to be T (n) = (n lg n).

Solving Recurrences
 The following methods are used for solving
recurrences:
◦ substitution method (only the idea)
◦ Iteration method (recursion-tree method)
◦ master method
 The Substitution method
 In the substitution method, we guess a
bound and then use mathematical
induction to prove our guess correct.
 The substitution method for solving

recurrences entails two steps:

1. Guess the form of the solution.
2. Use mathematical induction to find the constants
and show that the solution works.
t h en
, a nd tion
w e r d uc
a n s y i n
e
th rrect b
s s
Gue e it co
v
pro
Recurrence Examples
Power example

Merge sort
Iteration Method
 Iteration method:
◦ Expand the recurrence k times.
◦ Stop the recurrence at smallest value (0 or
1).
◦ Work some algebra to express as a
summation and solve the summation, if
any exist.
Master Theorem
 Given: a divide and conquer algorithm:
 An algorithm that divides the problem of size

n into a sub-problems, each of size n/b.

 Let the cost of each stage (i.e., the work to

divide the problem + combine solved

sub-problems) be described by the function
f(n).

 Then, the Master Theorem gives us a

cookbook for the algorithm’s running time:
Master Theorem
 ifT(n) = aT(n/b) + f(n) where a ≥ 1 & b > 1
(n > n 0), when n/ n 0 is a power of b
Then:

 Examples….
Master Theorem (Book Version)
 if T(n) = aT(n/b) + f(n) where a ≥ 1 & b > 1
T(n) can be bounded asymptotically as follows:

ε: Epsilon
ε >0
Master Theorem
 The function divides the problem of size n into a
subproblems each of size n/b.

 The subproblems are solved recursively each in

time T(n/b).

 The cost of dividing the problem and combining

the results of the subproblems is described by the
f(n).
 We interpret n/b to mean either ⎡n/b ⎤ or ⎣n/b ⎦, which

does not affect the asymptotic behaviour of the

recurrence.
Understanding Master
Theorem
 In each of the three cases, we are comparing
the largest term in f(n) with n logba , the
solution to the recurrence is determined by the
larger of the two functions.
◦ In case 1, if the function n log ba is the larger, then the
solution T(n) = Θ(n logb a ).
◦ In case 3, if the function f(n) is the larger, then the
solution is T(n) = Θ(f(n)).
◦ In case 2, if the two functions are the same size, then
the solution is T(n) = Θ(n logb a lg n) = Θ(f(n) lg n).
◦  two functions: T(n) = aT(n/b) + f(n)
Understanding Master
Theorem
 In case 1, not only must f(n) be smaller than
n logba, it must be polynomially smaller. That is
f(n) must be asymptotically smaller than n logba
by a factor of n ε for some constant ε > 0.

 In case 3, not only must f(n) be larger than

n logba, it must be polynomially larger ; That is
f(n) must be asymptotically Larger than n logba
by a factor of n ε for some constant ε > 0, in
addition satisfy the “regularity” condition that:
a f(n/b) ≤ c f(n). – (you will understand
using an Example)
Understanding Master
Theorem
 It is important to realize that the three cases do not cover all the
possibilities for f(n).
◦ There is a gap between cases 1 and 2 when f(n) is smaller than n logba
but not polynomially smaller.

◦ polynomially smaller : The ratio f(n)/nlogba is asymptotically less than n ε

for any positive constant ε >0.

◦ There is a gap between cases 2 and 3 when f(n) is larger than n logba but
not polynomially larger and satisfy the “regularity” condition .

 polynomially larger: The ratio f(n)/nlogba is asymptotically greater than nε for

any positive constant ε >0

◦If f(n) falls into one of these gaps, the master method cannot be
used to solve the recurrence.
Understanding Master
Theorem
 regulatory condition : is used to make sure that
the parent does at least as much as the children.
 This says that f(n) (the amount of work done in
the root) needs to be at least as big as the
sum of the work done in the lower levels
 But what if the function didn't fulfill the
regularity condition?
 Ex. cos(n)*cos (n) instead of n*n?
 Then the work done on the lower levels might
be bigger than the work done in the root so
you are not certain that the root dominates.
Using Master Method
Examples: Case 1
 T(n) = 9T(n/3) + n
 T(n) = T(2n/3) + 1
 T(n) = 3T(n/4) + n lg n
 T(n) = 2T(n/2) + n lg n
Why Master theorem cannot be
applied here !?
 Example: T(n) = T(2n/3)+T(n/3)+n

The master method works only for following type of

recurrences:
T(n) = aT(n/b) + f(n) where a >= 1 and b > 1