The Design and Analysis of Algorithms

Chapter 2:
Fundamentals of the
Analysis of Algorithm
Efficiency
Asymptotic Notations and Basic Efficiency Classes
Section 2.2. Asymptotic Notations
and Basic Efficiency Classes
 Classifying Functions by Their Asymptotic
Growth
 Formal Definitions of Theta, Little oh and
Little omega
 Big Oh and Big Omega
 Rules to Manipulate Big-Oh Expressions
 Typical Growth Rates

2
 Classifying functions by their
asymptotic growth
 Given a particular function g(n) , the set of all
functions can be partitioned into three sets:
 Little oh : o(g(n)): the set of functions f(n) that
grow slower than g(n)
 Theta: Θ (g(n)): the set of functions f(n) that
grow at same rate as g(n)
 Littleomega: ω(g(n)): the set of functions f(n)
that grow faster than g(n)

3
 Formal Definition of Theta
f(n) and g(n) have the same rate of
growth, if
lim( f(n) / g(n) ) = c, 0 < c < ∞, n → ∞

Notation: f(n) = Θ(g(n))

There exist constants c1 and c2 and a nonnegative integer n0 such that

c2g(n) ≤ f(n) ≤ c1g(n) for all n ≥ n0

4
 Formal Definition of Little oh
f(n) grows slower than g(n)
( or g(n) grows faster than f(n)) if
lim(f(n)/g(n)) = 0, n → ∞
Notation: f(n) = o(g(n))

There exists a constant c and a nonnegative integer n0 such that

f(n) < c.g(n) for all n ≥ n0

5
 Formal Definition of Little omega

f(n) grows faster than g(n)

( or g(n) grows slower than f(n)) if

lim(f(n)/g(n)) = ∞, n → ∞
Notation: f(n) = (g(n))

There exists a constant c and a nonnegative integer n0

such that
c.g(n) < f(n) for all n ≥ n0

6
 Big Oh and Big Omega

 O(g(n)) = o(g(n)  Θ(g(n))

 (g(n)) = ω (g(n))  Θ(g(n))

7
f(n) = ω (g(n))

f(n) grows faster

than g(n)
f(n) =Ω(g(n)
f(n) = Θ (g(n))

f(n ) grows with

same rate

f(n) =o(g(n)) f(n) = O(g(n))

f(n) grows slower
than g(n)

8
 Rules to Manipulate Big-Oh Expressions
 Let T1(N) = O(f(N)) and T2(N) = O(g(N))

 T1(N) + T2(N) = max(O(f(N)), O(g(N)))

where max(O(f(N)), O(g(N))) =

f(N) if g(N) = O(f(N))
g(N) if f(N) = O(g(N)),

 T1(N) * T2(N) = O(f(N) * g(N))

 If T(N) is a polynomial of degree k,
T(N) = Θ(Nk) = O(Nk)
 logkN = O(N) for any constant k.

9
Typical Growth Rates
C constant, we write O(1)
logN logarithmic
log2N log-squared
N linear
NlogN
N2 quadratic
N3 cubic
2N exponential
N! factorial

10