Beruflich Dokumente
Kultur Dokumente
(CLRS 3, Appendix A)
1 Algorithms matter!
Sort 10 million integers on
{ 1 GHZ computer (1000 million instructions per second) using 2n2 algorithm.
{ 100 MHz personal computer (100 million instructions per second) using 50n log n algo-
rithm.
Computer : 109 2(107 )2 = 200000 seconds 55 hours.
inst:
inst: per second
Personal computer : 7
50 10 log 10
8
7 inst:
< 78 = 5 7 3 = 105 seconds.
50 10 7 3
10 inst: per second 10
2 Asymptotic Growth
In the insertion-sort example we discussed that when analyzing algorithms we are
{ interested in worst-case running time as function of input size n
{ not interested in exact constants in bound
A good reason for not caring about constants and lower order terms is that the RAM model
is not completely realistic anyway (not all operations cost the same)
+
We want to express rate of growth of standard functions:
{ the leading term with respect to n
{ ignoring constants in front of it
k1 n + k2 n
k1 n log n n log n
k1 n2 + k2 n + k3 n2
We also want to formalize e.g. that a n log n algorithms is better than a n algorithm. 2
+
O-notation (Big-O)
{ you have probably all seen it intuitively dened but we will now dene it more carefully.
1
2.1 O-notation (Big-O)
O (g (n)) = ff (n) : 9 c; n0 > 0 such that jf (n)j cjg(n)j 8n n0 g
O() is used to asymptotically upper bound a function.
O() is used to bound worst-case running time.
cg(n)
f(n)
n0
Examples:
{1=3n2 3n 2 O(n2 ) because 1=3n2 3n cn2 if c 1=3 3=n which holds for c = 1=3
and n > 1.
{ k1 n + k2 n + k3 2 O (n ) because k1 n + k2 n + k3 < (k1 + jk2 j + jk3 j)n and for c >
2 2 2 2
Note:
{ When we say \the running time is O (n )" we mean that the worst-case running time is
2
() is used to asymptotically lower bound a function.
2
f(n)
cg(n)
n0
Examples:
{ 1=3n2 3n =
(n2 ) because 1=3n2 3n cn2 if c 1=3 3=n which is true if c = 1=6
and n > 18.
{ k1 n + k2 n + k3 =
(n ).
2 2
{ k1 n + k2 n + k3 =
(n) (lower bound!)
2
Note:
{ When we say \the running time is
(n )", we mean that the best case running time is
2
(n ) | the worst case might be worse.
2
Insertion-sort:
{ Best case:
(n)
{ Worst case: O (n )
2
3
c2g(n)
f(n)
c1g(n)
n0
Strongest notion
f (n) g(n) =) f (n) = (g (n)).
4
L'Hospital's Rule : If f (n) and g(n) are dierentiable and either f (n) and g(n) ! 1 or f (n)
and g(n) ! 0 as n ! 1, then
f (n) f 0 (n)
lim
n!1 g(n) = lim
!1 g0 (n) : n
Example: As n ! 1,
1 n
1 n
1+ n = exp ln 1 + n (1)
1
= exp n ln 1+ (2)
0 n1
ln 1 + 1
= exp @ 1=n A n
(3)
0 . 1
(1=n ) 1 + 2 1
! exp @ 1=n
A ; by L'Hospital's Rule
2
n
(4)
= exp lim 1
!1 1 + 1
n
(5)
n
= exp(1) = e: (6)
2.5 Growth rate of standard functions
Book introduces standard functions in section 2.2 (we will introduce them as we need them):
{ Polynomial of degree d: p(n) =
P a n where a ; a ; : : : ; a are constants (and
d i
i=1 i 1 2 d
\Growth order": log log n; log n; pn; n; n log log n; n log n; n log2 n; n2; n3 ; 2 n
3 Summations
When analyzing insertion-sort we used an arithmetic series
X n
n(n + 1)
k = 1+2+3+ + n = = (n2 )
=1
k
2
+
P n
k = (n2 )
k =1
5
The exact answer can be gotten by method used by Gauss as a boy in grade school: Write
the sum forwards, and immediately below it write the sum backwards, and then sum the n
columns. Each of the n columns sums to n + 1. Therefore, double the summation is n(n + 1).
QED.
Another way (proof by induction!):
{ Basis: n = 1 =)
P1 = 1
=1
k
( +1)
n n
2
= 122 = 1
{ Induction:
P
Assume it holds for n: =1 k = ( 2+1)
n n n
P +1 k = ( +1)( +2) = 1 n2 + 3 n + 1
Show it holds for n + 1: =1
k
n n n
k2 2 2
Proof:
X
n+1
X n
k = k + (n + 1)
k =1 k =1
n(n + 1)
= 2 + (n + 1)
= 12 n2 + 12 n + n + 1
= 12 n2 + 32 n + 1
x 1
= O(x ), for x > 1 n
k =0
xk =1+x xk xn+1
k =0 k =0
.
Proof by induction:
{ Basis: n = 1 =)
P1 x = 1 + x k
=0
n+1 1
= 2 11 = ( +1)(
k
x
1 x ( 1)
1)
x
x
=x+1 x
x
x
{ Induction:
P
Assume holds for n: =0 x = n+11 1
n k x
P +1 x = n+2 1
Show it holds for n + 1: =0
k
n k
x
x
k1 x
Proof:
X
n+1
Xn
xk = xk + xn+1
k =0 k =0
= xn+1 1
1 +x
n+1
6
= xn+1 1+x n+1
(x 1)
x 1
= xn+1 1+x n+2
xn+1
x 1
= xn+2 1
x 1
for some c.
{ Basis: n = 1 =)
P 1
3 =1+3 =4 x
k =0
c3 = c3
1
Ok if c > 4=3
{ Induction: P
Assume holds for n: P=0 3 c3
n
k
k n
Proof:
X
n+1
X n
3 = k
3 +3k n+1
k =0 k =0
c3 + 3 n n+1
= c3 n+1
(1=3 + 1=c)
c3 n+1
X1 n
Hn = k
k =1
= ln n +
+ 21n 121n2 + : : :
where
0:44742 : : : .
Upper bound (approximate by superior integral, as in handout for sum of squares)
X1 Z 1 n
1 + x dx
n
k k =1
1
= 1 + ln n ln 1
= 1 + ln n
Lower bound (approximate by inferior integral)
X1 Z 1 n n+1
k
x
dx
1
k =1
7
= ln(n + 1) ln 1
= ln(n +1)
= ln n 1 + n1
1
= ln n + ln 1 + n
= ln n + n1 1
2n2 + : : :
4 Growth review
O() used to asymptotically upper bound functions.
() used to asymptotically lower bound functions.
() used to asymptotically tight bound functions.
5 Summation review
We computed a number of sum's using:
{ Manipulation
{ Splitting and bounding terms ideas
{ Induction (!)
{ Approximation by an integral