CSE548 & AMS542: Analysis of Algorithms Topic: Randomized Quicksort Analysis Scribe: Atluri Vamsi Krishna

Lecturer: Rezaul A. Chowdhury Date: 16 april 2014

20.1

Chernoff Bounds: Revisited with an Example

Lower bounds Pr[X (1 )]
e (1 )(1)
2 3 2

Upper Bounds

Pr[X (1 + )]

e (1+ )(1+)
2 3 2

P r[X (1 )] e Pr[X ] e

P r[X (1 + )] e Pr[X + ] e

There are chernoff bounds for lower tail and for upper tail. Lets illustrate this with an example : Q : Suppose we have n bins,we are throwing balls into the bins. Each ball is thrown uniformly at random into each bin. So the question is after we throw n2 balls into the bins, Whats the number of balls in every bin? Sol : We have n bins U1 U2 U3 ...... Un We have n2 balls If we throw all n2 balls. How many balls each bin might have? we will show that no bin will have more than Let no of balls in each bin i be bi We want to show that
3n 2

balls

n 3n bi 2 2

On average the expected number of balls in each bin is n as there are n2 balls and n bins. We will use chernoff bounds to prove. We create n2 0 , 1 variables for 1 i n2 Xi = Then 1 0 ith ball lands in the choosen bin otherwise

P r(Xi ) =

1 n

E [Xi ] = 1 P r(X = i) + 0 P r(X = 0)

n2 n2

X=
i=1

Xi =
i=1 n2

E [Xi ] 1 n

= =
i=1 n2

n = n= We use the in chernoff bounds

2 n 3 2

P r(x (1 + )n) e

if =

1 2

1 1 2 P r(x (1 + )n) e 3 2 n 3 P r(x ( )n) e 12 2

So, 1 n e 12 1 n e 12 = 1 1+ + 1 n 1 + ( 12 ) +
n ( 12 ) 1 n 2 2! ( 12 )

1 n 3 3! ( 12 )

1 n 2= 2! ( 12 )

+ ....... 1 O( 2 ) n

P r(x

3n 1 holds for any of n bins) = n O( 2 ) 2 n 1 = O( ) n

P r(x

1 3n holds for any of n bins) = n O( 2 ) 2 n 1 = O( ) n

P r (x <

1 3n will hold for all bins) = 1 O( ) 2 n

n ) will near to 1 As n becomes larger and larger we have P r(x < 32 3n So the higher bound on bi is 2 with High Probability.

We will now prove Lower Bounds P r(x (1 )n) e

2 n 3

if =
2

1 2

1 1 2 n P r(x (1 )n) e 2 2 n 1 P r(x ( )n) e 8 2

1 n e8 1 n e 12

1 1+ + 1 n 1 + ( 12 ) + (n 8)
1 n 2 2! ( 8 )

1 n 3 3! ( 8 )

1 n 2= 2! ( 12 )

+ ....... 1 O( 2 ) n

P r (x

n 1 holds for any of n bins) = n O( 2 ) 2 n 1 = O( ) n

P r(x

1 n holds for any of n bins) = n O( 2 ) 2 n 1 = O( ) n

P r (x >

n 1 will hold for all bins) = 1 O( ) 2 n

As n becomes larger and larger we have P r(x > n 2 ) will near to 1 So the lower bound on bi is
n 2

with High Probability.

20.2

Expected Time of Randomized Quicksort : Revisited

Randomized Quicksort Algorithm Input: A set of numbers . ( i.e., all numbers are distinct ) Output: The numbers of sorted in increasing order. Steps: 1. Pivot Selection: Select a number x S uniformly at random. 2. Partition: Compare each number of S with x, and determine sets Sl = {y S | y < x}andSr = {y S | y > x}. 3. Recursion: Recursively sort Sl and Sr 4. Output: Output the sorted version of Sl , followed by x, followed by the sorted version of Sr Observations: If S = n and X is the total number of comparisons made in step 2 ( Partition ) across all ( original and recursive ) calls to RAND QS, then RAND QS sorts S in n + X time. Then all we need to do is determine E[X]. Let s1 , s2 , s3 , ......., sn be the elements of S in sorted order. Let Sij = si , si+1 , ......, sj for all1 i < j n. For 1 i < j n, LetXij =
n1 n

1 x

if si is compared to sj otherwise

T henX =
i=1 j =i+1 n1 n

Xij E [Xij ].

E [X ] =
i=1 j =i+1 n1 n

E [X ] =
i=1 j =i+1

P r[Xij = 1].

Observations: xij = 0 : Once a pivot x with si < x < sj is chosen, si and sj will never be compared at any subsequent time. xij = 1 : if either si or sj is chosen as a pivot before any other item in Sij then si will be compared with sj . Xij = 1 then si and sj are in same segment when comparision was made. Let the length of the segment in which both si and sj lie be m 1 So P r(si is choosen as pivot) = m Since si and sj are int the same segement m j i + 1 1 P r(si ) j 1+1 = P r(sj )

P r[Xij = 1] = P r(si ) + P r(sj ) 1 1 2 = + = ji+1 ij+1 ji+1

n1 n

Hence, E [X ] =
i=1 j =i+1 n1 n

P r[Xij = 1] 2 ji+1

=
i=1 j =i+1 n1 ni

=
i=1 k=1 n1 n

2 k+1 2 k

=
i=1 k=1 n1

=
i=1

O(logn)

= O(nlogn)

20.3

High Probability Bound on Rand QS

Input: A set of numbers . ( i.e., all numbers are distinct ) Output: The numbers of sorted in increasing order. Steps: 1. Pivot Selection: Select a number x S uniformly at random. 2. Partition: Compare each number of S with x, and determine sets Sl = {y S | y < x}andSr = {y S | y > x}. 3. Recursion: Recursively sort Sl and Sr 4. Output: Output the sorted version of Sl , followed by x, followed by the sorted version of Sr Let us x an element z in the original input set of size n We will trace the partition containing z for c ln n levels of recursion, where c is a constant that will be determined later. If a partitioning step divides S such that
|S | 4

| Sl |, | Sr |

3|S | 4

then that partition is a balanced partition.

If at any point z is in a partition of size k , after a balanced partitioning step it ends up in a partition of size at most 3 4 k.

Since the input size is n, after size

3 4
c 4 ln n

c 4 ln n

balanced partitions, z will end up in a partition of

n=

It turns out that after c ln n levels i.e 14 ln n levels the z will end up in a segment of size 1. That means if c 14, then z will end up in its nal sorted position in the output after undergoing balanced partitions. For 1 i c ln n, let Zi = 1 0 if the partition at recursion level i is balaced; otherwise.

c 4 ln 4 3

, which is 1 for c 14
c 4 ln n

3k But a balanced partition is obtained by choosing a pivot with rank between k 4 and 4 , where k is the size of the set being partitioned. Since each element of the set is chosen uniformly at random, a balancing pivot

will be chosen with probability

3k k 4 4

1 2

1 Hence P r[Zi = 1] = 1 2 . Thus E [Zi ] = P r [Zi = 1] = 2 . c ln n

Total number of balanced partitions,Z =

i=1 c ln n

Zi .

Then = E [Z ] =
i=1

E [Zi ] =

c ln n 2 .

Now applying chernoff lower bound 5 with = 1 2, P r[Z (1 ) ] e P r[Z (1 ) ] e

2 2 2 2 c

= e 16

ln n

= n 16 =

1 c n 16

1 For c = 32, we have P r[Z 8 ln n] n 2. This means that the probability that z fails to reach its nal sorted position even after 32 ln n levels of 1 recursion is n 2.

The probability that at least one of the n input elements fail to reach its nal sorted position after 32 ln n 1 1 levels of recursion is n n 2 = n. the probability that all n input elements reach their nal sorted positions after 32 ln n levels of recursion 1 is 1 n . But observe that the total amount of work done in each level of recursion is O (n). total workdone in 32 ln n levels of recursion is O (n ln n). Hence, with High Probability RAND QS terminates in O (n ln n) time.

20.4

Random Skip Lists

Searching in a sorted Linked List : We have a sorted linked list. we want to nd out whether an item exits in the linked list.

In worst case it takes n comparisions. So its of order O(n) 2-level Linked List: In 2 level linked list we divide the elements into n segments. Then we promote the rst element in each segment to the next level. So whenever we have to search for an element. we rst search in the rst level it is found then our search ends, otherwise we search in the next level.

 In the worst case it takes 2 n comparisions. So its order is O( n). 3-level Linked List: In this we divide the linked list in 3 n segments.The rst level contains n elements. The second level 2 contains n 3 elements. And the third contains 3 n. In the worst case search takes 3 3 n comparisions. So the order is O( 3 n) .

1 So similarly, k - level Linked List: takes k k n time. For k =log n: Search takes (log n).n log n = 2 log n time.

(log n)-level Linked List:

Search takes (log n) .n log n = 2 log n time. Observations : l 1. Let nl = no of items in level l. Then nl+1 = n 2 . 2. Let ml = nl nl+1 = no of items in level l that have not reached level l + 1. Then ml = Random Skip List

n 2l

In Random skip list we insert a sentinal at the start of the list. We then promote each non sentinel item of 1 . If l + 1 is non empty then promote the sentinel too. level l > 0 to level l + 1 with a probability 2

The height of the structure is log n with high probability. In the next lecture we will show that we can search with a high probability of log n.

References 1. Class slides

10