Quick Sort

The basic version of quick sort algorithm was invented by C. A. R.

Hoare in 196 and formally introduced quick sort in 196!. "t is used on
the #rinci#le of divide$and$conquer. %uick sort is an algorithm of choice
in many situations because it is not difficult to im#lement& it is a good
'general #ur#ose' sort and it consumes relatively fewer resources during
e(ecution.
Good points
"t is in$#lace since it uses only a small au(iliary stack.
"t requires only n log(n) time to sort n items.
"t has an e(tremely short inner loo#
)his algorithm has been sub*ected to a thorough mathematical
analysis+ a very #recise statement can be made about #erformance
issues.
Bad Points
It is recursive. Especially if recursion is not available, the
implementation is extremely complicated.
It requires quadratic (i.e., n
2
) time in the worst-case.
It is fraile i.e., a simple mista!e in the implementation can
o unnoticed and cause it to perform badly.

%uick sort works by #artitioning a given array A,p . . r- into two non$
em#ty sub array A,p . . q- and A,q.1 . . r- such that every key in A,p . .
q- is less than or equal to every key in A,q.1 . . r-. )hen the two
subarrays are sorted by recursive calls to %uick sort. )he e(act #osition
of the #artition de#ends on the given array and inde( q is com#uted as a
#art of the #artitioning #rocedure.

QuickSort
". If p # r then
2. q $artition (A, p, r)
%. &ecursive call to 'uic! (ort (A, p, q)
). &ecursive call to 'uic! (ort (A, q * r, r)

/ote that to sort entire array& the initial call %uick 0ort 1A& 1& length,A-2
As a first ste#& %uick 0ort chooses as #ivot one of the items in the array
to be sorted. )hen array is then #artitioned on either side of the #ivot.
3lements that are less than or equal to #ivot will move toward the left
and elements that are greater than or equal to #ivot will move toward the
right.

Partitioning the Array
4artitioning #rocedure rearranges the subarrays in$#lace.

PARTITION (A, p, r)
". x + A,p-
2. i + p-"
%. j + r*"
). while .&/E do
0. &epeat j + j-"
1. until A,j- 2 x
3. &epeat i + i*"
4. until A,i- 5 x
6. if i # j
"7. then exchane A,i- 8 A,j-
"". else return j

4artition selects the first key& A,p- as a #ivot key about which the array
will #artitioned5
6eys 7 A,p- will be moved towards the left .
6eys 8 A,p- will be moved towards the right.

)he running time of the #artition #rocedure is 1n2 where n 9 r $ p .1
which is the number of keys in the array.
Another argument that running time of 4AR)")":/ on a subarray of
si;e 1n2 is as follows5 4ointer i and #ointer j start at each end and move
towards each other& conveying somewhere in the middle. )he total
number of times that i can be incremented and j can be decremented is
therefore :1n2. Associated with each increment or decrement there are
:112 com#arisons and swa#s. Hence& the total time is :1n2.

Array of Same Elements
0ince all the elements are equal& the 'less than or equal' teat in lines 6
and < in the 4AR)")":/ 1A& p& r2 will always be true. this sim#ly means
that re#eat loo# all sto# at once. "ntuitively& the first re#eat loo# moves j
to the left+ the second re#eat loo# moves i to the right. "n this case& when
all elements are equal& each re#eat loo# moves i and j towards the
middle one s#ace. )hey meet in the middle& so q= Floor(p+r/2).
)herefore& when all elements in the array A,p . . r- have the same value
equal to Floor(p+r/2).

Performance of Quick Sort
)he running time of quick sort de#ends on whether #artition is balanced
or unbalanced& which in turn de#ends on which elements of an array to
be sorted are used for #artitioning.
A very good #artition s#lits an array u# into two equal si;ed arrays. A
bad #artition& on other hand& s#lits an array u# into two arrays of very
different si;es. )he worst #artition #uts only one element in one array
and all other elements in the other array. "f the #artitioning is balanced&
the %uick sort runs asym#totically as fast as merge sort. :n the other
hand& if #artitioning is unbalanced& the %uick sort runs asym#totically as
slow as insertion sort.

Best Case
)he best thing that could ha##en in %uick sort would be that each #artitioning stage
divides the array e(actly in half. "n other words& the best to be a median of the keys in
A,p . . r- every time #rocedure =4artition= is called. )he #rocedure =4artition= always
s#lit the array to be sorted into two equal si;ed arrays.
"f the #rocedure =4artition= #roduces two regions of si;e n>!. the recurrence relation is
then
)1n2 9 )1n>!2 . )1n>!2 . 1n2
9 !)1n>!2 . 1n2
And from case ! of ?aster theoremi

Worst case Partitioning
)he worst$case occurs if given array A,1 . . n- is already sorted. )he
4AR)")":/ 1A& p& r2 call always return # so successive calls to #artition
will s#lit arrays of length n& n$1& n$!& . . . & ! and running time
#ro#ortional to n . 1n$12 . 1n$!2 . . . . . ! 9 ,1n.!21n$12->! 9 1n
!
2. )he
worst$case also occurs if A,1 . . n- starts out in reverse order.




Randomized Quick Sort
"n the randomi;ed version of %uick sort we im#ose a distribution on
in#ut. )his does not im#rove the worst$case running time inde#endent of
the in#ut ordering.
"n this version we choose a random key for the #ivot. Assume that
#rocedure Random 1a, b2 returns a random integer in the range ,a, b2+
there are b$a.1 integers in the range and #rocedure is equally likely to
return one of them. )he new #artition #rocedure& sim#ly im#lemented
the swa# before actually #artitioning.

RANDOMIZED_PARTITION (A, p, r)
i @ RA/A:? 1p, r2
3(change A,p- B A,i-
return 4AR)")":/ 1A, p, r2
/ow randomi;ed quick sort call the above #rocedure in #lace of
4AR)")":/

RANDOMIZED_QUICKSORT (A, p, r)
"f # C r then
q @ RA/A:?"D3AE4AR)")":/ 1A, p, r2
RA/A:?"D3AE%F"C60:R) 1A, p, q2
RA/A:?"D3AE%F"C60:R) 1A, q+1, r2


Gike other randomi;ed algorithms& RA/A:?"D3AE%F"C60:R) has
the #ro#erty that no #articular in#ut elicits its worst$case behavior+ the
behavior of algorithm only de#ends on the random$number generator.
3ven intentionally& we cannot #roduce a bad in#ut for
RA/A:?"D3AE%F"C60:R) unless we can #redict generator will
#roduce ne(t.

Anal!i! o" Quick !ort
#or!t$ca!%
Get T1n2 be the worst$case time for %F"C6 0:R) on in#ut si;e n. He
have a recurrence
T1n2 9 ma(
17q7n$1
1T1q2 . T1n$q22 . 1n2 $$$$$$$$$ 1
where q runs from 1 to n$1& since the #artition #roduces two regions&
each having si;e at least 1.
/ow we guess that T1n2 7 cn
!
for some constant c.
0ubstituting our guess in equation 1.He get
T1n2 9 ma(
17q7n$1
1cq
!
2 . c1n $ q
!
22 . 1n2
9 c ma( 1q
!
. 1n $ q2
!
2 . 1n2
0ince the second derivative of e(#ression q
!
. 1n$q2
!
with res#ect to q is
#ositive. )herefore& e(#ression achieves a ma(imum over the range 17
q 7 n $1 at one of the end#oints. )his gives the bound ma( 1q
!
. 1n $
q2
!
22 1 . 1n $12
! 9
n
!
. !1n $12.
Continuing with our bounding of T1n2 we get
T1n2 7 c ,n
!
$ !1n$12- . 1n2
9 cn
!
$ !c1n$12 . 1n2
0ince we can #ick the constant so that the !c1n $12 term dominates the
1n2 term we have
T1n2 7 cn
!
)hus the worst$case running time of quick sort is 1n
2
2.

A&%ra'%$ca!% Anal!i!
"f the s#lit induced by RA/A:?"D3AE4AR)")":/ #uts constant
fraction of elements on one side of the #artition& then the recurrence tree
has de#th 1lgn2 and 1n2 work is #erformed at 1lg n2 of these level.
)his is an intuitive argument why the average$case running time of
RA/A:?"D3AE%F"C60:R) is 1n lg n2.
Get T1n2 denotes the average time required to sort an array of n elements.
A call to RA/A:?"D3AE%F"C60:R) with a 1 element array takes a
constant time& so we have T112 9 112.
After the s#lit RA/A:?"D3AE%F"C60:R) calls itself to sort two sub
arrays. )he average time to sort an array A,1 . . q- is T,q- and the
average time to sort an array A,q.1 . . n- is T,n$q-. He have
T1n2 9 1/n 1T112 . T1n$12 .
n$1
I
q91
T1q2 . T1n$q222 . 1n2 $$$ 1
He know from worst$case analysis
T112 9 112 and T1n $12 9 O1n
!
2
T1n2 9 1/n 1 112 . O1n
!
22 . 1>n
n$1
I
q91
1r1q2 . T1n $ q22
.
1n2
9 1/n
n$1
I
q91
1T1q2 . T1n $ q22 . 1n2 $$$$$$$ !
9 1>n,!
n$1
I
k91
1T1k2- . 1n2
9 !/n
n$1
I
k91
1T1k2 . 1n2 $$$$$$$$$ J
0olve the above recurrence using substitution method. Assume
inductively that T1n2 7 anlgn . b for some constants a K and b K .
"f we can #ick =a= and =b= large enough so that n lg n . b K )112. )hen for
n K 1& we have
T1n2 8
n$1
I
k91
!>n 1aklgk . b2 . 1n2
9 !a/n
n$1
I
k91
klgk - 1><1n
!
2 . !b>n 1n $12 . 1n2 $$$$$$$ L
At this #oint we are claiming that
n$1
I
k91
klgk 1/2 n
!
lgn - 1><1n
!
2
0tick this claim in the equation L above and we get
T1n2 !a>n ,1>! n
!
lgn $ 1><1n
!
2- . !/n b1n $12 . 1n2
anlgn $ an/L . !b + 1n2 $$$$$$$$$$ M
"n the above equation& we see that 1n2 . b and an/L are #olynomials
and we certainly can choose =a= large enough so that an>L dominates
1n2 . b.
He conclude that %F"C60:R)=s average running time is 1n lg(n)2.


Conclu!ion
%uick sort is an in #lace sorting algorithm whose worst$case running
time is 1n
2
2 and e(#ected running time is 1n lg n2 where constants
hidden in 1n lg n2 are small.
I()l%(%ntation
void quickSort(int numbers[], int array_size)
{
q_sort(numbers, 0, array_size - 1);
}
void q_sort(int numbers[], int let, int ri!"t)
{
int #i$ot, l_"ol%, r_"ol%;
l_"ol% & let;
r_"ol% & ri!"t;
#i$ot & numbers[let];
while (let ' ri!"t)
{
while ((numbers[ri!"t] (& #i$ot) )) (let ' ri!"t))
ri!"t--;
if (let *& ri!"t)
{
numbers[let] & numbers[ri!"t];
let++;
}
while ((numbers[let] '& #i$ot) )) (let ' ri!"t))
let++;
if (let *& ri!"t)
{
numbers[ri!"t] & numbers[let];
ri!"t--;
}
}
numbers[let] & #i$ot;
#i$ot & let;
let & l_"ol%;
ri!"t & r_"ol%;
if (let ' #i$ot)
q_sort(numbers, let, #i$ot-1);
if (ri!"t ( #i$ot)
q_sort(numbers, #i$ot+1, ri!"t);
S*%ll Sort
This algorithm is a sim#le e(tension of "nsertion sort. "ts s#eed comes
from the fact that it e(changes elements that are far a#art 1the insertion
sort e(changes only ad*acent elements2.
)he idea of the 0hell sort is to rearrange the file to give it the #ro#erty
that taking every h
th
element 1starting anywhere2 yields a sorted file.
0uch a file is said to be h$sorted.

S+E,,_SORT (A)
for h 9 1 to h N>9 do
for 1+ h K + h N9 J2 do
for i 9 h .1 to i n do
v 9 A,i
j 9 i
while 1j K h A/A A,j $ h- K v
A,i- 9 A,j $ h-
j 9 j $ h
A,j- 9 v
i 9 i . 1

)he function form of the running time for all 0hell sort de#ends on the
increment sequence and is unknown. Oor the above algorithm& two
con*ectures are n 1log n2
!
and n
1.!M
. Ourthermore& the running time is not
sensitive to the initial ordering of the given sequence& unlike "nsertion
sort.



0hell sort is the method of choice for many sorting a##lication because it
has acce#table running time even for moderately large files and requires
only small amount of code that is easy to get working. Having said that&
it is worthwhile to re#lace 0hell sort with a so#histicated sort in given
sorting #roblem.


IMP,EMENTATION
void s"ellSort(int numbers[], int array_size)
{
int i, ,, increment, tem#;
increment & -;
while (increment ( 0)
{
For (i&0; i ' array_size; i++)
{
, & i;
tem# & numbers[i];
while ((, (& increment) )) (numbers[,-
increment] ( tem#))
{
numbers[,] & numbers[, - increment];
, & , - increment;
}
numbers[,] & tem#;
}
if (increment./ *& 0)
increment & increment./;
else if (increment && 1)
increment & 0;
else
increment & 1;
}
}
-uck%t Sort

Bucket sort runs in linear time on the average. "t assumes that the in#ut
is generated by a random #rocess that distributes elements uniformly
over the interval ,& 12.
)he idea of Pucket sort is to divide the interval ,& 12 into n equal$si;ed
subintervals& or buckets& and then distribute the n in#ut numbers into the
buckets. 0ince the in#uts are uniformly distributed over 1& 12& we don=t
e(#ect many numbers to fall into each bucket. )o #roduce the out#ut&
sim#ly sort the numbers in each bucket and then go through the bucket
in order& listing the elements in each.
)he code assumes that in#ut is in n$element array A and each element in
A satisfies 7 A,i- 7 1. He also need an au(iliary array P, . . n $1- for
linked$lists 1buckets2.

-UCKET_SORT (A)
". n + lenth ,A-
2. 9or i : " to n do
%. Insert A,i- into list B,nA,i--
). 9or i : 7 to n-" do
0. (ort list B with Insertion sort
1. ;oncatenate the lists B,7-, B,"-, . . B,n-"- toether in
order.

Eample
Qiven in#ut array A,1..1-. )he array P,..9- of sorted lists or buckets
after line M. Pucket i holds values in the interval ,i>1& 1i .12>1-. )he
sorted out#ut consists of a concatenation in order of the lists first P,-
then P,1- then P,!- ... and the last one is P,9-.

Analysis
All lines e(ce#t line M take :1n2 time in the worst case. He can see
ins#ection that total time to e(amine all buckets in line M is :1n$12 i.e.&
:1n2.
)he only interesting #art of the analysis is the time taken by "nsertion
sort in line M. Get n
i
be the random variable denoting the number of
elements in the bucket !,i-. 0ince the e(#ected time to sort by
"/03R)":/E0:R) is :1n
!
2& the e(#ected time to sort the elements in
bucket P,i- is

",:1
!
n
i
2- 9 :1",
!
n
i
--

)herefore& the total e(#ected time to sort all elements in all buckets is


n$1
I
i9
O1",
!
n
i
-2 9 O
n$1
I
i9
1",
!
n
i
-2 $$$$$$$$$$$$ A


"n order to evaluate this summation& we must determine the distribution
of each random variable n
He have n elements and n buckets. )he #robability that a given element
falls in a bucket !,i- is 1>n i.e.& 4robability 9 p 9 1>n. 1/ote that this
#roblem is the same as that of 'Palls$and$Pin' #roblem2.
)herefore& the #robability follows the binomial distribution& which has
mean5 3,n
i
- 9 np 9 1
variance5 Rar,n
i
- 9 np11$ p2 9 1$ 1>n
Oor any random variable& we have
",
!
n
i
- 9 Rar,n
i
- . "
!
,n
i
-
9 1 $ 1>n . 1
!
9 ! $ 1>n
9 112
4utting this value in equation A above& 1do some tweaking2 and we have
a e(#ected time for "/03R)":/E0:R)& :1n2.

/ow back to our original #roblem
"n the above Pucket sort algorithm& we observe
)1n2 9 ,time to insert n elements in array A- . ,time to go
through au(iliary array !, . . n$1- S 10ort by
"/03R)":/E0:R)2
9 :1n2 . 1n$12 1n2
9 :1n2
)herefore& the entire Pucket sort algorithm runs in linear e(#ected time.