Sorting

Text
Read Weiss, 8.1 8.8 O(N2) sorting algorithms: Insertion, Selection, Bubble O(N log N) sorting algorithms HeapSort, MergeSort, QuickSort

Sorting

Assumptions
Array of elements Contains only integers Array contained completely in memory

2) O(N

Sorting Algorithms
Insertion Sort Selection Sort

Bubble Sort

Insertion Sort
Pseudo-code Algorithm
public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p Insertion Sort Strategy: Start with p=1. In each pass of the outer loop, determine where the pth element should be inserted in the sorted | subarray. Make room for it, if necessary, by sorted | unsorted sliding sorted elements down one. When th : 0 | 1 2 3 4 5 appropriate slot is found, insert p element. Increment p and repeat.

a :

| 15 | 4 |

13

21

10

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 15

p (insert pth element into sorted array) 1 2 3 4 5

4 13 2 21 10

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 15

p 1
4

2 13

3 2

4 21

5 10

tmp=4

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 15

p 1 j 4

2 13

3 2

4 21

5 tmp < a[j-1]! 10

tmp=4

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 15

p 1 j 15

2 13

3 2

4 21

5 Copy a[j-1] down! 10

tmp=4

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 j 15

p 1
15

2 13

3 2

4 21

5 j==0, exit inner loop. 10

tmp=4

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 j 4

p 1
15

2 13

3 2

4 21

5 Copy tmp. 10

tmp=4

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

| sorted | unsorted i : 0 1 | 2 3 4 5 | a : 4 15 |13 2 21 10 |

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 4

1 15

p (insert pth element into sorted array) 2 3 4 5

13 2 21 10

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 4

1 15

p 2 j 13

3 2

4 21

5 tmp < a[j-1]! 10

tmp=13

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 4

1 15

p 2 j 15

3 2

4 21

5 Copy a[j-1] down! 10

tmp=13

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 4

1 j 15

p 2
15

3 2

4 21

5 tmp >= a[j-1], exit loop! 10

tmp=13

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 4

1 j 13

p 2
15

3 2

4 21

5 Copy tmp! 10

tmp=13

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

| sorted | unsorted 0 1 2 | 3 4 5 | 4 13 15 | 2 21 10 |

Insertion Sort: Step Through

for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p

i : a :

0 4

1 13

2 15

p 3
2

4 21

5 Continue 10

Insertion Sort: Analysis

public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort

Count comparisons Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i (from j=p downto j>0)
p: 1 2 3 4 i (n-1) max #comparisons: 1 2 3 4 i (n-1) total number of comparisons 1 + 2 + 3 + + (n-1) = (n-1)n/2

Selection Sort
Pseudo-code Algorithm
public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort

public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort Selection Sort Strategy: In each pass of the outer

Selection Sort: Step Through

| | unsorted i : | 0 1 2 a : | 15 4 13
|

loop, select smallest value in unsorted subarray (i.e., from pth element on). Swap smallest element with pth element. Increment p and repeat.

3 2

4 21

5 10

public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort

Selection Sort: Analysis

Count comparisons. Assume a.length == n In general, for a given p the number of comparisons performed in the inner loop is (from j=p+1 to j<a.length) = (n-p-1)
p: 0 1 2 i (n-3)(n-2) max #comparisons: (n-1)(n-2)(n-3) (n-i-1) 2 1 total number of comparisons (n-1)+(n-2)+ + 2 + 1 = (n-1)n/2

Bubble Sort
Pseudo-code Algorithm
public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort

Bubble Sort: Step Through

public static void bubbleSort(Comparable for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort a[]) {

Bubble Sort Strategy: Outer loop starts with bottom of array (i.e. p=a.length-1). In each pass of outer loop, bubble largest element down by swapping adjacent elements (i.e., a[j] and a[j+1]) from the top whenever a[j] is larger. Decrement p |and repeat.

i : a :

0 15

1 4

unsorted | 2 3 4 5 | 13 2 21 10 | |

Bubble Sort: Analysis

public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort

Count comparisons. Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i (from j=0 to j<p)
p: (n-1) (n-2) (n-3) i 2 1 max #comparisons: (n-1) (n-2) (n-3) i 2 1 total number of comparisons (n-1)+(n-2) + + 2 + 1 = (n-1)n/2

Heaps and Heap Sort

A heap is a data structure that is particularly well-suited for implementing the Priority Queues Definition
Every leaf node is of height h or h - 1. Every leaf node of height h appears to the left of every leaf node of height h 1. The value in each node is greater than or equal to those in its children (if any).
27

Example

28

Array Representation of Heaps

29

Building a Max heap

30

Building a Max heap...

31

Algorithm
Algorithm Build_Max_Heap(A[1..n], n) // Input: Array A[1..n] of size n // Output: Array A[1..n] in heap format for i n/2 downto 1 do Max_Heapify(A, i, n) end Build_Max_Heap.

32

Algorithm...
Algorithm Max_Heapify(A[1..n], i, n) // Input: Array A[1..n] of size n with starting node i // Output: Array A[1..n] in heap format l 2*i r 2*i + 1 if l n and A[l] > A[i] largest l else largest i if r n and A[r] > A[largest] largest r if largest i exchange A[i] A[largest]

33

Heap Sort

34

Heap Sort...

35

Algorithm - HeapSort
Algorithm HeapSort(A[1..n], n) // Input: Array A[1..n] of size n. // Output: Sorted array A Build_Max_Heap(A, n) for i n downto 2 do Exchange A[1] A[i] Max_Heapify(A, 1, i - 1) end HeapSort.
36