http://wikistudent.

ws/Unisa

Sorting
1. Bubble sort
How it works: The highest number bubbles its way up (with each outer iteration), by swapping with adjacent elements on the way (the inner loop). Version 1: template<class T> void bubblesort( T* a, int n ) { for( int i = 1; i < n; i++ ) // see Notes on Outer Loop. { for( int j = 0; j < n - i; j++ ) // see Notes on Inner Loop. { if( a[j] > a[j + 1 ] ) swap( a[ j ], a[ j + 1 ] ); } } } Notes on Outer Loop: With each iteration, the next highest element is pushed to the right. The number of iterations is 1 LESS than the number of array elements! This is because when you get to the bottom, the last element doesn't need to be sorted since all the rest are. (It's the smallest). (Don't have the outer loop starting from i = 0, because array element n - 0 is out of bounds). Notes on Inner Loop: Start swapping from the first to second last element (first stop at the second last because it may be swapped with the next i.e. last) As i increases, the inner loop decreases, as the sorted top half gets bigger. (You don't need to touch the sorted elements!) Version 2: template<class T> void bubblesort( T* a, int n { for( int i = 1; i < n; i++ { for( int j = 1; j <= n { if( a[j - 1] > a[j] ) swap( a[ j - 1 ], a[ } } }

) ) i; j++ ) // see Notes on Inner Loop 2.

j ] );

Notes on Inner Loop 2: Because j starts at 1 here, you must check index j-1 with j, so that you include the first element. Here index j also reaches the very last element, to swap with the second last. With both methods, each element is compared with an adjacent one, including the very first and very last elements. The difference is that one method compares an element with the next one, and the other method compares an element with the previous one. In the examples above, the arrays can be sorted in decreasing order instead by changing the > sign in the if statement to a < sign.

http://wikistudent.ws/Unisa

Version 3: template< class T > void bubbleSort( T* a, int n ) { bool sorted = false; for( int i = 1; i < n && !sorted; i++ ) for( int j = 1; j <= n - i; j++ ) { sorted = true; if( a[ j - 1 ] > a[ j ] ) { swap( a[ j - 1 ], a[ j ] ); sorted = false; } } } This is an improved version that makes the bubbling stop when the array is sorted. A flag (sorted) is used to make it more efficient. Running time for versions 1 & 2: On the first iteration of the outer loop, the inner loop executes n-1 times. On the second iteration, the inner loop executes n-2 times, so the number of iterations is the sum of (n-1) + (n-2) + ... + 1 = O(n 2 ) The bubble sort algorithm always executes the same number of comparisons, regardless of the original order of the elements in the array. So it doesnt matter whether the array is already sorted or whether it is sorted in reverse order; it is still very slow. Sample sorting trace, using version 1: Input data: i = 1 j = 0 j = 1 j = 2 i = 2 j = 0 j = 1 i = 3 j = 0 5 4 4 4 3 3 2 4 5 3 3 4 2 3 3 3 5 2 2 4 4 2 2 2 5 5 5 5

2. Selection sort
How it works: The smallest number gets swapped with the one at the bottom (with each outer iteration), and its found by sequentially searching the remaining unsorted elements (in the inner loop). Version 1: template<class T> void selectionsort( { for( int i = 1; i for( int j = i; { if( a[j] < a[ swap( a[j], } }

T* a, int n ) < n; i++ ) j < n; j++ ) i - 1 ] ) a[ i - 1 ] );

http://wikistudent.ws/Unisa The above is not as optimal as the following two because there are too many function calls to swap(). Its better to find the minimum element in the rest of the array first, before swapping. Version 2: template<class T> void selectionsort( T* a, int n ) { for( int i = 0; i < n-1; i++ ) { int minIndex = i; for( int j = i + 1; j < n; j++ ) if( a[ j ] < a[ minIndex ] ) // if its smaller than the next smallest... minIndex = j; // now the minIndex is not in front, swap( a[ minIndex ], a[ i ] ); // so that the swap here works correctly. } // the subarray[0:i] is now sorted } The above is an improvement to the bubble sort because the swapping is postponed to the end of the inner loop. Version 3: template<class T> void selectionsort( T* a, int n ) { for( int i = n-1; i > 0; i-- ) { int largeIndex = 0; for( int j = 1; j <= i; j++ ) if( a[ largeIndex ] < a[ j ] ) largeIndex = j;

// if it's larger than a previous number... // make its index the largeIndex

if( i != largeIndex ) // if its not already in the right position... swap( a[ i ], a[ largeIndex ] );// swap the largest with the end value } } The third version is an improvement on the second one because it prevents the redundant swap by checking in the final if statement. It works the opposite way to the second one because of the backwards method: here the right-most elements are sorted first, from largest to smallest. Notes: In all the examples, the number of iterations of the outer loop is one LESS than the array size because once n-1 elements are sorted, the remaining one must be sorted too. In the examples above, the arrays can be sorted in decreasing order instead by changing the > sign in the if statement to a < sign. Running time: It has the same number of comparisons as the bubble sort, so its an O(n 2 ) algorithm. Sample sorting trace, using version 2: Input data: i = 0 i = 1 i = 2 i = 3 5 1 1 1 1 4 4 2 2 2 3 3 3 3 3 2 2 4 4 4 1 5 5 5 5

http://wikistudent.ws/Unisa

3. Insertion sort
How it works: With each iteration of the outer loop, the next element is inserted in its correct position (relative to all the elements on the left). The inner loop is there to determine exactly where the element should go and shifts up the higher elements by one. Version 1: template<class T> void insertionsort( T* a, int n ) { for( int i = 1; i < n; i++ ) // the 2nd -> end elements are to be inserted for( int j = 0; j < i; j++ ) // check the previous sorted ones if( a[ i ] < a[ j ] ) { T temp = a[ i ]; // create a temp before it's written over for( int k = i; k > j; k-- ) // shift larger elements up a[ k ] = a[ k - 1 ]; a[ j ] = temp; break; // so that you avoid unnecessary comparisons } } Version 2: template<class T> void insertionsort( T* a, int n ) { for( int i = 1; i < n; i++ ) { T temp = a[ i ]; int j; for( j = i; j > 0 && a[ j - 1 ] > temp; j-- ) a[ j ] = a[ j - 1 ]; a[ j ] = temp; } } Version 3: template<class T> void insertionsort( T* a, int n ) { for( int i = 1; i < n; i++ ) { T temp = a[ i ]; int j = i - 1; while( j >= 0 && a[ j ] > temp ) { a[ j + 1 ] = a[ j ]; j--; } a[ j + 1 ] = temp; } } Notes: The shifting up must be done backwards, with the loop control variable decrementing with each iteration. The forwards method wont work!

http://wikistudent.ws/Unisa Running time: The complexity of the insertion sort is O(n 2 ) because in the worst case (i.e. when the array is in reverse order) each new element requires all of the elements on its left to be shifted one position right. The number of iterations of the inner loop follows the pattern 1 + 2 + 3 + ... + (n - 1). The sum of this series is n(n-1)/2. If the array is already sorted, the insertion sort has complexity O(n) because the total number of iterations is: 1 + 1 + ... + 1 + 1 = n-1. Sample sorting trace: Input data: i = 1 i = 2 i = 3 i = 4 5 4 3 2 1 4 5 4 3 2 3 3 5 4 3 2 2 2 5 4 1 1 1 1 5

4. Shell sort
How it works: You first choose a jumping distance, d, so that you can compare every dth element, moving up 1 position at a time. The exact value of d is determined by the array size - different formulas can be used. The outer loop starts at position d, and ensures that all values with index amultiple-of-d smaller than the loop index are in fact smaller than the value at the loops index. If the value at the loops index happens to be smaller, the insertion sort technique is used (the inner while loop) to shift up the dth index values and insert the new value into its correct position. After each complete run of the for loop, d is made smaller, so that narrower comparisons can be made, until d = 1 and the regular insertion sort completes the job. Version 1: template<class T> void shellsort( T* a, int n ) { // first choose the jumping distance, d: int d = 1, j; while( d < n/9 ) // (if n < 18, you may as well do an insertion sort) d = 3*d + 1; // d, the jump distance, starts off quite small here while( d > 0 ) // loop till d is 1, when you compare all elements { for( int i = d; i < n; i++ ) // go from element d to the end, comparing { // i with every d'th element below the current value T t = a[ i ]; // this is the temporary to be swapped... j = i; // the index for the while loop starts at the temp's index while( j >= d && a[ j - d ] > t ) // j is first equal to d (when i=d) { a[ j ] = a[ j - d ]; // first round: j-d = 0 (first element index) j -= d; // if j is smaller than d, quit the while loop } // before accessing negative array subscripts a[ j ] = t; // insert the loop control variable in the right position } d /= 3; // decrement d appropriately (it must end up as 1 in the end) } } // the while loop with the insertion sort has while j >= 0. This is // replaced with while j >= d here.

http://wikistudent.ws/Unisa

Version 2: template<class T> void shellsort( T* a, int n ) { for( int d = n / 2; d > 0; d = ( d == 2 ? 1 : ( int )( d / 2.2 ) ) ) for( int i = d; i < n; i++ ) { T t = a[ i ]; int j = i; for( ; j >= d && t < a[ j - d ]; j -= d ) a[ j ] = a[ j - d ]; a[ j ] = t; } } Notes: Initially, with the first run of the for loop, you start off by comparing only 2 elements. Say d = 13; then you compare a[ 13 ] with a[ 0 ], a[ 14 ] with a[ 1 ] etc... until you get to 2d (26 in this example) where you now have 3 elements (indexes 0, 13, and 26 ) - the third of which must be correctly inserted. More notes: You need to choose a formula that ensures that d is eventually equal to 1! Running time: The average running time is unknown, but in the worst case, its O(n 2 ). Sample sorting trace: Data: Gap = 5 8 3 7 7 2 2 2 2 2 2 6 6 6 1 1 1 1 1 1 3 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 5 3 8 8 8 2 7 7 1

Gap = 3

3 Gap = 1 1

8 8 8 6

7 7 7

6 8

5. Merge sort
How it works: There are 2 functions: sort and merge. The sort() function doesnt actually do any sorting, but it calls merge() which merges two sorted arrays into sorted order. Initially it merges two one-element sub-arrays (which can be considered sorted!) into a two-element sorted array, and these sub-arrays get larger and larger until the original array you started with gets its sorted bottom and top halves merged. This is all done recursively, i.e. When you call sort() on the original array, the function is called again and again, on sub-arrays half the size every time, until the size = 1, which is when the recursive call starts unwinding and the sorting begins. Note: since the first recursive call is on the bottom half of the original array, the lower elements get entirely sorted first, and only after that the upper ones.

http://wikistudent.ws/Unisa

Version 1: template<class T> void merge( T* a, int n1, int n2 ) { T* temp = new T[n1 + n2]; // the array to store the sorted elements int i = 0, j1 = 0, j2 = 0; // the indexes of the sub-arrays while( j1 < n1 && j2 < n2 ) // while there are elements left in 1 array... temp[ i++ ] = ( a[ j1 ] <= a[ n1 + j2 ] ? a[ j1++ ] : a[ n1 + j2++ ] ); // the above line inserts the smaller value of the 2 sub-arrays into // the next position in the big temporary array. // the appropriate indexes are advanced after the insertion while ( j1 < n1 ) // this executes if there's still more in the first array temp[ i++ ] = a[ j1++ ]; while( j2 < n2 ) // this executes if there's still more in the second array temp[ i++ ] = ( a + n1 )[ j2++ ]; for( i = 0; i < n1 + n2; i++ ) a[ i ] = temp[ i ]; // transfer the elements into the original array delete [] temp; // delete the dynamic array } template<class T> void sort( T* a, int n ) { if( n > 1 ) { int n1 = n / 2; int n2 = n - n1; sort( a, n1 ); sort( a + n1, n2 ); merge( a, n1, n2 ); } }

// sort bottom half // sort top half // merge bottom and top

Version 2: template<class T> void merge( T* a, T* temp, int leftPos, int rightPos, int rightEnd ) { int leftEnd = rightPos - 1; int tempPos = leftPos; int numElements = rightEnd - leftPos + 1; while( leftPos <= leftEnd && rightPos <= rightEnd ) if( a[ leftPos ] <= a[ rightPos ] ) temp[ tempPos++ ] = a[ leftPos++ ]; else temp[ tempPos++ ] = a[ rightPos++ ]; while( leftPos <= leftEnd ) temp[ tempPos++ ] = a[ leftPos++ ]; while( rightPos <= rightEnd ) temp[ tempPos++ ] = a[ rightPos++ ]; for( int i = 0; i < numElements; i++, rightEnd-- ) a[ rightEnd ] = temp[ rightEnd ]; }

http://wikistudent.ws/Unisa

template<class T> void sort( T* a, T* temp, int left, int right ) { if( left < right ) { int mid = ( left + right ) / 2; sort( a, temp, left, mid ); sort( a, temp, mid + 1, right ); merge( a, temp, left, mid + 1, right ); } } Running time: The complexity is O(nlgn) because the merge sort works by repeatedly dividing the array in half until the pieces are singletons, and then it merges the pieces pair wise until a single piece remains. The number of iterations in the first part equals the number of times n can be halved. (lgn) The second part of the process reverses the first, and merging two pieces reverses halving them. So the second part also has lgn steps. The entire algorithm has 2lgn steps. Each step compares all n elements, so the total number of comparisons is 2nlgn, which is proportional to nlgn. Sample sorting trace, using version 1: Data: merge( merge( merge( merge( merge( merge( merge( 8 7 ) 5 ) ) ) 1 2 3 4 6 7 8 6 5 7 5 6 8 3 1 5 4 2 6 1 3 7 2 4 8 4 3 2 1

v, 1, 1 v+2, 1, v, 2, 2 v+4, 1, v+6, 1, v+4, 2, v, 4, 4

) 1 ) 1 1 2 )

6. Quick sort
How it works: A recursive function partitions an array into two, around a pivot, placing all elements smaller than the pivot on the left, and all elements larger than the pivot on the right. After each call, the pivot element is in the correct position. template<class T> void quicksort( T* a, int low, int high ) { if( low >= high ) return; T pivot = a[ high ]; int i = low - 1; int j = high; while( i < j ) { while ( a[ ++i ] < pivot ) ; // shift up i while( j >= 0 && a[ --j ] > pivot ) ; // shift down j if( i < j ) swap( a[ i ], a[ j ] ); }

http://wikistudent.ws/Unisa swap( a[ i ], a[ high ] ); quicksort( a, low, i - 1 ); quicksort( a, i + 1, high ); } Version 2: template<class T> int pivot( T* a, int start, int stop, int position ) { swap( a[ start ], a[ position ] ); int low = start + 1; int high = stop + 1; while( low < high ) if( a[ low ] < a[ start ] ) low++; else if( a[ --high ] < a[ start ] ) swap( a[ low ], a[ high ] ); swap( a[ start ], a[ --low ] ); return low; } template<class T> void quicksort( T* a, int low, int high ) { if( low >= high ) return; int pivotIndex = ( low + high ) / 2; pivotIndex = pivot( a, low, high, pivotIndex ); if( low < pivotIndex ) quicksort( a, low, pivotIndex ); if( pivotIndex < high ) quicksort( a, pivotIndex + 1, high ); } Version 3: template<class T> void quicksort( T* a, int low, int high ) { if( low + CUTOFF > high ) insertionsort( a, high - low ); else { int mid = ( low + high ) / 2; if( a[ mid ] < a[ low ] ) swap( a[ low ], a[ mid ] ); // now low < mid (median = ?) if( a[ high ] < a[ low ] ) swap( a[ low ], a[ high ] ); // now high > low (median = mid / high) if( a[ high ] < a[ mid ] ) swap( a[ mid ], a[ high ] ); // now mid = mid T pivot = a[ mid ]; swap( a[ mid ], a[ high - 1 ] ); // a[ high ] is already sorted int i, j; for( i = low, j = high - 1; ; ) { while( a[ ++i ] < pivot ) { } while( pivot < a[ --j ] ) { } // now i is at a position of a high element, and j at a low one if( i < j ) // if i & j haven't crossed yet... swap( a[ i ], a[ j ] ); else

http://wikistudent.ws/Unisa break; } swap( a[ i ], a[ high - 1 ] ); quicksort( a, low, i - 1 ); quicksort( a, i + 1, high ); } } Running time: It has O(nlgn) complexity, but in most cases is quicker than the merge sort. Each time round, the array is halved (lgn) and since each step compares all n elements, the entire process makes nlgn comparisons. If you use the highest (right-most) element as a pivot, the quick sort will degrade into an O(n 2 ) algorithm if the array is initially sorted (in either order). This is because the pivot element will always be an extreme value within its subarray.

// move the pivot back in place

7. Mean sort
This is a version of quick sort int mean( const vector < int > & a, int low, int high ) { int sum = 0; for( int i = low; i <= high; i++ ) sum += a[ i ]; return sum / ( high - low + 1 ); } void meansort( vector< int > & a, int low, int high ) { if( high > low ) { int pivot = mean( a, low, high ); int i, j; for( i = low-1, j = high+1; ; ) { while( a[ ++i ] < pivot ) {} while( pivot < a[ --j ] ) {} if( i < j ) swap( a[ i ], a[ j ] ); else break; } meansort( a, low, i-1 ); meansort( a, i, high ); } } void meansort( vector< int > & a ) { meansort( a, 0, a.size() - 1 ); }

8. Counting sort
How it works: This is similar to bucket sort (see below) in that a for loop counts the number of occurrences of each element in the input vector. (You now have a histogram of

10

http://wikistudent.ws/Unisa counts). When all the counts have been determined, the program loops through the counts vector, and overwrites the original data vector. This algorithm can only be used in a narrow set of circumstances. In particular, the elements must be integers, and must be drawn from a small range. The bucket sort algorithm is similar in structure to the integer counting sort algorithm, but removes some of these limitations. void countSort( vector< int > & data, int range ) { vector< int > counts( range, 0 ); vector< int >::iterator start = data.begin(); vector< int >::iterator stop = data.end(); // count the occurrences of each value: for( ; start != stop; ++start ) counts[ *start ]++; // now put values back into the vector: start = data.begin(); for( int i = 0; i < range; i++ ) for( int j = counts[ i ]; j > 0; j-- ) *start++ = i; // the inner loop adds element i (for each repetition of it) // to the next position in the vector // the outer loop is there to loop across all possible vector values } Running time: The running time depends on the number of data elements n, (data.size() ) and the largest value (range - 1). Loop1: The loop that copies values into the histogram is O(n), where n = the vector size. Loop2: The outer loop executes range times, and in the worst case (i.e. if all the numbers are the same) the inner loop could execute n times. You might think that the execution time is O(range * n) but this is not the case, because the assignment statement will be executed exactly n times. (Remember, youre just putting each of the n elements back in the vector). Thus, the total execution time is the maximum of n and range. This is often written as O(n + range). Sample sorting trace: Contents of the input vector, before the sort: data vector: 4 1 1 4 3 0 Contents of the counting vector, after the count: counts vector index: 0 1 2 3 4 counts vector values: 1 2 0 1 3 How i = i = i = i = i = the original vector gets its values back, in sorted order: 0 j = 1 0 1 j = 2 1 1 2 j = 0 3 j = 1 3 4 j = 3 4 4 4

9. Bucket sort

11

http://wikistudent.ws/Unisa How it works: The different elements in an array are separated into different buckets like this: A vector of size RANGE is declared, where RANGE is the set of different values that the array can contain. Each index value in this vector is a bucket; when you go through your array that needs to be sorted, you place each element in the appropriate bucket, i.e. if you find a number 5, you add 1 to the vector index [5] so that you know how many 5s are in the array. At the moment, your bucket vector contains different sums, which in total add up to the size of your input array. You now need to change this vector, so that each index value (increasing from 0 RANGE-1) indicates the position of that element in the output array. (Do this by incrementally adding up the buckets elements from left to right, and re-assigning them to the bucket as you go). The vector now has 0 in position [0] and the maximum array index number in position [RANGE-1] (if there is only one occurrence of the largest element!). After that, you need to make a temporary copy of the input array, which you go through, one (unsorted) element at a time, which you use to index the bucket to find its actual (sorted) position in the array. This array index position (that you found in the bucket) is then assigned the element in your temporary array. Note: For the duplicate elements not to overwrite the same array index position, you need to increment the buckets index! const int RANGE = 10; template<class T> void bucketsort( vector<T> & a ) { vector<int> offset( RANGE ); for( int j = 0; j < a.size(); j++ ) offset[ a[ j ] ]++; // Each index number of offset contains the number of occurrences of // that number in a int prevsum, sum = 0; for( int k = 0; k < RANGE; k++ ) { prevsum = sum; sum += offset[ k ]; offset[ k ] = prevsum; } // Each index number of offset contains the sum of occurrences of the // elements in a less than that number vector<int> b = a; for( int j = 0; j < a.size(); j++ ) { a[ offset[ b[ j ] ]++ ] = b[ j ]; } } Running time: O(N), because each of the N elements in the input array must be inspected for insertion into the bucket. Sample sorting trace: Contents of vector a: 6 6 0 4

Contents of offset after the first loop: Bucket: 0 1 2 3 4 5 6 Value: 1 0 0 0 3 0 2 Contents of offset after the second loop:

7 0

8 0

9 1

12

http://wikistudent.ws/Unisa Bucket: Value: 0 0 1 1 2 1 3 1 4 1 5 4 6 4 7 6 8 6 9 6

The final for loop: vector b: 6 6 j = 0 j = 1 j = 2 0 j = 3 4 j = 4 j = 5 j = 6

4 6

9 6

4 9 4

10. Radix sort

How it works: This is like the bucket sort, but the values in the input array can have any positive value, instead of being in a limited range. The numbers are sorted according to their nd right-most digits first, and then their 2 -last digits, etc always preserving the relative sorted order as you move left. Version 1: const int RANGE = 10; // digits in base 10 range from 0 -> 9 inclusive const int MAXPOWER = 6; // the max number of digits the values can have void radixSort( vector< int > & { for( int i = 0; i < MAXPOWER; { int p = int( pow( 10.0, i ) vector< int > offset( RANGE a ) i++ ) ); // multiply p by 10 with each iteration ); // this is the bucket that counts digits

for( int j = 0; j < a.size(); j++ ) offset[ a[ j ] / p % 10 ]++; // fill up the buckets! int prevsum, sum = 0; for( int k = 0; k < RANGE; k++ ) { prevsum = sum; sum += offset[ k ]; offset[ k ] = prevsum; } // the bucket vector now contains indexes in increasing order // with each iteration of the outer for loop, vector b is updated to a // progressively more sorted a: vector< int > b = a; for( int j = 0; j < a.size(); j++ ) a[ offset[ b[ j ] / p % 10 ]++ ] = b[ j ]; } } Version 2: void radixSort( vector<int> & values ) {

13

http://wikistudent.ws/Unisa bool flag = true; int divisor = 1; vector<int>::iterator start, stop; list<int>::iterator lstart, lend; while( flag ) { vector< list<int> > buckets( 10 ); flag = false; start = values.begin(); stop = values.end(); while( start != stop ) { int hashIndex = ( *start / divisor ) % 10; if( ( *start / divisor ) > 0 ) flag = true; buckets[ hashIndex].push_back( *start++ ); } start = values.begin(); for( int i = 0; i < 10; i++ ) { lstart = buckets[ i ].begin(); lend = buckets[ i ].end(); while( lstart != lend ) *start++ = *lstart++; } divisor *= 10; } } Running time: O(N) Sample sorting trace: In this example, let RANGE = 4 and MAXPOWER = 3 input vector a: i = 0 p = 1, a: offset: offset: a: i = 1 p = 10, a: offset: offset: a: i = 2 p = 100,a: offset: offset: a: 123 (123) 0 0 321 (321) 0 0 111 (111) 0 0 111 321 (321) 2 0 111 (111) 2 0 213 (213) 2 0 123 213 (213) 0 2 123 (123) 2 2 321 (321) 2 2 213 111 (111) 3 2 213 (213) 1 4 123 (123) 1 4 233 233 (233)

233 (233)

233 (233)

321

14

http://wikistudent.ws/Unisa

15