Divide and Conquer

Three Steps of The Divide and Conquer Approach

The most well known algorithm design strategy: 1. Divide the problem into two or more smaller subproblems.
2.

Conquer the subproblems by solving them recursively. Combine the solutions to the subproblems into the solutions for the original problem.
VANDANA M. LADWANI 2

3.

A Typical Divide and Conquer Case

a problem of size n

subproblem 1 of size n/2

subproblem 2 of size n/2

a solution to subproblem 1

a solution to subproblem 2

a solution to theVANDANAproblem original M. LADWANI

An Example: Calculating a0 + a1 +
ALGORITHM RecursiveSum(A[0..n-1])

+ an-1

//Input: An array A[0..n-1] of orderable elements //Output: the summation of the array elements if n > 1 return ( RecursiveSum(A[0.. n/2 1]) + RecursiveSum(A[n/2 .. n 1]) )

k Efficiency: (for n = 2 )


A(n) = 2A(n/2) + 1, n >1  A(1) = 0;

VANDANA M. LADWANI 4

General Divide and Conquer recurrence The Master Theorem

the time spent on solving a subproblem of size n/b.

T(n) = aT(n/b) + f (n), where f (n) (nk) k 1. a < b T(n) (nk) k 2. a = b T(n) (nk lg n ) k 3. a > b T(n) (nlog b a) Where a 1,b>1,k the0 spent on dividing the problem time
into smaller ones and combining their solutions.
VANDANA M. LADWANI 5

Divide and Conquer Examples



Sorting: mergesort and quicksort Tree traversals Binary search Matrix multiplication-Strassen s algorithm

VANDANA M. LADWANI

Mergesort
Algorithm:
 

The base case: n = 1, the problem is naturally solved. The general case:


 

Divide: Divide array A[0..n-1] in two and make copies of each half in arrays B[0.. n/2 - 1] and C[0.. n/2 - 1] Conquer: Sort arrays B and C recursively using merge sort Combine: Merge sorted arrays B and C into array A as follows:
(Example: 1 3 7, 2 4 5 9 )  Repeat the following until no elements remain in one of the arrays:  compare the first elements in the remaining unprocessed portions of the arrays  copy the smaller of the two into A, while incrementing the index indicating the unprocessed portion of that array  Once one of the arrays is exhausted, copy the remaining unprocessed elements from the other array into A.
VANDANA M. LADWANI 7

The Mergesort Algorithm

ALGORITHM Mergesort(A[0..n-1])
//Sorts array A[0..n-1] by recursive mergesort //Input: An array A[0..n-1] of orderable elements //Output: Array A[0..n-1] sorted in nondecreasing order if n > 1 //divide copy A[0.. n/2 1] to B[0.. n/2 1] copy A[ n/2 .. n 1] to C[0.. n/2 1] //conquer Mergesort(B[0.. n/2 1) Mergesort(C[0.. n/2 1) //combine Merge(B, C, A)
VANDANA M. LADWANI 8

The Merge Algorithm

ALGORITHM Merge(B[0..p-1], C[0..q-1], A[0..p+q-1])
//Merges two sorted arrays into one sorted array //Input: Arrays B[0..p-1] and C[0..q-1] both sorted //Output: Sorted Array A[0..p+q-1] of the elements of B and C i 0; j 0; k 0 while i < p and j < q do //while both B and C are not exhausted if B[i] <= C[j] //put the smaller element into A A[k] B[i]; i i + 1 else A[k] C[j]; j j + 1 k k+1 if i = p //if list B is exhausted first copy C[j..q-1] to A[k..p+q-1] else //list C is exhausted first copy B[i..p-1] to A[k..p+q-1]
VANDANA M. LADWANI 9

Mergesort Examples
 

83297154 72164

Efficiency?
Worst case: if the smaller element comes from alternating arrays.

VANDANA M. LADWANI

10

The Divide, Conquer and Combine Steps in Quicksort



Divide: Partition array A[l..r] into 2 subarrays, A[l..s-1] and A[s+1..r] such that each element of the first array is A[s] and each element of the second array is A[s]. (computing the index of s is part of partition.)


Implication: A[s] will be in its final position in the sorted array.

Conquer: Sort the two subarrays A[l..s-1] and A[s+1..r] by recursive calls to quicksort Combine: No work is needed, because A[s] is already in its correct place after the partition is done, and the two subarrays have been sorted.

VANDANA M. LADWANI

11

Quicksort
 

Select a pivot w.r.t. whose value we are going to divide the sublist. (e.g., p = A[l]) Rearrange the list so that it starts with the pivot followed by a sublist (a sublist whose elements are all smaller than or equal to the pivot) and a sublist (a sublist whose elements are all greater than or equal to the pivot ) (See algorithm Partition in section 4.2) Exchange the pivot with the last element in the first sublist(i.e., sublist) the pivot is now in its final position Sort the two sublists recursively using quicksort.

p A[i]p
VANDANA M. LADWANI

A[i] p

12

The Quicksort Algorithm

ALGORITHM Quicksort(A[l..r]) //Sorts a subarray by quicksort //Input: A subarray A[l..r] of A[0..n-1],defined by its left and right indices l and r //Output: The subarray A[l..r] sorted in nondecreasing order if l < r s Partition (A[l..r]) // s is a split position Quicksort(A[l..s-1]) Quicksort(A[s+1..r]
VANDANA M. LADWANI 13

The Partition Algorithm

ALGORITHM Partition (A[l ..r]) //Partitions a subarray by using its first element as a pivot //Input: A subarray A[l..r] of A[0..n-1], defined by its left and right indices l and r (l < r) //Output: A partition of A[l..r], with the split position returned as this function s value P A[l] i l; j r + 1; Repeat repeat i i + 1 until A[i]>=p //left-right scan repeat j j 1 until A[j] <= p//right-left scan if (i < j) //need to continue with the scan swap(A[i], a[j]) //no need to scan until i >= j swap(A[l], A[j]) return j VANDANA M. LADWANI 14

Quicksort Example
15 22 13 27 12 10 20 25

VANDANA M. LADWANI

15

Efficiency of Quicksort
Based on whether the partitioning is balanced.  Best case: split in the middle ( n log n)


C(n) = 2C(n/2) +

(n)

//2 subproblems of size n/2 each

Worst case: sorted array!



( n2) ( n log n)

C(n) = C(n-1) + n+1 //2 subproblems of size 0 and n-1 respectively

Average case: random arrays

VANDANA M. LADWANI

16

Binary Search

an Iterative Algorithm

ALGORITHM BinarySearch(A[0..n-1], K)
//Implements nonrecursive binary search //Input: An array A[0..n-1] sorted in ascending order and // a search key K //Output: An index of the array s element that is equal to K // or 1 if there is no such element

l 0, r n 1 while l e r do m (l + r) / 2 if K = A[m] return m else if K < A[m] r m 1 else l m+1 return -1

//l and r crosses over can t find K. //the key is found //the key is on the left half of the array // the key is on the right half of the array
17

VANDANA M. LADWANI

Binary Search

a Recursive Algorithm

ALGORITHM BinarySearchRecur(A[0..n-1], l, r, K)
//base case 1: l and r cross over can t find K if l > r return 1 else m (l + r) / 2 //base case 2: K is found if K = A[m] return m //general case: divide the problem. else if K < A[m] //the key is on the left half of the array return BinarySearchRecur(A[0..n-1], l, m-1, K) else //the key is on the left half of the array return BinarySearchRecur(A[0..n-1], m+1, r, K)
VANDANA M. LADWANI 18

Binary Search -- Efficiency



What is the recurrence relation? C(n) = C(n / 2) + 2

Efficiency C(n) ( log n)

VANDANA M. LADWANI

19

Binary Tree Traversals



Definitions


A binary tree T is defined as a finite set of nodes that is either empty or consists of a root and two disjoint binary trees TL and TR called, respectively, the left and right subtree of the root.


Examples.

The height of a tree is defined as the length of the longest path from the root to a leaf.

Problem: find the height of a binary tree.

VANDANA M. LADWANI 20

Find the Height of a Binary Tree

ALGORITHM Height(T)
//Computes recursively the height of a binary tree //Input: A binary tree T //Output: The height of T if T = return 1 else return max{Height(TL), Height(TR)} + 1

Base case? Efficiency?

C(n) = n + x =2n + 1
VANDANA M. LADWANI 21

Exercises
The following algorithm seeks to compute the number of leaves in a binary tree. Is it correct? ALGORITHM LeafCounter(T)
//Computes recursively the number of leaves in a binary tree //Input: A binary tree T //Output: The number of leaves in T if T = return 0 else return LeafCounter(TL)+ LeafCounter(TR)
VANDANA M. LADWANI 22

Binary Tree Traversals preorder, inorder, and postorder traversal



Binary tee traversal: visit all nodes of a binary tree recursively. Write a recursive preorder binary tree traversal algorithm.
Algorithm Preorder(T) //Implement the preorder traversal of a binary tree //Input: Binary tree T (with labeled vertices) //Output: Node labels listed in preorder if T write label of T s root Preorder(TL) Preorder(TR)

How many recursive calls ?M. LADWANI VANDANA

23

Multiplication of Large Integers (1)



Multiplication of two n-digit integers. # of digit multiplications 23 * 14 (n = 2) 123 * 456(n = 3) Multiplication of 2 n-bit integers 4 9 n2 # of additions 3 8 n2-1

Multiplication of a m-digit integer and a n-digit integer ( where n > m) can be modeled as the multiplication of 2 n-digit integers (by padding n m 0s before the first digit of the m-digit integer)
VANDANA M. LADWANI

24

Multiplication of Large Integers (2)



Do we have to make n2 digit multiplications?

 

n = 2: we only need 3 digit multiplications. What about n > 2?



Assume n is an even number. (What about the situations where n is an odd number?) Recursively compute the product of two integers, with the integer size (# of digits) reduced by half each time.

Efficiency of the recursive algorithm



Recurrence Relation

VANDANA M. LADWANI

25

Strassen s Matrix Multiplication



Brute-force algorithm
c00 c10 c01 = c11 a10 a11 a00 a01 * b10 b11 a00 * b01 + a01 * b11 a10 * b01 + a11 * b11 b00 b01

a00 * b00 + a01 * b10 = a10 * b00 + a11 * b10

8 multiplications 4 additions

Efficiency class: 5 (n3)

VANDANA M. LADWANI

26

Strassen s Matrix Multiplication



Strassen s Algorithm (two 2x2 matrices)

c00 c10 c01 = c11 a10 a11 a00 a01 * b10 b11 m3 + m5 m1 + m3 - m2 + m6 b00 b01

m1 + m4 - m5 + m7 = m2 + m4
      

m1 = m2 = m3 = m4 = m5 = m6 =

(a00 + a11) * (b00 + b11) (a10 + a11) * b00 a00 * (b01 - b11) a11 * (b10 - b00) (a00 + a01) * b11 (a10 - a00) * (b00 + b01)

7 multiplications 18 additions

m7 = (a01 - a11) * (b10 + b11) VANDANA M. LADWANI

27

Strassen s Matrix Multiplication



Two nxn matrices, where n is a power of two (What about the situations where n is not a power of two?)
C00 C10 C01 = C11 A10 A11 A00 A01 * B10 B11 M3 + M5 M1 + M3 - M2 + M6 B00 B01

M1 + M4 - M5 + M7 = M2 + M4

VANDANA M. LADWANI

28

Submatrices:


M1 = (A00 + A11) * (B00 + B11) M2 = (A10 + A11) * B00 M3 = A00 * (B01 - B11) M4 = A11 * (B10 - B00) M5 = (A00 + A01) * B11 M6 = (A10 - A00) * (B00 + B01) M7 = (A01 - A11) * (B10 + B11)
VANDANA M. LADWANI 29

Efficiency of Strassen s Algorithm



If n is not a power of 2, matrices can be padded with rows and columns with zeros Number of multiplications Number of additions Other algorithms have improved this result, but are even more complex
VANDANA M. LADWANI 30