CSCE 310: Divide and Conquer Algorithms

August 31, 2016

Ryan Patrick

Outline
Divide and Conquer Algorithms
Binary Tree Traversal

Divide/Conquer
Patrick

Removing Node(s)

Divide and
Conquer

Multiplication
Closest Pair & Quickhull
Branch & Bound
Dynamic Programming

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

29

Outline
Divide and Conquer Algorithms
Binary Tree Traversal
Multiplication
Closest Pair & Quickhull
Branch & Bound
Dynamic Programming

Divide/Conquer
Patrick
3

Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

29

General Procedure
Divide/Conquer
Patrick
4

Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

1. Break Problem into Subproblems

Branch/Bound

2. Solve Subproblems

Fast Concurrent
Object Localization
and Recognition

3. Combine Subsolutions

Dynamic
Programming

29

Algorithms Weve Seen

Divide/Conquer
Patrick
5

Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Merge Sort

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Quicksort

Dynamic
Programming

29

Binary Tree Traversal

Divide/Conquer
Patrick
Divide and
Conquer
6

Enumerate All Nodes in Binary Tree

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Different Orders Are Useful

Dynamic
Programming

29

Preorder Traversal
Divide/Conquer
Patrick

Divide and
Conquer
7

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

1. Process the Node

2. Process the Nodes Left Child (Recursively)
3. Process the Nodes Right Child (Recursively)

29

Inorder Traversal
Divide/Conquer
Patrick

Divide and
Conquer
8

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

1. Process the Nodes Left Child (Recursively)

2. Process the Node
3. Process the Nodes Right Child (Recursively)

29

Postorder Traversal
Divide/Conquer
Patrick

Divide and
Conquer
9

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

1. Process the Nodes Left Child (Recursively)

2. Process the Nodes Right Child (Recursively)
3. Process the Node

29

Removing All Nodes

Divide/Conquer
Patrick

Divide and
Conquer
10

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

Simpler than removing a single node

Use postorder traversal and process a node by deleting it

29

Multiplication
Divide/Conquer
Patrick
Divide and
Conquer

11

Public-Key Encryption Relies on Multiplying Large Prime Numbers

(Hundreds of Digits)

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

29

Multiplication
Clicker Question: How many multiplications are needed to multiply two 300-digit long numbers?
Divide/Conquer
Patrick
Divide and
Conquer

12

A 3002 = 90, 000?

B 10300 2997 ?
C

9972

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

= 994, 009?

Dynamic
Programming

D Other?

29

Karatsubas Method
Two-Digit Integers a1 a0 and b1 b0

ab
(a1 + a0 ) (b1 + b0 )
a1 b0 + a0 b1
ab

= (a1 10 + a0 ) (b1 10 + b0 )
= a1 b1 102 + (a1 b0 + a0 b1 ) 10 + a0 b0
= a1 b1 + (a1 b0 + a0 b1 ) + a0 b0
= (a1 + a0 ) (b1 + b0 ) a1 b1 a0 b0
= a1 b1 102 + ((a1 + a0 ) (b1 + b0 ) a1 b1 a0 b0 ) 10 + a0 b0

Complexity of Multiplication
Divide/Conquer

Regular Method


T (n) =4T n2 +O (n)
n2

Patrick

Katsuba Method


T (n) =3T n2 +O (n)

n1.59

Divide and
Conquer

14

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
5

x 10

Fast Concurrent
Object Localization
and Recognition

10
1.59

n2

Dynamic
Programming

8
7
6
5
4
3
2
1
0
0

100

200

300

400

500

600

700

800

900

1000

29

Multiplication
Clicker Question: How many multiplications are needed to multiply two 300-digit long numbers?
Divide/Conquer
Patrick
Divide and
Conquer

A 3002 = 90, 000?

15

B 10300 2997 ?

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound

C 9972 = 994, 009?

Fast Concurrent
Object Localization
and Recognition

D Other?

Dynamic
Programming

3001.59 8, 682
< 10% of 3002

29

Closest Pair
Divide/Conquer
Patrick
Divide and
Conquer

16

Closest Pair of Points in Left Half

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound

Closest Pair of Points in Right Half

Fast Concurrent
Object Localization
and Recognition

Closest Pair of Points within min (dleft , dright ) of Dividing Line

Dynamic
Programming

29

Quickhull
Divide/Conquer
Patrick
Divide and
Conquer

17

Convex Hull of Top Half (Plus Dividing Line)

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound

Convex Hull of Bottom Half (Plus Dividing Line)

Fast Concurrent
Object Localization
and Recognition

Union of Top and Bottom Hulls

Dynamic
Programming

29

Outline
Divide and Conquer Algorithms
Binary Tree Traversal
Multiplication
Closest Pair & Quickhull
Branch & Bound
Dynamic Programming

Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull
18

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

29

Branch & Bound Algorithms

Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Divide & Conquer algorithms leave you with a subproblems, each of size b

19

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Which subproblem do you solve first?

Dynamic
Programming

29

Branch & Bound Algorithms

Prioritize Subproblems
Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Work on the subproblem that is most likely to contain the solution (or the
best solution)
Leave less promising subproblems for later

20

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

Ignore a subproblem if its best possible solution is worse than the current
best solution

Dynamic
Programming

Whats promising?
. . . That depends on the problem . . .

29

Object Localization & Recognition

The Problem
Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Identifying what object is in an image is important

Branch/Bound
21

Identifying where an object is also important

Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

29

Object Localization & Recognition

Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
22

Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

Figure : Images from [Hes10]

29

Object Localization & Recognition

Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

An object in an image can be defined by . . .

1.
2.
3.
4.
5.

The
The
The
The
The

x coordinate of
x coordinate of
y coordinate of
y coordinate of
objects class

its
its
its
its

left-most point
right-most point
top-most point
bottom-most point

Branch/Bound
23

Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

29

Object Localization & Recognition

Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
24

Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

Figure : Figure 1 from [YLD09]

29

Object Localization & Recognition

Divide/Conquer
Patrick
Divide and
Conquer

If none of the boxes that you still have to look at could be as good as your
current, best result, youre done

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
25

Remove the best result and rerun the algorithm to find the second, third,
. . . , nth best result

Fast Concurrent
Object Localization
and Recognition

Dynamic
Programming

29

Outline
Divide and Conquer Algorithms
Binary Tree Traversal
Multiplication
Closest Pair & Quickhull
Branch & Bound
Dynamic Programming

Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition
26

29

Dynamic
Programming

Dynamic Programming
Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Divide & Conquer algorithms leave you with a distinct subproblems, each
of size b

Branch/Bound
Fast Concurrent
Object Localization
and Recognition

What if there is some overlap in the problems?

Can we solve some before others?

27

Can we save those results for later?

29

Dynamic
Programming

External References I
Divide/Conquer
Patrick
Divide and
Conquer

[Hes10] Rob Hess.

An open-source siftlibrary.
In Proceedings of the international conference on Multimedia, MM 10,
pages 14931496, New York, NY, USA, 2010. ACM.
[YLD09] Tom Yeh, J.J. Lee, and T. Darrell.
Fast concurrent object localization and recognition.
In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE
Conference on, pages 280 287. IEEE, IEEE, 2009/06// 2009.

Binary Tree Traversal

Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition
28

29

Dynamic
Programming

Any Questions?
Divide/Conquer
Patrick
Divide and
Conquer
Binary Tree Traversal
Removing Node(s)
Multiplication
Closest Pair &
Quickhull

Branch/Bound
Fast Concurrent
Object Localization
and Recognition
29

29

Dynamic
Programming