Lecture 24: From Dijkstra to Prim

Todays Topics: Dijkstras Shortest Path Algorithm Depth First Search Spanning Trees Minimum Spanning Trees Prims Algorithm

Covered in Chapter 9 in the textbook

R. Rao, CSE 373

Some slides based on: CSE 326 by S. Wolfman, 2000

Single Source, Shortest Path Problem

Given a graph G = (V, E) and a source vertex s in V, find the minimum cost paths from s to every vertex in V
2

A
9 3 8

B
1 1

C
3

Source
2

R. Rao, CSE 373

Dijkstras Shortest Path Algorithm

1. 2. 3.

Initialize the cost of each node to Initialize the cost of the source to 0
2

While there are unknown nodes left in the A 1 1 graph 9 1. Select the unknown node N with the 3 C lowest cost (greedy choice) 8 2 2. Mark N as known D 3 3. For each node A adjacent to N E If (Ns cost + cost of (N, A)) < As cost As cost = Ns cost + cost of (N, A) (Prev allows Prev[A] = N //store preceding node paths to be reconstructed) R. Rao, CSE 373 3

Dijkstras Algorithm (greed in action)

vertex A B C D E known No No Yes No No cost 0 Prev 2 vertex A B C D E known Yes Yes Yes Yes Yes cost 8 10 0 5 2 Prev D A E C

Initial
A
3 9 8 1

Final
B
1

C
3

D
R. Rao, CSE 373

E
4

Analysis of Dijkstras Algorithm

Main loop: While there are unknown nodes left in the graph |V| times 1. Select the unknown node N with the lowest cost O(|V|) 2. Mark N as known O(|E|) total 3. For each node A adjacent to N If (Ns cost + cost of (N, A)) < As cost As cost = Ns cost + cost of (N, A)

Total time = |V| (O(|V|)) +O(|E|) = O(|V|2 + |E|) Dense graph: |E| = (|V|2) ! Total time = O(|V|2) = O(|E|) Sparse graph: |E| = (|V|) ! Total time = O(|V|2) = O(|E|2) Quadratic! Can we do better?
R. Rao, CSE 373 5

Analysis of Dijkstras Algorithm

Yes! Use a priority queue to store vertices with key = cost |V| times: Select the unknown node N with the lowest cost deleteMin |E| times: As cost = Ns cost + cost of (N, A) decreaseKey 7 2 4 5 6

Total run time = ?

R. Rao, CSE 373 6

Analysis of Dijkstras Algorithm

Yes! Use a priority queue to store vertices with key = cost |V| times: Select the unknown node N with the lowest cost deleteMin |E| times: As cost = Ns cost + cost of (N, A) decreaseKey 7 2 4 5 6

Total run time = O(|V| log |V| + |E| log |V|)

R. Rao, CSE 373 7

Does Dijkstras Algorithm Always Work?

Dijkstras algorithm is an example of a greedy algorithm Greedy algorithms always make choices that currently seem the best
Short-sighted no consideration of long-term or global issues Locally optimal does not always mean globally optimal

In Dijkstras case choose the least cost node, but what if there is another path through other vertices that is cheaper? Can prove: Never happens if all edge weights are positive

R. Rao, CSE 373

The Cloudy Proof of Dijkstras Correctness

Least cost node Next shortest path from G inside the known cloud

THE KNOWN CLOUD

Source

If the path to G is the next shortest path, the path to P must be at least as long. Therefore, any path through P to G cannot be shorter!
R. Rao, CSE 373 9

Inside the Cloud (Proof)

Everything inside the cloud has the correct shortest path Proof is by induction on the # of nodes in the cloud:
Base case: Initial cloud is just the source with shortest path 0 Inductive hypothesis: cloud of k-1 nodes all have shortest paths Inductive step: choose the least cost node G ! has to be the shortest path to G (previous slide). Add kth node G to the cloud

R. Rao, CSE 373

10

Inside the Cloud (Proof)

Everything inside the cloud has the correct shortest path Proof is by induction on the # of nodes in the cloud:
Base case: Initial cloud is just the source with shortest path 0 Inductive hypothesis: cloud of k-1 nodes all have shortest paths Inductive step: choose the least cost node G ! has to be the shortest path to G (previous slide). Add kth node G to the cloud

But waitaminute!! What about negative weights??

Kevin Bacon
R. Rao, CSE 373

Gotcha!!
11

Negative Weights: Dijkstras Achilles Heel

2

A
3 9 -5

B
1

A
E

2 -5

B
1

10

C
-10

Dijkstra: C!D (cost = -5) Least cost path: C!E!D (cost = -8)
R. Rao, CSE 373

Negative cycles: Whats the shortest path from A to E? (or to B, C, or D, for that matter)
12

Depth First Search (DFS)

We used Breadth First Search for finding shortest paths in an unweighted graph
Use a queue to explore neighbors of source vertex, neighbors of each neighbor, and so on: 1 edge away, two edges away, etc.

Its counterpart: Depth First Search

A second way to explore all nodes in a graph

DFS searches down one path as deep as possible

When no new nodes available, it backtracks When backtracking, we explore side-paths that werent taken

DFS allows an easy recursive implementation

So, DFS uses a stack while BFS uses a queue

R. Rao, CSE 373

13

DFS Pseudocode

Pseudocode for DFS:

DFS(v) If v is unvisited mark v as visited print v (or process v) for each edge (v,w) DFS(w)

C D

DFS(C)
E

Works for directed or undirected graphs A Running time = O(|V| + |E|)

C DFS(C)

E
14

R. Rao, CSE 373

What about DFS on this graph?

What happens when you do DFS(142)?

322 321 326 370 341

143

142

378

Go as deep as possible, Then backtrack

R. Rao, CSE 373

421 401
15

We get a spanning tree

322 321 326 370 341

143

142

378 421 401

R. Rao, CSE 373

16

DFS and BFS may give different trees

B A B A D B E BFS(C) A D
R. Rao, CSE 373

DFS(C) C D E C

E
17

Spanning Tree Definition

Spanning tree: a subset of edges from a connected graph that:

touches all vertices in the graph (spans the graph) forms a tree (is connected and contains no cycles)
4 9 1 2 5 7

Minimum spanning tree: the spanning tree with the least total edge cost
18

R. Rao, CSE 373

Minimum Spanning Tree (MST)

We are given a weighted, undirected graph G = (V, E), with weight function w: E ! R mapping edges to real valued weights Problem: Find the minimum cost spanning tree
R. Rao, CSE 373

1 10 2 1 6 2 1 6 5
19

5 1 8 3 3 4

Why minimum spanning trees?

Lots of applications Minimize length of gas pipelines between cities Find cheapest way to wire a house (with minimum cable) Find a way to connect various routers on a network that minimizes total delay Etc

R. Rao, CSE 373

20

Prims Algorithm for Finding the MST

1. Starting from an empty tree, T, pick a vertex, v0, at random and initialize: V = {v0} and E = {} 2 1 6 2
R. Rao, CSE 373

v0
10 8 1 6

1 5 1 3 3 4

4 5
21

Prims Algorithm for Finding the MST

1. Starting from an empty tree, T, pick a vertex, v0, at random and initialize: V = {v0} and E = {} 2. Choose a vertex v not in V such that edge weight from v to a vertex in V is minimal (greedy again!) 1 6 2
R. Rao, CSE 373

v0
10 2 1 6

1 5

v
8 3

1 3 4

4 5
22

Prims Algorithm for Finding the MST

1. Starting from an empty tree, T, pick a vertex, v0, at random and initialize: V = {v0} and E = {} 2. Choose a vertex v not in V such that edge weight from v to a vertex in V is minimal (greedy again!) 3. Add v to V and the edge to E if no cycle is created

v0
10 2 1 6 2 1 6

1 5

v
8 3

1 3 4

4 5
23

R. Rao, CSE 373

Prims Algorithm for Finding the MST

1. Starting from an empty tree, T, pick a vertex, v0, at random and initialize: V = {v0} and E = {} 2. Choose a vertex v not in V such that edge weight from v to a vertex in V is minimal (greedy again!) 3. Add v to V and the edge to E if no cycle is created 4. Repeat until all vertices have been added
R. Rao, CSE 373

v0
10 2 1 6 2 1 6 8

1 5 1 3 3 4

4 5
24

Prims Algorithm for Finding the MST

1. Starting from an empty tree, T, pick a vertex, v0, at random and initialize: V = {v0} and E = {} 2. Choose a vertex v not in V such that edge weight from v to a vertex in V is minimal (greedy again!) 3. Add v to V and the edge to E if no cycle is created 4. Repeat until all vertices have been added
R. Rao, CSE 373

v0
10 2 1 6 2 1 6 8

1 5 1 3 3 4

4 5
25

Prims Algorithm for Finding the MST

1. Starting from an empty tree, T, pick a vertex, v0, at random and initialize: V = {v0} and E = {} 2. Choose a vertex v not in V such that edge weight from v to a vertex in V is minimal (greedy again!) 3. Add v to V and the edge to E if no cycle is created 4. Repeat until all vertices have been added
R. Rao, CSE 373

v0
10 2 1 6 2 1 6 8

1 5 1 3 3 4

4 5
26

Prims Algorithm for Finding the MST

1. Starting from an empty tree, T, pick a vertex, v0, at random and initialize: V = {v0} and E = {} 2. Choose a vertex v not in V such that edge weight from v to a vertex in V is minimal (greedy again!) 3. Add v to V and the edge to E if no cycle is created 4. Repeat until all vertices have been added
R. Rao, CSE 373

v0
10 2 1 6 2 1 6 8

1 5 1 3 3 4

4 5
27

Prims Algorithm for Finding the MST

1 Done! Total cost = 1 + 3 + 4 + 1 + 1 = 10 1 6
R. Rao, CSE 373

1 2 1 4 5
28

Next Class: Analysis of Prims Algorithm Kruskal takes a bow faster MST To Do: Programming Assignment #2 (Dont wait until the last few days!!!) Continue chapter 9
R. Rao, CSE 373 29