An Illustrative Introducction To Algorithms

Copyright
© 2019 by Dino Cajic

All rights reserved. This book or any portion thereof may not be reproduced or
used in any manner whatsoever without the express written permission of the
copyright owner except for the use of brief quotations in a book review.
First Edition 2019

Contents
Chapter 0: Introduction
Chapter 1: Big O Complexity
Chapter 2: Lower Bounds
Chapter 3: Master Theorem
Chapter 4: Bubble Sort
Chapter 5: Insertion Sort
Chapter 6: Divide and Conquer
Chapter 7: Merge Sort
Chapter 8: Quick Sort
Chapter 9: Heap Data Structure
9.1 Creating a heap from an array
9.2 Constructing Min-Heap from a Tree
9.3 Constructing Min-Heap from an Array
9.4 Constructing Max-Heap from a Tree
9.5 Constructing Max-Heap from an Array
9.6 Deleting a Node from Min-Heap
Chapter 10: Heap Sort
Chapter 11: Selection Sort
Chapter 12: Adjacency Matrix
Chapter 13: Adjacency List
Chapter 14: Edge List
Chapter 15: Depth First Search (DFS)
Chapter 16: Topological Sorting
Chapter 17: Breadth First Search (BFS)
Chapter 18: Greedy Algorithms
Chapter 19: Minimum Spanning Trees (MST)
Chapter 20: Kruskal’s Algorithm
Chapter 21: Prim’s Algorithm
Chapter 22: Binary Tree Traversal – Depth First – In Order
Chapter 23: Binary Tree Insertion/Deletion
23.1 Inserting a Node
23.2 Deleting a Leaf in the Tree
23.3 Deleting a Node with One Child
23.4 Deleting a Node with Two Children
Chapter 24: Planar Graphs
Chapter 25: Dynamic Programming: Longest Common Subsequence
Chapter 26: All-Pairs Shortest Path Matrix Multiplication
Chapter 27: All-Pairs Shortest Path Matrix Multiplication (Floyd-Warshall)
Chapter 28: Dijkstra's Algorithm: Single Source Shortest Path
Chapter 29: Bellman-Ford
Chapter 30: Johnson’s Algorithm
Chapter 31: Clockwise and Counterclockwise Line Segment Intersection
Chapter 32: Graham’s Scan
Chapter 33: Closest Pair of Points on a Plane – Divide and Conquer
Chapter 34: Voronoi graph and Delaunay Triangulation
Chapter 35: Maximum Independent Set
Chapter 36: Minimum Vertex Cover
Chapter 37: Maximum Clique
References
Chapter 0: Introduction
Let me make an educated guess: you learn best when the information is
presented to you visually. Me too. This book is going to attempt to teach you a
rather complex subject without the use of any words. Okay, maybe some words.
Most illustrations will have an explanation attached to them. There are three
approaches to reading this book:
1. Read about the algorithm online and then see if you can follow the
traversal illustrated in this book.
2. View the algorithm in action by going through the chapter designated for
the algorithm and then go online and read further about it.
3. Just view the algorithm as is outlined in this book. A significant amount of
time was dedicated to make these algorithms easy to follow.
Not everything in this book is on algorithm traversal, however, everything is
related to algorithms. Chapter 1 illustrates the Big O complexity for the
algorithms we discuss in this book. It does so by showing the best (Ω), average
(Θ) and worst (O) time complexity. You may have encountered Big O notation,
but a good portion of people have not encountered Big Omega (Ω) or Big Theta
(Θ) notation. Chapters 2 and 3 are dedicated to introducing you to those topics.
A few other chapters are dedicated to data structures that you may need to know
before proceeding to the specific algorithm that utilizes those data structures.
The book is intended for first time learners as well as a reference guide for all
programmers. The book does not contain any code since it’s readily available
online. The goal of this book is to get you acquainted with the concept of each
algorithm. If you can visualize it, you can code it.
Chapter 1: Big O Complexity
The graph is self-explanatory. Try to stick away from the algorithms that have a
time complexity in red and attempt to stay with the ones in orange if you have no
other choice. Yellow and green is where you would like to optimally reside. With
some extra research, you’ll soon see that even O( n log(n) ) is good enough.
ALGORITHM BEST Ω AVERAGE Θ BIG O
Binary Tree Search
Ω( n ) Θ( n ) O( n )
In-Pre-Post- Order
Binary Search Ω( 1 ) Θ( log(n) ) O( log(n) )
Bubble Sort Ω( n ) Θ( n 2 ) O( n 2 )
Breadth First Search
Ω( |V| + |E| ) Θ( |V| + |E| ) O( |V| * |V| )
(BFS)
Depth First Search
(DFS) – Directed Ω( |V| + |E| ) Θ( |V| + |E| ) O( |V| + |E| )
Graph
Depth First Search
(DFS) – Undirected Ω( |V| + |E| ) Θ( |V| + |E| ) O( |V| + 2|E| )
Graph
Heap Sort Ω( n ) Θ( n log(n) ) O( n log(n) )
Insertion Sort Ω( n ) Θ( n 2 ) O( n 2 )
Kruskal’s Algorithm Ω( E log(E) ) Θ( E log(E) ) O( E log(E) )
Merge Sort Ω( n log(n) ) Θ( n log(n) ) O( n log(n) )
Prim’s Algorithm Ω( V log(V) ) Θ( E log(V) ) O( V 2 log(V) )
Quick Sort Ω( n log(n) ) Θ( n log(n) ) O( n 2 )
Selection Sort Ω( n 2 ) Θ( n 2 ) O( n 2 )
Chapter 2: Lower Bounds
What are the lower bounds of an algorithm? Instead of giving you a formal
definition, let’s look at weather. Most weather forecasters will provide you with
the high and the low; the accurate temperature is somewhere in the middle. The
high is your upper bound and the low is your lower bound. When relating this
concept to algorithms, certain problems have provable lower bounds meaning
that you will be incapable of providing a better answer than what was previously
proven. For example, the minimum number of comparisons that you would need
to sort 5 numbers is 7. Let’s look at a few additional examples.
Example 1
You have 81 bottles of water. A bottle of water can be heavier than the others,
lighter than the others, or equal in weight as the rest. What is the lower bound?
The formula to find the lower bounds of a problem is ceiling(logb n) , where b
is the number of possible outcomes and n is the number of possible answers.
Most times b is going to be either 2 or 3. In the example above, you have 3
possibilities: heavier, lighter or equal. In most cases, the second variable, n , is
more difficult to compute. In this example it’s straight forward: you have 81
bottles of water, so you have 81 possible answers. We’ll look at another example
where it’s not so straight forward later.
ceiling(log3 81) = 4
Example 2
You flip a coin 64 times. What is the lower bound?
In this example, b would equal 2 since during each flip we get either heads or
tails. The number of possible answers, n, is 128. Why 128? We flip the coin 64
times and during each flip we can get either heads or tails, so the possible
number of answers is 128.
Example 3
Find the minimum number of comparisons required to sort 5 numbers. The
difficulty increases here but it’s still not bad. Once you see this type of problem,
you apply the same logic. There is a total of 5! or 120 possible outcomes. A
binary sort tree will have 7 levels for the sorting procedure.
ceiling(log2 n!)
ceiling(log2 5!)
Chapter 3: Master Theorem
There are many times when you have to write a recurrence relation for a divide
and conquer problem. The example below shows the recurrence relation for a
binary search algorithm where you divide the problem by half and you
accomplish a constant amount of work at each level:
T(n) = T( ) + O(1)
The general formula for the master theorem states that T(n) = aT( )+ O(nc )
with T(0) = 0 and T(1) = Ө(1). There are three conditions to the master
theorem:
If c < logb a, then T(n) = Ө(nlog b a )

If c = logb a, then T(n) = Ө(nc logn)
If c > logb a, then T(n) = Ө(nc )
Pretty simple. Let’s look at a couple of examples.
Example 1
T(n) = T( ) + n
log10/7 1 < n, therefore T(n) = Ө(n)
Example 2
T(n) = 4T(n/3) + nlogn
log3 4 = 1.26185950... Since n grows faster than log(n), we get Ө(nlog 3 4 )
Example 3
T(n) = 16T( ) + 9999n2

Let’s look at this problem in more detail.
a = 16
b = 2
f(n) = 9999n2
f(n) = O(nc ), where c = 2
logb a = log2 16 = 4 > c
Therefore, T(n) = Ө(nlog b a ) = Ө(n4 )

Example 4
Sometimes the Master Theorem doesn’t work. We may just have to solve it
recursively. Let’s look at this example.
T(n) = T(n-2) + n2
= T(n-4) + (n-2)2 + n2
= Ө(1) + … + (n-4)2 + (n-2)2 + n2
= Ө(1) + 12 + 22 + 32 + … + n2
“1 2 + 2 2 + 3 2 + … = n”

= Ө(n•n2 )
= Ө(n3 )
Again, these chapters are not meant to be lengthy; they exist to simplify your
life. If you need more details on anything that was covered in this book, the
internet is your friend.
Chapter 4: Bubble Sort
The Bubble Sort algorithm sorts an array of items in increasing order. It steps
through the array and compares the adjacent elements. If they are not sorted, it
swaps them. It then keeps repeating the procedure until the array is fully sorted.
Let’s jump into an example.
The following elements are compared during the first iteration: 0 and 1, 1 and 2,
2 and 3, 3 and 4, and 4 and 5. In the first comparison of the first iteration,
elements 0 and 1 are compared. Since 3 is less than 7, the elements remain in
their current positions.
Next, elements at indices 1 and 2 are compared. Since 7 is larger than 1, the
elements are swapped.
Next, elements at indices 2 and 3 are compared. Since 7 is larger than 4, the
Moving to elements 3 and 4, the algorithm compares the values 7 and 6. Since 7
is larger than 6, the values are swapped.
Lastly, elements at indices 4 and 5 are compared. Since 7 is less than 5, the
This completes the first iteration. We know that the last element is fully sorted.
In the next iteration the last element will not be compared. So, in iteration two
the following comparisons will occur: 0 and 1, 1 and 2, 2 and 3, and 3 The
Bubble Sort algorithm begins the second iteration. Values at indices 0 and 1 are
compared. Since 3 is greater than 1, the elements are swapped.
Next, the algorithm compares the values 3 and 4. Since 3 is less than 4, the
values remain in their current positions.
Values at indices 2 and 3 are compared. Since 4 is less than 6, the values are not
swap.
The Bubble Sort algorithm compares the values at indices 3 and 4. Since 6 is
greater than 5, the elements are swapped.
By inspecting the array visually, we can quickly see that the array is sorted, but
the Bubble Sort algorithm keeps going since it doesn’t know that it’s finished. It
starts the third iteration. Since 1 is less than 3, the values are not swapped.
Next, 3 and 4 are compared. Since 3 is less than 4, the values are not swapped.
Finally, values 4 and 5 are compared. Since 4 is less than 5, the values remain in
their current positions. This finishes the third iteration. The Bubble Sort
algorithm guarantees that it has found the last 3 numbers in the sequence.
Bubble Sort starts the fourth iteration. Since 1 is less than 3, the values are not
swapped.
Next, the algorithm compares values at indices 1 and 2. Since 3 is less than 4,
the elements remain in their current positions. This ends the fourth iteration. The
algorithm guarantees that the last 4 digits have been sorted. One more iteration
to go.
The Bubble Sort algorithm starts working on the fifth iteration. It has only one
comparison: the comparison between values 1 and 3. Since 1 is less than 3, the
values remain in their current positions and the algorithm finishes the fifth
iteration.
There are no further comparisons to be performed. The Bubble Sort algorithm
has sorted the array in n-1 iterations, which in this case equals 5.
The following example was deliberately chosen to show that even though the
Bubble Sort algorithm completed the sort in 2 iterations, n-1 iterations were still
needed to finish the execution of the algorithm. It can be deduced that the
Bubble Sort algorithm is not efficient for large data sets; it always runs in O(n2 ).
Chapter 5: Insertion Sort
The insertion sort starts with the value at index 1 and compares it to the value at
index 0. If the value at index 1 is less than the value at index 0, the values are
swapped. The index is incremented, and the procedure is repeated. The best way
to see this is with an example. We’ll start off with the array 4, 7, 1, 2, 8, 5.
The algorithm starts at index position 1.
The first comparison is executed. The insertion sort algorithm starts at index 1
and compares it to the index before, which in this case is index 0.

Considering 7 is not less than 4, the values will remain at their current positions.
Since the algorithm reached index 0, it needs to increment the current position to
start the next comparison. It is now at index 2. The algorithm starts the
comparison between the value at index 2 and the value at index 1.
Since the value at index 2 is less than the value at index 1, the algorithm is going
to swap the values at those indexes. The index is decremented by 1, so it’s back
at index 1 now. It compares the value at index 1 with the value at index 0.

Since the value at index 1 is less than the value at index 0, it’s going to swap
those two values. The insertion sort algorithm has reached the end of the
comparison cycle. It increments the starting index value from 2 to 3.
The algorithm compares the value at index 3 with the value at index 2.

Since the value at index 3 was less than the value at index 2, it swaps the values.
It decrements the index from 3 to 2 and starts the comparison with index 1.
Since the value at index 2 is less than the value at index 1, the algorithm swaps
the two values. It decrements the index from 2 to 1 and compares that value with
the value at index 0.

Since the value at index 1 is not smaller than the value at index 0, the values stay
in their current spots. The starting index value is incremented to 4.
The algorithm starts its comparison between the values at index 4 and index 3.

Since the value at index 4 is not less than the value at index 3, no swap occurs.
It’s also worth mentioning that it is known that the values before index 3 are
smaller than the value at index 3; there is no reason to compare the value at
index 4 with those values since they’re guaranteed to be smaller than the value at
index 3 already. The starting index is incremented from 4 to 5.
The algorithm compares the value at index 5 with the value at index 4.
Since the value at index 5 is less than the value at index 4, the two values are
swapped. The index is decremented and the value at index 4 is compared to the
value at index 3.
Since the value at index 4 is less than the value at index 3, the two values are
swapped, and the index is decremented from 4 to 3. The value at index 3 is
compared to the value at index 2.
Since the value at index 3 is larger than the value at index 2, the values are not
swapped. The algorithm has reached the end and the array is sorted: 1, 2, 4, 5, 7,
8.
Chapter 6: Divide and Conquer

Divide-and-conquer is a strategy for solving certain computational problems.
The divide-and-conquer algorithm breaks a problem down recursively into two
or more sub-problems. They’re broken down until the sub-problem can be
solved. The solution of each sub-problem is combined to provide a solution to
the main problem.
The divide-and-conquer algorithm is important in solving large, complicated
problems that can be broken down recursively. It divides the large problem into a
series of subproblems. It conquers these subproblems by solving them
recursively. It then combines the answers to these subproblems to get the answer
to the main problem. A few of the more well-known divide-and-conquer
algorithms are:
Binary Search
Quick Sort
Merge Sort
The best way to understand the divide-and-conquer method is with an example.

Shown below is the divide-and-conquer approach to sorting a list. Although we
haven’t yet covered it, this example shows the merge-sort algorithm. It takes n
items and keeps dividing them into n/2 items until each item is by itself. Merge-
sort then merges the items and sorts them along the way. Merge sort will be
discussed in the next chapter.
Chapter 7: Merge Sort
The Merge Sort algorithm sorts an array of items in ascending order. Merge Sort
is a divide-and-conquer type of algorithm and was introduced briefly in chapter
6. The algorithm takes n elements and recursively keeps dividing them until it
reaches a state where there is only one element that it needs to focus on. Once
that state is reached, it starts to combine the elements and each time it combines
them, it sorts them along the way. Let’s dive into an example. We’ll start off with
an array containing 8 elements.
The Merge Sort algorithm starts recursively dividing the array into n/2 sub-
arrays. After the first division, the array is divided into 2 sub-arrays.
The algorithm continues to divide the sub-arrays. Both the left and right sub-
arrays are divided, and the algorithm generates four new sub-arrays.
There is one final division that must be done on all four sub-arrays to create
eight individual elements.
Now that the Merge Sort algorithm broke down the array into individual
elements, the elements can be recombined and sorted along the way. Elements 0
and 1 will be merged, then 2 and 3, 4 and 5, and 6 and 7. We’ll create 4
placeholders to help visualize this better.
First, elements 0 and 1 are compared. Since 35 is larger than 12, element 1 is
added first into the new sub-array followed by 35.
Next, the algorithm compares values at indices 2 and 3. Since 24 is less than 29,
24 is added to the new sub-array first, followed by 29.
Next, elements at indices 4 and 5 are compared. Since 1 is less than 13, 1 is
added first and then 13.
Finally, values at indices 6 and 7 are compared. Since 3 is less than 45, 3 is
added first to the subarray.
This completes the first merge. There are still two more to go.
During the second merge, the values are compared between adjacent arrays. To
start, 12 is compared with 24. There is no need to compare 12 to 35 since it’s
already sorted in that list.
Since 12 was added to the list first, the left array index is incremented. The value
at index 1 is compared to the value at index 2. Since 24 is less than 35, 24 is
added to the list next.
The index in the second array is incremented and values 35 and 29 are
compared. Since 29 is less than 35, 29 is added next. Since there are no other
comparisons that 35 can do, 35 is added to the list.
The Merge Sort algorithm moves to the next two sub-arrays to merge. The first
indices from each subarray are compared. Since 1 is less than 3, 1 is added to the
list next.
The index value in the left array is incremented and 13 is compared to 3. Since 3
is less than 13, 3 is added next to the list.
The index value of the right array is incremented and 13 is compared to 45.
Since 13 is less than 45, 13 is added first and then 45. This completes the
merging of four subarrays into 2 subarrays. One more merge to go.
The Merge Sort algorithm starts by comparing the first elements in each array. In
this example, the value at index 0 is compared to the value at index 4. Since 1 is
less than 12, 1 is added to the array first.
Since the value was added from the right array, the index value of the right array
is incremented. The next comparison is between 12 and 3. Since 3 is less than
12, 3 is added to the list next.
Since 12 is less than 13, 12 is added to the list next.
The index value of the left array is incremented and 24 is compared to 13. Since
13 is less than 24, 13 is added to the list next.
Since 24 is less than 45, 24 is added to the list next.
29 is less than 45, 29 is added to the list next.
35 is less than 45, 35 is added next followed by 45.
The two subarrays have been merged completely and the final array is fully
sorted.
Chapter 8: Quick Sort
Like most sorting algorithms, the Quick Sort algorithm sorts an array of items in
ascending order. There are numerous approaches to Quick Sort. This chapter will
focus on one method. This approach selects the left-most element of the array as
the pivot. It then compares it against the last element in the array. If the pivot is
less than the element it’s being compared to, that element becomes the pivot and
the left side is incremented. If the pivot is greater than the element it’s being
compared to, the index value of that element is incremented. This occurs until
the left and the right sides meet. Once the values meet, the element where they
met is said to be sorted. Left and right partitions are created, and new pivot
points are generated.
This version of the Quick Sort algorithm will follow these rules:
Pivot is on the left (the pivot is the selected element that the Quick
Sort algorithm compares all other elements to)
Pivot is compared with the right element (P < R)
If the right element is larger
The values of the right element and the pivot are
swapped
The right element becomes the new pivot
Left index is incremented
If the right element is smaller
Right index value is decremented
Pivot is on the right
Pivot is compared with the left element (P > L)
If the pivot value is larger
Increment the left index value
If the pivot value is smaller
The values of the left element and the pivot are
swapped
The left element becomes the new pivot
Right index is decremented
Pivot, left, and right all point to the same element
That element is now sorted
Create left and right partitions
Set new pivot, left, and right for both partitions starting
with the left
Repeat above procedure
This is only one way to do the Quick Sort algorithm. We’ll now move to an
example.
Index 0 is set as the pivot value. The left element is set as the pivot value and the
right element is set as the last element in the array. The pivot value is compared
to the right element. Since the right value is less than the pivot value, the right
value is swapped with the pivot value, and the right value becomes the new pivot
value; left index is incremented.
Next, the new pivot value is compared to the value at index 1. Since the pivot is
greater than the left value, the pivot remains in the current position and the left
element is incremented.
The Quick Sort algorithm compares pivot to left. Since the pivot value is not
greater than the left value, the pivot value is swapped with the left value. Index 2
becomes the new pivot and the right index is decremented.
Next, the new pivot at index 2 is compared the value at index 4. Since 5 is not
less than 3, the values are swapped, and the left index value is incremented.
The Quick Sort algorithm compares the pivot value to the left value. Since the
pivot value is larger than the left value, the left index is incremented.
Now, both the left and the right arrows point to the same element. The algorithm
knows that this value is sorted. Left and right partitions are created.
The algorithm starts working on the left partition. The value at index 0 becomes
the new pivot. The left arrow points to the pivot value and the right arrow points
to the last element in the left partition.
The pivot value is compared to the right value. Since the pivot is larger than the
right value, the two values are swapped. Index 3 becomes the new pivot and the
left index value is incremented.
The algorithm compares the pivot value to the value at index 1. Since the pivot
value is larger, the left index value is incremented.
The algorithm compares the pivot value to the value at index 2. Since the pivot
value is larger, the left index value is incremented.
The left and right arrows have met at the same element. The Quick Sort
algorithm now knows that the value at index 3 is sorted. New partitions are
created. Since there is no unsorted value to the right of the pivot, only the left
partition is created.
The algorithm starts working on the new left partition since it’s still within the
old left partition. It assigns index 0 as the pivot value and points the left arrow to
it. The right arrow points to the last element within the new left partition.
The algorithm starts by comparing the pivot value to the right value. Since the
pivot value is less than the right value, the right index is decremented.
Next, the values at indices 0 and 1 are compared. Since the pivot value is
smaller, the right value is decremented.
The left and right arrows have met at index 0. The Quick Sort algorithm knows
that the value at index 0 is now sorted. New partitions are created. Since there is
no value to the left of the pivot, only the right partition is created.
Since the algorithm is still within the left partition, it starts working on the new
right partition. The pivot and the left arrow both point to index 1; the right arrow
points to index 2.
The values at indices 1 and 2 are compared. Since the pivot value is less than the
right value, the right index is decremented.
The left and right arrows have met at index 1. The algorithm knows that the
value at index 1 is sorted. New partitions are created. Since there are no unsorted
elements to the left of index 1 within this partition, only the right partition is
created.
Since the Quick Sort is still within the first left partition, it starts working on the
new right partition. It points the pivot, left arrow, and right arrow to index 2
since that is the only element in this partition.
The arrows have met at index 2, so the algorithm knows that the value at index 2
is now sorted.
The Quick Sort algorithm has finally made its way out of the first left partition
and starts working on the right partition. It points the pivot, left arrow, and right
arrow to the only element available.
Since both arrows point to the same element, the algorithm concludes that the
value at index 5 is now sorted.
Since there are no other partitions to sort, the Quick Sort algorithm ends.
Chapter 9: Heap Data Structure
9.1 Creating a heap from an array
The heap denotes an ordered binary tree. A heap can be built from a one-
dimensional array. In this one-dimensional array:
n represents the index of the parent node

n = 1, 2, 3, …
2n represents the index of the left child
2n + 1 represents the index of the right child
If we have the following array, we can construct a heap from it following the
rules outlined above.
Since there are 6 elements, the heap will have 6 nodes. Although the indices of
the array start at 0, when numbering the elements in the heap, the first element
starts at 1. We’ll follow the formula to construct the heap. The root node is
located at n = 1, which in this case is 7.
The children of 7 are located at 2n and 2n+1. Since the root node is at n = 1, the
left child is located at 2•1 = 2, and the right child is located at (2•1) + 1 = 3. The
left child is 2 and the right child is 9.
The heap is built from left to right. The next elements to be added are the left
and right children of 2. The node with the value of 2 is located at n = 2. The left
child is located at 2•2 = 4, and the right child is located at (2•2) + 1 = 5. The left
child is 4 and the right child is 5.
The last element that needs to be added is under number 9; this will be the left
child of number 9. Number 9 is located at node 3. The left child of node 3 is
located at 2n = 2•3 = 6. The value at n = 6 is 3. The heap is constructed from the
array.
9.2 Constructing Min-Heap from a Tree
We’ll look at two types of heaps: max-heap and min-heap. The root node (top-
most node) of a min-heap contains the smallest value. Each child is larger than
the parent. Once the heap was built in the previous example, a min-heap can be
constructed from it. To build a min-heap, we’ll start from floor(n/2) node. The
reason for the floor is due to uneven number of nodes. If the number of nodes
was 7, the starting position would be at floor(7/2) = floor(3.5) = 3.
For our example, there are 6 nodes:
floor(6/2) = floor(3) = 3
The starting position will be at the third node. We’ll check to see if there is a
child node that has a smaller value than 9. Since there is, the two nodes are
swapped.
We’re done with the 3rd node and we move to the node before, node 2. At node
2, there are two children. A comparison is made between the two children and
the smallest node is selected, in this case node 4. Next, 2 is compared to 4. Since
2 is already smaller than 4, the nodes are left in place.
We’re done with the 2nd node and we move to the node before, node 1. At node
1, there are two children. A comparison is made between the two children and
the smallest node is selected, in this case the node with the value of 2.
Next, 7 is compared to 2. Since 2 is smaller than 7, the nodes are swapped.

Now that we have reached the root node, we must iterate through the heap once
more to see if we finished generating a min-heap. So, we’ll repeat the procedure
again. We begin at the third node.
Since 3 is already smaller than 9, we can move to the 2nd node. A comparison is
made between the two children. Since 4 is smaller, 7 is compared to 4.
Since 4 is smaller than 7, the two nodes are swapped. The comparison node
moves from the second node to the first node.
A comparison is made between the two children. Since 3 is smaller, 2 is

compared to 3. The first node is already smaller, so the nodes remain in their
current positions.
One final iteration needs to occur to verify that the min-heap is created. We start
back at the third node. Since the third node doesn’t have any children that are
smaller than itself, we move to the second node. Similarly, the second node
doesn’t have any children that are smaller than it. We finally move back to the
first node and verify that the first node also doesn’t have any children that are
smaller than it. The min-heap tree has been created.
9.3 Constructing Min-Heap from an Array
We looked at the construction of the min-heap tree from an already constructed
tree. Min-heap can also be constructed directly from an array. If we look at the
array that we used in section 9.1, we’ll see how easy it is to construct a min-heap
from the array.
We start by adding the first node, 7.
Heaps are built from left to right. The next element that’s added is 2.
Since this is a min-heap, all the children nodes must be smaller than the parent
node. Number 2 is smaller than 7, so the two nodes are swapped.
Next, element 9 is added. Since 9 is larger than 2, no swapping is necessary.

Next, element 4 is added.
Since 4 is smaller than 7, the two nodes are swapped.

Element 4 is then checked with element 2 to make sure it’s not smaller. Since it’s
not, the elements remain in their current positions. Next, element 5 is added.
Since element 4 is already smaller than element 5, the nodes remain in their
current positions.
Finally, element 3 is added.

Since element 3 is smaller than element 9, the two nodes are swapped.
Element 3 is also compared to element 2. Since element 3 is larger than element

2, the two nodes remain in their current positions.
There are no additional array elements. This completes the construction of the
min-heap tree.
9.4 Constructing Max-Heap from a Tree
The root node of a max-heap contains the largest value. Each child is smaller
than the parent. We’ll start with the same array that we had in the previous few
sections.
We’ll first construct the heap.
Next, we’ll start constructing the max-heap. Just like when we were constructing
the min-heap, we’ll start with the 3rd node.
floor(6/2) = floor(3) = 3
Since 9 has only one child node, 9 will be compared to 3. If the parent is less
than the child, the nodes will be swapped. Since 9 is larger than 3, the nodes are
not swapped. We move to the 2nd node.
The 2nd node has two children. The larger of the two is 5, so 5 is compared with
2. Since 5 is larger than 2, the two values are swapped.
The comparison node is decremented, and we start the comparison with the 1st
node.
The 1st node has two children. The children are compared. Since 9 is larger than
5, 9 is compared with 7. Number 9 is larger than 7, so the two nodes are
swapped.
We’ll iterate through it once more to make sure that Max-Heap has been
achieved. We’ll start with the 3rd node. Since the parent, 7, is larger than its child,
the nodes are kept in place.
The comparison node is decremented and the 2nd node is compared with its
children. Since the parent is larger than both of its children, the nodes remain in
The comparison node is decremented and the 1st node is compared with its
children. Since the parent is larger than both of its children, the nodes remain in
Since there are no changes in the tree, we can conclude that the max-heap tree
has been created.
9.5 Constructing Max-Heap from an Array

Let’s use the same array that we used to construct the min-heap.
We start by adding the first node, 7.
We move top-to-bottom, left-to-right and we add the 2nd node, 2.
Since 7 is larger than 2, the nodes remain in their current position. Next, 9 is
added to the heap.
Since 9 is larger than 7, the two nodes are swapped.
Next, 4 is added as a child of 2.

Since 4 is larger than 2, the two nodes are swapped.
We observe that 4 moved up, so 4 is also compared to 9. Considering that 4 is

smaller than 9, they’re kept in place. The next value to be added is 5.
Since 5 is larger than 4, the two values are swapped.

Since 5 moved up, 5 is compared to 9. Seeing that 5 is smaller than 9, we’ll keep
the two values in place. Finally, 3 is added.
Considering that 3 is smaller than 7, the two values remain in their positions.
This completes the construction of the Max-Heap from an array.
9.6 Deleting a Node from Min-Heap
We’ll begin by stating that inserting a node into the min-heap has been outlined
in sections 9.2 and 9.3. As a quick recap, you add it as the last element in the
tree. You compare it with its parent, and if its larger, you swap the parent and the
child. You keep comparing it with each parent going up the tree until you reach a
point where either the parent is smaller than the node you’re inserting, or the
node has reached the root.
We’ll start by deleting the root node. The reason we’re deleting the root node is
due to the underlying concept of the heap: the priority queue. The element with
the highest priority is removed from the priority queue.
When deleting the root node, the last node becomes the new root node.
The root element might not be in the correct spot on the min-heap. Since 48 is
larger than its children, it needs to trickle down into the correct position. We’ll
follow the same approach as we did in Section 9.2.
A comparison is made of the two children, 4 and 7, and the smallest value is
chosen. Since 4 is smaller than 48, they swap positions.
Next, 5 and 8 are compared to obtain the smallest child of 48 again. Considering
5 is the smallest child, 48 is compared to 5. Node 5 is smaller than 48, so the two
nodes are swapped.
Finally, 11 and 6 are compared. Node 6 is the smallest child, so it’s compared to
48. Node 48 is larger; therefore, the two nodes are swapped. Node 48 has no
more comparisons to make. Min-Heap is achieved again after the deletion of the
root node.
Chapter 10: Heap Sort

Now that we know how to create a max-heap, we’ll move into sorting an array
of values using heap sort. To sort the array using heap-sort, we’ll keep creating
max-heaps. Each time a max-heap is created, the root is removed and is
considered sorted. Let’s start with the following example.
The first step is to construct a tree from the array.

We’ll follow the same procedure as outlined in section 9.4. To begin, 5 is
swapped with 9 and then 5 is swapped with 7 to generate a max-heap.
Considering that the tree is just a visual representation of the array, we’ll show
you what the array looks like after the initial max-heap sort.
The root node is swapped with the last item in the array. The last item is now
sorted and is removed from the tree.

A new heap is created. The root node is swapped with the largest of its children,
which is 7, and a new max-heap is constructed.
The root node is swapped with the last node in the tree and is removed from the
tree. We now have two nodes that are sorted: 7 and 9.

We create the new max-heap by swapping 1 with the largest of its children,
which is 5.
Since the new max-heap has been created, the root node is swapped with the last
element in the tree, which is 2. Number 5 is now sorted.

The process continues by swapping 2 with the largest of its children, 3,
achieving max-heap.
The root node is swapped with the last element effectively sorting element 3.

The procedure is done one more time. The root node is swapped with the largest
of its children, 2, creating a max-heap.
The root node is swapped with the last node sorting element 2.

Since there’s only one element left in the array, that element is marked as sorted
and so is the array.
This completes the heap sort process.

Chapter 11: Selection Sort
The Selection Sort algorithm sorts an array by looking for the smallest item and
moving it to the front of the list. That’s really all you have to know. Let’s jump
into an example.
Selection Sort starts by setting the first value as the minimum value.
It then starts comparing it with each value in the array. If a lower value is found,
the minimum value is updated, and the traversal continues until the end of the
array is reached. During the first comparison, the value at index 1 is compared to
the minimum value.
Since the value is smaller than the minimum value, the minimum value is
updated, and the index is incremented. Next, 2 is compared to the minimum
value. Since 2 is not less than the minimum value, the minimum value stays the
same, and the index is incremented.
Next, 7 is compared to 1. Number 1 is still smaller.
Selection Sort compares 1 and 4. Since 1 is smaller, the minimum value remains.
The algorithm has reached the end of the array for the first traversal. The first
element is swapped with the element that contains the minimum value. In this
case, 5 is swapped with 1. The first element is marked as sorted.
The algorithm starts the second traversal from the element at index 1. The value
at element 1 is set as the minimum value.
The first comparison of the second iteration is between 5 and 2. Since 2 is

smaller than the minimum value, the minimum value is updated to 2.
Next, 2 is compared to 7. Since 7 is larger than 2, the minimum value stays the
same.
The index is incremented and 4 is compared to 2. Since 2 is smaller, the
minimum value stays the same.
The algorithm has reached the end of the array for the second traversal. The
second element is swapped with the element that contains the minimum value. In
this case, 5 is swapped with 2. The second element is marked as sorted.
The algorithm starts with the third iteration. It sets the minimum value as the
first unsorted element in the array. The value at element 2 is set as the minimum
value.
The Selection Sort algorithm starts doing the third traversal. The first
comparison is between 5 and 7. Since 5 is smaller than 7, the algorithm keeps 5
as the minimum value.
The index is incremented and 4 is compared to the minimum value. Since 4 is

smaller than the minimum value, the minimum value is updated to 4.
The algorithm has finished the third traversal. The value at index 2 is swapped
with the minimum value. The first three elements have been marked as sorted.
The fourth iteration starts by setting the element at index 3 as the minimum
value.
A comparison is made between 7 and 5. Since 5 is less than 7, the minimum

value is updated to 5.
The Selection Sort algorithm has reached the end of the array. It swaps the
element at index 3 with the minimum value. The first four elements are sorted.
In the next traversal, there is only one element left. Since there’s only one
element left, there is no need to compare it to anything else. The Selection Sort
algorithm marks the last element as sorted and completes the sorting procedure.
Chapter 12: Adjacency Matrix
The adjacency matrix is a square matrix that’s used to represent a graph. The
elements that are next to each other represent adjacent vertices. Why would you
want to create an adjacency matrix? To save time. The adjacency matrix is much
more efficient when trying to figure out the adjacent nodes in a graph. Let’s look
at an example of how someone would create an adjacency matrix from a directed
graph.
Although the graph looks complicated, creating an adjacency matrix is a simple

process. Since there are 6 vertices, the adjacency matrix will have 6 rows and 6
columns. Each entry is initially filled with zeros.
Starting at vertex 1, there is one outbound edge to vertex 6. So, the first entry in
the adjacency matrix is at row 1, column 6. We change the 0 to 1 to indicate that
there is an edge from vertex 1 to vertex 6.
Next, we look at vertex 2. Vertex 2 has two outbound edges: one outbound edge
to 1, and another outbound edge to 5. We’ll update the adjacency matrix by
adding a one for entry (2,1) and entry (2,5).
Vertex 3 has two outbound edges: one outbound edge leads to vertex 2, and the
other outbound edge goes to vertex 5. We update the matrix entries (3,2) and
(3,5).
Vertex 4 has one outbound edge to vertex 6, so we update entry (4,6).
We move to vertex 5, which has 4 outbound edges. Entries (5,1), (5,3), (5,4), and
(5,6) are updated.
Finally, we examine vertex 6. Vertex 6 has 3 outbound edges.
Entries (6,1), (6,2), and (6,5) are updated in the adjacency matrix. This
completes the creation of the adjacency matrix.
The adjacency matrix is a tool that you can use if you’re trying to figure out
whether the graph is symmetric. If we look at each entry, Axy , and compare it to
Ayx , we can conclude that this graph is not symmetric. A symmetric graph
would have Axy = Ayx for each entry. For example, entry (2,1) has a value of 1,
while entry (1,2) has a value of 0.
If you look at the diagonal, you can quickly tell that this graph has no self-loops.
If there were any self-loops (a vertex with an edge pointing to itself), the
diagonal entry for that vertex would show a 1.
To find the adjacent vertices for some vertex, you just have to scan that
particular row and see if there are any non-zero entries. For vertex 5 the adjacent
vertices are 1, 3, 4, and 6.
Chapter 13: Adjacency List

The adjacency list is another way to represent adjacent vertices. Why would you
want to create an adjacency list? Again, to save time. The adjacency list is much
more efficient when trying to figure out the adjacent nodes in a graph.
Adjacency list also takes up less space than the adjacency matrix. In the
adjacency matrix, each node must contain a value, regardless of whether it
contains an edge to another node or not. The adjacency list only contains nodes
that are connected. Let’s look at an example of how someone would create an
adjacency list from a directed graph.
We’ll start by creating 6 elements in the array to represent the 6 nodes. Each
array element will store a linked list.
Going through the graph, vertex 1 is connected to vertex 6. So, we update the
adjacency list for vertex 1. The loop at the end indicates that the there are no
additional nodes that vertex 1 points to. So, it points to NIL.
Vertex 2 points to two other vertices: vertex 1 and vertex 5. We update the list to
have vertex 2 point to 1 and then to 5.
Vertex 3 points to 2 and 5.

Vertex 4 points to vertex 6.
Vertex 5 points to vertices 1, 3, 4, and 6.

Finally, vertex 6 points to vertices 1, 2, and 5.
By observation, we can quickly see that a significant amount of space is saved

by switching to an adjacency list from an adjacency matrix. The time complexity
stays roughly the same.
Chapter 14: Edge List

The last type of graph representation that we’ll discuss is the edge list. The edge
list is another way to represent adjacent vertices. Why would you want to create
an edge list? Again, to save time. The edge list is much more efficient when
trying to figure out the adjacent nodes in a graph. Let’s look at an example of
how someone would create an edge list from a directed graph. We’ll use the
same directed graph as we’ve done in the past 2 chapters.
The edge list contains a list of edges in alphanumerical order. We start by filling
in all the outbound edges from vertex 1. Vertex 1 has one outbound edge to
vertex 6, so that edge is stored.
Edge List
1-6
Moving through the graph in alphanumerical order, the next vertex that will be
considered is vertex 2. Vertex 2 has two outbound edges: to vertex 1 and to
vertex 5. Both of those edges will be recorded into the edge list table.
Edge List
1-6
2-1
2-5
Vertex 3 has two outbound edges: to vertex 2 and vertex 5.

Edge List
1-6
2-1
2-5
3-2
3-5
Vertex 4 has one outbound edge to vertex 6.

Edge List
1-6
2-1
2-5
3-2
3-5
4-6
Vertex 5 has 4 outbound edges:
Edge 5-1
Edge 5-3
Edge 5-4
Edge 5-6
All of the edges are stored into the edge list.
Edge List
1-6
2-1
2-5
3-2
3-5
4-6
5-1
5-3
5-4
5-6
Finally, vertex 6 has 3 outbound edges to vertices 1, 2, and 5. They’re added to
the edge list.
Edge List
1-6
2-1
2-5
3-2
3-5
4-6
5-1
5-3
5-4
5-6
6-1
6-2
6-5
The edge list is complete and contains all the directed edges. If the graph was
weighted, an additional column can be created next to each edge to contain the
weight of that edge. i.e. (1-6 | 12)
Chapter 15: Depth First Search (DFS)

The Depth First Search algorithm traverses the graph and explores each adjacent
node before backtracking and moving to the next node. It utilizes the stack data
structure. Remember, the stack uses the last-in-first-out (LIFO) approach. We
push items onto the stack and pop items from the top of the stack. We’ll keep
track of the stack and will show the vertices in the order of which they were
discovered.
We’ll begin our traversal at vertex A. Vertex A has been discovered, so it’s added
to the discovery list. Vertex A is also pushed onto the stack and is marked as
visited.
Vertex A is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex A. Vertex A is connected to the unvisited nodes B and G. The
DFS algorithm pushes the next node onto the stack, which is B in this case since
it comes next alphabetically. Vertex B is also added to the discovery list and is
marked as visited.
Vertex B is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex B. Vertex B is connected to the unvisited nodes C and G. The
DFS algorithm pushes the next node onto the stack, which is C in this case since
it comes next alphabetically. Vertex C is also added to the discovery list and is
marked as visited.
Vertex C is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex C. Vertex C is connected to the unvisited node E. The DFS
algorithm pushes E onto the stack. Vertex E is also added to the discovery list
and is marked as visited.
Vertex E is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex E. Vertex E is connected to the unvisited nodes G, J, and K.
The DFS algorithm pushes G onto the stack. Vertex G is also added to the
discovery list and is marked as visited.
Vertex G is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex G. Vertex G is connected to the unvisited node J. The DFS
algorithm pushes J onto the stack. Vertex J is also added to the discovery list and
is marked as visited.
Vertex J is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex J. Vertex J is connected to the unvisited nodes D and F. The
DFS algorithm pushes D onto the stack. Vertex D is also added to the discovery
list and is marked as visited.
Vertex D is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex D. Vertex D is connected to the unvisited nodes F and K. The
DFS algorithm pushes F onto the stack. Vertex F is also added to the discovery
Vertex F is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex F. Vertex F is connected to the unvisited node K. The DFS
algorithm pushes K onto the stack. Vertex K is also added to the discovery list
At this point all nodes have been discovered. Vertex K is at the top of the stack.
Since there are no other unvisited nodes that can be reached from vertex K, it’s
popped from the top of the stack. The algorithm backtracks to vertex F since it’s
on top of the stack now.
There are no other unvisited nodes that can be reached from vertex F, so it’s
popped from the top of the stack. The algorithm backtracks to vertex D since it’s
There are no other unvisited nodes that can be reached from vertex D, so it’s
popped from the top of the stack. The algorithm backtracks to vertex J since it’s
There are no other unvisited nodes that can be reached from vertex J, so it’s
popped from the top of the stack. The algorithm backtracks to vertex G since it’s
There are no other unvisited nodes that can be reached from vertex G, so it’s
popped from the top of the stack. The algorithm backtracks to vertex E since it’s
There are no other unvisited nodes that can be reached from vertex E, so it’s
popped from the top of the stack. The algorithm backtracks to vertex C since it’s
There are no other unvisited nodes that can be reached from vertex C, so it’s
popped from the top of the stack. The algorithm backtracks to vertex B since it’s
There are no other unvisited nodes that can be reached from vertex B, so it’s
popped from the top of the stack. The algorithm backtracks to vertex A since it’s
There are no other unvisited nodes that can be reached from vertex A, so it’s
popped from the top of the stack. The algorithm attempts to backtrack, but the
stack is empty. This signals the DFS algorithm that the traversal is complete.
This was a good example of how the DFS algorithm discovers all vertices and
backtracks at the end. Let’s look at another quick example where the DFS
algorithm backtracks throughout the traversal.
The sample graph that we’ll use is like the previous one, but with fewer edges.
We’ll start the DFS algorithm at vertex A. Vertex A has been discovered, so it’s
added to the discovery list. Vertex A is also pushed onto the stack and is marked
as visited.
Vertex A is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex A. Vertex A is connected to the unvisited nodes B and G. The
DFS algorithm pushes the next node onto the stack, which is B in this case since
it comes next alphabetically. Vertex B is also added to the discovery list and is
marked as visited.
nodes from vertex B. Vertex B is connected to the unvisited nodes C and G. The
DFS algorithm pushes the next node onto the stack, which is C in this case since
it comes next alphabetically. Vertex C is also added to the discovery list and is
marked as visited.
There are no other unvisited nodes that can be reached from vertex C, so it’s
nodes from vertex B. Vertex B is connected to the unvisited node G. The DFS
algorithm pushes the next node onto the stack, which is G in this case since it
comes next alphabetically. Vertex G is also added to the discovery list and is
marked as visited.
Vertex G is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex G. Vertex G is connected to the unvisited node E. The DFS
algorithm pushes E onto the stack. Vertex E is also added to the discovery list
Vertex E is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex E. Vertex E is connected to the unvisited nodes J and K. The
DFS algorithm pushes J onto the stack. Vertex J is also added to the discovery
nodes from vertex J. Vertex J is connected to the unvisited nodes D and F. The
DFS algorithm pushes D onto the stack. Vertex D is also added to the discovery
Vertex D is currently on top of the stack. The algorithm checks all the unvisited
nodes from vertex D. Vertex J is connected to the unvisited node K. The DFS
algorithm pushes K onto the stack. Vertex K is also added to the discovery list
There are no other unvisited nodes that can be reached from vertex K, so it’s
popped from the top of the stack. The algorithm backtracks to vertex D since it’s
There are no other unvisited nodes that can be reached from vertex D, so it’s
nodes from vertex J. Vertex J is connected to the unvisited node F. The DFS
algorithm pushes F onto the stack. Vertex F is also added to the discovery list
There are no other unvisited nodes that can be reached from vertex F, so it’s
There are no other unvisited nodes that can be reached from vertex J, so it’s
popped from the top of the stack. The algorithm backtracks to vertex E since it’s
There are no other unvisited nodes that can be reached from vertex E, so it’s
popped from the top of the stack. The algorithm backtracks to vertex G since it’s
There are no other unvisited nodes that can be reached from vertex G, so it’s
There are no other unvisited nodes that can be reached from vertex B, so it’s
popped from the top of the stack. The algorithm backtracks to vertex A since it’s
There are no other unvisited nodes that can be reached from vertex A, so it’s
popped from the top of the stack. The algorithm attempts to backtrack, but the
stack is empty. This signals the DFS algorithm that the traversal is complete.
We’ll look at one additional DFS sorting approach in the next chapter.
Chapter 16: Topological Sorting
Topological sorting is the result you get when you sort the tree in order of
decreasing finish times. What this means is that you run the Depth First Search
algorithm. Once it’s complete, you’ll have two sets of values: discovery times
and finishing times. Finishing times sorted in reverse will generate the
topologically sorted list. Since there’s an entire chapter dedicated to traversing a
tree using the DFS algorithm, we’ll work with a smaller example.
DFS starts traversing the tree at vertex A. Since it has been discovered, we’ll
mark it with the number 1. Number 1 is used to indicate that it has been
discovered first.

The algorithm checks all the unvisited nodes from vertex A. Vertex A is
connected to the unvisited nodes B and C. The DFS algorithm visits vertex B
next. Vertex B is marked as visited and receives a discovery time of 2.
The algorithm checks all the unvisited nodes from vertex B. Vertex B is
connected to the unvisited nodes C and D. The DFS algorithm visits vertex C
next. Vertex C is marked as visited and receives a discovery time of 3.
The algorithm checks all the unvisited nodes from vertex C. Vertex C is
connected to the unvisited nodes E and F. The DFS algorithm visits vertex E
next. Vertex E is marked as visited and receives a discovery time of 4.
Vertex E does not have any additional edges that are connected to unvisited
nodes. It backtracks to vertex C. With the backtrack, vertex E receives a
finishing time of 5. The finishing time, 5, is displayed after the discovery time of
4.
The algorithm checks all the unvisited nodes from vertex C again. Vertex C is
connected to the unvisited node F. The DFS algorithm visits vertex F next.
Vertex F is marked as visited and receives a discovery time of 6.
Vertex F does not have any additional edges that are connected to unvisited
nodes. It backtracks to vertex C. With the backtrack, vertex F receives a
6.
Vertex C does not have any additional edges that are connected to unvisited
nodes. It backtracks to vertex B. With the backtrack, vertex C receives a
3.
The algorithm checks all the unvisited nodes from vertex B again. Vertex B is
connected to the unvisited node D. The DFS algorithm visits vertex D next.
Vertex D is marked as visited and receives a discovery time of 9.
Vertex D does not have any additional edges that are connected to unvisited
nodes. It backtracks to vertex B. With the backtrack, vertex D receives a
finishing time of 10. The finishing time, 10, is displayed after the discovery time
of 9.
Vertex B does not have any additional edges that are connected to unvisited
nodes. It backtracks to vertex A. With the backtrack, vertex B receives a
finishing time of 11. The finishing time, 11, is displayed after the discovery time
of 2.
Vertex A does not have any additional edges that are connected to unvisited
nodes. It attempts to backtrack but has no additional nodes to backtrack to. With
the backtrack, vertex A receives a finishing time of 12. The finishing time, 12, is
displayed after the discovery time of 1. The DFS algorithm completes.
To get the discovery times list, you place the items in order of ascending
discovery time.
To get the topologically sorted list, you place the items in order of descending
finishing times. A was the last item to be marked as complete, so it goes in first,
followed by B and then D. Although the first two elements were the same in
both lists, element 3 changes from C to D. Next, vertex C was marked as
complete, followed by vertices F and E. This completes the topologically sorted
list.
Chapter 17: Breadth First Search (BFS)
The Breadth First Search algorithm is like the Depth First Search algorithm with
the exception that it uses a queue rather than a stack. BFS is technically the
opposite of DFS since the BFS algorithm explores the deepest nodes first before
backtracking to explore the shallower nodes. An element can be enqueued or
dequeued. The queue works on a first-in-first-out principle (FIFO), which means
that the element that was enqueued first will be dequeued first. BFS works like
this:
Select the first node

Explore all unvisited nodes and enqueue them
Once all unvisited, adjacent nodes have been visited from the first
node, dequeue the second node.
Move to the next enqueued node and repeat the process
Let’s take a look at the Breadth First Search Algorithm in action.
The first operation that the BFS algorithm does is that it selects vertex A as the
starting point. Why vertex A? Because that’s the vertex that we’ve decided to
start with. Vertex A is added to the discovery list and is set as the current node.
The current node is represented with the arrow. It is marked as visited but It is
not enqueued.
The vertices that are adjacent to it, B and G, are enqueued in alphabetical order.
Vertex B is added to the discovery list, followed by vertex G. Both vertices are
marked as visited.
Since there are no additional, undiscovered, adjacent nodes, this completes the
discovery phase for vertex A. Since vertex B was enqueued first, it’s dequeued
now and selected as the node from which the traversal continues.
The process repeats. All adjacent, unvisited vertices from the current node are
enqueued. Vertex B has one undiscovered node: vertex C. Vertex C is enqueued,
added to the discovery list, and marked as visited.
discovery phase for vertex B. Since vertex G was enqueued next, it’s dequeued
All adjacent, unvisited vertices from the vertex G are enqueued. Vertex G has
two undiscovered nodes: vertices E and J. Both vertices are enqueued, added to
the discovery list, and marked as visited.
discovery phase for vertex G. Since vertex C was enqueued next, it’s dequeued
There are no additional, undiscovered, adjacent nodes, so this completes the
discovery phase for vertex C. Since vertex E was enqueued next, it’s dequeued
All adjacent, unvisited vertices from the vertex E are enqueued. Vertex E has one
undiscovered node: vertex K. Vertex K is enqueued, added to the discovery list,
and marked as visited.
discovery phase for vertex E. Since vertex J was enqueued next, it’s dequeued
All adjacent, unvisited vertices from the vertex J are enqueued. Vertex J has two
undiscovered nodes: vertices D and F. Both vertices are enqueued, added to the
discovery list, and marked as visited.
Although the algorithm has discovered all the nodes in the graph, it cannot finish
until the queue is empty. There are no additional, undiscovered, adjacent nodes
from vertex J; this completes the discovery phase for vertex J. Since vertex K
was enqueued next, it’s dequeued now and selected as the node from which the
traversal continues.
discovery phase for vertex K. Since vertex D was enqueued next, it’s dequeued
discovery phase for vertex D. Since vertex F was enqueued next, it’s dequeued

discovery phase for vertex F. Since the queue is empty, this completes the
traversal process.
Chapter 18: Greedy Algorithms

The Greedy Method is an approach for solving certain types of optimization
problems. The greedy algorithm chooses the optimum result at each stage. While
this works majority of the times, there are numerous examples where the greedy
approach is not the correct approach. For example, let’s say that you’re taking
the greedy algorithm approach to earning money at a certain point in your life.
You graduate high school and have two options:
1. Get a job that only requires a high school diploma

2. Go to college and then get a job after you graduate from college
The greedy approach would choose the first option. You instantly get money
versus spending money to get an education. Although you’re likely to get more
money with a college education in the future, the algorithm only examines your
choices at that moment. Let’s look at an example of the greedy algorithm in
action. We have a directed graph with weighted edges.
If we’re trying to make our way from A to J, the greedy algorithm will examine
all the paths that are immediately connected to A, which are edges to B and C.
The weight of edge A-C is smaller than A-B, so the algorithm chooses A-C.
We’ll keep a table with the greedy and optimal paths during each point. After the
first decision, the greedy algorithm is winning. Its current weight is 3 while the
current weight of the optimal path is 4.
Greedy
Greedy Path Optimal Path Optimal Weight
Weight
A-C 3 A-B 4
The greedy algorithm is now at vertex C and has the option to use edge C-F or
C-G. It chooses C-G since the immediate weight is lower.
Greedy
Weight
A-C-G 3+3 A-B-E 4+5
At this point the greedy algorithm is still winning. The weight of the edges
utilized by the greedy algorithm totals 6, while the weight of the edges utilized
by the optimal path totals 9. The greedy algorithm is now at vertex G. It has only
one path to vertex I, so it must take it. It gets a big penalty in weight by adding
12 to its current weight.
Greedy
Weight
A-C-G-I 3+3+12 A-B-E-H 4+5+2
The current weight of the greedy algorithm, 18, has now surpassed the weight of
the optimal algorithm, 11. The greedy algorithm has one more edge that it can
take to J. Unfortunately, it gets another big penalty by adding on an additional
weight of 9.

Greedy Path Greedy Weight Optimal Path Optimal Weight
A-C-G-I-J 3+3+12+9 = 27 A-B-E-H 4+5+2+1 = 12
Both the greedy algorithm and the optimal algorithm went from A to J but
utilized different paths. In the end, the optimal algorithm won since its total
weight was more than twice as less as that of the greedy algorithm. Why would
the greedy algorithm even exist then? There are situations that we’ll cover where
the greedy algorithm is the most efficient algorithm. Some algorithms that utilize
the greedy approach are Kruskal’s algorithm, Prim’s algorithm, and Dijkstra’s
algorithm. We’ll cover these in later chapters.
Chapter 19: Minimum Spanning Trees (MST)

The Minimum Spanning Tree connects all vertices utilizing the minimum path.
We won’t go into detail since the next two chapters will cover MSTs in detail,
but we’ll show a quick example of what an MST looks like. We’re not going to
take any algorithmic approach, but instead use intuition in this example.
We can see that there are two negative edges in this graph, so most likely they’ll
be part of the minimum spanning tree. We’ll start by adding those edge.
From there, we can add edges from vertices D, E, and F. Vertex D and vertex F
both have edges with weights of 1, so we’ll add both of those to the graph.
We have to make sure that each time we add an edge to the minimum spanning
tree, no cycles appear. What is a cycle? Imagine that we add the edge E-G next;
the E-F-G cycle appears.
We’ll remove that edge since it’s not part of the Minimum Spanning Tree. If we
observe the graph, vertices C, D, E, F, and G have been added, but vertices A
and B have not. Vertex A can be connected either through edge A-C or edge A-
B. Since edge A-C is lower in weight, we’ll add that edge.
Edge B has quite a number of options. It can enter the MST via edge A-B, B-C,
B-D, or B-F. Since B-D has the smallest weight, it’s added to the tree.
Let’s double check to make sure that our solution is correct. We’ll do this by
trying to find other optimum routes to each vertex from another vertex, i.e.
vertex A. If we want to go from A to B, we could utilize edge A-B, which has a
weight of 5, or a series of edges A-C-E-F-D-B, which has a weight of -10. We
can repeat this procedure for other routes to B, as well as for other routes to
other vertices. We’ll quickly see that the edges that we chose to include in the
MST are the optimal edges. These edges create the optimal route between all
vertices. We’ll take a more algorithmic approach in the next few chapters in
finding the minimum spanning tree.
Chapter 20: Kruskal’s Algorithm
Kruskal’s algorithm generates a minimum spanning tree. The way it works is
that it keeps selecting the least costly edge until a minimum spanning tree is
created. The only constraint is that a cycle doesn’t appear once the least costly
edge is selected. Sound familiar? Let’s look at an example.
There are 10 vertices and each vertex has at least one edge connecting to it.
We’ll look through the edges until we find the least costly edge. We can quickly
see that edge F-I has the least costly edge, so we’ll include it first.
Next, there are two edges with a weight of 2: C-D and G-H. Neither of them
forms a cycle, so we’ll select one of them: C-D.
The next least costly edge is edge G-H. Since it doesn’t form a cycle, we’ll
include it into the MST.
The next least costly weight is 3; there are 2 edges with a cost of 3: edges A-C
and I-J. We’ll choose edge A-C and include it into the formulation of the MST
since it comes next lexically.
Edge I-J is also included since it’s not involved in the construction of a cycle in
the graph.
The next least costly edge weight is 4. There are two edges with a weight of 4:
edges B-D and F-J. Edge B-D is added to the MST formation.
Although edge F-J does have the next least costly edge, it cannot be added to the
list since it creates a cycle. This is what the cycle would look like if we added
that edge.
We’ll remove that edge and continue with the creation of the MST.
There are two possibilities next. We can either add edge C-G or F-H. We’ll add
the edge C-G since it comes next alphabetically and does not form a cycle.
Edge H-F can also be added since it’s not part of a cycle.
Only one more vertex needs to be added to the MST: vertex E. The next lowest
edge weight is 6. Edge E-F contains a weight of 6. Once added, it does not form
a cycle.
Kruskal’s algorithm ends since all vertices have been added to the MST.
Chapter 21: Prim’s Algorithm
Prim’s Algorithm is similar to Kruskal’s Algorithm. It’s a greedy algorithm that
finds the MST (minimum spanning tree) for a weighted, undirected graph.
Starting at an arbitrary vertex, the algorithm builds the MST one vertex at a time
where each vertex takes the shortest path from the root node. Let’s move to our
example.
Although we can begin with any vertex, we’ll choose vertex A for this example.
Vertex A has direct edges to B and I. The cost to vertex B is 7 and the cost to
vertex I is 15. Since Prim’s algorithm is a greedy algorithm, it chooses the edge
with the lowest weight, which is edge A-B in this case. The numbers in green
represent the order of vertex discovery.
Vertices A and B can now see vertices C and I. Vertex B has access to C and
vertex A has access to I. Looking at vertex C, the distance from B to C is 5; the
distance from A to I is 15. Prim’s algorithm chooses the smallest edge, which in
this case is from B-C.
The algorithm must consider all the edges that are visible from all of the already
discovered vertices. The undiscovered, visible edges now are A-I, C-E, C-D, and
C-G. By comparing the weights of each edge, Prim’s algorithm concludes that
the lowest weight is from vertex C to vertex E, so it chooses that edge.
The algorithm now has access to edges A-I, C-D, C-G, and E-D. Edge E-D has
the lowest cost, so the algorithm chooses it next.
Prim’s algorithm now must consider the following edges: A-I, C-G, and D-F.
Edge C-D is not considered since both vertices have already been found. If that
edge is chosen, a cycle would be created. Edge C-G contains the smallest
weight, so it’s chosen next.
There are three undiscovered vertices left. Prim’s algorithm will choose the least
costly edge from the following: A-I, D-F, G-F, and G-I. Edge G-F contains the
least costly edge, so it’s chosen next.
Prim’s looks at edges A-I, F-H, and G-I. Since G-I is the smallest edge, Prim’s
chooses it next.
Finally, vertex H can be accessed via edge F-H or edge I-H. Since edge I-H is
smaller, Prim’s algorithm chooses it.
Since all the vertices have been found, the algorithm ends. The minimum
spanning tree has been found. The order of vertex discovery is as follows: A, B,
C, E, D, G, F, I, H.
Chapter 22: Binary Tree Traversal – Depth First – In
Order
There are three types of depth first binary tree traversals:
Pre-order
<root><left><right>
In-order
<left><root><right>
Post-order
<left><right><root>
In this chapter we’ll focus on the second case: in-order. We can see that starting
from the root node, we’ll visit all the nodes in the left subtree, come back to the
root node and then visit all the vertices in the right subtree. As is customary with
all other chapters in this book, let’s quickly move to an example.
Next to each node, we’ll put a couple of reminders for ourselves:
left subtree
visit/print node
right subtree
We’ll begin with the root node.

At this point, we can cross out the left subtree label and start moving down the
left subtree.
Since vertex B also has a left subtree, according to the rules, we must visit its left
subtree next. So, we cross out the left subtree label on vertex B and move to
vertex D.
Vertex D also has a left subtree, so we visit its left subtree next. We cross out the
left subtree label on vertex D and move to vertex H.
We can quickly see that vertex H does not have any other left subtrees. We can
safely cross that out.
We’ll return to visit/print node H.

An attempt is made to visit node H’s right subtree, but vertex H has no right
subtree. The right subtree label is crossed out and we return to node D. We are
now visiting node D after visiting its left subtree. We can add node D to the
output.
We move to node D’s right subtree. Node D only has one node in its right
subtree, node I.
We can quickly see that vertex I does not have any other left subtrees. We can
safely cross that out.
We then return to visit/print node I.
An attempt is made to visit node I’s right subtree, but vertex I has no right
subtree. The right subtree label is crossed out and we return to node D.
We’re done with node D’s right subtree, so we return to B. We are now visiting
node B after visiting its left subtree. We can add node B to the output.
We move to node B’s right subtree. Node B only has one node in its right
subtree, node E.
We attempt to visit node E’s left subtree, but it doesn’t have a left subtree. Once
we cross out the left subtree label, we visit vertex E and print out its value.
We attempt to visit node E’s right subtree, but it does not have one. We return to
vertex B.
We’re done with B’s right subtree, so we return to node A. Since we’re visiting
node A after visiting its left subtree, we will add node A to the output.
We next visit node A’s right subtree.
According to the algorithm, we have to visit node C’s left subtree first.
Since node F does not have a left subtree, we return to node F and print out its
value.
Node F does not have a right subtree, so we return to node C. Since we’re
returning to node C after visiting its left subtree, we add node C to the output.
The algorithm calls for us to visit node C’s right subtree next.
An attempt is made to visit node G’s left subtree. Since node G has no left
subtree, we return to node G and print it out.
We attempt to visit G’s right subtree, but it has none, so we return to node C.
We’re done with node C’s right subtree, so we return to node A.
Now that we have made it back to the root node after visiting its right subtree,
the algorithm ends. Pre-order and post-order traversals follow similar guidelines.
Pre-order binary tree traversal displays the node first and then visits the left
subtree followed by the right subtree. Post-order binary tree traversal visits the
left and right subtrees, and then outputs the node.
Chapter 23: Binary Tree Insertion/Deletion
There are two primary binary tree operations: insertion and deletion. We’ll
briefly explore both of those features in this chapter.
23.1 Inserting a Node
When inserting a node, remember this rule: nodes will only be added as leaves in
the tree. Let’s assume we have the following tree.
If we look at this tree, you’ll notice that at each node, values in the left subtree
are smaller than the value of the node, and values in the right subtree are larger
than the node’s value.
Let’s say we wanted to insert the value 35. Can you predict where it would be in
the tree? Let’s walk through it and see where it ends up.
We start by comparing 35 to the root node’s value, 20. Since 35 is larger than the
value at the root node, we move into the right subtree.
Next, we compare 35 to 40. Since 35 is less than 40, we move into its left
subtree.
Next, we compare 35 to 33. Since 35 is greater than 33, we attempt to move into
node 33’s right subtree.
Since there is no right subtree at node 33, we insert node 35 there.
23.2 Deleting a Leaf in the Tree
When deleting a leaf in the tree, the structure of the tree does not change. If we
examine the tree that we used in the previous section, the following nodes can be
deleted safely without changing the overall structure of the tree.
The leaves circled in red can simply be removed without modifying the existing
leaf structure. Nodes that pointed to the deleted leaf node will no longer point to
that node. In the next section, we’ll examine what happens when we delete a
node that’s not a leaf.
23.3 Deleting a Node with One Child
Let’s examine which nodes have one child. In our example, only node 33 has
one child. All other nodes have two children.
If we removed node 47 from the list, we would have two nodes that have 1 child
each; node 40 would have one immediate child and node 33 would have one
immediate child. However, let’s keep the example as is. It is important for the
reader to have this visual in their head though.
So, how would we delete node 33 from the tree shown above? That’s simple.
Node 33’s parent, node 40, would point to node 33’s child after it severs the edge
with node 33.
Node 35 would move into the place of node 33. If we examine the binary search
tree, we can safely conclude that everything in the left subtree of each parent is
less than the parent, and everything in the right subtree of each parent is greater
than the parent.
23.4 Deleting a Node with Two Children
What would happen if we wanted to remove node 40 from the tree below?
Since node 40 has two children, we can’t simply point to its child and hope it
works. There’s an algorithmic approach to this problem too. The way we remove
this node is by replacing it the next largest value. How do we get the next largest
value? By going into the right subtree, and then following the left subtree until
we get to the leaf node.
Let’s look at that more closely.
We enter node 40’s right subtree.
We then follow the left subtree path until we get to a leaf node. So, we go to
node 43.
Since node 43 is a leaf node, we swap node 40 with node 43.
We can now remove node 40 from the tree.
If we examine the tree, we can see that the binary tree properties still hold. All
nodes in the left subtree are smaller than the parent, and all nodes in the right
subtree are larger than the parent. Just remember, to remove a node that has two
children, start by looking in its right subtree. The node on the far left in its right
subtree is the node that will replace the node you’re trying to remove.
Chapter 24: Planar Graphs

A planar graph is a graph that can be drawn on the plane with no intersecting
arcs. The edges can intersect only at endpoints. Let’s look at a couple of planar
graphs.
Let’s also take a quick look at a graph that is not planar. This is the famous K5
graph. We’ll attempt to make it planar.
After step 1, we can see that intersecting edges still exist. Let’s see what step 2
and 3 look like.
We can see that in step 2, it still looks promising. There’s only one additional
intersecting edge that must be dealt with. However, once we attempt to move it
in any direction, there will always be an intersect. You can see that occurs in step
3. K5 is therefore a non-planar graph.
K5 graph is a famous non-planar graph; K3,3 is another. Draw out the K3,3 graph
and attempt to make it planar. You’ll quickly see that it’s not possible.
Do we have to attempt to make the planar graph by redrawing it every single

time? No. There is the Euler formula. Euler’s formula states that the number of
vertices minus the number of edges plus the number of faces must equal 2 on a
planar graph.
v - e + f = 2
Let’s test this with a couple of small examples.
The graph on the left has 4 vertices, 4 edges and 2 faces. The outside of the
graph is also considered a face. If we plug those values into Euler’s equation, we
get 4 – 4 + 2 = 2. So, this is a planar graph. The example on the right has 4
vertices, 6 edges and 4 faces. If we plug those values into Euler’s equation, we
get 4 – 6 + 4 = 2. Since 2 equals 2, we can see that the graph on the right is a
planar graph as well.
There’s another simple trick to keep in mind. Complete graphs (Kn), where each
vertex is connected to all other vertices in the graph, are not planar if n ≥ 5. So,
K5 , K6 , K7 , …, Kn graphs are not planar. Complete bipartite graphs (Km,n) are
not planar if m ≥ 3 and n ≥ 3. We can quickly verify that the K3,3 graph is not
planar then.
Of course, it’s not always that simple. The general rule of thumb is that if you
can find a K3,3 or a K5 subgraph, then the graph is not planar. Let’s see if we
can prove that the following graph is not planar.
After observing the graph, we can quickly see that vertices 2, 3, 5, 7, 8, and 9
have 3 edges connecting to each of them. We can move towards the direction of
proving that this will be a K3,3 graph. Let’s draw those points on a plane.
If we can rearrange the graph above so that each of those points encompasses all
the other vertices and still maintains three edges each, then that sufficiently
proves that this is a K3,3 graph and is not planar.
We’ll start by connecting vertices 2 and 7 since there is a direct connection.
Node 2 is also directly connected to node 3.
We need node 2 to be connected to one additional node. It looks like if we follow

its path to the left, it connects to node 9 through nodes 12, 11, and 10. To make it
clearer, to get to node 9, we’ll take a path from node 2 through edge 2-1, then
through edge 1-12, then through edge 12-11, then through edge 11-10, and
finally through edge 10-9.
Moving on to edge 3, it’s connected to node 8 directly.
When comparing the graph on the left with the graph on the right, we can see
that in the left graph node 3 is connected to node 4. However, in our graph on the
right, node 4 does not exist. But, node 5 exists. We can see that node 3 is
connected to node 5 through node 4.
Moving on to node 5, it’s connected directly to node 9.

Node 5 is also connected indirectly to node 7 through node 6.
Node 7 is connected directly to node 8.
And finally, node 8 is connected directly to node 9.
Since each of the vertices in the graph we produced on the right is connected to 3
edges, this shows that the graph produced is a K3,3 graph, which we know for a
fact is non-planar. Therefore, the graph on the left is non-planar as well.
Chapter 25: Dynamic Programming: Longest Common
Subsequence
Another type of problem that arises in computer science is finding the longest
common subsequence. If you have the following string, BEADCA and ABCDA,
what is the longest common subsequence? You can probably quickly figure out
that the longest common subsequence of these two strings is BDA. It is possible
to have multiple subsequences that are of equal length that would work. Let’s see
how we would find the longest common subsequence in this problem.
We begin by constructing a matrix. One sequence goes on top of the matrix and
the other goes on the left side. For every row/column match, we’ll enter a 1. If
there is no match, we’ll enter a zero, unless the adjacent cell is already higher
than zero, in which case we enter that value. For all other matches where the
adjacent cell is already greater than one, 1 will be added to the new cell. Let’s
see what that looks like in practice.
We’ll start by populating the first row and first column with zeros.
We then start with the first row, B. Row B is compared with the column A. Since
there’s no match, a zero is entered for that spot.
The next comparison is between row B and column B. Since they match, we’ll
add a value of 1, which is added from the cell (_,A).
We find out that there is no match when comparing row B to column C. We
carry the 1 from cell (B,B).
Similarly, we carry the 1 from cell (B,C) to cell (B,D) and to cell (B,A).
We move to the second row. For row E, we compare it to the first column. Since
they’re not the same, we enter a zero for that cell. We move to column B. Even
though there is no match, we can carry the 1 from cell (B,B).
Since there are no other column values that match row E, we can carry the 1
across
We move down to row A. Row A and column A are identical, so we enter a 1 in
cell (A,A).
Moving, through the next values do not match: A does not match B. However,
we can bring down 1 from above.
Row A does not match with either C or D, so we’ll just move the 1 from cell
(A,B) to those cells.
The last cell does match. We’ll add 1 to the top left cell (E,D) and enter the new
value of 2 into cell (A,A).
We move to the next row down. Row D has no match with the first column, so
we’ll just bring the 1 down. It also has no matches with the next two columns, so
we’ll move 1 from cell (D,A) to cells (D,B) and (D,C).
Row D does match column D, so we’ll add 1 to the cell that’s located in the top
left corner, (A,C). Whenever there’s a match, you will always add 1 to the top
left cell and insert it into the new cell. Those transitions will aid you in creating
the longest common subsequence.
Row D does not match column A, so we’ll just move the largest value into its
location.
Moving on to row C, the first cells receive a value of 1 since that is the largest
value surrounding them, and the cells don’t match.
The next cell does match, so we’ll add 1 to the value located in the top-left cell
and add it to the new cell.
The next two cells do not match, so we’ll just move the largest value into those
cells. We’ll explain why we moved from above for cell (C,D) and from the left
for cell (C,A) when we complete this table. For now, just know that we’re trying
to get the longest path possible.
On the last row, A does match column A, so we’ll add 1 to the top-left cell and
insert the value into the new cell.
For cell (A,B) we’ll move the largest value into it.
Cell (A,C) contains a larger value above than to the left of it, so we’ll move the
larger value from above into it. As a rule of thumb, if the value to the left is the
same as the value above, move the value from above. That doesn’t mean that
you shouldn’t account for other paths coming from the left. It could be that an
equal size subsequence comes from the left.
The next cell’s row does not match the column value, so we’ll move the largest
value surrounding it into the new cell.
Finally, the last cell’s row and column values do match, so we’ll add 1 to the top-
left cell and enter the new value there.
We know that the largest common subsequence is going to contain 3 letters, so
we’ll try to follow the path up, starting at the last transitional cell (the cell with
the 1 added to it from the top left cell) and see what those values are going to be.
We can quickly see that the longest common subsequence is going to be B,D,A.

You’re presented with a graph and your goal is to find all-pairs shortest paths
using dynamic programming.
The first step is to create a matrix where the number of rows and columns equals
the number of vertices and then to populate it with initial data. For each vertex
that can be reached by one hop, you will populate it with the edge weight. If a
vertex cannot be reached with one hop, you will populate it with infinity and if a
vertex is trying to reach itself, you will populate it with zero. Let’s walk through
the algorithm to populate the initial matrix.
A1 =
We’ll start by populating all the zeros. The zeros will be down the diagonal.
Why? Looking at the first entry at A1 (1,1), you’re trying to go from vertex 1 to
vertex 1. What’s the weight of that? Zero. Same occurs for (2,2), (3,3), and (4,4)
A1 =
Let’s examine A1 (1,2) next. You can get from vertex 1 to 2 in one hop. The edge
weight is 10.
A1 =
Next, we’ll look at A1 (1,3). Since you cannot get to vertex 3 from vertex 1 in
one hop, you enter infinity.
A1 =
Let’s fill in the rest of the fields.
From 1 to 4, A0 (1,4) = 5
From 2 to 1, A0 (2,1) = ∞
From 2 to 3, A0 (2,3) = ∞
From 2 to 4, A0 (2,4) = 9
From 3 to 1, A0 (3,1) = -2
From 3 to 2, A0 (3,2) = 4
From 3 to 4, A0 (3,4) = ∞
From 4 to 1, A0 (4,1) = ∞
From 4 to 2, A0 (4,2) = -3
From 4 to 3, A0 (4,3) = 1
A1 =
Next, let’s look at the algorithm programmatically and visually.
for (k = 1 to k = 4) where k = number of vertices
Cij = min(Aik + Akj )
We start with trying to obtain the value for A0 (1,1). If we follow the for loop, k
will be equal to 1, then 2, then 3 and finally 4 while the values of i and j do not
change.
Cij = min(Ai1 + A1j , Ai2 + A2j , Ai3 + A3j , Ai4 + A4j )
C11 = min(A11 + A11 , A12 + A21 , A13 + A31 , A14 + A41 )
Next, plug in the values for those entries.
C11 = min(0 + 0, 10 + ∞ , ∞ + (-2), 5 + ∞ )
The minimum value is 0. So, we take the value of zero and plug it into A1 (1,1)
A1 =
Next, we’ll examine A1 (1,2), but this time we’ll do it visually. What we did in
the previous example is go through the row and the column corresponding to the
entry. Since we looked at entry (1,1), we added the values from the first row and
the first column and then got the minimum of those entries.
Let’s start off by highlighting the first row and second

column since we’re working on entry (1,2).
A1 =
If we walk through the algorithm we add (1,1) to (1,2), (1,2) to (2,2), (1,3) to
(3,2) and (1,4) to (4,2).
A1 =
A1 =
A1 =
A1 =
C12 = min(0+10, 10 + 0, ∞ + 4, 5 + (-3)) = min(10, 10, ∞ , 2)
The minimum is 2 so we update A2 (1,2) with 2.
A2 =
Let’s fill out the rest of the cells visually as well for the first row. A2 (1,3) is
next.
→
→ →
C13 = min(0 + ∞ , 10 + ∞ , ∞ + 0, 5 + 1) = min( ∞ , ∞ , ∞ , 6) = 6
A2 =
The minimum is 6 so we update A2 (1,3) with 6. A2 (1,4) is next.
→
→ →
C14 = min(0 + 5, 10 + 9, ∞ + ∞ , 5 + 0) = min(5, 19, ∞ , 5) = 5
A2 =
The minimum is 5 so we update A2 (1,4) with 5. A2 (2,1) is next.
→
→ →

C21 = min( ∞ + 0, 0 + ∞ , ∞ + (-2), 9 + ∞) = ∞
The value is still infinity, so we update A2 (2,1) with ∞ . A2 (2,2) is next.
A2 =
For the remainder of the iteration in constructing A2 , we’ll list the items and you
can verify them by following through the matrix A1 .
A1 =
A2 (2,2) = min( ∞ + 10, 0 + 0, ∞ + 4, 9 + (-3)) = min( ∞ , 0, ∞ , 6) = 0
A2 (2,3) = min( ∞ + ∞ , 0 + ∞ , ∞ + 0, 9 + 1) = min( ∞ , ∞ , ∞ , 10) = 10
A2 (2,4) = min( ∞ + 5, 0 + 9, ∞ + ∞ , 9 + 0) = min( ∞ , 9, ∞ , 9) = 9
A2 (3,1) = min((-2) + 0, 4 + ∞ , 0 + (-2), ∞ + ∞ ) = min(-2, ∞ , -2, ∞ ) = -2
A2 (3,2) = min((-2) + 10, 4 + 0, 0 + 4, ∞ + (-3)) = min(8, 4, 4, ∞ ) = 4
A2 (3,3) = min((-2) + ∞ , 4 + ∞ , 0 + 0, ∞ + 1) = min( ∞ , ∞ , 0, ∞ ) = 0
A2 (3,4) = min((-2) + 5, 4 + 9, 0 + ∞ , ∞ + 0) = min(3, 13, ∞ , ∞ ) = 3
A2 (4,1) = min( ∞ + 0, (-3) + ∞ , 1 + (-2), 0 + ∞ ) = min( ∞ , ∞ , -1, ∞ ) = -1
A2 (4,2) = min( ∞ + 10, (-3) + 0, 1 + 4, 0 + (-3)) = min( ∞ , -3, 5, -3) = -3
A2 (4,3) = min( ∞ + ∞ , (-3) + ∞ , 1 + 0, 0 + 1) = min( ∞ , ∞ , 1, 1) = 1
A2 (4,4) = min( ∞ + 5, (-3) + 9, 1 + ∞ , 0 + 0) = min( ∞ , 6, ∞ , 0) = 0
The final product looks like the following:
A2 =
We keep repeating the procedure until we’re able to observe one of two
occurrences:
The entries from the previous matrix to the current matrix don’t
change
There is a negative value in the diagonal. This indicates a negative
cycle and the values will decrease indefinitely.
What exactly is the A2 matrix? It’s the result of modified matrix multiplication of
two A1 matrices. The next matrix to find is A4 . That will be accomplished by
multiplying two A2 matrices. Let’s run through the entire process.
A4 (1,1) = min(0 + 0, 2 + ∞ , 6 + (-2), 5 + (-1)) = min(0, ∞ , 4, 4) = 0
A4 (1,2) = min(0 + 2, 2 + 0, 6 + 4, 5 + (-3)) = min(2, 2, 10, 2) = 2
A4 (1,3) = min(0 + 6, 2 + 10, 6 + 0, 5 + 1) = min(6, 12, 6, 6) = 6
A4 (1,4) = min(0 + 5, 2 + 9, 6 + 3, 5 + 0) = min(5, 11, 9, 5) = 5
A4 (2,1) = min( ∞ + 0, 0 + ∞ , 10 + (-2), 9 + (-1)) = min( ∞ , ∞ , 8, 8) = 8
A4 (2,2) = min( ∞ + 2, 0 + 0, 10 + 4, 9 + (-3)) = min( ∞ , 0, 14, 6) = 0
A4 (2,3) = min( ∞ + 6, 0 + 10, 10 + 0, 9 + 1) = min( ∞ , 10, 10, 10) = 10
A4 (2,4) = min( ∞ + 5, 0 + 9, 10 + 3, 9 + 0) = min( ∞ , 9, 12, 9) = 9
A4 (3,1) = min((-2) + 0, 4 + ∞ , 0 + (-2), 3 + (-1)) = min(-2, ∞ , -2, 2) = -2
A4 (3,2) = min((-2) + 2, 4 + 0, 0 + 4, 3 + (-3)) = min(0, 4, 4, 0) = 0
A4 (3,3) = min((-2) + 6, 4 + 10, 0 + 0, 3 + 1) = min(4, 14, 0, 4) = 0
A4 (3,4) = min((-2) + 5, 4 + 9, 0 + 3, 3 + 0) = min(3, 13, 3, 3) = 3
A4 (4,1) = min((-1) + 0, (-3) + ∞ , 1 + (-2), 0 + (-1)) = min(-1, ∞ , -1, -1) = -1
A4 (4,2) = min((-1) + 2, (-3) + 0, 1 + 4, 0 + (-3)) = min(1, -3, 5, -3) = -3
A4 (4,3) = min((-1) + 6, (-3) + 10, 1 + 0, 0 + 1) = min(5, 7, 1, 1) = 1
A4 (4,4) = min((-1) + 5, (-3) + 9, 1 + 3, 0 + 0) = min(4, 6, 4, 0) = 0
A4 =
Since the matrix changed from the previous version, we’re still not finished.
Time to move to A8 .
A8 (1,1) = min(0 + 0, 2 + 8, 6 + (-2), 5 + (-1)) = min(0, 10, 4, 4) = 0
A8 (1,2) = min(0 + 2, 2 + 0, 6 + 0, 5 + (-3)) = min(2, 2, 6, 2) = 2
A8 (1,3) = min(0 + 6, 2 + 10, 6 + 0, 5 + 1) = min(6, 12, 6, 6) = 6
A8 (1,4) = min(0 + 5, 2 + 9, 6 + 3, 5 + 0) = min(5, 11, 9, 5) = 5
A8 (2,1) = min(8 + 0, 0 + 8, 10 + (-2), 9 + (-1)) = min(8, 8, 8, 8) = 8
A8 (2,2) = min(8 + 2, 0 + 0, 10 + 0, 9 + (-3)) = min(10, 0, 10, 6) = 0
A8 (2,3) = min(8 + 6, 0 + 10, 10 + 0, 9 + 1) = min(14, 10, 10, 10) = 10
A8 (2,4) = min(8 + 5, 0 + 9, 10 + 3, 9 + 0) = min(13, 9, 13, 9) = 9
A8 (3,1) = min((-2) + 0, 0 + 8, 0 + (-2), 3 + (-1)) = min(-2, 8, -2, -1) = -2
A8 (3,2) = min((-2) + 2, 0 + 0, 0 + 0, 3 + (-3)) = min(0, 0, 0, 0) = 0
A8 (3,3) = min((-2) + 6, 0 + 10, 0 + 0, 3 + 1) = min(4, 10, 0, 4) = 0
A8 (3,4) = min((-2) + 5, 0 + 9, 0 + 3, 3 + 0) = min(3, 9, 3, 3) = 3
A8 (4,1) = min((-1) + 0, (-3) + 8, 1 + (-2), 0 + (-1)) = min(-1, 5, -1, -1) = -1
A8 (4,2) = min((-1) + 2, (-3) + 0, 1 + 0, 0 + (-3)) = min(1, -3, 1, -3) = -3
A8 (4,3) = min((-1) + 6, (-3) + 10, 1 + 0, 0 + 1) = min(5, 7, 1, 1) = 1
A8 (4,4) = min((-1) + 5, (-3) + 9, 1 + 3, 0 + 0) = min(4, 6, 4, 0) = 0
A8 =
Since there are no changes between A4 and A8 , no further action is necessary.
(Floyd-Warshall)
The goal of the Floyd-Warshall algorithm is to find the shortest paths in a tree
using Dynamic Programming. Let’s get started with the same example as the one
in the previous chapter. The result should be the same.

The Floyd-Warshall algorithm starts by gathering the initial weights and placing
them into a matrix. Since there are 4 vertices, the matrix will have 4 rows and 4
columns.

Starting with vertex 1, the algorithm gathers the initial weights from 1 to the
other 3 vertices. It will place a 0 for itself and infinity if there is no direct path.
The edge weight from 1 to itself is 0. The edge weight from 1 to 2 is 10. Since
there is no direct path from 1 to 3 the algorithm places infinity as the
placeholder. The edge weight from 1 to 4 is 5.
D0 =
The first row is complete and the Floyd-Warshall algorithm will work on vertex
2 next which corresponds to the second row. There is no direct path from 2 to 1,
so infinity is placed. The weight from 2 to itself is 0. There is no direct path from
2 to 3, so infinity is placed. Lastly, the edge weight from 2 to 4 is 9.
D0 =
The Floyd-Warshall algorithm will work on vertex 3 next which corresponds to
the third row. The edge weight from 3 to 1 is -2. The edge weight from 3 to 2 is
4. The edge weight from 3 to itself is 0. Finally, the edge weight from 3 to 4 is
infinity since there is no direct path from 3 to 4.
D0 =
The final row corresponds to the last vertex, which is vertex 4. The sample
principle is applied. The weight from 4 to 1 is infinity since there is no direct
path from 4 to 1. The edge weight from 4 to 2 is -3. The edge weight from 4 to 3
is 1 and the weight from 4 to itself is 0.
D0 =
Now that the initial matrix, D0 , is created, the Floyd-Warshall algorithm will
generate matrices D1 through D4 . Although exhaustive, this book will provide
you with each step of this process. To generate D1 , the algorithm will use D0 and
see if there is a shorter approach from 2 to 3, for example, by going from 2 to 1
and then from 1 to 3. Since the algorithm is building D1 , the first row and the
first column are locked and cannot be modified. Looking at the diagonal, they
are all 0’s, so the diagonal will not change since there are no self-loops present in
the provided graph. Before starting, D1 will look like the following matrix.
D1 =
The blank fields in the matrix are the ones that the Floyd-Warshall algorithm will
focus on.
To get the value for D1 row 2, column 3, the following operations are performed:
1. The value for (2,3) is retrieved from D0

2. The value D0 (2,3) is compared with the sum of the values D0 (2,1) +
D0 (1,3). Vertex 1 is the intermediate vertex for this graph.
a. D0 (2,3) = ∞
b. D0 (2,1) = ∞
c. D0 (1,3) = ∞
3. D0 (2,3) = D0 (2,1) + D0 (1,3) since ∞ = ∞ + ∞
4. Infinity is kept in place for D1 (2,3)
D1 =
Next, the algorithm will focus on D1 (2,4)
D0 (2,4) � D0 (2,1) + D0 (1,4)
9 < ∞ + 5
D0 (2,4) < D0 (2,1) + D0 (1,4)
Since D0 (2,4) is less than D0 (2,1) + D0 (1,4), the value of 9 is placed into D0
(2,4).
D1 =
Next, the algorithm will focus on D1 (3,2).
D0 (3,2) � D0 (3,1) + D0 (1,2)
4 < -2 + 10
Since D0 (3,2) is less than D0 (3,1) + D0 (1,2), 4 is preserved for D1 (3,2).
D1 =
Next, the algorithm focuses on D1 (3,4).
D0 (3,4) � D0 (3,1) + D0 (1,4)
∞ > -2 + 5
Since D0 (3,4) > D0 (3,1) + D0 (1,4), field D1 (3,4) is updated with 3.
D1 =

D0 (4,2) � D0 (4,1) + D0 (1,2)
-3 < ∞ + 10
Since D0 (4,2) < D0 (4,1) + D0 (1,2), -3 is preserved for D1 (4,2).
D1 =
Finally, the Floyd-Warshall algorithm looks at D1 (4,3).
D0 (4,3) � D0 (4,1) + D0 (1,3)
1 < ∞ + ∞
Since D0 (4,3) < D0 (4,1) + D0 (1,3), 1 is preserved for D1 (4,3).
D1 =
After all those steps, D1 is finally complete. Next, the Floyd-Warshall algorithm
will create D2 , D3 , and D4 . To create D2 , the algorithm takes the D1 matrix as
the starting point and fills in the data that is guaranteed not to change. The fields
that are guaranteed not to change in this step are the diagonals, the second row,
and the second column values.
D2 =
The Floyd-Warshall algorithm starts by examining D2 (1,3) by going through 2.
D1 (1,3) � D1 (1,2) + D1 (2,3)
∞ = 10 + ∞
Since D1 (1,3) = D1 (1,2) + D1 (2,3), ∞ is preserved for D1 (1,3).
D2 =
D1 (1,4) � D1 (1,2) + D1 (2,4)
5 < 10 + 9
Since D1 (1,4) < D1 (1,2) + D1 (2,4), 5 is preserved.
D2 =
D1 (3,1) � D1 (3,2) + D1 (2,1)
-2 < 4 + ∞
Since D1 (3,1) < D1 (3,2) + D1 (2,1), -2 is preserved.
D2 =
D1 (3,4) � D1 (3,2) + D1 (2,4)
3 < 4 + 9
Since D1 (3,4) < D1 (3,2) + D1 (2,4), 3 is preserved for D2 (3,4).
D2 =
Next, the Floyd-Warshall algorithm focuses on D2 (4,1) by going through 2.
D1 (4,1) � D1 (4,2) + D1 (2,1)
∞ = -3 + ∞
Since D1 (4,1) = D1 (4,2) + D1 (2,1), there is no change from D1 to D2 .
D2 =
Finally, the algorithm examines D2 (4,3).
D1 (4,3) � D1 (4,2) + D1 (2,3)
1 < -3 + ∞
D2 =
The Floyd-Warshall algorithm completed the D2 matrix; D3 is next. To create D3
, the algorithm takes the D2 matrix as the starting point and fills in the data that is
guaranteed not to change. The fields that are guaranteed not to change in this
step are the diagonals, the third row, and the third column values.
D3 =
D2 (1,2) � D2 (1,3) + D2 (3,1)
10 < ∞ + (-2)
D3 =
D2 (1,4) � D2 (1,3) + D2 (3,4)
5 < ∞ + 3
D3 =
D2 (2,1) � D2 (2,3) + D2 (3,1)
∞ = ∞ + (-2)
Since D2 (2,1) = D2 (2,3) + D2 (3,1), there is no change.
D3 =
D2 (2,4) � D2 (2,3) + D2 (3,4)
9 < ∞ + 3
D3 =
D2 (4,1) � D2 (4,3) + D2 (3,1)
∞ > 1 + (-2)
Since D2 (4,1) > D2 (4,3) + D2 (3,1), D3 (4,1) is updated to -1.
D3 =
Finally, the algorithm examines D3 (4,2).
D2 (4,2) � D2 (4,3) + D2 (3,2)
-3 < 1 + 4
Since D2 (4,2) < D2 (4,3) + D2 (3,2), -3 is preserved and the algorithm finishes
the construction of the D3 matrix.
D3 =
The Floyd-Warshall algorithm has finally made it to D4 . To construct D4 , the
algorithm takes the D3 matrix as the starting point and fills in the data that is
guaranteed not to change. The fields that are guaranteed not to change in this
step are the diagonals, the fourth row, and the fourth column values.
D4 =
D3 (1,2) � D3 (1,4) + D3 (4,2)
10 > 5 + (-3)
Since D3 (1,2) > D3 (1,4) + D3 (4,2), D4 (1,2) is updated to the new value of 2.
D4 =
D3 (1,3) � D3 (1,4) + D3 (4,3)
∞ > 5 + 1
D4 =
D3 (2,1) � D3 (2,4) + D3 (4,1)
∞ > 9 + (-1)
D4 =
D3 (2,3) � D3 (2,4) + D3 (4,3)
∞ > 9 + 1
D4 =
D3 (3,1) � D3 (3,4) + D3 (4,1)
-2 < 3 + (-1)
Since D3 (3,1) < D3 (3,4) + D3 (4,1), -2 is preserved.
D4 =
Finally, the Floyd-Warshall algorithm focuses on D4 (3,2).
D3 (3,2) � D3 (3,4) + D3 (4,2)
4 > 3 + (-3)
Since D3 (3,2) > D3 (3,4) + D3 (4,2), D4 (3,2) is updated to the new value of 0,
and the algorithm is complete.
D4 =
Looking over the entire process, the Floyd-Warshall algorithm produces the
following matrices.
D0 = → D1 =
→ D2 = →
D3 = → D4 =
If you examine the values of matrix D4 by comparing it to the tree, you’ll see
that the shortest path is found from each vertex to each vertex. Some vertices can
be reached by one hop, others by two, and some even by three.
Chapter 28: Dijkstra's Algorithm: Single Source Shortest
Path
Dijkstra’s Algorithm can be applied to either a directed or an undirected graph to
find the shortest path to each vertex from a single source. It can only be used
with non-negative edge weights.
A table is generated to track the distance from the source (S) vertex. The
distance to each vertex is initialized to infinity except for the source itself.
Another variable, π, is used to track the previous vertex from which the specified
vertex was reached.
S A B C D E F G H I
d 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
π - - - - - - - - - -
Dijkstra’s algorithm starts with the source, S, and relaxes the edges that are
connected directly to it in alphabetical order. The first edge to be relaxed is S to
A. The value for A is updated from infinity to 5 and the predecessor is updated
to S. The edge S-A is also marked as complete.
S A B C D E F G H I
d 0 5 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
π - S - - - - - - - -
Next, the edge S-I is relaxed. The distance to I is updated to 15 and the
predecessor is marked as S. The edge S-I is marked as visited. All of the
outbound edges from S have been visited, so S is marked as completed.
S A B C D E F G H I
d 0 5 ∞ ∞ ∞ ∞ ∞ ∞ ∞ 15
π - S - - - - - - - S
Dijkstra’s algorithm moves from vertex S to vertex A. Vertex A is the chosen
since it’s the closest, visited vertex to the source. The same process is repeated:
all of the immediate, outbound vertices are visited from vertex A. Those vertices
are C and E. Vertex C is visited first since it comes first alphabetically. The
distance to C is updated to 8 since vertex A can be reached in 5 units and vertex
C can be reached in another 3 units. The predecessor is set to A and the edge A-
C is marked as visited.
S A B C D E F G H I
d 0 5 ∞ 8 ∞ ∞ ∞ ∞ ∞ 15
π - S - A - - - - - S
Vertex E is visited next. The distance to vertex E is updated to 7 (S-A-5 + A-E-

2) and the predecessor of E is updated to A. Since all of the outbound edges
from A have been visited, vertex A is marked as complete.
S A B C D E F G H I
d 0 5 ∞ 8 ∞ 7 ∞ ∞ ∞ 15
π - S - A - A - - - S
Dijkstra’s algorithm chooses the next closest vertex. Vertex I is currently 15

units away from the source, vertex E is 7 units away and vertex C is 8 units
away. Vertex E is the closest, so it’s chosen next. Since there are no outbound
edges from vertex E, it is marked as complete.
Since vertex C is 8 units away and vertex I is 15 units away, vertex C is chosen
next. Vertex C has 3 outbound edges: C-B, C-D, and C-H. Vertex B is visited
first since it’s alphabetically the next one on the list. The distance to B is updated
to 15 (S-A-C-8 + C-B-7) and the predecessor is set as C. The edge C-B is
marked as visited.
S A B C D E F G H I
d 0 5 15 8 ∞ 7 ∞ ∞ ∞ 15
π - S C A - A - - - S
Dijkstra’s algorithm visits vertex D next. The distance to vertex D is updated to

14 (S-A-C-8 + C-D-6) and the predecessor of D is set as C. The edge C-D is
marked as visited.
S A B C D E F G H I
d 0 5 15 8 14 7 ∞ ∞ ∞ 15
π - S C A C A - - - S
The last outbound edge from C is visited, to vertex H. The distance to vertex H
is updated to 11 (S-A-C-8 + C-H-3) and the predecessor to H is updated to C.
Since all the outbound edges have been visited from C, vertex C is marked as
complete.
S A B C D E F G H I
d 0 5 15 8 14 7 ∞ ∞ 11 15
π - S C A C A - - C S
Dijkstra’s algorithm chooses the next vertex. Vertex I is 15 units away from the
source, vertex B is 15 units away, vertex D is 14 units away and vertex H is 11
units away. Since vertex H is the closest vertex to the source, it is chosen next.
Vertex H has two outbound edges: H-C and H-D. Vertex C is visited first. Vertex
C can already be reached in 8 units so there is no point to reach it in 20 units.
The distance to vertex C stays the same and the edge H-C is marked as visited.
Vertex D is visited next via edge H-D. Since the distance to vertex D via edge H-
D (12 units) is shorter than the distance to vertex D via edge C-D (14), the
distance to vertex D is updated to 12; the predecessor to vertex D is updated to
H. The edge H-D is marked as complete; since all the outbound edges from H
have been visited, H is marked as complete.
S A B C D E F G H I
d 0 5 15 8 12 7 ∞ ∞ 11 15
π - S C A H A - - C S
Dijkstra’s algorithm chooses the next closest edge by examining all the
available, non-visited vertices. Vertex I can be reached in 15 units, D can be
reached in 12 units and B can be reached in 15 units. Since D is the closest
vertex to the source, vertex D is chosen. Vertex D has one outbound edge, D-I.
The distance to vertex I from D is 14. Since vertex I can be reached in 14 units
via edge D-I, the distance to vertex I is updated to 14 and the predecessor is set
to D. Edge D-I is marked as visited. All of the outbound edges from vertex D
have been visited and vertex D is marked as complete.
S A B C D E F G H I
d 0 5 15 8 12 7 ∞ ∞ 11 14
π - S C A H A - - C D
Dijkstra’s algorithm examines all the edges that can be visited. Vertex I is 14
units away and vertex B is 15 units away. Since vertex I is the closest one to the
source, vertex I is chosen next.
Vertex I has no outbound edges so vertex I is marked as complete. Vertex B is
the only other available, non-visited edge from the source, so vertex B is chosen
next.
Vertex B has one outbound edge: B-F. Vertex F can be reached in 19 units (S-A-
C-B-15 + B-F-4). The distance to F is updated to 19 and the predecessor is set to
B. The edge B-F is marked as visited and since there are no other outbound
edges from vertex B, it is marked as complete.
S A B C D E F G H I
d 0 5 15 8 12 7 19 ∞ 11 14
π - S C A H A B - C D
Vertex F is the next closest vertex to the source. Vertex F has one outbound edge,
F-G. Vertex G can be reached in 21 units. The distance to G is updated to 21 and
the predecessor of G is set to F. The edge F-G is marked as visited and since
there are no other outbound edges from F, vertex F is marked as complete.
S A B C D E F G H I
d 0 5 15 8 12 7 19 21 11 14
π - S C A H A B F C D
Vertex G is the last unvisited vertex. Vertex G has one outbound edge, from G to
B. The distance to vertex G through edge G-B is 21 units. Since the distance to B
through edge C-B is closer than through edge G-B, the distance to B is kept at
15. The edge G-B is marked as visited and since all the outbound edges from G
have been visited, vertex G is marked as complete. All the vertices in the
directed graph have been visited, so Dijkstra’s algorithm is finished. The list
below shows the shortest path to each vertex from the source.
S A B C D E F G H I
d 0 5 15 8 12 7 19 21 11 14
π - S C A H A B F C D
Chapter 29: Bellman-Ford
The Bellman-Ford algorithm finds the shortest path to each vertex in the directed
graph from the source vertex. Unlike Dijkstra’s algorithm, Bellman-Ford can
have negative edges.
To begin, all the outbound edges are recorded in a table in alphabetical order.
Edge A- A- B- C- C- F- G- H- S-
C E F B H G B D A
Weight -3 2 -5 7 -3 2 4 1 5
Like Dijkstra’s algorithm, a table recording the distance to each vertex and the
predecessor of each vertex is created. The distances for each vertex, except the
source vertex, is initialized to infinity. Distance is represented by the variable d
and the predecessor is represented by the variable π.
S A B C D E F G H
d 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
π - - - - - - - - -
The Bellman-Ford algorithm will iterate through each of the edges. Since there
are 9 edges, there will be up to 9 iterations. During each iteration, the specific
edge is relaxed. During the first iteration, the cost to get to vertex C from A is -3.
The current distance from the source to A is infinity. When -3 is added to
infinity, the result is infinity, so the value of C remains infinity. Similarly, from A
to E, the cost is 2, however, since the distance to A is infinity, the value of E
remains the infinity. Looking at edges B-F, C-B, C-H, F-G, G-B and H-D all
yield the same result, infinity. The last edge, S-A, yields a different result. The
weight of edge S-A is 5. The current distance to S is 0, so the distance from S to
A is 0 + 5 = 5. The predecessor to A is set to S. After the first iteration, Bellman-
Ford found the path to A from S.
Iteration 1
S A B C D E F G H
d 0 5 ∞ ∞ ∞ ∞ ∞ ∞ ∞
π - S - - - - - - -
Since all the edges have been relaxed, Bellman-Ford starts on the second
iteration. Although each edge is relaxed, the only edges that matter are the edges
from S and from A since the distance to those vertices is already known. The
distance to all other vertices is infinity. Looking at the table containing the edges,
we start by relaxing edge A-C.
C E F B H G B D A
Weight -3 2 -5 7 -3 2 4 1 5
The weight of edge A-C is -3. The current distance to vertex A is 5 via edge S-A,
so the distance to vertex C is 5 + (-3) = 2. The predecessor of C is A. The weight
of edge A-E is 2. The distance to E is 5 + 2 = 7 via edge S-A. The predecessor of
E is updated to A. Edge B-F cannot be relaxed yet.
Iteration 2
S A B C D E F G H
d 0 5 ∞ 2 ∞ 7 ∞ ∞ ∞
π - S - A - A - - -
Edge C-B can be relaxed since we know the distance to C. The distance to B is 2
+ 7 = 9 and the predecessor of vertex B is C. Edge C-H can be relaxed since we
know the distance to C. The distance to H is 2 + (-3) = -1 and the predecessor of
vertex H is vertex C.
Iteration 2
S A B C D E F G H
d 0 5 9 2 ∞ 7 ∞ ∞ -1
π - S C A - A - - C
Edge F-G cannot be yet be relaxed. Edge G-B cannot be relaxed. Edge H-D can
be relaxed since we know the distance to vertex H is -1. The distance to vertex D
is -1 + 1 = 0 and the predecessor to vertex D is vertex H.
Iteration 2
S A B C D E F G H
d 0 5 9 2 0 7 ∞ ∞ -1
π - S C A H A - - C
The distance to A from edge S-A is already 5 so no update is necessary. This
ends iteration 2.
C E F B H G B D A
Weight -3 2 -5 7 -3 2 4 1 5
During the third iteration, the Bellman-Ford algorithm examines all the edges
again. Edges A-C and A-E yield the same results. Edge B-F can now be relaxed.
The distance to B is 9, so the distance to vertex F is 9 + (-5) = 4. The predecessor
to F is B.
Iteration 4
S A B C D E F G H
d 0 5 9 2 0 7 4 ∞ -1
π - S C A H A B - C
Edges C-B and C-H yield the same results, so the table remains the same. Edge
F-G can now be relaxed. The distance to vertex F is 4, so the distance to vertex
G is 4 + 2 = 6. The predecessor of G is F.
Iteration 4
S A B C D E F G H
d 0 5 9 2 0 7 4 6 -1
π - S C A H A B F C
Edge G-B can now be relaxed. The distance to vertex G is 6, so the distance to B
is 6 + 4 = 10. Since vertex B can be reached with a shorter distance by going
through edge C-B, the table remains the same.
C E F B H G B D A
Weight -3 2 -5 7 -3 2 4 1 5
During the fourth iteration, all the edges are examined. The algorithm sees that
there are no changes, so the algorithm ends on the fourth iteration.
If this graph had a negative cycle, after the iteration is repeated n-1 times,
theoretically the Bellman-Ford algorithm should have found the shortest paths to
all vertices. During the nth iteration, where n represents the number of vertices, if
there is a negative cycle, the distance to at least one vertex will change. Let’s
look at a quick example.
Edge A-B B-C C-A S-A S-B

Weight 5 2 -10 3 6
The table with the distances and the predecessors is constructed. The distances
are initialized to infinity for vertices A, B and C. The distance to S is 0.
SABC
d 0 ∞∞∞
π - - - -
Looking at the first edge, A-B cannot be relaxed yet and neither can edge B-C
nor edge C-A. Edge S-A can be relaxed. The distance to S is 0, so the distance to
A is 0 + 3 = 3. The predecessor of A is S.
Iteration 1
S A B C
d 0 3 ∞ ∞
π - S - -
Edge S-B can also be relaxed. The distance to vertex B is 0 + 6 = 6. Vertex B’s
predecessor is S.
Iteration 1
S A B C
d 0 3 6 ∞
π - S S -
The first iteration is complete. During the second iteration, all of the edges are
examined again.

Weight 5 2 -10 3 6
Edge A-B can be relaxed during the second iteration. The distance to A is 3, so
the distance to vertex B is 3 + 5 = 8. Since the distance to B is already less than
the new value, the value of B is retained. Edge B-C can be reached in 6 + 2 = 8.
Vertex C’s predecessor is vertex B.
Iteration 2
S A B C
d 0 3 6 8
π - S S B
Edge C-A is examined next. The distance to C is 8 units, so the distance to A via
edge B-C is 8 + (-10) = -2. Since the distance to A via edge C-A is less than the
distance to A via S-A, the distance to A is updated.

Iteration 2
S A B C
d 0 -2 6 8
π - C S B
Edges S-A and S-B yield nothing better, so the second iteration is complete. The
third iteration starts.
Weight 5 2 -10 3 6
Edge A-B is relaxed. The distance to A is currently -2, so the distance to B via
edge A-B is -2 + 5 = 3. Since the distance to B is less via A-B than S-B, the
distance is updated to 3. Vertex B’s predecessor is updated to vertex A.
Iteration 3
S A B C
d 0 -2 3 8
π - C A B
Edge B-C is relaxed next. The current distance to B is 3, so the distance to C is 3
+ 2 = 5. The distance to C is updated to 5.
Iteration 3
S A B C
d 0 -2 3 5
π - C A B
Edge C-A is relaxed. The distance to C is 5 + (-10) = -5. The distance to vertex A
is updated to -5 units.
Iteration 3
S A B C
d 0 -5 3 5
π - C A B
Edges S-A and S-B yield no better results. At this time, all shortest paths should
have been found. If we examine another iteration, there should be no changes.
Weight 5 2 -10 3 6
Edge A-B is relaxed. The distance to A is -5 so the distance to B is -5 + 5 = 0.
The distance to B is updated to 0. Since the value changes on the nth iteration,
values will change on the n+1th iteration as well; values will continue to change
indefinitely. If we examine the graph closely, we can see that A-B-C yields a
negative value: 5 + 2 + (-10) = -3.
Chapter 30: Johnson’s Algorithm
Johnson’s algorithm finds the shortest paths between all pairs of vertices in a
directed graph. It converts negative edge weights into non-negative edge links. It
does this by using the Bellman-Ford algorithm to remove all negative weights. It
then allows Dijkstra’s algorithm to be used on the new graph.
Let’s start with an example. Since this graph contains negative edges, Dijkstra’s
algorithm cannot be applied to it yet. To begin, the edge links must be
transformed to contain non-negative numbers.
Johnson’s algorithm starts off by selecting a source vertex. The problem with
choosing an existing vertex is that it might not be able to reach all the vertices in
the graph. To guarantee that a single source vertex can reach all vertices in the
graph, a new vertex is introduced. The new vertex, S, is introduced to all vertices
in the graph. Although the weight from the source to each vertex doesn’t matter,
as long as they’re all consistent, by convention a weight of 0 is applied to each
edge from the source.
Johnson’s algorithm computes the shortest paths from vertex S to all the vertices
on the graph. Johnson’s algorithm utilizes a single source shortest path algorithm
to get the new values. Since there are negative edge weights in the directed
graph, Bellman-Ford is the algorithm that’s used to process this computation.
To begin, all the outbound edges are recorded in a table in alphabetical order.
A- B- B- B- C- C- D- D- E- F- F- G-
Edge
D A D F A B F G C C E E
Weight 11 -7 -5 3 17 3 12 9 5 -5 4 -3
S- S- S- S- S- S-
Edge S-F
A B C D E G
Weight 0 0 0 0 0 0 0
The distance to each vertex is initialized to infinity except for the source vertex
that’s initialized to 0.
S A B CD E F G
d 0 ∞∞∞∞∞∞ ∞
π - - - - - - - -
The edges are relaxed in order. Edges A-D, B-A, B-D, B-F, C-A, C-B, D-F, D-G,
E-C, F-C, F-E and G-E have current distances of infinity. If you add any number
to infinity, the number is still infinity. Edges that start with the source vertex, S,
have a distance of 0 to S, so distances from S can be computed. During the first
iteration, edges S-A, S-B, S-C, S-D, S-E, S-F and S-G are the only edges that
matter. Looking at the graph, the distance to all vertices from the source is 0. The
predecessor to all vertices after the first iteration is S.
Iteration 1 S A B C D E F G
d 0 0 0 0 0 0 0 0
π - S S S S S S S
Starting the second iteration, all the edges are relaxed again. Since we have a
detailed chapter on the Bellman-Ford algorithm, we’ll go through the Bellman-
Ford portion of this algorithm quickly.
Distance from source to
Edge Update
vertex
A-D 0 + 11 = 11 It’s quicker to get to D via S-D
B-A 0 + (-7) = -7 -7 is smaller, so update to -7
B-D 0 + (-5) = -5 -5 is smaller, so update to -5
B-F 0 + 3 = 3 It’s quicker to get to F via S-F
C-A 0 + 17 = 17 It’s quicker to get to A via S-A
C-B 0 + 3 = 3 It’s quicker to get to B via S-B
D-F -5 + 12 = 7 It’s quicker to get to F via S-F
D-G -5 + 9 = 4 It’s quicker to get to G via S-G
E-C 0 + 5 = 5 It’s quicker to get to C via S-C
F-C 0 + (-5) = -5 -5 is smaller, so update to -5
F-E 0 + 4 = 4 It’s quicker to get to E via S-E
G-E 0 + (-3) = -3 -3 is smaller, so update to -3
S-A 0 + 0 = 0 It’s quicker to get to A via S-B-A
S-B 0 + 0 = 0 No change
S-C 0 + 0 = 0 It’s quicker to get to C via S-F-C
S-D 0 + 0 = 0 It’s quicker to get to D via S-B-D
S-E 0 + 0 = 0 It’s quicker to get to E via S-G-E
S-F 0 + 0 = 0 No change
S-G 0 + 0 = 0 No change
The table below lists the shortest distances to each vertex after the second
iteration.
d 0 -7 0 -5 -5 -3 0 0
π - B S F B G S S
You might have noticed that all distances that are computed from S directly will
not change, so during the third and subsequent iterations, edges from S will not
be included in the table.
Edge Update
vertex
A-D -7 + 11 = 4 It’s quicker to get to D via S-B-D
B-A 0 + (-7) = -7 No change
B-D 0 + (-5) = -5 No change
B-F 0 + 3 = 3 No change
C-A -5 + 17 = 12 It’s quicker to get to A via S-B-A
C-B -5 + 3 = -2 -2 is smaller, so update to -2
D-F -5 + 12 = 7 It’s quicker to get to F via S-F
D-G -5 + 9 = 4 It’s quicker to get to G via S-G
E-C -3 + 5 = 2 It’s quicker to get to C via S-F-C
F-C 0 + (-5) = -5 No change
F-E 0 + 4 = 4 It’s quicker to get to E via S-G-E
G-E 0 + (-3) = -3 No change
The table below lists the shortest distances to each vertex after the third iteration.
d 0 -7 -2 -5 -5 -3 0 0
π - B C F B G S S
Bellman-Ford starts on the fourth iteration.
Edge Update
vertex
A-D -7 + 11 = 4 No change
-9 is smaller, so update to -9 (S-F-C-B-
B-A -2 + (-7) = -9
A)
-7 is smaller, so update to -7 (S-F-C-B-
B-D -2 + (-5) = -7
D)
B-F -2 + 3 = 1 No change
C-A -5 + 17 = 12 No change
C-B -5 + 3 = -2 No change
D-F -7 + 12 = 5 No change
D-G -7 + 9 = 2 No change
E-C -3 + 5 = 2 No change
F-C 0 + (-5) = -5 No change
F-E 0 + 4 = 4 No change
G-E 0 + (-3) = -3 No change
The table below lists the shortest distances to each vertex after the fourth
iteration.
d 0 -9 -2 -5 -7 -3 0 0
π - B C F B G S S
Bellman-Ford starts on the fifth iteration.
Edge Update
vertex
A-D -9 + 11 = 2 No change
B-A -2 + (-7) = -9 No change
B-D -2 + (-5) = -7 No change
B-F -2 + 3 = 1 No change
C-A -5 + 17 = 12 No change
C-B -5 + 3 = -2 No change
D-F -7 + 12 = 5 No change
D-G -7 + 9 = 2 No change
E-C -3 + 5 = 2 No change
F-C 0 + (-5) = -5 No change
F-E 0 + 4 = 4 No change
G-E 0 + (-3) = -3 No change
The table below lists the shortest distances to each vertex after the fifth iteration.
d 0 -9 -2 -5 -7 -3 0 0
π - B C F B G S S
After the fifth iteration, there are no changes, so we can end the Bellman-Ford
portion of Johnson’s algorithm. The distance to each vertex from vertex S has
been updated on the graph. Initially, the shortest distance from S-A was 0, but
after running the Bellman-Ford algorithm, the shortest distance to A is -9 via S-
F-C-B-A.
The source vertex is unnecessary for the remainder of this example, so we’ll
drop the S and its edges.
Johnson’s algorithm then applies the following formula for the reweighting
calculations:
C’e = Ce + pu - pv
In English, the new length (C’ ) is equal to the original length (C ) plus the
e e
weight of its tail (p ) minus the weight of its head (p ). Johnson’s algorithm
u v
does this for each of the edges.
A- B- B- B- C- C- D- D- E- F- F- G-
Edge
Weight 11 -7 -5 3 17 3 12 9 5 -5 4 -3
d 0 -9 -2 -5 -7 -3 0 0
π - B C F B G S S
For A-D, Johnson’s algorithm starts with its original length of 11, adds the
weight of the tail (A), which the algorithm determined was -9, and subtracts the
weight of the head (D), which the algorithm determined was -7.
(C’e )A-D = 11 + (-9) – (-7) = 9
The procedure is repeated for all the edges.
(C’e )B-A = -7 + (-9) – (-9) = 7
(C’e )B-D = -5 + (-2) – (-7) = 0
(C’e )B-F = 3 + (-2) – 0 = 1
(C’e )C-A = 17 + (-5) – (-9) = 21
(C’e )C-B = 3 + (-5) – (-2) = 0
(C’e )D-F = 12 + (-7) – 0 = 5
(C’e )D-G = 9 + (-7) - 0 = 2
(C’e )E-C = 5 + (-3) – (-5) = 7
(C’e )F-C = -5 + 0 – (-5) = 0
(C’e )F-E = 4 + 0 – (-3) = 7
(C’e )G-E = -3 + 0 – (-3) = 0
A- B- B- B- C- C- D- D- E- F- F- G-
Edge
C’ e 9 7 0 1 21 0 5 2 7 0 7 0
The graph is updated with the new weights.
Now that all weights are non-negative, Dijkstra’s algorithm can be applied to
each vertex to compute all shortest paths. To see an example of Dijkstra’s
algorithm, refer to chapter 28.
Chapter 31: Clockwise and Counterclockwise Line
Segment Intersection
If you’re given a problem to find out whether two line-segments intersect, there
is an easy approach to this. It involves finding the orientation of the line
segments. If they’re different, they intersect.
Each line segment has a start-point and an end-point. Our first line will be
represented by (a1 ,b1 ) and our second line will be represented by (a2 ,b2 ).
To test if they intersect, we need to verify the following condition:
(a1 ,b1 ,a2 ) and (a1 ,b1 ,b2 ) are oriented

differently, AND
(a2 ,b2 ,a1 ) and (a2 ,b2 ,b1 ) are oriented
differently.
What does this mean exactly? Let’s jump into an example and find out.
It’s clear to us that the two line-segments intersect, but it’s not clear to your
computer. We need to test it with the condition outlined above. Let’s walk
through each step.
First, we’ll look at the orientation of (a1 ,b1 ,a2 ). We can see that this orientation
is clockwise.
Next, we move to (a1 ,b1 ,b2 ). The orientation is counterclockwise, therefore, (a1
,b1 ,a2 ) and (a1 ,b1 ,b2 ) are oriented differently.
We’ll have to test the second part to make sure that the orientation between those
pairs is also different. The orientation for (a2 ,b2 ,a1 ) is counterclockwise.
Next, we move to (a2 ,b2 ,b1 ). The orientation is clockwise, therefore, (a2 ,b2 ,a1 )
and (a2 ,b2 ,b1 ) are oriented differently.
Since both (a1 ,b1 ,a2 ) and (a1 ,b1 ,b2 ) are oriented differently, and (a2 ,b2 ,a1 ) and
(a2 ,b2 ,b1 ) are oriented differently, we know that the lines intersect.
Chapter 32: Graham’s Scan
Graham’s scan is a method for finding the convex hull that encompasses all
points on the plane. Below you’ll see an example of a convex hull.
Graham’s scan starts by finding the point with the lowest y coordinate. If there
are multiple points on the y-coordinate, the point with the smallest x-value is
chosen. The points are sorted by polar angles in counterclockwise rotation.
Graham’s scan utilizes the stack. Points are pushed onto the stack. If the point is
not part of the convex hull, the point is popped from the stack. Let’s look at an
example.
We have a set of points. It’s clear which point has the lowest y-coordinate value.
From there, points are ordered in increasing angle.
Now we can follow Graham’s scan to find out which points create the convex
hull. Point 0 is pushed onto the stack.
Point 1 is pushed onto the stack immediately after.
The next point to be added to the stack is 2. A line is made from point 1 to point
2.
Whenever a left turn is made, the point is presumed to be part of the convex hull.
We can clearly see a left turn being made to reach 2 from point 1. To get to point
3, another left turn is made. Currently, point 3 is part of the convex hull. A line
segment is drawn from point 2 to 3 and 3 is pushed onto the stack.
We make a right turn going to point 4. We’ll draw the line to point 4 but will not
push it onto the stack.
Whenever a right turn is made, Graham’s scan algorithm pops the previous value
from the stack and compares the new value with the top of the stack again. In
this case, we’ll pop 3 from the top of the stack and we’ll see if going from point
2 to point 4 creates a left bend. In this case it does, so we’ll draw a line segment
from 2 to 4 and push 4 onto the stack.
Since going from 4 to 5 creates a left turn, we’ll push 5 onto the stack. Point 5 is
currently part of the convex hull.
Moving from point 5 to 6 creates a left-hand turn, so we’ll push 6 onto the stack.
Point 6 is currently part of the convex hull.
To get to point 7, we must make a right-hand turn at 6.

Point 6 is popped from the stack and the turn is examined from point 5 to point
7. Since we make a left hand turn from point 5 to point 7, we push point 7 onto
the stack.
We attempt to push point 8 onto the stack. To get to point 8, we make a left at
point 7, therefore point 8 is added to the stack. Point 8 is currently part of the
convex hull.
Going to point 9 requires a right-hand turn at point 8.

Since there’s a right-hand turn, point 8 is popped from the stack and point 9 is
compared with point 7.
To get to point 9 from point 7 requires another right-turn, so we pop point 7 from
the stack too and compare point 9 to point 5. We make a left-hand turn at point 5
to get to point 9, so 9 is pushed onto the stack.
Next, we make a left turn to get to point 10. Point 10 is currently part of the
convex hull.
A right turn is required to get to point 11 from point 10.

Since a right turn is taken at point 10, point 10 is popped from the stack and the
path to point 10 from point 9 is examined. Since a left turn is made at point 9 to
get to point 11, point 11 is pushed onto the stack.
A left turn is made at point 11 to get to point 12. Point 12 is therefore pushed
onto the stack and is currently considered part of the convex hull.
A right turn is required to go to point 13 from point 12.
Point 12 is popped from the stack and the path to point 13 from point 11 is
examined. Since a left turn is made at point 11, point 13 is pushed onto the stack.
A left turn is made at point 13 to get to point 14, so point 14 is pushed onto the
stack.
A right turn is required to go from point 14 to point 15.
Since a right turn was made at point 14, point 14 is popped from the stack. The
path to point 15 from point 13 is examined next. A left turn is made at point 13
to get to point 15, so point 15 is pushed onto the stack.
Going from point 15 to the starting point 0 requires a left turn. Since the initial
point was the point that we needed to reach to complete the convex hull, the
algorithm ends.
The points that are needed to create the convex hull are 0-1-2-4-5-9-11-13-15.
Chapter 33: Closest Pair of Points on a Plane – Divide
and Conquer
If we’re given a set of points that lie on a plane, how can we figure out which
pairs are the closest pair of points on the plane? The points are frequently stored
in an array. One approach is to take the divide and conquer method. How? By
following the steps below:
Count the number of points on the plane

Divide the points into two equal groups and draw a line through the
middle. If there are an odd number of points, a line will go through at
least one of the points.
Get the smallest distance between two points on the left, and the
smallest distance between two points on the right.
Whatever the smallest distance is, δ, create an area to the left of the
midline and to the right of the midline that’s equal to that distance.
Start at the bottom of the newly created area and compare the points
that are on the left with the points that are on the right. Note, not all
points have to be compared.
Let’s look at an example. We have a plane with 16 points.
Step one says to draw a line down the middle so that the left plane and the right
planes can be created. There are 8 points on the left and 8 points on the right.
Next, we’ll find the closest pair of points on the left and the closest pair of points
on the right
Next, we’ll find the smallest distance of the two. Since they’re both equal to 2,
we’ll set δ = 2.
An area to the left of the midline with δ = 2 is shaded as well as an area to the
right of the midline.
We’ll start with the bottom point and move our way up. Only points that are not
in the same area will be compared since the points that are in the same area have
already been compared. Comparing the bottom point with the point next up in
the opposite side yields a distance of 3. That point will not be included. There’s
no point in comparing other points from the bottom point since we already know
that they’re further away.
We move to the second point.
The next closest point to the second point is in the same region, so we move to
the third point. The next closest point on the opposite side is 2 units away. Since
it’s within the smallest distance constraint, we’ll include that point in calculating
the closest distance between points.
We move to point 4 and find the distance to point 5 on the left. The distance is 5
so that path is excluded.
Finally, we find the distance from point 5 on the left to point 6 on the right. The
distance is 3, so that path is excluded as well.
We reach the final point. There are no additional points that we can find the
distance to, so the algorithm ends. The closest distance between two points is 2.
Chapter 34: Voronoi graph and Delaunay Triangulation
Voronoi diagrams help us find the closest points from an arbitrary point. Let’s
start by displaying a set of points on a plane.
The Voronoi diagram draws boundaries between each of those points. Around
each point, a region will develop called Voronoi regions. Let’s see how we
would draw those boundaries. Each boundary should be in the middle of two
points. Let’s begin by drawing a boundary between point 1 and point 2.
The boundary should be directly in the middle separating the two points. Let’s
move to creating a boundary between points 2 and 3.
Wherever the points intersect, we’ll remove that line segment.
We’ll create the boundary between point 2 and 6 next.

We also have to create a boundary between point 1 and point 6. This will require
trimming on a couple of different borders.
The final product looks like the following.
Let’s move on and create a border between 2 and 7.
Next, we’ll create a border between points 6 and 7.


Let’s draw a border between points 5 and 9.
A couple of more points to go. We’ll draw the boundary for points 3 and 5 next.
We’ll draw a boundary between points 4 and 5.
From the looks of the diagram, there are a couple of boundaries that were not
accounted for, the boundary between point 1 and 3 and the boundary between
point 8 and 9. Let’s quickly adjust those and complete the Voronoi diagram.

Once we have the regions defined, we can draw the lines that make up the
Delaunay triangulation. The simple rule of thumb is, if the points share a border,
draw a line connecting them. Point 1 shares a boundary with points 2, 3, and 6,
so we’ll draw lines connecting those points.
Although it may look like the line crosses two boundary lines when going from
point 1 to 3, we can quickly modify that line to show where it will cross the
border only once.
Point 2 shares boundaries with unconnected points 3, 6, and 7; it’s already
connected to point 1.
Point 3 shares boundaries with unconnected points 4, 5, and 7.

Point 4 shares a boundary with the unconnected point 5.
Point 5 shares boundaries with unconnected points 7 and 9.


Point 9 is already connected to all other points that share its boundary, so this
completes the construction of the Delaunay triangles. If we knew the weight of
each edge, we could construct a minimum spanning tree after the creation of the
Delaunay triangles.
How would be find the Voronoi diagram for a set of points in a rectilinear
metric? Let’s start with a small set of points on a grid.
The distance from p1 to p2 is 4 units regardless of the path that you take.
The middle point going through either route is 2 units away.
We’ll start by drawing a line connecting those two points.

To separate the points equally, we’ll draw lines protruding from each endpoint
that maximizes the area of each point.
Next, we’ll examine the boundary for points 1 and 3. This will create a border
similar to the one separating points 1 and 2. Again, regardless of the path that
you take, to get to the midpoint between those two points will require 2 units of
travel.
We’ll connect those two points and will shift the bottom vertical line.
Next, we must draw a boundary between points 2 and 3. The midpoint is 1 unit
away, so a horizontal line will be drawn separating those two points.
A border needs to be constructed between points 2 and 4. The distance to point 4
from point 2 is 6 units. The midpoint of the trajectory is 3 units from points 2
and 4.
We will draw a line connecting those two points and will draw vertical lines
from the endpoints.
There is still the boundary between points 3 and 4 that needs to be constructed.
Since the distance between points 3 and 4 is the same as the distance between
points 2 and 4, a similar approach is taken to construct the boundary between
them. The midpoint is 3 units away, so we’ll draw a line that connects the
midpoints from the top and from the bottom. The right-bottom, vertical line will
be moved to the new endpoint.
We can again construct the Delaunay triangles by connecting the points that
share a boundary. Point 1 shares boundaries with points 2 and 3.
All points are now connected to their immediate neighbors. This completes the
Delaunay triangulation. Since we know the distance to each point, we can
construct a minimum spanning tree. There are a couple of variations of the
minimum spanning tree for this problem, but we’ll choose one. The total length
of this MST is 12.
Chapter 35: Maximum Independent Set

Given a graph, a subset of the vertices is an independent set if there are no edges
between the vertices in the subset. Let’s take a look at a quick example to see
what this means.
We can see that there is an edge between the following points:
Points 1 and 2
Points 1 and 3
Points 1 and 4
Points 2 and 3
Points 2 and 5
Points 3 and 5
Points 4 and 5
Points 4 and 6
Points 5 and 6
When trying to find the maximum independent set, we’ll try to find the
maximum amount of points that are not connected directly. Where do we start?
Usually, we’ll look for a point that is connected to two other points. In this case,
the only point that meets the criteria is point 6. Point 6 is connected to points 4
and 5.
We know that since points 4 and 5 are connected to point 6, they cannot be part
of the maximum independent set, and any edge connecting those points is
severed.
We can see that there are three points left in the graph: points 1, 2, and 3. If we
choose either of the points, the other two points will automatically not be part of
the maximum independent sets. That’s fine since there can be multiple maximum
independent sets. For this example, we’ll choose vertex 1. Vertices 2 and 3 will
be crossed out and all edges to which they’re connected to will be severed. The
maximum independent set for this example is 1,6.
Chapter 36: Minimum Vertex Cover
To find the minimum vertex cover, you’re trying to find the minimum number of
vertices that will include all the edges.
We can start with the densest vertex on the graph and hope that we get lucky.
We can choose vertex 3 next. Through some trial and error, you’ll see that you
will have to choose 2 points from points 1, 2, and 3 regardless of which point
you choose next.
We can choose vertex 1 next to get the edges connecting 1 and 2 and 1 and 4.
Finally, we must choose either vertex 4 or vertex 6. We’ll choose vertex 4. Since
all the edges are now included, this completes the minimum vertex cover. The
minimum vertex cover consists of vertices 1, 3, 4, and 5.
Chapter 37: Maximum Clique
Maximum clique is a technique used to find the largest vertex cluster where each
vertex is connected to each other. Let’s look at an example. We’ll examine each
vertex and see what the greatest cluster of the following graph will be.
Let’s look at the first vertex. Vertex 1 is connected to vertices 2, 4, and 6. We

need to make sure that each of those vertices has a connection to one another as
well.
Is vertex 2 connected to vertex 4? Yes

It looks like all of the vertices are connected to each other, so the current
maximum clique that we’re observing is composed of 4 vertices: 1, 2, 4, and 6.
Let’s examine vertex 2. Vertex 2 is connected to vertices 1, 4, 3, and 5. We need
to make sure that each of those vertices has a connection to one another as well.
Is vertex 1 connected to vertex 3? No

There’s no point in continuing. It looks like we cannot use that vertex as the
center-point of the max-clique creation. To have vertex 2 be the center-point of a
maximum clique, all the vertices would need to be connected as is illustrated
below.
Let’s examine vertex 3. Vertex 3 is connected to vertices 2 and 8.

To be part of a clique, the edge connecting vertex 2 to 8 would need to exist.
Let’s examine vertex 4. Vertex 4 is connected to vertices 1, 2, 4, 5, and 6. We

well.

We cannot use vertex 4 as the center-point of the max-clique creation. To create
a clique, vertex 1 must have an edge to vertex 5.

well.
Is vertex 4 connected to 6? Yes
Is vertex 4 connected to 7? No
a clique, all the vertices would need to be connected as is illustrated below.

well.

a clique, all the vertices would need to be connected as is illustrated below.
Let’s examine vertex 7. Vertex 7 is connected to vertices 5 and 6. We need to
make sure that each of those vertices has a connection to one another as well.

Vertices 5, 6, and 7 do form a clique, however, we already found a larger clique
than this one, so this is not a maximum-clique.
Let’s examine vertex 8. Vertex 8 is connected to vertices 3 and 5. We need to

make sure that each of those vertices has a connection to one another as well.
a clique, vertex 3 would need to be connected to vertex 5.
We have completed looking at each of the vertices. The maximum clique that we
were able to find was the one consisting of vertices 1, 2, 4, and 6.
References
Cormen, Thomas H., et al. Introduction to Algorithms. Mit Press, 2009.
Sedgewick, Robert, and Kevin Wayne. Algorithms. Pearson Education, 2011.
Bhargava, Aditya Y. Grokking Algorithms: an Illustrated Guide for
Programmers and Other Curious People. Manning., 2016.
Bentley, J. L., et al. A General Method for Solving Divide-and-Conquer
Recurrences. Carnegie-Mellon University. Department of Computer Science,
1978.
Knuth, Donald E. The Art of Computer Programming. Addison-Wesley, 1998.
Dijkstra, E. W. “A Note on Two Problems in Connexion with Graphs.”
Numerische Mathematik, vol. 1, no. 1, 1959, pp. 269–271.,
doi:10.1007/bf01386390.
Tarjan, Robert E. Data Structures and Network Algorithms. Society for
Industrial and Applied Mathematics, 1983.
Boyer, John M., and Wendy J. Myrvold. “On the Cutting Edge: Simplified O(n)
Planarity by Edge Addition.” Graph Algorithms and Applications 5, vol. 8, no.
3, 2004, pp. 241–273., doi:10.1142/9789812773289_0014.

An Illustrative Introducction To Algorithms

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

An Illustrative Introducction To Algorithms

Hochgeladen von

Copyright:

Verfügbare Formate

Copyright

© 2019 by Dino Cajic

First Edition 2019

If c < logb a, then T(n) = Ө(nlog b a )

T(n) = 16T( ) + 9999n2

Therefore, T(n) = Ө(nlog b a ) = Ө(n4 )

The algorithm starts at index position 1.

Chapter 6: Divide and Conquer

The best way to understand the divide-and-conquer method is with an example.

n represents the index of the parent node

Next, 7 is compared to 2. Since 2 is smaller than 7, the nodes are swapped.

A comparison is made between the two children. Since 3 is smaller, 2 is

Next, element 9 is added. Since 9 is larger than 2, no swapping is necessary.

Since 4 is smaller than 7, the two nodes are swapped.

Finally, element 3 is added.

Element 3 is also compared to element 2. Since element 3 is larger than element

We’ll first construct the heap.

9.5 Constructing Max-Heap from an Array

We start by adding the first node, 7.

We move top-to-bottom, left-to-right and we add the 2nd node, 2.

Next, 4 is added as a child of 2.

We observe that 4 moved up, so 4 is also compared to 9. Considering that 4 is

Since 5 is larger than 4, the two values are swapped.

Chapter 10: Heap Sort

The first step is to construct a tree from the array.

This completes the heap sort process.

Next, 7 is compared to 1. Number 1 is still smaller.

The first comparison of the second iteration is between 5 and 2. Since 2 is

The index is incremented and 4 is compared to the minimum value. Since 4 is

A comparison is made between 7 and 5. Since 5 is less than 7, the minimum

Although the graph looks complicated, creating an adjacency matrix is a simple

Finally, we examine vertex 6. Vertex 6 has 3 outbound edges.

Chapter 13: Adjacency List

Vertex 3 points to 2 and 5.

Vertex 5 points to vertices 1, 3, 4, and 6.

By observation, we can quickly see that a significant amount of space is saved

Chapter 14: Edge List

Vertex 3 has two outbound edges: to vertex 2 and vertex 5.

Vertex 4 has one outbound edge to vertex 6.

Vertex 5 has 4 outbound edges:

Chapter 15: Depth First Search (DFS)

Select the first node

There are no additional, undiscovered, adjacent nodes, so this completes the

Chapter 18: Greedy Algorithms

1. Get a job that only requires a high school diploma

Chapter 19: Minimum Spanning Trees (MST)

Next to each node, we’ll put a couple of reminders for ourselves:

We’ll begin with the root node.

We’ll return to visit/print node H.

We can now remove node 40 from the tree.

Chapter 24: Planar Graphs

Do we have to attempt to make the planar graph by redrawing it every single

Node 2 is also directly connected to node 3.

We need node 2 to be connected to one additional node. It looks like if we follow

Moving on to node 5, it’s connected directly to node 9.

Node 7 is connected directly to node 8.

And finally, node 8 is connected directly to node 9.

Chapter 26: All-Pairs Shortest Path Matrix Multiplication

Let’s start off by highlighting the first row and second

The minimum is 2 so we update A2 (1,2) with 2.

C13 = min(0 + ∞ , 10 + ∞ , ∞ + 0, 5 + 1) = min( ∞ , ∞ , ∞ , 6) = 6

The minimum is 6 so we update A2 (1,3) with 6. A2 (1,4) is next.

C14 = min(0 + 5, 10 + 9, ∞ + ∞ , 5 + 0) = min(5, 19, ∞ , 5) = 5