Algorithm Design Methods

Decision Problems & complexity class

❂ A decision problem is a problem where the answer is
always yes or no.
❂ A complexity class is the set of all of the computational
problems which can be solved using a certain amount of a
certain computational resource.
P, NP and NP-complete problems
❂ The complexity class P is the set of decision problems that
can be solved by a deterministic machine in
polynomial time. This class corresponds to an intuitive idea
of the problems which can be effectively solved in the worst
cases.
❂ Problems that can be solved in polynomial time is called
tractable.
❂ Problems that cannot be solved in polynomial time is called
intractable.
NP PROBLEMS
❂ The complexity class NP is the set of decision problems that
can be solved by a non-deterministic machine in polynomial
time. This class contains many problems that people would
like to be able to solve effectively, including the
Boolean satisfiability problem, the
Hamiltonian path problem and the vertex cover problem.
All the problems in this class have the property that their
solutions can be checked efficiently.
Contd…
❂ The halting problem is a decision problem about properties
of computer programs on a fixed Turing-complete model of
computation. The question is, given a program and an
input to the program, whether the program will eventually
halt when run with that input. In this abstract framework,
there are no resource limitations of memory or time on the
program's execution; it can take arbitrarily long, and use
arbitrarily much storage space, before halting. The
question is simply whether the given program will ever halt
on a particular input.
Contd…
❂ int main(void) { while (1); } – doesnot halt goes in infinite
loop
❂ int main(void) { printf("Hello, world"); return 0; } – halts
very quickly
❂ Are there decidable but intractable problems?
❂ Yes, in cryptography and AI
NP complete
❂ Problems are designated "NP-complete" if their solutions can be quickly
checked for correctness, and if the same solving algorithm used can solve all
other NP problems. They are the most difficult problems in NP in the sense that
a deterministic, polynomial-time solution to any NP-complete problem would
provide a solution to every other problem in NP (and conversely, if any one of
them probably lacks a deterministic polynomial-time solution, none of them has
one). Problems in NP-complete are known as NP-complete problems. A more
formal definition is given below.
❂ One example of an NP-complete problem is the subset sum problem which is:
given a finite set of integers, determine whether any non-empty subset of them
sums to zero. A supposed answer is very easy to verify for correctness, but
there is no known efficient algorithm to find an answer; that is, all known
algorithms are impractically slow for large sets of integers.
NP complete
❂ A decision problem D is NP-complete if:
❂ D is in NP, and
❂ Every problem in NP is reducible to D.
❂ Hamiltonian problem:
In the mathematical field of graph theory, a Hamiltonian
path is a path in an undirected graph which visits each
vertex exactly once. A Hamiltonian cycle (or Hamiltonian
circuit) is a cycle in an undirected graph which visits each
vertex exactly once and also returns to the starting vertex.
Determining whether such paths and cycles exist in graphs
is the Hamiltonian path problem which is NP-complete.
P, NP Complete and NP hard
NP hard
❂ A problem H is NP-hard if and only if there is an
NP-complete problem L that is
polynomial time Turing-reducible to H.
❂ An example of an NP-hard problem is the decision problem
SUBSET-SUM which is this: given a set of integers, does
any non empty subset of them add up to zero? That is a yes/
no question, and happens to be NP-complete. Another
example of an NP-hard problem is the optimization
problem of finding the least-cost route through all nodes of
a weighted graph. This is commonly known as the
Traveling Salesman Problem
Tackling Difficult Combinatorial Problems

There are two principal approaches to tackling difficult

combinatorial problems (NP-hard problems):

❂ Use a strategy that guarantees solving the problem exactly

but doesn’t guarantee to find a solution in polynomial time

❂ Use an approximation algorithm that can find an

approximate (sub-optimal) solution in polynomial time
 Exact Solution Strategies
❂ exhaustive search (brute force)
• useful only for small instances

❂ dynamic programming
• applicable to some problems (e.g., the knapsack problem)

❂ backtracking
• eliminates some unnecessary cases from consideration
• yields solutions in reasonable time for many instances but worst case is
still exponential

❂ branch-and-bound
• further refines the backtracking idea for optimization problems
Backtracking & Branch and Bound
❂ Improvement over exhaustive search
❂ It construct one candidate solution at a time and evaluate
the partially constructed solution; if no potential values of
the remaining components can lead to a solution, the
remaining is not generated.
❂ It uses state space tree.
❂ It terminates a node as soon as it can be guaranteed that no
solution to the problem can be obtained by considering
choices that corresponds to the descendants.
Comparison

Branch and bound Backtracking

❂ Applicable to ❂ Not constrained but
Optimization problem mostly applied to Non-
❂ The state space tree is Optimization problem
generated using best first ❂ The state space tree is
rule. generated using DFS
Backtracking
❂ Construct the state-space tree
• nodes: partial solutions
• edges: choices in extending partial solutions

❂ Explore the state space tree using depth-first search

❂ “Prune” nonpromising nodes

• dfs stops exploring subtrees rooted at nodes that cannot
lead to a solution and backtracks to such a node’s
parent to continue the search
Example: n-Queens Problem

Place n queens on an n-by-n chess board so that no two of them

are in the same row, column, or diagonal

1 2 3 4
1 queen 1
2 queen 2
3 queen 3
4 queen 4
State-Space Tree of the 4-Queens Problem
Hamiltonian circuit problem
❂ In the mathematical field of graph theory, a Hamiltonian
path is a path in an undirected graph which visits each
vertex exactly once. A Hamiltonian cycle (or Hamiltonian
circuit) is a cycle in an undirected graph which visits each
vertex exactly once and also returns to the starting vertex.
Determining whether such paths and cycles exist in graphs
is the Hamiltonian path problem which is NP-complete.
Example: Hamiltonian Circuit Problem

a b

c f

d e
Hamiltonian Circuit Problem
❂ It starts at vertex a.
❂ 3 adjacent nodes can be visited, tie is resolved, we selected
b.
❂ From b, algorithm proceeds c, then d and then e and finally
f, which is an dead end.
❂ Then Backtrack from f to e, then e to d, then d to c, which
provides first alternate to pursue. From c we can go to e
and f, which is also dead end, backtrack to e and from e we
can go to d.
❂ Backtrack to b, the alternate is f,e,c,d and then reach a.
Sum of Subset problem
❂ Problem statement: Let, S = {S1, ………..Sn} be a set of n
positive integers, then we have to find a subset whose sum is
equal to given positive integer d. For example, for
❂ S = {1, 2, 5, 6, 8} and d = 9, there are two solutions: {1, 2, 6}
and {1, 8}.
❂ It is always convenient to sort the sets elements in
ascending order. That is
❂ S1≤ S2 ≤ S3 ………………≤ Sn
❂ The state space tree can be constructed as a binary tree as
in fig. for instance S = {3, 5, 6, 7} and d=15.
Steps:
❂ Step 1: The root of the tree represents the starting point, with no decisions about the
Given elements.
❂ Step 2: Its left and right children represent, inclusion and exclusion of S1 in the set being
Sought.
❂ Step 3: Going to the left from a node of the first level corresponds to inclusion of S2,
While going to right corresponds to exclusion.
❂ Step 4: A path from the root to a node at the ith level of the tree indicates which of the
first i numbers have been included in the subsets represented by that node.
❂ Step 5: We record the value of s’, the sum of these numbers in the node. If s’ is equal to
d, we have a solution to the problem and stop. If all the solutions need to be
found, continue by backtracking to the node’s parent. If s’ is not equal to d, we
can terminate the node as nonpromising if either of the two equaities holds:
❂ s’ + si+1 > d (the sum’s too large)
n
s '+ ∑sj < d
j =i +1

❂ (the sum’s too small)

Sum of Subset problem

0
with 3 w/o 3

3 0
with 5 w/o 5 with 5 w/o 5

8 3 5 0
with 6 w/o 6 with 6 w/o 6 with 6 w/o 6 X
0+13<15

14 8 9 3 11 5
X with 7 w/o 7 X X X X
14+7>15 9+7>15 3+7<15 11+7>14 5+7<15
15 8
solution X
8<15