Lecture 3

Engineering Systems Design and Analysis
(ENG 504)
504)
Lecture 3
/

Email: amhm@zu.edu.eg / amhm162@gmail.com
Webpage: http://www.staff.zu.edu.eg/amhm/
In the previous lectures
AI Paradigms.
Solving Problems by Search
Tree Search Algorithms (just names!)
Local Search Algorithms
Genetic Algorithms
Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)
Zagazig Univ-Eng. Faculty
2013/2014
Agenda
Tree Search Algorithms:

Depth-first search
Branch and bound algorithm
Examples:
Travelling Salesman Problem
Scheduling Problems
2013/2014
Introduction
Travelling Salesman Problem (TSP):

Five cities: A,B,C,D,E
B
A
Given: cost of travelling between each pair.
1
8
D
4
C
2
3
E
Goal: find the cheapest tour starting from A

then passes all other cities once, then back to A.
2013/2014
Introduction
4
Solutions of TSP?
You may select the cheapest way out of each city until
you are back in the origin.

(Greedy search, only one iteration, but not always the best solution)
You may test all possible tours, then select the cheapest one.
(Exhaustive search, so expensive)
You may setup only the first possible tour, then search for other tours
that could be better than the current best one.
(Branch and Bound, will save effort in most cases!)
You can also try genetic algorithms. (may succeed)
2013/2014
Introduction
5
Solutions of TSP?
Branch and Bound: is an enhanced version of the so-called back tracking search.
It can be applied when the solution can be represented by a search tree:
root A
node
node: partial solution
Father & child ?
Leaf: one complete solution

2013/2014
Introduction
6
Solutions of TSP?
Branch and Bound: methodology:
For each node, if the current partial solution is not

promising, then back to the father node and test
another child.
Always keep the best complete solution so far.
Thus we need first to know how to traverse a tree in (a depth first fashion
vs. breadth first fashion).
Depth--first search
Depth
2013/2014
Depth--first search (DFS)

Depth
Methodology: starts at the root and explores as far as possible

(until one leaf) along each branch, then back to the first father node and
branch to another child.
Suppose that this tree is already constructed

Numbers show the order of traversing nodes
2013/2014

Depth
Pseudo-code:
DFS(G,v) ( v is the vertex where the search starts )
Stack S := {}; Visited V:={}
push (S, v);
While (S is not empty) do
u := pop (S);
If ( u is not in V) Then
insert (V, u);
for each neighbor w of u that is not in V, push (S, w);
End If
End While
END DFS()
2013/2014

Depth
Notice:
Stack: is a data structure (like an array for ex.) where elements that enters
the stack first will leave it at last also (LIFO order).
push(S,v): inserts the element v in the stack S.
pop(S): returns or gets the first element in the stack S.
insert(V,u): is a function that puts the vertex u in the array V.

By the end of DFS(): the array V contains the visited vertices in order.

Depth
2013/2014
10
Example #1: show how the DFS algorithm is executed?

That means that you have to follow every
step in the shown pseudo code.
root A
For each iteration, show:

D
contents of the stack S

contents of the array visited V
when DFS() stops.
C
E
2013/2014

Depth
11
Example #1: solution:

initially: S={ }, V={ }
root A
iteration #1: S={A}

u=A
S={ }
V={A}, S={B,C}
iteration #2: u=B
S={C}
V={A,B}, S={C,D}
iteration #3: u=C
S={D}
V={A,B,C}, S={D,D}
iteration #4: u=D
S={D}
V={A,B,C,D}, S={D,E}
iteration #5: u=D
S={E}
D has been visited once before !
iteration #6: u=E
S={ }
V={A,B,C,D,E} .While ends and thus DFS() stops.

Depth
2013/2014
12
Exercise #1: show how the DFS algorithm can be executed?
B
root A
E
D
C
2013/2014
Breadth--first search (BFS)

Breadth
13
Uses a queue data structure: FIFO.

Generates first all children of a node then move to another node
unless it is a leaf.
Breadth--first search (BFS)

Breadth
2013/2014
14
Exercise #1: show how the BFS algorithm can be executed?
B
root A
E
D
C
2013/2014
Branch and Bound

Algorithm
Branch and Bound Algorithm (B&B)
15
Methodology:
Methodology
For each node, if the current partial solution is not
promising, then back to the father node and test
another child. (prune this subtree)
Always keep the best complete solution so far.
Here we need two functions:

Cost function: evaluates a complete solution.
Bound function: estimates a lower bound of the cost of a partial
solution at some node.
2013/2014
16
Branching: refers to exploring the subtree of an active node

Bounding: refers to estimating a bound on the solutions in the subtree
rooted at the active node.
root
19
unfeasible
Cost=17
One solution
15
Best solution
prune
Main steps of a basic B&B
2013/2014
17
1.
Compute a lower bound (lb) for the starting node(s).
2.
Insert partial solutions (PS) in memory and rank them from best to worst w.r.t. the lb.
3.
Remove PS with rank 1 (and re-rank others).
4.
If lb(PS) is greater than cost of best solution so far (BSSF) then

4.1 Remove it from memory (and re-rank others).
4.2 Go to step 3.
5.
Generate each feasible solution starting from the removed PS.
6.
If this PS leads to only one solution then

5.1 Complete this solution and compute the cost.
5.2 Update BSSF.
7.
Repeat steps 2-> 6 until the memory of PSs is empty.
8.
Report BSSF as the optimal (best) solution.
2013/2014
18
Example #2: find the shortest tour such that B must be visited
before C?
It is easier for you to try to apply your technique
for such small-scale problem before
implementing a computer program
D
4
to do the job for you.
C
2
So it is required in this example to build
3
E
the search tree for this graph using B&B technique.
2013/2014
19
Example #2
#2: solution:
First step is always to find a clever bound function that can help to
prune as many as possible branches in the search tree.
Each class of problems may require different bound strategy.
For TSP, the following function works (in most cases):
The initial bound for the minimum tour is the sum of the minimum
outgoing edges from each vertex.
Lower bound: lb(u,v) = Length from u to v + sum of min outgoing
edges for vertices from v to root node.
2013/2014
Example #2
#2: solution:
20
A
1
D,E
7
15
E,D
24
C
2
12
16
E,C
19
10
D
-
10
13
14
14
18
11
17
C,E
16
12
B
18
E
3
B,C
24
14
24
BSSF
numbers in left cell show order of traversing.

numbers in right cell show the lb,
cost of complete solutions are in red.
Pruned PSs are in brown.
2013/2014
21
Example #2
#2: solution:
The cheapest tour (with minimum cost) = {A,B,D,E,C}
Computations of lower bounds
lb(A)=1+3+1+1+3+2=10
lb
rank evolution
PS
10 A
12 A,B
lb(A,D)=5+3+1+3+2=14
14 A,D
2 ,1
lb(A,B,C)=9+1+3+2=15
17 A,E
3,4 ,3, 2, 1
Cost(A,D,E,B,C) = 24
15 A,B,C
2 ,1
PSs in memory are listed according to entering time.
16 A,B,D
3, 2 ,1
By the end of execution, this memory is empty.
18 A,B,E
5, 4 ,3, 2
18 A,D,B
6 ,5 ,4,3
14 A,D,E
lb(A,B)= 3+3+1+3+2=12
Removed and pruned because the lb > cost of current BSSF.
Memory
2013/2014
22
Exercise #2
#2:
Given n tasks and n agents.
Each agent has a cost to complete each task
Assign each agent one task
Minimize cost
Example of the solution:
(C,1),(A,2),(D,3),(B,4)
1
11
14
11
17
A
B
C
D
2
12
15
17
14
3
18
13
19
20
4
40
22
23
28
2013/2014
23
Exercise #3
#3:
Starting from A, find the shortest tour for this graph.
Construct the search tree using B&B algorithm.
Show why you may do pruning if this is the case at
any step.
What is the order of traversing the vertices?
2013/2014
Thanks

Lecture 3

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Lecture 3

Hochgeladen von

Copyright:

Verfügbare Formate

Engineering Systems Design and Analysis

In the previous lectures

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Tree Search Algorithms:

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Travelling Salesman Problem (TSP):

Given: cost of travelling between each pair.

Goal: find the cheapest tour starting from A

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

you are back in the origin.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

node: partial solution

Father & child ?

Leaf: one complete solution

Zagazig Univ-Eng. Faculty

Branch and Bound: methodology:

For each node, if the current partial solution is not

Zagazig Univ-Eng. Faculty

Depth--first search (DFS)

Methodology: starts at the root and explores as far as possible

Suppose that this tree is already constructed

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Depth--first search (DFS)

Zagazig Univ-Eng. Faculty

Depth--first search (DFS)

insert(V,u): is a function that puts the vertex u in the array V.

Zagazig Univ-Eng. Faculty

Depth--first search (DFS)

Example #1: show how the DFS algorithm is executed?

For each iteration, show:

contents of the stack S

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Depth--first search (DFS)

Example #1: solution:

iteration #1: S={A}

iteration #2: u=B

iteration #3: u=C

iteration #4: u=D

iteration #5: u=D

iteration #6: u=E

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Depth--first search (DFS)

Exercise #1: show how the DFS algorithm can be executed?

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Breadth--first search (BFS)

Uses a queue data structure: FIFO.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Breadth--first search (BFS)

Exercise #1: show how the BFS algorithm can be executed?

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Branch and Bound

Branch and Bound Algorithm (B&B)

Here we need two functions: