Heuristic search

Points
Definitions
Best-first search
Hill climbing
Problems with hill climbing
An example: local and global
heuristic functions

Simulated annealing
The A* procedure
Means-ends analysis

CSI 4106, Winter 2005

Heuristic search, page 1

Definitions
Heuristics (Greek heuriskein = find, discover): "the study
of the methods and rules of discovery and invention".
We use our knowledge of the problem to consider some
(not all) successors of the current state (preferably just
one, as with an oracle). This means pruning the state
space, gaining speed, but perhaps missing the solution!
In chess: consider one (apparently best) move, maybe a few -- but
not all possible legal moves.
In the travelling salesman problem: select one nearest city, give up
complete search (the greedy technique). This gives us, in polynomial
time, an approximate solution of the inherently exponential problem; it
can be proven that the approximation error is bounded.

CSI 4106, Winter 2005

Heuristic search, page 2

Definitions (2)
For heuristic search to work, we must be able to rank
the children of a node. A heuristic function takes a state
and returns a numeric value -- a composite assessment
of this state. We then choose a child with the best score
(this could be a maximum or minimum).
A heuristic function can help gain or lose a lot, but
finding the right function is not always easy.
The 8-puzzle: how many misplaced tiles? how many
slots away from the correct place? and so on.
Water jugs: ???
Chess: no simple counting of pieces is adequate.

CSI 4106, Winter 2005

Heuristic search, page 3

Definitions (3)
The principal gain -- often spectacular -- is the
reduction of the state space. For example, the
full tree for Tic-Tac-Toe has 9! leaves. If we
consider symmetries, the tree becomes six
times smaller, but it is still quite large.
With a fairly simple heuristic function we can
get the tree down to 40 states. (More on this
when we discuss games.)
Heuristics can also help speed up exhaustive,
blind search, such as depth-first and breadthfirst search.

CSI 4106, Winter 2005

Heuristic search, page 4

Best-first search
The algorithm
select a heuristic function (e.g., distance to the goal);
put the initial node(s) on the open list;
repeat
select N, the best node on the open list;
succeed if N is a goal node; otherwise put N on the
closed list and add N's children to the open list;
until
we succeed or the open list becomes empty (we fail);
A closed node reached on a different path is made open.
NOTE: "the best" only means "currently appearing the best"...

CSI 4106, Winter 2005

Heuristic search, page 5

Hill climbing
This is a greedy algorithm: go as high up as possible
as fast as possible, without looking around too much.
The algorithm
select a heuristic function;
set C, the current node, to the highest-valued initial
node;
loop
select N, the highest-valued child of C;
return C if its value is better than the value of N;
otherwise set C to N;

CSI 4106, Winter 2005

Heuristic search, page 6

Problems with hill climbing

1.

Local maximum, or the foothill problem: there is a

peak, but it is lower than the highest peak in the
whole space.

2.

The plateau problem: all local moves are equally

unpromising, and all peaks seem far away.

3.

The ridge problem: almost every move takes us

down.
Random-restart hill climbing is a series of hillclimbing searches with a randomly selected start
node whenever the current search gets stuck.

See also simulated annealing -- in a moment.

CSI 4106, Winter 2005

Heuristic search, page 7

A hill climbing example

Goal

state

CSI 4106, Winter 2005

Initial

state

Move

Move

Move

2a

2b

Heuristic search, page 8

A hill climbing example (2)

A local heuristic function
Count +1 for every block that sits on the correct
thing. The goal state has the value +8.
Count -1 for every block that sits on an incorrect
thing. In the initial state blocks C, D, E, F, G, H
count +1 each. Blocks A, B count -1 each , for the
total of +4.
Move 1 gives the value +6 (A is now on the correct
support). Moves 2a and 2b both give +4 (B and H
are wrongly situated). This means we have a local
maximum of +6.

CSI 4106, Winter 2005

Heuristic search, page 9

A hill climbing example (3)

A global heuristic function
Count +N for every block that sits on a correct
stack of N things. The goal state has the value
+28.
Count -N for every block that sits on an incorrect
stack of N things. That is, there is a large
penalty for blocks tied up in a wrong structure.
In the initial state C, D, E, F, G, H count -1, -2,
-3, -4, -5, -6. A counts -7 , for the total of -28.

continued

CSI 4106, Winter 2005

Heuristic search, page 10

A hill climbing example (4)

Move 1 gives the value -21 (A is now on the
correct support).
Move 2a gives -16, because C, D, E, F, G, H
count -1, -2, -3, -4, -5, -1.
Move 2b gives -15, because C, D, E, F, G
count -1, -2, -3, -4, -5.
There is no local maximum!
Moral: sometimes changing the heuristic
function is all we need.

CSI 4106, Winter 2005

Heuristic search, page 11

Simulated annealing
The intuition for this search method, an
improvement on hill-climbing, is the
process of gradually cooling a liquid.
There is a schedule of "temperatures",
changing in time. Reaching the "freezing
point" (temperature 0) stops the process.
Instead of a random restart, we use a
more systematic method.

CSI 4106, Winter 2005

Heuristic search, page 12

Simulated annealing (2)

The algorithm, one of the versions

select a heuristic function f;

set C, the current node, to any initial node;
set t, the current time, to 0;
let schedule(x) be a table of "temperatures;
loop
t = t + 1;
if schedule(t) = 0, return C;
select at random N, any child of C;
if f(N) > f(C), set C to N, otherwise
set C to N with probability e(f(N)-f(C))/schedule(t);

CSI 4106, Winter 2005

Heuristic search, page 13

A few other search ideas

Stochastic beam search:
select w nodes at random; nodes with higher
values have a higher probability of selection.
Genetic algorithms:
generate nodes like in stochastic beam search,
but from two parents rather than from one.
(This topic is worthy of a separate lecture...)

CSI 4106, Winter 2005

Heuristic search, page 14

The A* procedure
Hill-climbing (and its clever versions) may miss an optimal solution.
Here is a search method that ensures optimality of the solution.
The algorithm
keep a list of partial paths (initially root to root, length 0);
repeat
succeed if the first path P reaches the goal node;
otherwise remove path P from the list;
extend P in all possible ways, add new paths to the list;
sort the list by the sum of two values: the real cost of P till now,
and an estimate of the remaining distance;
prune the list by leaving only the shortest path for each node
reached so far;
until
success or the list of paths becomes empty;

CSI 4106, Winter 2005

Heuristic search, page 15

The A* procedure (2)

A* requires a lower-bound estimate of the
distance of any node to the goal node. A
heuristic that never overestimates is also
called optimistic or admissible.
We consider three functions with values 0:
g(n) is the actual cost of reaching node n,
h(n) is the actual unknown remaining cost,
h'(n) is the optimistic estimate of h(n).

CSI 4106, Winter 2005

Heuristic search, page 16

The A* procedure (3)

Here is a sketchy proof of the optimality of the goal node
found by A*. Let m be such a non-optimal goal:
g(m) > g(nk)
and let n0, n1, ..., nk be a path leading to nk.
Since m is a goal, h(m) = 0 and thus h'(m) = 0.
Every step of A* selects the next ni in such a way that
g(ni) + h'(ni) g(ni) + h(ni) = g(nk) < g(m) = g(m) + h'(m)
In other words, g(ni) + h'(ni) < g(m) + h'(m), so that A*
cannot have selected m.

CSI 4106, Winter 2005

Heuristic search, page 17

The A* procedure (4)

A simple search problem: S is the start node, G is the
goal node, the real distances are shown.
3

A
5

5
2

3.5

A lower-bound estimate of the distance to G could be

as follows (note that we do not need it for F):
10.1

11.5

3.4

9.2

CSI 4106, Winter 2005

B 5.8

7.1

3.5

Heuristic search, page 18

Hill-climbing
happens to
succeed here:

The A* procedure (5)

S
10.1

9.2

D
10.1

11.5

S
5.8

CSI 4106, Winter 2005

7.1

E
9.2

3.5

7.1

0.0

Heuristic search, page 19

S
3+10.1

A*, step 1

CSI 4106, Winter 2005

The A* procedure (6)

4+9.2

Heuristic search, page 20

A*, step 2

13.1

13.2

3+4+5.8

CSI 4106, Winter 2005

The A* procedure (7)

3+5+9.2

Heuristic search, page 21

A*, step 3
12.8

B
3+4+4+3.4

CSI 4106, Winter 2005

The A* procedure (8)

13.1

13.2

17.2

D
3+4+5+7.1

Heuristic search, page 22

S
13.1

13.2

A*, step 4
12.8

B
14.4

CSI 4106, Winter 2005

The A* procedure (9)

17.2

4+5+10.1

19.1

4+2+7.1

Heuristic search, page 23

S
13.1

13.2

A*, step 5
12.8

B
14.4

17.2

19.1

19.1

13.1

E
4+2+5+5.8

CSI 4106, Winter 2005

The A* procedure (10)

4+2+4+3.5

Heuristic search, page 24

S
13.1

13.2

A*, step 6
12.8

B
14.4

The A* procedure (11)

17.2

19.1

19.1

13.1

E
16.8

13.5

CSI 4106, Winter 2005

Heuristic search, page 25

CSI 4106, Winter 2005

m(G)

d(x)

=4

=2

m(K)

d(x)

=4

=3

d(x)

=1

=4

=0

=1

m(L)

d(x)

=5

m(M)

The A* procedure (12)

The 8-puzzle with

optimistic heuristics.

=2

m(D)

m(J)

=4

=3

m(I)

=3

m(F)
1

m(H)

=3

=3

m(C)

m(E)

=0

=5

d(x)

=4

m(B)
1

m(A)

m(N)

d(x)

=2

=5

A, ..., N: states;
m = misplaced;
d = depth;
f(x) = m(x) + d(x).

Heuristic search, page 26

Means-ends analysis
The idea is due to Newell and Simon (1957): work by
reducing the difference between states, and so
approaching the goal state. There are procedures,
indexed by differences, that change states.
The General Problem Solver algorithm
set C, the current node, to any initial node;
loop
succeed if the goal state has been reached, otherwise
find the difference between C and the goal state;
choose a procedure that reduces this difference, apply
it to C to produce the new current state;

CSI 4106, Winter 2005

Heuristic search, page 27

Means-ends analysis (2)

An example: planning a trip.

P
miles

D
> 1000
i
s
100-1000
t
a 10-100
n
c
<1
e
Prerequisite

CSI 4106, Winter 2005

Plane

Train

Car

Taxi

e
Walk

X
X

At plane At train

At car

At taxi

Heuristic search, page 28