Formalizing the notion of satisficing

Shlomo Zilberstein
Department of Computer Science
University of Massachusetts
What is satisficing?
• A Scottish word which means satisfying.
• Proposed by Herbert Simon in 1957 to
denote decision making that searches
until an alternative is found that meets
the agent’s aspiration level criterion.
• Aspiration level is borrowed from
psychology, where it denotes a dynamic,
context-dependent criterion typically
acquired by experience.
2
Satisficing in nature
“It appears probable that, however adaptive the
behavior of organisms in learning and choice
situations, this adaptiveness falls far short of the
ideal “maximizing” postulated in economic theory.
Evidently, organisms adapt well enough to
“satisfice”; they do not, in general, “optimize”.”
• Is satisficing more than a vague principle?
Can it be formalized?
• What sort of satisficing models are
appropriate for computer systems?
3
Why use satisficing in AI?
• Problem complexity makes it impossible
to reach the provably best decision (e.g.
chess, planning under uncertainty, VLSI
layout, etc.)
• Computing the best answer may be
feasible, but it is not economical (e.g.
mobile robot navigation, scheduling,
image processing, etc.)
4
Formalizing the notion of satisficing
Simon’s definition is vague and therefore
hard to use as a design principle.
• The aspiration level tells us nothing about
the problem solving technique.
• What is a “good enough” solution?
• How can a computer measure that?
• Should a satisfactory solution be reached
directly or by iterative refinement?
• How to evaluate a satisficing agent?
5
Achieving satisficing
Satisficing can be achieved by:
• design - the designer of a system
determines the aspiration level.
• run-time deliberation - the agent
determines the aspiration level.
• adaptation - the agent learns the
aspiration level.
6
Why should it be formalized?
• Understand the pros and cons of different
solutions.
• Develop reusable satisficing techniques
and architectures.
• Make satisficing more widely acceptable.
• Put optimality results in context!
7
Notions of satisficing
• Satisficing by an agent versus satisficing
by the system architect (“design choices”).
• Satisficing as an immediate property
versus satisficing as an emergent property.
• Common to all approaches is an explicit or
implicit choice to seek suboptimal
solutions and to terminate the computation
when a particular solution quality is
reached.
8
Approaches to satisficing
i. Satisficing = approximate reasoning
ii. Satisficing = approximate modeling
iii. Satisficing = optimal meta-reasoning
iv. Satisficing = bounded optimality
v. Satisficing = a combination of the
above approaches
9
Satisficing vs. computational savings
• A necessary condition for satisficing is
static or dynamic solution evaluation
criteria leading to sub-optimal solutions.
≠ Limiting computational resources:
– Depth-limited search, RTA*
≠ Minimizing computational effort while
seeking the best answer:
– Admissible heuristic search (A*, AO*, IDA*, SMA*)
– Proving dominance of an action (B*, Protos)
– Pruning (alpha-beta search, bidirectional search)
10
Evaluating satisficing models
• Justifying the designer’s choices
(architectural constraints)
• Justifying the agent’s choices (optimizing
solution quality)
• Ease of implementation
• Overall performance
None of the existing approaches has a
clear advantage over the others!
11
i. Satisficing = approximate reasoning
• Heuristic methods (not necessarily
admissible ones) are used to choose
actions, evaluate solutions and terminate
search.
• In what sense is approximate reasoning a
satisficing approach?
12
e.g. Heuristic evaluation in games
• Designed to return a reasonable measure
of “goodness” within a reasonable
amount of time.
• Corresponds to Simon’s original
interpretation of satisficing (heuristic rules
used to determine the agent’s aspiration
level in expert systems).
13
Pros and cons
• Satisficing is relatively easy to achieve.
• But system performance is totally
unconstrained; almost anything counts as
satisficing.
• Sometimes, however, it’s the best we can
do!
14
ii. Satisficing = approximate modeling
• A tractable but approximate model of the
problem domain is used to derive a
solution.
• Solution may be optimal for the simplified
problem, but it is not necessarily optimal
with respect to the original problem.
• Tractable models can be created using
abstraction or by making simplifying
assumptions.
15
e.g. Naive Bayesian Classifiers
• Making a strong independence
assumptions that features are conditionally
independent given a class label.
• Some correlation between features can be
modeled (by incrementally pairing features).
– Word sense disambiguation [Pedersen, 1998]
– Medical diagnosis [Provan & Singh, 1996]
– Rating Web pages [Pazzani et al., 1996]
– Tree Augmented NB [Friedman & Goldszmidt, 1996]
16
e.g. Planning under uncertainty
• Making the Markov assumption about
dynamical systems so that planning can
be modeled as an MDP.
• Approximating an MDP using a limited
horizon and abstract states.
– The envelope technique [Dean et al., 1993]
– Control strategies for stochastic planning
[Tash & Russell, 1994]
17
Optimizing in the real-world
“In complex real-world situations, optimization
becomes approximate optimization since the
description of the real-world is radically
simplified until reduced to a degree of
complication that the decision maker can handle.
Satisficing seeks simplification in a somewhat
different direction, retaining more of the detail of
the real-world situation, but settling for a
satisfactory, rather than approximate-best,
decision.”
- Herbert Simon
18
iii. Satisficing = optimal meta reasoning
-
• Meta-reasoning in this context is the
process of monitoring and control of the
modules that compute the result.
• In what sense is optimal meta-reasoning a
satisficing process?
• Myopic versus non-myopic optimization of
computational actions.
19
e.g. Control of contract algorithms
Given a contract algorithm, its conditional
performance profile, and a time-dependent
utility function, run the algorithm for a
fixed contract time that maximizes the
expected quality of the result.
Theorem [Zilberstein, 1993]: The fixed-contract
monitoring strategy is optimal when the
domain has predictable utility, and the
system has a deterministic CPP.
20
e.g. Control of interruptible algorithms
Given an interruptible anytime algorithm, its
CPP, and a time-dependent utility
function, run the algorithm as long as the
marginal value of computation is positive.
Theorem [Zilberstein, 1993]: Monitoring using the
value of computation is optimal when the
CPP is monotonically increasing and
concave down, and the cost of time is
monotonically increasing and concave up.
21
e.g. Monitoring policies
Given Pr(qj|qi,∆t) and U(qj,tk) compute π(qj,tk) →(∆t,m)
by optimizing the following value function:
V(qi, tk) = argmax∆t,m{
Σj Pr(qj | qi, ∆t) U(qj, tk+∆t) if m = stop,
Σj Pr(qj | qi, ∆t) V(qj, tk+∆t) − C if m=monitor}
Theorem [Hansen & Zilberstein, 1996]: A monitoring
policy that maximizes the above value
function is optimal when quality
improvement is Markovian.
22
e.g. Control of progressive reasoning
Given a set of progressive processing units,
the problem is how to construct a
schedule of the PRUs that maximizes the
comprehensive utility of the system.
Theorem [Mouaddib and Zilberstein, 1998]: Given a
monotonic progressive processing
problem P, the optimal policy for the
corresponding MDP is an optimal
schedule for P.
23
 End
Level/Value
PRU #2
F
Monitoring Policy 2/3 M
3.0
M
3.0
M
3.0
.4 .4 .4
.6 .6 .6
1/2 E E E
A graphical 5.0 5.0 3.2
representation of a 0/0 E
.4
.6
E
.4
.6
5.0 3.9
monitoring policy for
a progressive
F
processing problem PRU #1
with two PRUs. 2/2 M
7.0
.6
.4
1/3 E M
8.1 8.0
.3
.7
0/0 E
8.0
4 3 2 1 0
24 Remaining Time
e.g. Selection of multiple methods
Design-to-time [Garvey & Lesser, 1993]
• Use multiple methods for solving each
subproblem.
• Weigh the interaction between methods.
• Assume some amount of predictability of
method quality and duration.
• Plan which methods to execute, then
schedule and monitor execution.
25
Pros and cons
• It is feasible and practical.
• Requires optimality at the meta-level but
does not impose any restrictions on base
level computations.
• Why should we accept this as a good
model for satisficing?
26
The case for using anytime algorithms
• A “least commitment approach” - delay
the decision on quality/resource tradeoff
to run-time (simplifies design).
• Fine granularity of computational actions.
• Low run-time overhead (By constructing
PPs and monitoring policies off-line).
• But, incurs the overhead of producing
gradually improving results.
27
iv. Satisficing = bounded optimality
Bounded optimality [Russell, 1998] requires that
the agent program outperforms any other
possible program on a given architecture.
lopt = argmaxl∈L V(Agent(l,M),E,U)
A shift from optimizing action selection to
optimizing over programs within a certain
architecture.
28
e.g. Provably bounded optimal agents
General approach:
– Specify a class of machines
– Specify a construction method
– Prove that the construction is bounded optimal
One example [Russell and Subramanian, 1995] is a DP
construction algorithm that generates a
bounded optimal program for a restricted
class of agent architectures, in which a
program consists of a sequence of
decision procedures.
29
e.g. From contract to interruptible alg.
Theorem [Zilberstein, 1993]: For any contract
algorithm A, an interruptible algorithm B
can be constructed such that
QB(4t) ≥ QA(t).
Theorem [Charpillet, 1997]: Given any contract
algorithm A, there is no interruptible
anytime algorithm B that can be
constructed from A such that
QB(kt) ≥ QA(t) and k < 4.
30
Pros and cons
• BO represents a well-defined constraint
optimization problem that has a solution.
• But it really shifts the intractable
computational task from the agent to its
designer.
• Desirable, but in most cases very hard to
establish.
31
Optimality of programs
“There are few opportunities for proving
optimality, or even theoretical dominance of one
program over another. For the most part,
evaluation is carried out by examining
performance in comparison with the performance
of other programs and human professionals on
standard tasks.”
-- Simon, On how to decide what to do
32
Asymptotic bounded optimality
Definition: the agent’s program is
asymptotically bounded optimal if it needs
a k-times faster (larger) machine to have a
worst-case behavior as good as that of
any other program in all environments.
Problem: in some simple situations this
leads to counter intuitive results:
Prog1: time=1, quality=.99
Prog2: time=100, quality =1
and assume U(q,t) monotonic in t and q.
33
Summary
• There are several different senses of
“satisficing” and they lead to different
agent designs and performance.
• No existing approach is provably superior
to the others.
• The approach based on meta-level control
of anytime algorithms seems attractive
because it is both practical and well-
motivated.
34