Agents and Environments

Good Behavior: Concept of Rationality

Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Overview
Intelligent Agents
Sanjay Singh
Department of Information and Communication Technology
Manipal Institute of Technology, Manipal University
Manipal-576104, INDIA
sanjay.singh@manipal.edu

February 19, 2016

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality

Nature of Environments

Agent Types

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments I

Agents and Environments II

Agents include humans, robots, softbots, thermostats, etc.

The agent function maps from percept histories to actions:
f : P A
The agent program runs on the physical architecture to produce
f

Figure 1 : Agent interaction with environment through sensors and

actuators.
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Vacuum-cleaner world I

A vacuum-cleaner agent
Table 1 : 1. Partial tabulation of a simple agent function for the
vacuum-cleaner world shown in Figure 2

Figure 2 : A vacuum-cleaner world with just two locations.

Percepts: location and contents, e.g., [A, Dirty ]

Actions: Left, Right, Suck, NoOp
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Action
Right
Suck
Left
Suck
Right
Suck
..
.

[A, Clean], [A, Clean], [A, Clean]

[A, Clean], [A, Clean], [A, Dirty ]
..
Intelligent Agents
. Sanjay Singh, MIT Manipal-576104

Right
Suck
..
.

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

A vacuum-cleaner agent

Overview

function Reflex-Vacuum-Agent(location, status])

if status = Dirty then
Suck
else if location=A then
Right
else if location=B then
Left
end if
end function
What is the right function?
Can it be implemented in a small agent program?

Sanjay Singh, MIT Manipal-576104

Percept Sequence
[A, Clean]
[A, Dirty ]
[B, Clean]
[B, Dirty ]
[A, Clean], [A, Clean]
[A, Clean], [A, Dirty ]
..
.

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality

Nature of Environments

Agent Types

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Rationality

Rationality

A rational agent is one that does the right thing.

If the sequence is desirable then the agent has performed well.
This notion is captured by a performance measure that
evaluates any given sequence of environment state.
What is rational at any time depends on following four things:
The
The
The
The

performance measure that defines the criteria of success.

agents prior knowledge of the environment.
action that the agent can perform.
agents percept sequence to date.

Definition: For each possible percept sequence, a rational agent

should select an action that is expected to maximize its
performance measure, given the evidence provided by the percept
sequence and whatever built-in knowledge the agent has.
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Fixed performance measure evaluates the environment

sequence
one point per square cleaned up in time T ?
one point per clean square per time step, minus one per move?
penalize for > k dirty squares?
A rational agent chooses whichever action maximizes the
expected value of the performance measure given the percept
sequence to date
Rational 6= omniscient
percepts may not supply all relevant information
Rational 6= clairvoyant
action outcomes may not be as expected
Hence, rational 6= successful
Rational = exploration, learning, autonomy
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Overview

Specifying the task environment: PEAS

Agents and Environments

Good Behavior: Concept of Rationality

Nature of Environments

To design a rational agent, we must specify the task environment

Performance measure
Environment
Actuators

Sensors
What is PEAS for the task of designing an automated taxi?

Agent Types

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

PEAS

PEAS for Internet shopping agent

Consider, e.g., the task of designing an automated taxi:

Performance measure safety,fast, destination, profits,
legality, comfort, . . .

Performance measure: price, quality, appropriateness,

efficiency

Environment Chandni Chowk streets/freeways, traffic,

pedestrians, weather, . . .

Environment: current and future WWW sites, vendors,

shippers

Actuators steering, accelerator, brake, horn, speaker/display,

...

Actuators: display to user, follow URL, fill in form

Sensors: HTML pages (text, graphics, scripts)

Sensors video, accelerometers, gauges, engine sensors,

keyboard, GPS, . . .
What is PEAS for the task of Internet shopping agents?

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Sanjay Singh, MIT Manipal-576104

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

PEAS Description

Environment Types

Table 2 : Examples of agent types and their PEAS description

Agent Type
Medical
diagnosis
system

Satellite image analysis system

Part-picking robot
Refinery controller

Interactive tutor

Performance Measure
Healthy patient, reduced
cost

Correct image categorization

% of parts in correct bins
Purity, yield, safety

Students score on test

Sanjay Singh, MIT Manipal-576104

Environment
Patient, hospital, staff

Downlink from
orbiting satellite
Conveyor belt
with parts, bins
Refinery, operators
Set of students,
testing agency

Intelligent Agents

Actuators
Display
of
questions,
tests, diagnosis,treatments,
referrals
Display of scene
categorization
Jointed arm and
hand
Valves, pumps,
heaters, displays
Display of exercises, suggestions, correction

Sensors
Keyboard entry
of
symptoms,
findings,
patients answer
Color pixel arrays
Camera, joint
angle sensors
Temperature,
pressure, chemical sensors
Keyboard entry

Fully observable vs. partially observable

Single agent vs. multiagent

Deterministic vs. stochastic

Episodic vs. sequential

Static vs. dynamic

Discrete vs. continuous

Known vs. unknown

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Which is the hardest combination of environment characteristics to

deal with?
partially observable

Table 3 : Examples of task environments and their characteristics

multiagent
stochastic
sequential
dynamic
continuous

Task Environment

Observable

Agents

Deterministic

Episodic

Static

Discrete

Crossword Puzzle
Chess
Poker
Backgammon
Taxi driving
Medical Diagnosis
Image Analysis
Part-picking robot
Refinery controller
Interactive tutor

Fully
Fully
Partially
Fully
Partially
Partially
Fully
Partially
Partially
Partially

Single
Multi
Multi
Multi
Multi
Single
Single
Single
Single
Multi

Deterministic
Deterministic
Stochastic
Stochastic
Stochastic
Stochastic
Deterministic
Stochastic
Stochastic
Stochastic

Sequential
Sequential
Sequential
Sequential
Sequential
Sequential
Episodic
Episodic
Sequential
Sequential

Static
Semi
Static
Static
Dynamic
Dynamic
Semi
Dynamic
Dynamic
Dynamic

Discrete
Discrete
Discrete
Discrete
Continuous
Continuous
Continuous
Continuous
Continuous
Discrete

unknown
Driving a rented car in a new country with unfamiliar geography
and traffic laws is a lot more exciting!!!!!!!

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Overview

Structure of Agent

Agents and Environments

Good Behavior: Concept of Rationality

Nature of Environments

Agent Types

Sanjay Singh, MIT Manipal-576104

The objective of AI is to design an agent program that

implements the agent function.
agent=agent program + architecture
function Table-Driven-Agent(percept)
persistent: percepts, table
append percept to the end of percepts
action LOOKUP(percepts,table)
return action
end function

Intelligent Agents

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Structure of Agent

Agent types

Let p be the set of possible percepts and T be the life time of the
agent (the total number of percepts it will receive).
The number of entries in the lookup table is given by
#LookUp Table Entries =

T
X

Simple reflex agents

|p|t

t=1

10150

Even the lookup table for chess would have

entries. The
number of atoms in the observable universe is less than 1080 !!!
The key challenge of AI is to find out how to write a program that
produce rational behavior from a small program rather than a vast
table.

Sanjay Singh, MIT Manipal-576104

Four basic types in order of increasing generality:

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Model-based reflex (reflex agents with state)

Goal-based agents; and
Utility-based agents
All these can be turned into learning agents

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Simple reflex agents

function Simple-Reflex-Agent(percept)
persistent: rules, a set of condition-action rules
state INTERPRET-INPUT(percept)
rule RULE-MATCH(state, rules)
action rule.ACTION
end function
A simple reflex agent. It acts according to a rule whose condition
matches the current state, as defined by the percept.

Figure 3 : Schematic diagram of a simple reflex agent.

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Infinite loops are unavoidable in partially observable (PO)

environments
Randomization can help to escape from infinite loop
Is there any effective way to handle PO?
Yes, keep track of the part of the world it cant see now

Agent should maintain internal state that depends on

percept history and reflects at some of the unobserved aspect
of the current state
Updating internal state information requires two kinds of
knowledge encodings
How the world evolves independent of the agent
How the agents own actions affect the world

Knowledge about how the world works-is called a model

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Model based reflex agent

function Model-Based-Reflex-Agent(percept)
persistent: state, model, rules, action
state UPDATE-STATE(state, action, percept, model)
rule RULE-MATCH(state,rules)
action rule.ACTION
end function
A model based reflex agent keeps track of the current state of the
world using an internal model. It then choose an action in the
same way as the reflex agent.

Figure 4 : A model-based reflex agent.

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Goal-based agents

Utility-based agent

Figure 5 : A Goal-Based-Agent.
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Figure 6 : A Utility-Based-Agent.
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Learning agents
So far agent program description in very high-level terms only,
consisting of various components whose function is to answer :
What is the world like now?
What action should I do?
What do my actions do?
How does these components work?
Basic distinctions among the various ways that components
can represent the environments that the agent inhabit

Figure 7 : A general learning agent.

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

How the components of agent programs work

Atomic representation: The algorithms underlying search,
game-playing, HMM and MDP all work with atomic
representation.
Factored representation: CSP, Propositional Logic,
Planning, Bayesian Networks and Machine Learning
algorithms are based on factored representation.
Figure 8 : Three ways to represent states and the transition between
them. (a) Atomic representation: a state (such as B or C) is a black box
with no internal structure; (b) Factored representation: a state consists
of a vector of attribute values; values can be Boolean, real valued, or one
of a fixed set of symbols. (c) Structured representation: a state includes
objects, each of which may have attributes of its own as well as
relationships to other objects.
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Agents and Environments

Good Behavior: Concept of Rationality
Nature of Environments
Agent Types

Summary
Agents interact with environments through actuators and
sensors.
The agent function specifies the action taken by the agent in
response to any percept.
The performance measure evaluates behavior of the agent
in an environment.
A rational agent maximizes expected performance.
PEAS descriptions define task environments.
Agent programs implement (some) agent functions.
Environments are categorized along several dimensions:
observable? deterministic? episodic? static?
discrete? single-agent?

Several basic agent architectures exist:

reflex, reflex with state, goal-based, and
utility-based
Sanjay Singh, MIT Manipal-576104

Intelligent Agents

Structured representation: Relational Database, First Order

Logic, Knowledge Based Learning and NLP

Sanjay Singh, MIT Manipal-576104

Intelligent Agents