Multi-Agent Pathfinding,

Challenges and Solutions

Guni Sharon
:Joint work with
Roni Stern, Ariel Felner, Nathan Sturtevant, Meir
Goldenberg.

Ben-Gurion University University of Denver

.ISE Dpt .C.S. Dpt
Motivation
Robotics
Video games
Transportation applications
Warehouse management
Product assembly
Example
Automated junction
Example from Peter Stones web site
Example
DNA Lab on a Chip
Duke University
Example
Warehouse management
 Multi-Agent Path Finding
Input
A graph with N states
A set of K agents each with start and goal state
Actions
An agent can move or wait
Task
A path for each agent
Constraints
Paths shouldnt conflict
Agents cannot be in the same location at the same time
Target
Minimize the sum of travel costs
 MAPF a continuum
K=1 (Navigation in explicit graphs) K=N-1 (Tile puzzle)

K (# agents)

? What about 0< K < N-1

How do we apply techniques from the extreme
cases to the general case?
How do N, K, b and influence the hardness of
the problem and the related algorithms?
Main approaches

MAPF
solvers

Non- Optimal
optimal
Main approaches

MAPF
solvers

Non- Optimal
optimal

Search based Procedure

(sub-optimal) based
Search based sub-optimal
Agents are planned individually
Fast
Sub optimal
Not complete
Forsake optimality in order to obtain
practical running time.
 Cooperative A* (David Silver)
A simple decoupled MAPF solver

CA*:
1. For each agent
1.2 p plan a path to goal //ignore other agents
1.3 for each step in p //(time,location) pair
1.3.2 block p for other agents
Return list of paths //path per agent
 Cooperative A* (David Silver)

Step 1:
Plan blue agent
 Cooperative A* (David Silver)

Step 1:
Plan blue agent

0
 Cooperative A* (David Silver)

Step 1:
Plan blue agent

1 0
 Cooperative A* (David Silver)

Step 1:
Plan blue agent

1 0
 Cooperative A* (David Silver)

Step 1:
3
Plan blue agent
DONE!
Step 2:
Plan red agent
2

1 0
 Cooperative A* (David Silver)

Step 1:
3
Plan blue agent
DONE!
Step 2:
Plan red agent
2

0 1 0
 Cooperative A* (David Silver)

Step 1:
3
Plan blue agent
DONE!
Step 2:
Plan red agent
2
Wait

0 1 0
 Cooperative A* (David Silver)

Step 1:
3
Plan blue agent
DONE!
Step 2: 2
Plan red agent
1

0 1 0
 Cooperative A* (David Silver)

Step 1:
3
Plan blue agent
DONE!
Step 2: 2 3
Plan red agent
1 2

0 1 0
 Cooperative A* (David Silver)

Step 1:
3
Plan blue agent
DONE!
Step 2: 2 3
Plan red agent
4
1 2
DONE!

Step N:
0 1 0
Plan N agent
Cooperative A* (David Silver)

Not complete
Not optimal
Main approaches

MAPF
solvers

Non- Optimal
optimal

Search based Procedure

(sub-optimal) based
Procedure based
If then procedures
Very fast (polynomial)
Complete in most cases
Not optimal, not even close
Solve extremely hard problems.
Procedure algorithm
Another procedure
Procedure algorithms for MAPF
Multi-Agent Path Planning, Wang & Botea, IJCAI, 2009
Complete for slidable grids

Push and swap, Luna & Bekris, IJCAI, 2011

Complete for graphs where at least two vertices are always
unoccupied

Tree-based agent swapping strategy, Khorshid at el, SOCS, 2011

Complete for tree type graphs
Main approaches

MAPF
solvers

Non- Optimal
optimal
Optimal approach
Agents are planned together
Optimal solution (focus of our research)
Complete
Exponentially hard
NP-Complete (makespan variant)
[P. Surynek, AAAI, 2010]

Use coupled setting

Traditional Coupled setting
A state is a set of locations, one per agent
!This state has 25 neighboring states

possible 5
moves

!Stay possible 5
moves

!Stay
 Search Tree Growth
a b c
Root
a,i
d e f

g h i
d,f d,h d,i b,f ...

g,i g,c g,e a,f a,i a,c ...

 Heuristic
?How far am I from the goal

Sum of Individual Costs (SIC)

A commonly used heuristic ([Standley, 2010/2011] ,
[Sharon et al, 2011] )
Sum the distance of all agents from their goal

SIC=3+3=6
Solution #1:

Given:
State <position(a1), position(a2), position(ak)>
Operators per state All single agent moves combinations
Admissible heuristic Sum of Individual Costs
Use A*
*A
State.g: distance of the state from the initial state
Keep track of distance while progressing (like in dijkstra)
State.h: assumption of distance remaining to goal
SIC heuristic
State.f: state.g + state.h
Length assumption of the best path through the state
Run a best-first search according to f values
Optimal solution (if h is admissible)
complete

!A* is optimally efficient

?Problem solved
Problem #1: Exponential BF
The branching factor is 5k (k is the # of agents)
On a problem with only 20 agents:
Branching factor = 95,367,431,640,625
A* cant expand even the root!!!

Cannot solve even with a perfect heuristic!

Most of these states are irrelevant to the search (f>C*)
 r
3

Open

Irrelevant states
 r
3

Open
r
3

Irrelevant states
 r
3

Open
r
3

Irrelevant states
 r
3
a b c d
3 3 5 6

Open
r
3

Irrelevant states
 r
3
a b c d
3 3 5 6

Open
a b c d
3 3 5 6

Irrelevant states
 r
3
a b c d
3 3 5 6

Open
a b c d
3 3 5 6

Irrelevant states
 r
3
a b c d
3 3 5 6

Open
a b c d
3 3 5 6

Irrelevant states
 r
3
a b c d
3 3 5 6

Open
b c d
3 5 6

Irrelevant states
 r
3
a b c d
3 3 5 6

Open
b c d
3 5 6

Goal
found

Irrelevant states
 r
3
a b c d
3 3 5 6

Open
b c d
3 5 6

Goal Surplus
found nodes

Irrelevant states
 Solution #2:
Operator Decomposition
AAAI 2010, Standley.

Increasing Cost Tree Search

IJCAI 2011, SoCS 2011, Guni Sharon, Roni Stern, Meir Goldenberg, Ariel
Felner
Enhanced Partial-Expansion A*
AAAI 2012, Ariel Felner, Meir Goldenberg, Guni Sharon, Roni Stern, Tal Beja,
Nathan R. Sturtevant, Robert Holte, Jonathan Schaeffer
*EPEA
On expansion, generate only the best successors
Avoids generation of states with f-value>C*
Generates a factor of b less states then A*
In MAPF b is exponential in the number of agents
EPEA* can obtain an exponential speedup
r EPEA* progress
3

Open
r EPEA* progress
3

Open
r
3
r EPEA* progress
3

Open
r
3
 r EPEA* progress
3
a b c d
3 3 5 6

Open
r
3
 r EPEA* progress
3
a b c d
3 3 5 6

Open
a b
3 3

We add children with f=3

 r EPEA* progress
5
a b c d
3 3 5 6

Open
a b
3 3

We collapse the others into r

 r EPEA* progress
5
a b
3 3

Open
a b r
3 3 5

We collapse the others into r

 r EPEA* progress
5
a b
3 3

Open
a b r
3 3 5

Assuming a and b have no children

r EPEA* progress
5

Open
r
5
 r EPEA* progress
5

Open
r
5

We re-expand r but now with F=5

 r EPEA* progress
5
a b c d
3 3 5 6

Open
r
5

We only want children with f=5

 r EPEA* progress
5
a b c d
3 3 5 6

Open
c
5

We only want children with f=5

 r EPEA* progress
5
a b c d
3 3 5 6

Open
c
5

We collapse the others (d) into r

r EPEA* progress
6
c
5

Open
c r
5 6
 r EPEA* progress
6
c
5

Open
c r
5 6

Assuming c has no children

r EPEA* progress
6

Open
r
6
 r EPEA* progress
6

Open
r
6

We re-expand again but now with F=6

 r EPEA* progress
6
a b c d
3 3 5 6

Open
r
6
 r EPEA* progress
6
a b c d
3 3 5 6

Open
r
6
We only want children with f=6
 r EPEA* progress
6
d
6

Open
d
6

R is now fully expanded and moved to CLOSED

Getting the right successors
Given an f value how do we generate the right successors?
For each agent
For each legal move
Calculate the effect of the move on the f value

f Operators
0 North, West
1 Wait
2 South, East

Generate states by using all sets of movement that sum up to

the wanted change in the f value
 EPEA* - Conclusions

EPE-A* only generate the necessary nodes

Has extra overhead per state
Generates substantially less states
Substantial time savings.

?Problem solved
76
Problem #2: Enormous state space

The size of the state space is | V |k

Allocate k agents to|V | locations
No perfect (or even close) heuristic
A large portion of the state space must be explored
Cant solve problems with many agents
Solution #3 : Independence Detection

Simple ID:
1. Plan for each agent independently
2. If agents collide
2.1 Group them together
2.2 Plan for these agents together (with A*)
2.3 Goto 2
Standley 2010 1 2
Extremely effective !
2

?Problem solved 1
Problem #3: Almost Independent

21ndst step

Each agent has m different optimal paths

All of them go through C at time step 2
ID will group both agents
*Group agents and solve with A
Optimal solution = 7
A* will expand all states with f < 7

Wait

Every m2 combinations of (Ai,Bj) will be expanded!

Conflicts and Constraints

Conflict: < Agent A, Agent B, place X, time T >

Constraint: < agent A, place X, time T >

Solve conflict:
< Agent A, Agent B, place X, time T >
Option 1: Constraint A - < A,X ,T>
Option 2 : Constraint B - < B , X , T >
Solution #4:
Conflict Based Search

Conflict Based Search:

1. Plan for each agents separately
2. Validate plans
3. If the plans of agents A and B conflict
Constrain A to avoid the conflicts
or
Constrain B to avoid the conflict
Must check both constraints
Required to ensure optimality

Conflict
No Constraints {A,B}

Conflict Conflict
Constrain B Constrain A
CBS Example

Expand
Root

Conflict
{C,2,1,2}

Wait OKOK
Replan
Expand1 Not Goal
Goal Test Replan 2

Goal
 Example analysis
CBS: Expand O(m)
A* : Expand O(m2)
All m2 (Ai,Bj) combinations

For this problem CBS will expand less states then A*

 Example #2 (not as nice)
4 optimal solutions for each agent
Each optimal solutions combination conflicts
 Example #2 Analysis
CBS: Expands 71 states in total (= 4 * 16 + 7)
4 conflicts 16 constraint tree (CT) nodes
Each CT node expands 4 single agent states
The goal CT node expands 7 single agent states
4 * 16 + 7 = 71 low-level states in total
A*: Expands 8 states in total

CBS expands more states by a factor of 9

 A* or CBS?
A* is exponential in the number of agents
CBS is exponential in the number of conflicts
No universal winner
Depends on the problem properties

!!!Use Both
Solution #5:
Mata-Agent Conflict Based Search

1. Plan for each agents separately

2. If the plans of agents A and B conflict
2.1 If ShouldMerge(A,B)
2.1.1 unite A and B and solve with A*
2.2 Else
2.2.1 Constrain A to avoid the conflicts
2.2.2 or
2.2.3 Constrain B to avoid the conflict
Other concept: The increasing cost
tree search

Convert the optimization problem to a set of decision problems

Instead of:
What is the shortest solution?
Ask:
Is there a solution of length L?
Be more specific
Is there a solution where:
Agent 1 goes L1 steps
Agent 2 goes L2 steps

Agent n goes Ln steps
 ICTS
two level algorithm

High- Is there a solution

level ?Whatwithabout
costs this
3 3 4
3

Low-
!!YES
NO level

3 3 4

3
 ICTS: High level
The high level searches the Increasing Cost Tree (ICT)
defined as follow:

Node a cost vector (cost per agent)

Operators one agents cost is increased by one.
Root The minimal individual costs (SIC).

Find
No a
solution

)Tree size=O
(
 ICTS: Low level
Is there a solution with length (L1,L2,,Lk) ?
We can use techniques similar to arc/path consistency (CSP)
Usually prunes a large portion of the state space

Is there a solution of length (2,2,2)

Agent 1 must go through A
Then agent 2 must go through B
Then agent 3 has no path of length 2
Return false A B
 ICTS - conclusion
A different concept from A*
Never generates states with f>C*
Can potentially prune a significant portion of the state space
Does a lot of repetitions in the high-level nodes
Exponential amount of high-level nodes
In many cases still state-of-the-art
 Summary: No universal
winner
:We observed the following tendencies
A* - exponential in #agents (NK)
1 2 3
best in areas dense with agents 4 5 6 7
8 9 10 11
12 13 14 15

ICTS - exponential in
best in open areas with few agents

CBS exponential in #conflicts

best in areas with many bottlenecks
 Future work

Rigorous analysis of how the different parameters

influence the different algorithms

Dynamic changing of polices based on the domain.

Other definitions for the problem:

Weighted graphs
A set of goals for a set of agents
!Thank you