IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO.

Y, MONTH 1995 1

Asynchronous Parallel Discrete Event Simulation

Yi-Bing Lin and Paul A. Fishwick, senior member

Abstract | Complex models may have model components that PDES speeds up otherwise serial computations dur-
distributed over a network and generally require signicant ing a simulation. The second reason for PDES (distributed
execution times. The eld of parallel and distributed simu- model constraint) is based on a situation where models for
lation has grown over the past fteen years to accommo-
date the need of simulating the complex models using a
distributed versus sequential method. In particular, asyn-
chronous parallel discrete event simulation (PDES) has been
widely studied, and yet we envision greater acceptance of
this methodology as more readers are exposed to PDES in-
troductions that carefully integrate real-world applications.
With this in mind, we present two key methodologies (con-
servative and optimistic) which have been adopted as solu-
system components are stored in physically dierent loca-
tions. The other author (Fishwick) is building a prototype
distributed simulation of a process plant where each plant
component is ultimately co-located with the manufacturer
of that component.
The paper proceeds as follows. First, in Section II, we
tions to PDES systems. We discuss PDES terminology and
methodology under the umbrella of the personal communi-
cations services application.
Keywords |parallel algorithm, distributed simulation, syn-
chronization, virtual time, network communications
dene our terms within the PDES area and demonstrate
the generic approach to distributed simulation. In Sec-
tion III, we introduce the PCS application and demon-
strate the need for synchronization of incoming messages
to a given process. There are two key approaches to syn-
chronization. Method 1, dened in Section IV, is termed
I. Introduction the conservative method since it ensures that the causal re-

O UR purpose is to introduce the basic technical con-

cepts of distributed simulation of event-based models
(so called discrete event models), and to tie these generic
lation among time consecutive events will be maintained
at all times during the simulation. Method 2 is dened
in Section V, and identies the optimistic method. In this
approach, the causal relation can be broken with subse-
concepts to a specic application: personal communica-
tions services (PCS). Several introductory articles have
been presented in the literature such as Fujimoto [1], Nicol
et al. [2] and Richter et al. [3]. These papers have helped to
disseminate the asynchronous parallel discrete event simu-
lation (PDES) methodology for a wide readership. Our ap-
proach is similar but stresses a single real world application
quent xing of state variables. We close in Section VI with
directions for the future of PDES.
Throughout this paper, we use three font styles to rep-
resent dierent concepts. The typewriter type style rep-
resents attributes or methods (e.g., SendMessage()) of ob-
jects. The italic type style represents variables such as LP
for discussing the methodology of PDES. By dening the or p. The serif type style represents event types such as
methodology and all PDES terminology within the context CallArrival.
of the PCS application, this paper serves both as a tutorial
to PDES and as an introduction to PCS simulation mod- II. Parallel Discrete Event Simulation
eling. PCS is a rich enough application to illustrate most A. Basic Terminology
basic PDES concepts.
The processing elements in PDES can either be of a par- We begin by dening terms which are commonly found
allel or distributed nature. An MIMD machine with mul-
tiple asynchronous elements performing message passing is
an example of a parallel machine. Distributed elements
normally refers to local or wide area networks composed
of inter-connected set of heterogeneous workstations and
computers. PDES is used for one of two reasons: 1) one
wants to execute a model faster than is possible in a se-
quential machine, or 2) one must model in a distributed
fashion because of a constraint that a process (i.e., com-
in the simulation and PDES elds. These terms will be
revisited in Section 3 when we assign the terms to the
PCS application. The study of any physical system to
be simulated begins with the creation of a model. Such
a model can be in one of several types [6]: 1) concep-
tual, 2) declarative, 3) functional, 4) constraint, 5) spatial
or 6) multimodel. One begins with a conceptual model
which describes qualitative terms and class hierarchies for
the system. In many ways, the conceptual model \orga-
putation) must be distributed rather than localized to a
single processor. One author (Lin) has demonstrated var-
ious speedups possible on a distributed memory architec-
ture for the PCS application [4], [5]. There is no question
ect an orientation to system construction;
Yi-Bing Lin is Professor with the Dept. of Computer Science and
Information Engineering, National Chiao Tung University. E-mail:
liny@csie.nctu.edu.tw.
Paul A. Fishwick is Associate Professor with the Dept. of Com-
nizes" the denition of attributes, methods and general
characteristics of each system component without going so
far as to ascribe dynamics to components. The next four
model types re
a system may be constructed as a Petri net [7], queuing
model [8] or as a cellular automaton [9] for instance. The
last model type (multimodel) permits the integration of ba-
sic model types to create a model composed of component
models [10], [11] where each component model represents
puter and Information Sciences, University of Florida. E-mail: sh-
wick@cis.u
.edu . a level of abstraction for the system.
2 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995
The PCS area, to be discussed in Section 3, uses a spa-
tial model in that the system is viewed as a hexagonal dis-
cretization of a large two-dimensional space representing
an area where cellular communications are to be imple-
mented. Spatial models can be executed in several ways
including time slicing, event scheduling and parallel and
distributed. Our approach will be to use a parallel and
distributed approach to model execution, while using the
concept of event scheduling within each process. Speaking
do work then the communications overhead becomes less
critical. The kind of work ideally suited in simulation is a
sequential simulation within the LP, composed of the usual
FEL and event routines. Thus, the distributed simulation
is hybrid in form with sequential simulation coinciding |
and synchronized with| distributed simulation. The LVT
of this more substantial LP is updated by removing the
highest priority event (lowest timestamp) from the FEL
and executing the associated event routine. Some (or all)
of process, we must dene this term appropriately. Model of these event routines will contain scheduling commands
components for a PCS implementation will be a collection to place events with new times back into the FEL. Some
of hexagonal cells. Other model types, such as a queu- event routines will involve messages to be issued through
ing model, are composed of other components (facilities). the output buer(s) to a target LP.
A logical process (LP) is dened as a set containing basic
model components, so a PCS logical process will be a set
of hexagons, or just one hexagon. A physical process or
processor is a set of logical processes mapped in a way that
conforms to the architecture of the parallel/distributed sys-
tem.
B. Object Oriented Implementation
A PDES consists of several PDES objects or LPs. These
LPs execute asynchronously with coordination to complete
a simulation run. To implement the objects in an LP (as
described in Section II-A), the attributes and methods of
An LP contains several objects: the LP are classied into four categories (see Table I):
 Local Virtual Time (LVT): time associated with the
LP. The LP does not know another LPs time unless [Table 1 about here.]
communicated via a message.  A clock mechanism indicates the progress of the LP.
 Future Event List (FEL): event list used when there An attribute LVT represents the timestamp of the
are internal events posted within the LP itself. event that just occurred in the LP. The LVTUpdate()
 Event: an item within the FEL. method updates LVT to advance the \clock" of the LP.
 Message: an item sent from one LP to another.  A FEL mechanism processes the events occurring in
The FEL is composed of events, where an event combines the LP. The FEL is basically a priority queue with one
the following objects: 1) time stamp, 2) token, 3) event attribute and three methods. An attribute eventList
type. The time stamp re
ects when the event is to oc- maintains the events to occur in the future. The
cur. An event's occurrence correlates with the execution Enqueue() method inserts a time-tagged event into
of an event routine for that LP. The token is associated eventList so that eventList maintains its ordered
with whatever is
owing through the network of LPs. For sequence. The Dequeue() method deletes the event
the PCS application, portables (i.e., mobile phones)
ow with the minimum timestamp in eventList. The
through the system. An event type species what will Cancel() method deletes the event with a specied
happen to the token (arrival, boundary crossing, depar- timestamp in eventList.
ture, incoming call). An LP has input channels and output  A synchronization mechanism interacts with other
channels where each channel has a rst-in/rst-out (FIFO)
buer associated with it. A message is equivalent to an
event that must be moved from one LP to another. Mes-
sages which simply enter an FEL and are processed are
generally called events. When an event must be issued to
another LP, it becomes a message. The relationship among
the above terms are shown in Figure 1.
[Figure 1 about here.]
Messages arrive in one of several input channel buers
and are routed directly to the LPs FEL. Note that simple
LPs to coordinate the execution of PDES. The
ReceiveMessage() method receives messages from
other LPs (these messages will be inserted into the
FEL for processing). The method ExecuteMessage()
executes events in the FEL. The SendMessage()
method sends output message (generated by the ex-
ecution of events) to their destination LPs.
It is probably more appropriate to consider
ExecuteMessage() as a method of the FEL. However,
this method is aected by the PDES synchronization
mechanisms to be described later. Thus the method
LPs may involve a calculation such as 1) taking the times- is classied as part of the synchronization mechanism.
tamp from an incoming message, 2) adding a value to this  An application mechanism represents a sub-model for
timestamp, and 3) sending the new message to the output
buers. Such an LP would not have any need of an FEL
and would be a \pure" distributed simulation. This kind of
technique, however, is wasteful of the computing elements
since there will be a large price to pay in communications
overhead among inter-LP communication. A simple addi-
y [12], [13], [14],
tion is not sucient to warrant a distributed approach. On
the other hand, if the processing element can be made to
a specic simulation application to be simulated by
the LP (to be elaborated).
C. PDES Implementation Platforms
PDES systems have been implemented in dierent par-
allel architectures such as BBN Butter
Sequent [15], [16], [17], JPL Mark III [18], Simulated Stan-
ford Dash Multiprocessor [19], Transputers [20], [21], CM-
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 3

1/CM-5 [22], KSR [23], and iPSC/860 [24]. PDES has scheme, if no channel is available in the new cell, then the
also been implemented in workstations connected by a lo- call will be dropped or forced terminated immediately.
cal area network [4] which is widely available in both the The PCS example is probably more realistic to the reader
industrial and the academic environments. if we add some geometry to these moving vehicles (porta-
bles). Unfortunately, whether a vehicle moves from one
III. Personal Communication Services cell to another cannot be simply determined by the phys-
We use personal communication service (PCS) network ical movement of the vehicle. We also need to consider
simulation to illustrate PDES functionality. A PCS net- the radio propagation. It is possible that the connection
work [25], [26] provides low-power and high-quality wire- to a vehicle changes from one port to another even if the
less access for PCS subscribers or portables. The service vehicle is stationary { the change of radio signal strength
area of a PCS network is populated with a number of radio may result in re-connecting the vehicle to a dierent port.
ports. Every radio port covers a sub-area or cell. The port According to the PCS network measurement methods, we
is allocated a number of channels (time slots, frequencies, determine that the movement (in the sense of port con-
spreading codes or a combination of these). A portable nection) of a vehicle is best characterized by the residence
occupies a channel for an incoming/outgoing call. If all time1 distribution and the destination cell routing proba-
channels are busy in the radio port, the call is blocked. bility. The reader may image that this movement model
In PCS network planning, PCS network modeling (usually is equivalent to a simple path approach where a vehicle
conducted by simulation experiments) is required to inves- moves straight with an angle. The angle determines the
tigate the usage of radio resources. Since PCS network destination cell and the residence time is the product of a
simulation is time-consuming, PDES eectively speeds up constant speed and the diameter of the cell2 .
the process of PCS network simulation. Specically, To map the PCS model into PDES, the cells in the PCS
 The size of the PCS network under study is usually network are represented by cell objects derived from the
large (e.g., thousands of cells). A typical sequential PDES objects (i.e., LPs). These LPs are then mapped
PCS simulation run takes over 20 hours, while the cor- to processors for execution (see Figure 2). A cell LP
responding PCS PDES takes less than 3 hours using has the following attributes and methods (i.e., the ap-
8 processors [4]. plication mechanism of a general LP): A constant at-
 Another popular parallel approach, the parallel inde- tribute channelNo represents the total number of chan-
pendent replicated simulation [27], [28], [29] (running nels in a radio port. An attribute idleChannelNo repre-
multiple simulation replications concurrently) does not sents the number of idle radio channels. A portableList
work for PCS simulation. In most cases, the PCS de- collects the information of all portables reside in the cell.
signer only is interested in the behavior of the PCS There are ve methods in the cell object: CallArrival(),
network at the engineered workload (e.g., the work- CallCompletion(), PortableMoveIn(),
load at which the blocking probability is 1%). To cal- PortableMoveOut(), and Handoff(). These methods will
ibrate the simulation at the engineered workload, the be elaborated later.
setup of input parameters for the next simulation run [Figure 2 about here.]
is dependent on the previous run.
Now we describe the PCS model and its mapping to the The portables in the PCS network are represented by
corresponding PDES. For demonstration purposes, we de- the portable objects. A portable object consists of four at-
scribe a simplied PCS model without considering the de- tributes:
tails of the radio signal propagation issues (such as Rayleigh  The busy attribute indicates the status of the portable.
fading, co-channel interference, and so on). We assume If busy=YES then the portable is in a conversation.
that there are S cells in the PCS network, and on the aver-  The callArrivalTime attribute represents the next
age, there are n portables in a cell. Every port is allocated call arrival time.
some number of channels. A portable resides at a cell for  The callCompletionTime attribute represents the
a period of time which is a random variable with some completion time of the current phone call when
distribution (e.g., exponential [30], [31], [32]). Then the busy=YES . If busy=NO , the callCompletionTime at-
portable moves to a neighbor cell based on some routing tribute is meaningless.
function (e.g., equal routing probabilities for all neighbors).  The portableMoveOutTime attribute represents the
The call arrivals to a portable is a random process (e.g., time when the portable moves out of the current cell.
Poisson), and is independent of the portable's movement. There are two categories of events in a PDES. An internal
A call is connected if a channel is available. Otherwise, the event is scheduled and executed at the same LP (the event
call is blocked. When a portable moves from one cell to represents the interaction between a cell and a portable
another while a call is in progress, the call requires a new within the cell in our PCS example), and an external event
channel (in the new cell) to continue. This procedure of is scheduled by one LP and is executed by another LP.
changing channels is called hando or automatic link trans- 1
fer (ALT). Several hando schemes have been proposed in cell.Residence time refers to the time that a portable resides within a
the literature [33], [34], [35]. In this paper, we consider 2 But note that our movement model is practical { it is used to ap-
the simplest scheme called non-prioritized scheme. In this proximate real radio systems, while the simple path approach cannot.
4 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

Thus, after its creation, an internal event is inserted in end case

the FEL by using the Enqueue() method, and an external g
event is considered as a message, and is sent to the des- The methods invoked in ExecuteMessage() are described
tination LP by using the SendMessage() method. In the below. When the event type of m is CallArrival, the follow-
PCS PDES, there are three internal event types and one ing action is taken.
external event type. The internal event types are described
below.
 CallArrival: Either the port (the cell) or the portable CallArrival( ) p f
initiates a call setup. A radio link is required to con- if p
.busy=YES then
nect the port and the portable. If no radio link is /* A call is already in progress when the new */
available or the portable is already busy with another /* call arrives at LVT. In other words, a busy line */
conversation, the call is dropped. /* occurs and the new call arrival is ignored. */
 CallCompletion: A phone call completes, and the radio update the busy line statistic;
link between the port and the portable is disconnected. else /* I.e., p.busy=NO. */
 PortableMoveOut: The portable moves out of a cell. If if idleChannelNo = 0 then
the portable is in a conversation, the radio link be- /* The call arrival is blocked. */
tween the portable and the port is disconnected. update the blocking statistic;
We treat the CallArrival event type as an internally gen- else /* I.e., idleChannelNo> 0. */
erated event based on a probability distribution. This is idleChannelNo idleChannelNo -1;
just an abstraction of the actual situation where arrivals p.busy YES;
are sent from outside the LP to one of the LPs input chan- generate the call holding time t, and
nels. Therefore, a more detailed simulation would involve p:callCompletionTime LVT + t;
\electromagnetic messages" re ecting the true nature of end if
incoming calls. The use of a probability function is an ab- end if
straction for this underlying process. generate the next inter-call arrival time t0 and compute
The external event type is described below. the next call arrival time as
 PortableMoveIn: A portable moves in a new cell. If the p:callArrivalTime LVT + t0 .
portable is in a conversation, then a new radio link invoke ScheduleNewEvent(p);
between the cell (port) and the portable is required. /* Schedule a new event (to be described). */
If no radio channel is available, the call is forced ter- g
minated. Note that the busy line and call blocking statistics are out-
In PDES, the execution of a PortableMoveOut event at put measures of the PCS simulation (not shown in our
a logical process LPA always results in the scheduling PDES example) 3 .
of a PortableMoveIn event for the destination logical pro- When the event type of m is CallCompletion, the follow-
cess LPB . This event type is external (to LPB ), and the ing action is taken.
scheduling of the event requires communication between
LPA and LPB .
An event/message m is of the format CallCompletion( ) p f
/* Release occupied channel at call completion. */
m = (timeStamp; p; eventType ) idleChannelNo idleChannelNo+1;
p.busy=NO;
where eventType represents the type of the event, invoke ScheduleNewEvent(p);
timeStamp represents the (simulated) time when the event /* Schedule a new event (to be described). */
occurs, and p is the pointer which points to the corre- g
sponding portable p. The execution of the event mes-
sage m at a cell object LP is described as follows. The When the event type of m is PortableMoveIn, the following
LP .ExecuteMessage() method invokes dierent methods action is taken.
according to the event type of m (the Pascal-like \case"
statement is used in the following denition). PortableMoveIn(p) f
if p.busy=YES then /* A hando occurs. */
ExecuteMessage(m) f invoke Handoff(p); /* To be described. */
LVTUpdate( m:timeStamp) end if
/* I.e., LVT m:timeStamp. */ generate the portable residence time t and compute the
case (m:eventType ) of next move time p.portableMoveOutTime LVT+t;
CallArrival: invoke CallArrival(m:p); invoke ScheduleNewEvent(p)
CallCompletion: invoke CallCompletion(m:p); 3 Call blocking is a major performance measure of a PCS net-
PortableMoveIn: invoke PortableMoveIn(m:p); work. A PCS Network is usually engineered at 1% or 2% blocking
PortableMoveOut: invoke PortableMoveOut(m:p); probabilities.
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 5

/* Schedule a new event (to be described). */ end if

g else /* I.e., p.busy=YES. The portable is busy at */
The method Handoff() is described below. /* LVT. The next event occurring to p is either a */
/* call arrival, a call completion or a cell crossing */
/* movement. */
Handoff( )p f ts min(p.callArrivalTime,
ifidleChannelNo=0 then p.callCompletionTime,
/* No channel is available to connect the */ p.portableMoveOutTime);
/* hando call i.e., the hando fails. */ if ts = 0p.callArrivalTime then
update the forced termination statistic m :eventType0 CallArrival;
(not shown in our PDES example); else if 0ts = p.callCompletionTime then
p.busy=NO; m :eventType0 CallCompletion;
else /* The hando succeeds. */ else m0:eventType0 PortableMoveOut;
idleChannelNo idleChannelNo-1; end if
end if end if
g g
If the event type of m is PortableMoveOut, the following Consider the example illustrated in Figure 3. In this g-
action is taken. ure, a portable is represented by a car (although in many
PCS systems, portables are carried by pedestrians). A call
arrival is represented by a phone connected to the car. A
PortableMoveOut(p) f call completion is represented by a cross (disconnection) on
if p.busy=YES then the phone line.
idleChannelNo idleChannelNo+1; At time 0, portable p1 is at cell A. At time 10, a phone
/* When a communicating portable moves */ call for p1 occurs. The call completes at time 13, and the
/* to a new cell, it releases the occupied */ portable moves to cell B at time 16. At time 20, another
/* channel of the old cell. */ phone call for p1 arrives. At time 24, p1 moves to cell C
end if (and a hando occurs).
determine the destination cell (LP 0 ) to which the
portable moves ; [Figure 3 about here.]
generate an output message In PDES, cells A, B, and C are simulated by logical pro-
m0 = (LVT; p; PortableMoveIn); cess LPA , LPB , and LPC respectively. At the beginning of
invoke SendMessage(m0; LP 0); the simulation, the attributes of p1 are
/* A PortableMoveIn event is scheduled for LP 0. */
g busy = NO;
callArrivalTime = 10;
Note that the timestamp of m0 is the same as that of m.
In our implementation, the execution of the event mes- callCompletionTime =?;
sage m results in the scheduling of exactly one future event portableMoveOutTime = 16
m0 . When m is executed, one or more attributes of the cor-
responding portable are updated. Then the next event for and an event
the portable is determined based on the updated values of m1 = (10; p1 ; CallArrival)
the attributes. If the event type of m is PortableMoveOut,
then a PortableMoveIn event message with the same times- is scheduled and inserted in the FEL of LPA . When the
tamp is scheduled for the destination LP as described in LVT of LPA advances to 10, m1 is executed by invoking
the denition of PortableMoveOut(). For the other three LPA .CallArrival(p1). Suppose that an idle channel ex-
event types, the message m0 = (ts; p; eventType0) is deter- ists. The call is connected and the call holding time for the
mined by invoking ScheduleNewEvent(): conversation is generated (which is 3, or the call comple-
tion time is 10+3=13). The next call arrival time is also
generated (which is 20 in Figure 3). Thus the attributes of
ScheduleNewEvent(p) f p1 are modied as
if p.busy=NO then
/* The portable is idle at LVT. The next */ busy = YES;
/* event occuring to p is either a call arrival */ callArrivalTime = 20;
/* or a cell crossing movement. */ callCompletionTime = 13;
ts min(p.callArrivalTime,
p.portableMoveOutTime); portableMoveOutTime = 16
if ts = 0p.callArrivalTime then and a new event
m :eventType0 CallArrival;
else m0:eventType0 PortableMoveOut; m2 = (13; p1; CallCompletion)
6 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

is scheduled and inserted in LPA 's FEL. When the LVT IV. Conservative Method
of LPA advances to 13, m2 is executed. The method The conservative simulation [36] is conservative in the
LPA .CallCompletion(p1) is invoked and the attributes of sense that it does not execute an event before it ensures
p1 are modied as that the local causality rule is satised. The conservative
busy = NO;
simulation follows two rules: the input waiting rule and the
output waiting rule. It also assumes that
callArrivalTime = 20;  the messages are received in the order they are sent
callCompletionTime =?; (the FIFO communication property), and
portableMoveOutTime = 16  the communication channels among LPs are xed and
never change during the simulation. In Figure 4(b),
and a new event LPA (LPC ) has one output channel directed to LPB ,
and LPB has two input channels (one from LPA and
m3 = (16; p1 ; PortableMoveOut) one from LPC ).
is scheduled. At LVT 16, m3 is executed. The method A. Basic Synchronization Mechanism
LPA .PortableMoveOut(p1) is invoked to determine the In a conservative simulation, every logical process LP
destination cell (which is B in Figure 3), and a message repeats the following two steps.
m4 = (16; p1 ; PortableMoveIn) Step 1. LP waits to select an input message m from
its input channels (extra data structures are required to
is sent from LPA implement
ing LP .
input channels in a logical process) by invok-
ReceiveMessage(). This method is implemented
to LPB by invoking LPA .SendMessage(m4; LPB ). Note based on the input waiting rule to be described. The
that the portable p1 migrates to LPB when m4 is sent. (In method inserts m into LP 's FEL.
GIT/Bellcore's PCS implementation [4], a message is part Step 2. Let ts be the timestamp of m.
of a portable object, and sending a message automatically LP .ExecuteMessage() is invoked to process all events
migrates the corresponding portable object.) When LPB 's in the FEL with timestamps no larger than ts in non-
LVT advances to 16, it executes m4 . The next portable decreasing timestamp order. The execution may invoke
move time is generated (which is 24). The attributes of p1 LP .SendMessage() to send output messages. This method
are modied as is implemented based on the output waiting rule to be
busy = NO;
callArrivalTime = 20;
callCompletionTime =?;
portableMoveOutTime = 24
and a new event m5 = (20; p1 ; CallArrival) is scheduled.
described. If the termination condition is satised (e.g.,
LP .LVT>5000), then exit the loop. Otherwise go to Step
1.
The waiting rules are described as follows.
The input waiting rule. An LP does not process any
input message until it has received at least one message
from each of its input channels. The input message with
A PDES is correct if the following rule is satised. the smallest timestamp is selected for processing. Figure 5
Local Causality Constraint: Every LP processes shows how the input message is selected for the PCS sim-
events in nondecreasing timestamp order. ulation.
The major problem of PDES is that the logical processes [Figure 5 about here.]
are executed at dierent speeds. Consider the scenario in
Figure 4 that portable p1 moves from cell A to cell B at Figure 5(a) illustrates a PCS system where 6 portables
time 20 with an ongoing phone call (i.e., a hando call), p1 ; p2 ; p3 ; p4 ; p5 , and p6 move from cells B, C, D, E, F,
and portable p2 moves from cell C to cell B at time 13 with and G to cell A at times 30, 10, 26, 4, 12, and 14, respec-
an ongoing phone call (see Figure 4(a)). tively. In the PDES model (see Figure 5(b)), the Portable-
[Figure 4 about here.] MoveIn events of p1 ; :::; p6 are represented by the messages
m1 ; :::; m6 sent to LPA . By the input waiting rule, m4 is
Consider the PDES scenario in Figure 4(b). LPA sends the next message to be executed in LPA .
a PortableMoveIn event (message) m1 (for p1 ) with times- Assume that all messages sent from one LP to another
tamp 20 to LPB . Later LPC sends m2 (for p2 ) with times- are in non-decreasing timestamp order (this property will
tamp 13 to LPB . If LPB executes m1 before m2 arrives, be guaranteed by the output waiting rule to be described
then the modications to LPB .idleChannelNo is out of the next), then the input waiting rule ensures that the times-
timestamp order, and the local causality rule is violated. tamp of the selected message is no larger than any input
Thus the simulation result is not correct. messages to be processed in the future.
To solve this problem, the executions of the logical pro- The output waiting rule. An LP does not send an
cesses must be synchronized. The remainder of this paper output message to another LP until it ensures that no out-
describes two popular asynchronous synchronization mech- put messages with smaller timestamps will be scheduled
anisms, the conservative and the optimistic methods. (at LP) in the future. Assume that all input messages are
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 7

handled in non-decreasing timestamp order (the property LPC does not send out any output message before it re-
is guaranteed by the input waiting rule). The output wait- ceives an input message from LPB , and LPB does not send
ing rule is satised if an LP only sends output messages
with timestamps no larger than its current LVT value.
Consider the following PCS example. Portables p1 ; p2
and p3 move into cell A at times 10, 20, and 30, and move
out of the cell at times 29, 24, and 36, respectively (see
Figure6(a)). This situation occurs since a portable, once
inside cell A, may take a dramatically dierent from other
portables. Some portables may stay in the same physical
location for a period while other portables continue moving
towards an adjacent cell to A.
[Figure 6 about here.]
In PDES, m1 ; m2 , and m3 are input messages represent-
ing the arrivals of p1 ; p2 and p3 , respectively (see Step 1
in Figure 6(b)). When m1 is processed, a move event
m01 for p1 is scheduled with timestamp 29 (see Step 2 in
Figure 6(b)). In the conservative simulation, m01 cannot
be sent to the destination LP immediately, or the output
waiting rule may be violated. In our PCS PDES imple-
mentation, the portable move is simulated by two types
of events: a PortableMoveOut event and a PortableMoveIn
event. In Figure 6(b), m0i and m00i represent the Portable-
MoveOut event and PortableMoveIn event of portable pi ,
respectively. When the event m0i is scheduled, it is in-
serted in LPA 's FEL. When the LVT of LPA advances to
the portable \move time" (i.e., the timestamp of m0i ), m0i
is processed, which results in sending the PortableMoveIn
event m00i (with the timestamp of m0i ) to the destination. In
Figure 6 (b), m002 and m001 are sent after Step (3) and before
Step (4); i.e., when LPA is sure that next input message to
be handled has timestamp larger than m01 and m02 . Note
that m002 is sent before m001 is.
Since the output waiting rule is guaranteed by using the
two \move" event types, the conservative SendMessage()
method simply sends the output message to the destina-
tion. Note that for other applications, a dierent conserva-
tive SendMessage() method may be required to implement
the output waiting rule.
The correctness of the conservative simulation can be
proved by induction on the interaction of the two waiting
rules.
out any output message before it receives an input message
from LPA (i.e., before m1 is processed). Thus the PDES is
in the deadlock situation. Two deadlock resolutions have
been proposed: deadlock avoidance [36] and deadlock de-
tection/recovery [37], [38]. It has been shown [39] that the
cost of deadlock detection/recovery is much higher than
deadlock avoidance. This article will focus on the deadlock
avoidance mechanism.
In a PCS network, a portable is expected to reside in
a cell for a period of time before it moves. Assume that
every portable resides in a cell for at least six time units
before it moves to a new cell. The information that \a
portable resides in a cell for at least 6 time units" is used
in the deadlock avoidance mechanism to predict when an
LP will receive an input message, and \6 time units" is
referred as the lookahead value. The lookahead information
is carried by the control messages called null messages. A
null message does not represent any event in the simulated
system. Instead, it is used to break deadlock as well as to
improve the progress of a conservative simulation.
In Figure 7(b) , at the beginning of PDES, the LVTs
of the three LPs are 0, and a PortableMoveOut event m1
with timestamp 8 is in LPA 's FEL. At time 0, LPA sends a
null message with timestamp 0+6=6 (the LVT value plus
the lookahead value) to LPB (see Figure 7(c)). The null
message implies that no portable will move in cell B earlier
than time 6. Thus, the LVT of LPB advances to 6 when
the null message arrives (Figure 7 (d)). Since no portable
arrives at cell B before time 6, it implies that no portable
will move out of cell B before time 12 and LPB sends a null
message with timestamp 12 to LPC . After the sending of
several null messages, LPA will eventually receive a null
message with timestamp larger than 8 (see Figure 7 (e)),
and by the input and output waiting rules, m1 is sent from
LPA to LPB and the deadlock is avoided (see Figure 7 (f)).
C. Exploiting Lookahead
It is important to exploit the lookahead to improve the
progress of a conservative simulation. Experimental studies
have indicated that the larger the lookahead values, the
better the performance of the conservative simulation [39].
Based on the techniques proposed in [40], [41], [42], we give
B. Deadlock and Deadlock Avoidance
The input waiting rule may result in deadlock (LPs are
waiting for input messages from each other and cannot
progress) even if the simulated system is deadlock free.
Consider a three-cell PCS network (see Figure 7(a)).
[Figure 7 about here.]
There is one portable in the network, and the portable
moves in the path A ! B ! C ! A. At time 0, the
portable is in cell A. The portable moves form cell A to
cell B at time 8. In the conservative simulation, a Portable-
MoveOut event m1 is scheduled in LPA initially (see Fig-
ure 7(b)). By the input waiting rule, LPA waits for an in-
put message from LPC before it can process m1 . Similarly,
three PCS examples for lookahead exploration. The rst
two examples assume single cell entrance and exit. The
single entrance/exit PCS model has been used in modeling
highway cellular phone systems [43]. The results can be
easily generalized for multiple entrances and exits. The
techniques introduced can be combined to exploit greater
lookahead.
1. Lookahead Method 1 (FIFO): In a large scale PCS
network, a cell may only cover a street, and the porta-
bles leave the cell in the order they move in (the FIFO
property; see Figure 8(a)).
[Figure 8 about here.]
Consider the corresponding FIFO LP for cell A in
PDES. The lookahead for the LP can be derived by
8 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

a presampling technique proposed by Nicol [41]. The (a) For j = 1, 1  3  min[0 + 9] = 9 ? No.
idea is to presample the residence times of the arrival (b) For j = 2, 2  3  min[0 + 9; 3 + 4] = 7 ? No.
portables. (c) For j = 3, 3  3  min[0 + 9; 3 + 4; 6 + 1] = 7 ? Yes.
If the FEL is not empty, then the next departure time From this procedure, we derive
= 7 by using the
can be easily computed. In the PCS PDES, the move-
out timestamp of a portable is computed and stored
in portableMoveOutTime of the portable object at the
time when the PortableMoveIn event is processed. The
FIFO property guarantees that the next departure
time is the minimum of the portableMoveOutTime
values of portable objects in the FEL. Thus, the pre-
computed next departure times can be used as the
rst three pre-sampled residence times.
3. Lookahead Method 3 (Minimum Residence Time): If
the FIFO portable movement property does not pre-
serve, and  does not exist (or is too small to be use-
ful), then the technique proposed in the previous ex-
ample may not work. In a PCS simulation, the total
number N = S  n of portables is an input parameter.
To compute the next lookahead value for an LP, it suf-
lookahead.
If the FEL of the LP is empty at timestamp LP .LVT,
then the lookahead can be generated by the same pre-
sampling technique. Since the portable will arrive at
the cell later than LP .LVT, it will leave the cell later
than LP:LVT + t (where t is the presampled portable
residence time). The FIFO property guarantees that
after time LP .LVT, no portables will depart earlier
than LP .LVT+t, and the LP may send null messages
with this timestamp to the downstream LPs.
2. Lookahead Method 2 (Minimum Inter-Boundary
Crossing Time): Consider the example in Figure 8(b)
where the FIFO portable movement property in the
previous example does not hold. In practice, the in-
ter arrival times to the cell (for the portables from
the same entrance) cannot be arbitrary small. In-
stead, a minimum cell crossing time  is assumed.
Let pi (i = 1; 2; 3; :::) be the ith portable arrival af-
ter time LP .LVT. The portable residence time for pi
is ti . Then the departure time of pi is later than
LP:LVT + (i ? 1) + ti , and the next departure time at
the cell after LP .LVT is later than LP:LVT +
where
 

= 1min
i<1
(i ? 1) + ti (1)
Since  > 0, there exists j such that
 
j  1min (i ? 1) + ti =

ij
ces to sample the next N portable residence times,
and (1) is re-written as [42]

= 1min t
iN i
The last two examples may require a large number of op-
erations to generate a lookahead value. In [40], O(1) al-
gorithms have been proposed to generate the lookahead
values.
When the ExecuteMessage() method processes a null
message in
an LP, it invokes a method ComputeLookahead() to com-
pute the timestamp of the output (null) messages. The
ComputeLookahead() method may implement the looka-
head exploiting techniques described above. Then the new
null message is sent to some or all output channels by in-
voking the SendMessage() method.
V. Optimistic Method
The optimistic simulation [44] is optimistic in the sense
that it handles the arrival events aggressively. When a mes-
sage m arrives at an LP, LP .ReceiveMessage() simply in-
serts m in the input queue (the optimistic simulation termi-
nology for the FEL). The logical process assumes that the
events already in its input queue are the \true" next events.
The ExecuteMessage() method proceeds to execute these
events in timestamp order, and SendMessage() is invoked
whenever an output message is scheduled . When a mes-
sage arrives at the LP, the timestamp of the message may
be less than some of the events already executed. (This
arrived message is referred to as a straggler.) The opti-
In other words, to compute
it suces to consider the mism was unjustied, and therefore a method Rollback()
rst j presampled residence timestamps in (1). Fig-
ure 9 displays a situation where we employ formula
(1).
[Figure 9 about here.]
Four portables arrive using times 10, 14, 19 and 22.
Let  = 3 so that we know that no two consecutive
portable arrivals will be less than 3. The residence
times for the portables are placed in parentheses in
Figure 9. The variable j is increased by 1 until the
is invoked by ExecuteMessage() to cancel the erroneous
computation. To support rollback, data structures such as
the state queue and the output queue are required (to be
elaborated).
Several strategies for cancelling incorrect computation
were surveyed by Fujimoto [45]. Two popular cancellation
strategies called aggressive cancellation [44] and lazy can-
cellation [46] are described in this section.
above inequality is satised. Suppose that LPA needs
to send a null message to its downstream before it
receives the PortableMoveIn event for p1 . The resi-
dence times of the subsequent arriving portables are
pre-samples as t1 = 9; t2 = 4; t3 = 1; t4 = 5... Our
algorithm proceeds as follows:
A. Cancellation Strategies
Consider the example in Figure 10.
[Figure 10 about here.]
For simplicity, assume that cell C has one radio chan-
nel (i.e., LPC .channelNo=1 in PDES). In this example,
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION
portable p2 moves from cell B to cell C at time 10 (event
1), and make a phone call at time 13. The call is completed
at time 21. Portable 1 moves from cell A to cell C at time
16 (event 2), and attempts to make a phone call at time
20. Since the only radio channel is used by portable 2, the
call attempt from portable 1 is blocked. Portable 1 moves
from cell C to cell D at time 24. Figures 11, 12, and 13
[Figure 11 about here.]
illustrate the data structures of LPC (the logical process
corresponding to cell C) assuming that message m1 (the
message that represents event 2) arrives at LPC earlier
than message m5 (the message that represents event 1)
does. In LPC , a state queue and an output queue are main-
tained to supported rollback. In our example, the state
variable (attribute) for LPC is the number of idle channels
LPC .idleChannelNo. The state variable is checkpointed
and saved in the state queue from time to time. The snap-
shots in the state queue are used to recover the state of
LPC when rollbacks occur. The output queue records the
anti-messages of the output messages that have been sent
from LPC . The anti-messages are used to annihilated false
messages sent in the incorrect computation.
In Figure 11(a), LPC receives m1 that is inserted in
LPC 's input queue. Initially, the output queue of LPC
is empty, and the value of LPC .idleChannelNo at times-
tamp 0 is saved. After m1 is executed, the system state
at timestamp 16 is checkpointed, and a call arrival event
(message m2 ) is scheduled for LPC itself (see Figure 11(b)).
Note that after its execution, m1 is kept in the input queue
(this message may be re-executed if a rollback occurs). A
pointer in the input queue indicates the next event to be
executed. The anti-message m?2 of m2 is saved in LPC 's
output queue. The message m?2 is identical to m2 except
9
and LPC .Rollback() is invoked. Two strategies for can-
celling incorrect computation are described below.
Aggressive Cancellation. When a straggler arrives,
aggressive cancellation assumes that the out-of-order com-
putation, as well as all other computations that may
have been aected by this computation are not correct.
Thus, the out-of-order computation is recomputed, and
LPC .Rollback() cancels the aected computations imme-
diately by sending anti-messages. In our example, a roll-
back of LPC at timestamp 10 occurs. In Figure 13 (b), the
anti-messages m?2 ; m?3 , and m?4 are
[Figure 13 about here.]
deleted from the output queue, and are sent to their des-
tinations to annihilate false messages m2 ; m3 , and m4 , re-
spectively. After the rollback (see Figure 13 (c)), messages
m2 and m3 (and m4 in LPD ) are removed from the in-
put queue. The state of LPC at timestamp 0 is re-stored.
Then LPC .ExecuteMessage() resumes the simulation by
executing m5 .
Lazy Cancellation. It is possible that the erroneous
computation still generated correct output messages. In
that case, it is not necessary to cancel the original message
that was sent. In lazy cancellation, logical processes do
not immediately send the anti-messages for any rolled back
computation. Instead, they wait to see if the reexecution
of the computation causes any of the same messages to be
regenerated. If the same message is recreated, there is no
need to cancel the original. Otherwise, an anti-message
is sent. In our example, lazy cancellation applies to three
situations.
1. If portable p2 arrives at cell C (LPC ) at time 10 and
leaves cell C at time 28 without making any phone call
(see Figure 14(a)) then the arrival of m5 in Figure 13
that it includes a destination eld (in the original optimistic
or Time Warp algorithm [44], the sender and the destina-
tion are recorded in both the output message and the cor-
responding anti-message for
ow control). To summarize, object.) Thus messages m1 ; m2 , and m3 do not need
the ExecuteMessage() method for the optimistic simula-
tion saves the system state after an event execution (note
that the state may be saved after several event executions),
and the executed event is not deleted from the input queue.
The SendMessage() method saves the anti-messages in the
output queue when it sends an output message.
After m2 is executed, the number of idle channel is decre-
mented by 1, and
LPC :idleChannelNo = 0
is saved in the state queue. A PortableMoveOut event m3
is scheduled at timestamp 24, and its anti-message m?3 is
stored in the output queue (see Figure 12(a)).
[Figure 12 about here.]
When m3 is executed, a PortableMoveIn message m4 is
sent to LPD (see Figure 12 (b)). After m4 is sent, the strag-
gler m5 (the event that p2 moves in LPC at timestamp 10)
ected the correct channel
arrives. Since LPC .LVT=24, the out-of-order execution is
detected (see Figure 13 (a)) by LPC .ReceiveMessage(),
(a) will not aect the executions of m1 ; m2 , and m3 .
(Note that in PDES, whether a call for p2 occurs in
the interval [10,28] can be detected in the portable
to be reexecuted after m5 is executed. This is called
jump forward or lazy reevaluation [1].
[Figure 14 about here.]
In this case, LPC .ReceiveMessage() simply inserts
m5 in the input queue, and the pointer of the input
queue points to m5 . LPC .ExecuteMessage() executes
m5 and the pointer jumps directly after m3 without
re-executing m1 ; m2 , and m3 .
2. The call for p2 does not block the call for p1 if
p2 's call completes before p1 's call arrives (see Fig-
ure 14 (b)) or p2 's call overlaps p1 's call but LPC has
two or more radio channels (i.e., LP:channelNo  2;
see Figure 14(c)). In these cases, the channel uti-
lization (not shown as a state variable in our ex-
ample) changes, but the subsequent messages (i.e.,
m2 ; m3 , and m4 ) scheduled due to the execution of
m1 are not aected. Thus, messages m1 ; m2 , and
m3 are re-executed to re
utilization. No anti-messages need to be sent (i.e.,
m?2 ; m?3 , and m?4 are not sent out). Like the previ-
10 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995
ous case, LPC .ReceiveMessage() simply inserts m5
in the input queue. After m5 has been executed,
LPC .ExecuteMessage() will re-execute m1 ; m2 , and
m3 without re-generating any output messages.
If lazy cancellation does succeed most of the time, then
the performance of the optimistic simulation is improved
by eliminating the cost of cancelling the computation which
would have to be reexecuted. If lazy cancellation fails, then
the performance degrades, because erroneous computations
are not cancelled as early as possible. In our PCS simula-
tion, we may exploit situations that lazy cancellation does
not fail (as described above), and a logical process can be
switched between aggressive cancellation and lazy cancel-
lation to reduce the rollback cost.
B. Memory Management
To support rollback, it is necessary to save the \history"
(the already executed elements in the input, the output,
and the state queues) of a logical process. However, it
may not be practical to save the whole history of a logical
process because memory is likely to be exhausted before
the simulation completes. Thus, it is important that we
only save \recent history" of logical processes to reduce
the memory usage.
Memory management for the optimistic simulation is
based on the concept of global virtual time (GVT). The
GVT at (execution) time t is the minimum of the times-
tamps of the not-yet executed messages (these messages are
either in the input queue or are in transit) in the optimistic
consider the example in Figure 15. In this example, we
ignore the phone call events and assume that all Portable-
MoveIn/PortableMoveOut events must be executed in their
timestamp order in the optimistic simulation. We further
assume that the state variable of a logical process is the
number of portables move in the corresponding cell after
time 0.
[Figure 15 about here.]
Portable 1 moves from cell C to cell A at time 4 and moves
from cell A to cell B at time 60. Portable 2 moves from
cell C to cell B at time 10. Portable 3 moves from cell B to
cell A at time 20. Portable 4 moves from cell A to cell C
at time 8. Portable 5 moves from cell A to cell C at time
7. Portable 6 moves from cell A to cell B at time 1 and
moves from cell B to cell C at time 15.
Figure 16 illustrates the
elements in the input/output/state queues of LPA ; LPB ,
and LPC after all transient messages arrive at their desti-
nations, and the GVT value (which is 8 = min(60; 20; 8))
is found.
[Figure 16 about here.]
Figure 17 illustrates the
elements in the input/output/state queues of LPA ; LPB ,
and LPC after the fossil collection procedure is completed.
[Figure 17 about here.]
simulation at time t. (Several other operational denition
of GVT are given in [47], [48].) It has been pointed out [44]
that at any given time t, a logical process cannot be rolled
back to a timestamp earlier than the GVT at t. Therefore
the storage for all messages with timestamps smaller than
the GVT value can be reclaimed for other usage. The pro-
cess of reclaiming the storage for the obsolete elements is
called fossil collection.
The GVT computation is not trivial in a distributed sys-
tem because it may be dicult to capture the messages in
transit. Several GVT algorithms have been developed in
the systems with the FIFO communication property [49]
or without the FIFO communication property [50], [51].
In GIT/Bellcore PCS PDES (where eight workstations
are connected by a local area network), all logical processes
are frozen during GVT computation. By utilizing the low
level communication mechanism, all transient messages are
guaranteed to arrive at their destinations before the GVT
computation starts. The fossil collection procedure works
as follows. A coordinator initiates the procedure by freez-
ing the execution of every logical process. After all tran-
sient messages arrive at their destinations, every logical
process reports its local minimum value (the minimum of
the timestamps of all unprocessed messages in the input
queue) to the coordinator. The coordinator then compute
All messages with timestamps smaller than 8 were fossil
collected. Note that fossil collection for the state queue is
not exact the same as that for the input/output queues.
In the state queue, the element with the largest times-
tamp smaller than the GVT value (i.e., 8) must not be
removed (see Figure 17). The other elements with times-
tamps smaller than 8 are removed.
C. Performance Evaluation
The performance of an optimistic PCS PDES implemen-
tation has been investigated in [4]. In this study, a version
of Time Warp has been developed that executes on 8 DEC
5000 workstations connected by an Ethernet.
In the experiments, speedp was used as the output mea-
sure where the sequential simulator used the same priority
queue mechanism as that of PDES for managing the pend-
ing set of events, but did not have the state saving, rollback
and fossil collection overheads associated with the PDES
implementation. 1024 cells are simulated for 2.5105 sim-
ulated seconds. Figure 18 shows the performance of the
optimistic PDES.
[Figure 18 about here.]
the GVT value as the minimum of the received local mini- The gure indicates good performance of PDES for the
mums. The GVT value is broadcast to all logical processes PCS application. PDES is particularly ecient when the
for fossil collection. number of portables is large, the cell residence time is long,
To illustrate the storage reclaimed in fossil collection, and the call interarrival time is short.
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION
VI. Future Directions for PDES
This tutorial describes the asynchronous parallel dis-
crete event simulation (PDES) mechanisms and optimiza-
tion techniques by examples of personal communications
services (PCS) network simulation. We described the con-
servative and the optimistic PDES mechanisms and several
optimizations tailored for the PCS simulation. The per-
formance of the optimistic method was brie
y discussed. Distributed Simulation, pp. 21{26, March 1991.
Since the conservative optimizations (tailored for PCS) in-
troduced in this paper are new and were not previously
reported, no performance studies have been conducted. In-
vestigating the performance of these optimizations will be
one of our future research directions.
The optimization techniques described in the paper are
general and apply to other simulation applications such as
11
[10] Fishwick, P.A. and Zeigler, B. P., \A Multimodel Methodol-
ogy for Qualitative Model Engineering", ACM Transactions on
Modeling and Computer Simulation, vol. 2, no. 1, pp. 52{81,
1992.
[11] Fishwick, P.A., \A simulation environment for multimodeling",
Discrete Event Dynamic Systems: Theory and Applications, vol.
3, pp. 151{171, 1993.
[12] Ebling, M., Di Loreto, M., Presley, M., Wieland, F. and Jeer-
son, D., \An Ant Foraging Model Implemented on the Time
Warp Operating System", Proc. 1991 SCS Multiconference on
[13] Hontalas, P., Beckman, B., Diloreto, M., Blume, L., Reiher, P.,
Sturdevant, K., Warren, L., Wedel, J., Wieland, F. and Jeerson,
D., \Performance of the Colliding Pucks Simulation on the Time
Warp Operating Systems (Part 1: Asynchronous Behavior &
Sectoring)", Proc. 1989 SCS Multiconference on Distributed
Simulation, pp. 3{7, March 1989.
[14] Fujimoto, R.M., \Time Warp on a Shared Memory Multipro-
cessor", Proc. 1989 International Conference on Parallel Pro-
cessing, vol. Volume III, pp. 242{249, August 1989.
battleeld simulation, VLSI simulation, queueing network [15] Ayani, R. and Rajaei, H., \Parallel simulation of a general-
ized cube multistage interconnection network", Proc. 1990 SCS
simulation and computer architecture simulation. How- Multiconference on Distributed Simulation, pp. 60{63, January
ever, these optimization techniques may need to be tailored 1990.
for specic applications. Many studies have devoted to this
issue (see [1], [2], [52], [53], [54] and references therein). The
PCS example can be seen as being a member of a larger
[16] Thomas, G.S. and Zahorjan, J., \Parallel simulation of perfor-
mance Petri Net: Extending the domain of parallel simulation",
Proc. 1991 Winter Simulation Conference, pp. 564{573, 1991.
[17] Reed, D.A. and Malony, A., \Parallel Discrete Event Simulation:
The Chandy-Misra Approach", Proc. 1988 SCS Multiconference
class of simulation model where one rst discretizes the
spatial domain into a grid, and then simulates moving en-
tities from one grid cell to another. In this sense, the PCS
problem is isomorphic to the problems of particle/n-body
simulation.
An important research direction that has not been fully
exploited is the building of user-friendly PDES environ-
ments. Such an environment should provides convenient
tools to develop simulation application. Methods should
also be provided to tailor general optimization techniques
on Distributed Simulation, pp. 8{13, February 1988.
[18] Wieland, F., Hawley, L., Feinberg, A., Di Loreto, M., Blume, L.,
Reiher, P., Beckman, B., Hontalas, P., Bellenot, S. and Jeer-
son, D., \Distributed Combat Simulation and Time Warp: The
Model and Its Performance", Proc. 1989 SCS Multiconference
on Distributed Simulation, pp. 14{20, March 1989.
[19] Soule, L. and Gupta, A., \An Evaluation of the Chandy-Misra-
Bryant Algorithm for Digital Logic Simulation", ACM Transac-
tions on Modeling and Computer Simulation, vol. 1, no. 4, pp.
308{347, 1991.
[20] Beazner, D., Lomow, G. and Unger, B., \A parallel simulation
environment based on Time Warp", To appear in International
Journal in Computer Simulation, 1995.
to t a specic simulation application. We anticipate that
these user-friendly environments can be constructed by the
object-oriented models described in [6].
Acknowledgments
C. Carothers and Y.C. Wong provided useful comments
to improve the quality of this paper.
References
[1] Fujimoto, R.M., \Parallel Discrete Event Simulation", Com-
munications of the ACM, vol. 33, no. 10, pp. 31{53, October
1990.
[2] Nicol, D. M. and Fujimoto, R. M., \Parallel simulation today",
Annals of Operations Research, vol. 53, pp. 249{286, December
1994.
[3] Richter, R. and Walrand, J.C., \Distributed Simulation of Dis-
crete Event Systems", Proceedings of the IEEE, vol. 77, no. 1,
[21] Turner, S. and Xu, M., \Performance evaluation of the bounded
Time Warp algorithm", The 6th Workshop on Parallel and Dis-
tributed Simulation, 1992.
[22] Lubachevsky, B., \Ecient Distributed Event-Driven Simu-
lations of Multiple-Loop Networks", Communications of the
ACM, vol. 21, no. 2, March 1989.
[23] Ghosh, K., Panesar, K., Fujimoto, R.M. and Schwan, K.,
\PORTS: A parallel, optimistic, real-time simulator", Proc. 8th
Workshop on Parallel and Distributed Simulation, 1994.
[24] Gaujal, G., Greenberg, A.G. and Nicol, D.M., \A sweep algo-
rithm for massively parallel simulation of circuit-switched net-
works", Journal of Parallel and Distributed Computing, vol. 18,
no. 4, pp. 484{500, 1993.
[25] Cox, D.C., \Personal communications { A viewpoint", IEEE
Commun. Mag., vol. 128, no. 11, pp. 8{20, 1990.
[26] Cox, D.C., \A radio system proposal for widespread low-power
tetherless communications", IEEE Trans. Commun., vol. 39,
no. 2, pp. 324{335, February 1991.
[27] Glynn, P.W. and Heidelberger, P., \Analysis of Initial Transient
Deletion for Parallel Steady-State Simulation", SIAM Journal
pp. 99{113, January 1989. on Scientic and Statistical Computing, vol. 13, no. 4, pp. 904{
[4] Carothers, C., Fujimoto, R.M., Lin, Y.-B. and England, P., \Dis- 922, 1992.
tributed Simulation of PCS Networks Using Time Warp", Proc. [28] Heidelberger, P., \Discrete Event Simulations and Parallel Pro-
International Workshop on Modeling, Analysis and Simulation
of Computer and Telecommunication Systems, pp. 2{7, 1994.
[5] Carothers, C., Lin, Y.-B. and Fujimoto, R.M., \A Re-dial Model
for Personal Communications Services Network", To appear in
45th Vehicular Technology Conference, 1995.
[6] Fishwick, P.A., Simulation Model Design and Execution: Build-
ing Digital Worlds, Prentice Hall, 1995.
[7] Peterson, J.L., Petri Net Theory and the Modeling of Systems,
Prentice-Hall, Inc., Englewood Clis, N.J., 1981.
[8] Law, Averill M. and Kelton, David W., Simulation Modeling &
Analysis, McGraw Hill, 1991, Second edition.
[9] Tooli, T. and Margolus, N., Cellular Automata Machines: A
New Environment for Modeling, MIT Press, 2nd edition, 1987.
cessing: Statistical Properties", SIAM Journal on Scientic and
Statistical Computing, vol. 9, no. 6, pp. 1114{1132, November
1988.
[29] Lin, Y.-B., \Parallel Independent Replicated Simulation on A
Network of Workstations", To appear in SIMULATION, 1995.
[30] Wong, W.C., \Packet reservation multiple access in a metropoli-
tan microcellular radio environment", IEEE J. Select. Areas
Commun., vol. 11, no. 6, pp. 918{925, 1993.
[31] Wong, W.C., \Dynamic allocation of packet reservation multiple
access carriers", IEEE Trans. Veh. Technol., vol. 42, no. 4, 1993.
[32] Lin, Y.-B., \Determining the user locations for personal com-
munications networks", IEEE Trans. Veh. Technol., vol. 43, no.
3, pp. 466{473, 1994.
12 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995
[33] Lin, Y.-B., Mohan, S. and Noerpel, A., \Channel Assignment
Strategies for Hand-o and Initial Access fora PCS Network",
IEEE Personal Communications Magazine, vol. 1, no. 3, pp.
47{56, 1994.
[34] Lin, Y.-B., Mohan, S. and Noerpel, A., \Queueing Priority
Channel Assignment Strategies for Hando and Initial Access
for a PCS Network", IEEE Trans. Veh. Technol., vol. 43, no. 3,
pp. 704{712, 1994.
[35] Lin, Y.-B., Noerpel, A. and Harasty, D., \Sub-rating Channel
Assignment Strategy for Hand-os", To appear in IEEE Trans.
Veh. Technol., 1995.
[36] Chandy, K.M. and Misra, J., \Distributed Simulation: A Case
Yi-Bing Lin received his BSEE degree from
National Cheng Kung University in 1983, and
his Ph.D. degree in Computer Science from the
University of Washington in 1990. Between
1990 and 1995, he was with the Applied Re-
search Area at Bell Communications Research
(Bellcore), Morristown, NJ. In 1995, he was ap-
pointed full professor of Department and Insti-
tute of Computer Science and Information En-
gineering, National Chiao Tung University. His
current research interests include design and
analysis of personal communications services network, distributed
simulation, and performance modeling. He is a subject area editor
Study in Design and Verication of Distributed Programs",
IEEE Trans. on Software Engineering, vol. SE-5, no. 5, pp. 440{
452, September 1979.
[37] Chandy, K.M. and Misra, J., \Asynchronous Distributed Simu-
lation via a Sequence of Parallel Computations", Communica-
tions of the ACM, vol. 24, no. 11, pp. 198{206, April 1981.
[38] Misra, J., \Distributed Discrete-Event Simulation", Computing
Surveys, vol. 18, no. 1, pp. 39{65, March 1986.
[39] Fujimoto, R.M., \Performance Measurements of Distributed
Simulation Strategies", Proc. 1988 SCS Multiconference on Dis-
tributed Simulation, pp. 14{20, February 1988.
[40] Lin, Y.-B. and Lazowska, E.D., \Exploiting Lookahead in Par-
allel Simulation", IEEE Trans. on Parallel and Distributed Sys-
tems, vol. 1, no. 4, pp. 457{469, October 1990.
[41] Nicol, D.M., \Parallel Discrete-Event Simulation of FCFS
Stochastic Queueing Networks", Proc. ACM SIGPLAN Sympo-
sium on Parallel Programming: Experience with Applications,
Languages and Systems, pp. 124{137, 1988.
[42] Wagner, D.B. and Lazowska, E.D., \Parallel Simulation of
Queueing Networks: Limitations and Potentials", Proc. 1989
of the Journal of Parallel and Distributed Computing, an associate
editor of the International Journal in Computer Simulation, an as-
sociate editor of SIMULATION, a member of the editorial board of
International Journal of Communications, a member of the editorial
board of Computer Simulation Modeling and Analysis, Program Co-
Chair for the 8th Workshop on Distributed and Parallel Simulation,
and General Chair for the 9th Workshop on Distributed and Parallel
Simulation.
Paul A. Fishwick is an associate professor
in the Department of Computer and Informa-
tion Sciences at the University of Florida. He
received the BS in Mathematics from the Penn-
sylvania State University, MS in Applied Sci-
ence from the College of William and Mary,
and PhD in Computer and Information Science
from the University of Pennsylvania in 1986.
He also has six years of industrial/government
production and research experience working at
Newport News Shipbuilding and Dry Dock Co.
ACM SIGMETRICS and Performance '89 Conference, pp. 146{ (doing CAD/CAM parts denition research) and at NASA Langley
155, 1989. Research Center (studying engineering data base models for struc-
tural engineering). His research interests are in computer simulation
[43] Kuek, S.S. and Wong, W.C., \Ordered Dynamic Channel As- modeling and analysis methods for complex systems. He is a senior
signment Scheme with Reassignment in Highway Microcells", member of the IEEE and the Society for Computer Simulation. He is
IEEE Trans. Veh. Technol., vol. 41, no. 3, pp. 271{277, 1992. also a member of the IEEE Society for Systems, Man and Cybernetics,
[44] Jeerson, D., \Virtual Time", ACM Transactions on Program- ACM and AAAI. Dr. Fishwick founded the comp.simulation Internet
ming Languages and Systems, vol. 7, no. 3, pp. 404{425, July news group (Simulation Digest) in 1987, which now serves over 15,000
1985. subscribers. He was chairman of the IEEE Computer Society techni-
cal committee on simulation (TCSIM) for two years (1988-1990) and
[45] Fujimoto, R.M., \Optimistic Approaches to Parallel Discrete he is on the editorial boards of several journals including the ACM
Event Simulation", Transactions of the Society for Computer Transactions on Modeling and Computer Simulation, IEEE Trans-
Simulation, vol. 7, no. 2, pp. 153{191, June 1990. actions on Systems, Man and Cybernetics, The Transactions of the
[46] Gafni, A., \Rollback Mechanisms for Optimistic Distributed Society for Computer Simulation, International Journal of Computer
Simulation", Proc. 1988 SCS Multiconference on Distributed Simulation, and the Journal of Systems Engineering.
Simulation, pp. 61{67, February 1988.
[47] Jeerson, D., \Virtual Time II: The Cancelback Protocol for
Storage Management in Time Warp", Proc. 9th Annual ACM
Symposium on Principles of Distributed Computing, pp. 75{90,
August 1990.
[48] Lin, Y.-B., \Memory Management Algorithms for Parallel Simu-
lation", Information Sciences, vol. 77, no. 1, pp. 119{140, 1994.
[49] Lin, Y.-B., \Determining the Global Progress of Parallel Simu-
lation", Information Processing Letters, vol. 50, 1994.
[50] Samadi, B., Distributed Simulation, Algorithms and Perfor-
mance Analysis, PhD thesis, Computer Science Department,
University of California, Los Angeles, 1985.
[51] Mattern, F., \Ecient Distributed Snapshots and Global Vir-
tual Time Algorithms for Non-FIFO Systems", Journal of Par-
allel and Distributed Computing, vol. 18, no. 4, pp. 423 { 434,
1993.
[52] Fujimoto, R.M., \Parallel Discrete Event Simulation: Will the
Field Survive?", ORSA Journal on Computing, vol. 5, no. 3,
1993.
[53] Arvind, D., Bagrodia, R. and Lin, Y.-B., Ed., Proc. 8th Work-
shop on Parallel and Distributed Simulation. ACM, 1994.
[54] Bailey, M. and Lin, Y.-B., Ed., Proc. 9th Workshop on Parallel
and Distributed Simulation. ACM, 1995.
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 13

List of Figures
1 Anatomy of a logical process (LP). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 Cells, logical processes, and processors. A PCS cell is represented by a logical process (LP) in PDES. More
than one LP may be mapped to a processor for execution. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3 A simple PCS example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4 PDES synchronization problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5 The input waiting rule. In (a), the number below a car represents the time when the portable crosses the
cell boundary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6 The output waiting rule. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7 Deadlock and deadlock resolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
8 Examples for lookahead exploiting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
9 Portables entering and leaving cell A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
10 A PCS example for optimistic PDES. Events 1 and 2 will be represented by messages m5 and m1 respectively
in the optimistic PDES (see the next gures). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
11 The data structures of LPC before/after rollback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
12 The data structures of LPC before/after rollback (cont.). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
13 The data structures of LPC before/after rollback (cont.). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
14 Situations when lazy cancellation applies (in these situations, t2 > t1 ). . . . . . . . . . . . . . . . . . . . . 27
15 An PCS example for fossil collection in optimistic PDES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
16 The optimistic PDES before fossil collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
17 The optimistic PDES after fossil collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
18 Speedup of the Optimistic PDES (The call holding time is exponentially distributed with mean 3 minutes.
Eight processors are used in the parallel simulation.) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
14 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

LP LVT schedule
FEL
schedule

Input Channel Output Channel

Buffers Buffers
Event
Routines
next_event

Fig. 1. Anatomy of a logical process (LP).

LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 15

F
E G
A
D B Cells

C
(Hexagonal PCS Network Model)

LPF
LPE LPG
LPA Logical Processes
LPD LPB
LPC (Parallel Simulation Software)

1 2 3 Processors (Physical Processes)

(Multiprocessor Hardware)
Fig. 2. Cells, logical processes, and processors. A PCS cell is represented by a logical process (LP) in PDES. More than one LP may be
mapped to a processor for execution.
16 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

A C

0 10 13 16 20 24 time

Legend:

portable (cross-boundary) movement call arrival call completion

Fig. 3. A simple PCS example.
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 17

A LPA
m1
p1 B
20
LPB
C p2
13

LPC m2

(a) The PCS model. (b) The PCS PDES.

Fig. 4. PDES synchronization problem.

18 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

B
LPB
C G
p1
LPC (m1,30) LPG
30 p6
p2
10
14 (m2,10) (m6,14)
A LPA
p5 (m3,26) (m5,12)
p3
12
26
4 LPD (m4,4) LPF
p4
D F

LPE
E

(a) The PCS model. (b) The PCS PDES.

Fig. 5. The input waiting rule. In (a), the number below a car represents the time when the portable crosses the cell boundary.
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 19

p3 p2 p1 A p3 p1 p2

30 20 10 36 29 24

portable arrivals portable departures

(a) The portables do not depart in the order they arrive.

LVT: 0 LVT: 10
FEL: (m’1,29)
FEL: empty

(m3,30) (m2,20)
(m3,30) (m2,20) (m1,10)
LPA LPA

Step 1. Before m1 is processed Step 2. After m1 is processed

LVT: 30
LVT 20 FRL: (m’3,36)
FRL: (m’2,24) (m’1,29)

(m3,30) (m’’1,29) (m’’2,24)

LPA
LPA

Step 3. After m2 is processed Step 4. After m3 is processed

(b) The PCS PDES

Fig. 6. The output waiting rule.

20 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

LVT: 0
FEL: empty
B
LPB

8
A LPA

LVT: 0 LPC
FEL: (m1,8)
C LVT: 0
FEL: empty

(a) The three-cell PCS model. (b) Deadlock in PDES.

LVT: 0
FEL: empty LVT: 6
FEL: empty

LPB LPB
(null, 6)

LPA LPA (null, 12)

LVT: 0 LPC
LVT: 0 LPC FEL: (m1,8)
FEL: (m1,8)
LVT: 0
LVT: 0 FEL: empty
FEL: empty

(d) LPB sends a null message to LPC .

(c) LPA sends a null message to LPB .

LVT: 6
LVT: 6 FEL: empty
FEL: empty

LPB (m1, 8) LPB

LPA LPA
(null, 18)

LVT: 0 LPC
FEL: (m1,8) LVT: 18 LPC
FEL: empty
LVT: 12
FEL: empty LVT: 12
FEL: empty
(e) LPC sends a null message to LPA .
(f) m1 is processed and sent to LPB .

Fig. 7. Deadlock and deadlock resolution.

LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 21

p3 p2 p1 A p3 p2 p1

portable arrivals portable departures

(a) The portables depart in the order they arrive.

p2 p1
A

portable arrivals
t

(b) Non-zero inter-boundary crossing time (t >  ).

Fig. 8. Examples for lookahead exploiting.

22 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

p4 p3 p2 p1 A p4 p3 p1 p2

22(5) 19(1) 14(4) 10(9) 27 20 19 18

τ=3
portable arrivals portable departures

Fig. 9. Portables entering and leaving cell A.

LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 23

A D

event 2
C
Portable 1
0 16 20 24 time

B event 1

Portable 2
10 13 21 time

Legend:

portable movement call arrival call completion call blocking

Fig. 10. A PCS example for optimistic PDES. Events 1 and 2 will be represented by messages m5 and m1 respectively in the optimistic
PDES (see the next gures).
24 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

m2
LPA m1 LPA

LPC LPD LPC LPD

LPB LPB
pointer
Input Queue
m1 m2
timestamp 16 20
pointer portable p1 p1
portable call
Input Queue event move
in arrival

m1
timestamp 16 State Queue
State Queue
portable p1 timestamp 0 16
portable
event move
in no. idle ch. 1 1

State Queue Output Queue

timestamp m-2
0
no. idle ch. 1
timestamp 20
portable p1
call
Output Queue event arrival

null destination LPC

(a) Before m1 is executed. (b) After m1 is executed.

Fig. 11. The data structures of LPC before/after rollback.

LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 25

m3 LPA
LPA m4
LPC LPD
LPC LPD
LPB
LPB

pointer pointer
Input Queue Input Queue

m1 m2 m3 m1 m2 m3
timestamp 16 20 24 timestamp 16 20 24
portable p1 p1 p1 portable p1 p1 p1
portable call portable portable portable
move move call
event in arrival out event move
in arrival
move
out

State Queue State Queue

timestamp 0 16 20 timestamp 0 16 20 24
no. idle ch. 1 1 0
no. idle ch. 1 1 0 0

Output Queue
Output Queue
m-2 m-3
m-2 m-3 m-4
timestamp 20 24
timestamp 20 24 24
portable p1 p1
portable portable p1 p1 p1
event call move
arrival out portable portable
call move move
destination LPC LPC event arrival out in

destination LPC LPC LPD

(a) After m2 is executed.

(b) After m3 is executed.

Fig. 12. The data structures of LPC before/after rollback (cont.).

26 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

m-2 m-3
LPA
m-4
LPC LPD
LPB

(b) The rollback in progress.

LPA

m5 LPC LPD
LPB
LPA
pointer
Input Queue

m5 m1 m2 m3 LPC LPD
timestamp 10 16 20 24

portable p1 p1
portable
p1 p1
portable
LPB
portable call move
move move
event in in arrival out

State Queue
timestamp 0 16 20 24
pointer
no. idle ch. 1 1 0 0 Input Queue

Output Queue
m5 m1
m-2 m-3 m-4
timestamp 10 16
timestamp 20 24 24
portable p1 p1
portable
p1
portable portable p1 p1
call move move
event arrival out in
portable portable
LPC LPC LPD move move
destination event in in

(a) Before the rollback. State Queue

timestamp 0
no. idle ch. 1

Output Queue null

(c) After the rollback.

Fig. 13. The data structures of LPC before/after rollback (cont.).

LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 27

phone conversation interval residual time interval

real time
t2
(portable p2)

t1
(portable p1)

10 16 20 24 28 virtual time (timestamp) in LP

C

(a) Situation 1. The number of radio channels in LPC is 1.

real time
t2
(portable p2)

t1 (portable p1)

10 12 14 16 20 23 24 virtual time (timestamp) in LPC

(b) Situation 2. The number of radio channels in LPC is 1.

real time
t2 (portable p2)

t1 (portable p1)

10 16 17 20 21 22 24 virtual time (timestamp) in LPC

(c) Situation 3. The number of radio channels in LPC is 2.

Fig. 14. Situations when lazy cancellation applies (in these situations, t2 > t1 ).
28 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

B
p1
60
1

7 p3
8
20
C 4 p5 p4

10 p2

p6
15

Fig. 15. An PCS example for fossil collection in optimistic PDES.

LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 29

LPA LPB
pointer
pointer
Input Queue
Input Queue
m0 m1 m2 m3 m4
m5 m6 m7
timestamp 1 4 7 8 60
timestamp 1 15 20
portable p6 p1 p5 p4 p1
portable portable portable portable portable p6 p6 p3
portable
move move move move move portable
event out in out out out portable
move
portable
move move
event in out out

State Queue
State Queue
timestamp 0 1 4 7 8 timestamp 0 1 15
move-in count 0 0 1 1 1
move-in count 0 1 1

Output Queue
Output Queue
m-5 m-9 m-10 m-4
m-6 m-11
timestamp 1 7 8 60
timestamp 15 15
portable p6 p5 p4 p1
portable portable portable portable portable p6 p6
event move
in
move move
in
move
out portable portable
in event move
out
move
in
destination LPB LPC LPC LPA
destination LPB LPC

LPC
pointer
Input Queue

m10 m12 m11 Output Queue

m8 m9
m-1
timestamp 4 7 8 10 15
timestamp 4
portable p1 p5 p4 p2 p6
p1
portable
move
portable
move
portable
move
portable
move
portable
move portable
event out in in out in portable
move
in
event
State Queue destination LPA

timestamp 0 4 7
move-in count 0 0 1

Fig. 16. The optimistic PDES before fossil collection.

30 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

LPA LPB LPC

pointer pointer
pointer
Input Queue
m6 m7 m10 m12 m11
m3 m4
timestamp 15 20 8 10 15
8 60
p6 p3 p4 p2 p6
portable p4 p1
portable portable portable portable portable
portable portable move move move move move
move move in out in
event out out out out

State Queue
1 15 7
timestamp 7 8
1 1 1
move-in count 1 1

Output Queue
m-10 m-4 m-6 m-11
null
8 60 15 15
timestamp
p4 p1 p6 p6
portable
portable portable portable portable
move move move move
event in out out in

LPC LPA LPB LPC

destination

Fig. 17. The optimistic PDES after fossil collection.

LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 31

8

.....................

7
.......................


......................
......................


...................................
.............................................
.............................................


........................................ ............................................
........................................ .............................................
...................................... .............................................
....................................

6
.................................... .............................................


.................................... ............................................
................................... .............................................
.................................... .............
....................................

 ....................................

Sp
...... ....................................


...... ....................................
................................

5
....... .....................
......
......
......
......

ee
......
......
......
......

4
......
.......
......


......
..............

du
....................................................
.....................................................
.....................................................
.....................................................

3
.....................................................


.....................................................
.....................................................

p
.................

: the mean cell residence time = 15 minutes

2 : the mean cell residence time = 45 minutes
: the mean cell residence time = 75 minutes
1
0
5 15 20 25 30 10
Call interarrival time (minutes)
(a) The expected number of portables per cell is 50.
8 
.........................
...........................


...........................

..........................

7
............................
................................ ...........................


................................... ..........................
.................................. ...........................
....................... ............................
....................... ...........................
........................ ...........................
....................... ..........................
....................... ............................

6
....................... ...........................
....................... ...........................
........................ ..........................


....................... ...........................


....................... ............................
....................... ......

Sp
...................................... .......................
......................................

.....................................................
........................
.......................

5
.......................................................................... ........................


.......................................................................... .......................
.......................................................................... .................
..........................................................................
.......................................................................

ee
4
du
p 3
: the mean cell residence time = 15 minutes
2 : the mean cell residence time = 45 minutes
: the mean cell residence time = 75 minutes
1
0
5 15 20 25 30 10
Call interarrival time (minutes)
(b) The expected number of portables per cell is 75.

Fig. 18. Speedup of the Optimistic PDES (The call holding time is exponentially distributed with mean 3 minutes. Eight processors are
used in the parallel simulation.)
32 IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, VOL. XX, NO. Y, MONTH 1995

List of Tables
I Attributes and methods of an LP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
LIN AND FISHWICK: ASYNCHRONOUS PARALLEL DISCRETE EVENT SIMULATION 33

Mechanism Attributes Methods

Clock LVT LVTUpdate()
FEL eventList Enqueue()
Dequeue()
Cancel()
Synchronization ReceiveMessage()
SendMessage()
ExecuteMessage()
Application to be elaborated to be elaborated

TABLE I
Attributes and methods of an LP.