IEEE TRANSACTIONS ON COMPUTERS, V O L . C-30, NO.

2, FEBRUARY 1981 161

To invoke an operation on a declared instance functional register Request ^

requires notation of the following form
MASTER Reply SLAVE
SHFTCNT ( 1 , 0 , 1, (1)
In this case the counter is incremented if the variable a = 1. It will Data Bus
f f
be noted that the functional register will be incremented when Fig. 1. Nonredundant system model.
enp Kent = 1. For this particular operation of SHIFCNT, enp.ent
= a, 1, and, thus, enp ent = a. It will be noted that the four bit data S V
vector A had no effect on this particular invocation. Any 4-bit vector M A S T E R S L A V E
could have entered in expression (3) in place of A. sv
*
VI. CONCLUSIONS

In AHPL III we have developed a language which is capable of

expressing both structure and the sequencing of activity. This lan S V
guage extends the application of hardware description languages to M A S T E R S L A V E
the interconnections MSI and LSI parts, as well as to VLSI chip r^-Hsv
design. This has been accomplished while remaining consistent with
AHPL II in all important respects.

REFERENCES

[ 1 ] F. J. Hill and G. R. Peterson, Digital Systems: Hardware Organization

and Design, New York, 1978.
[2] R. E. Swanson, Z. Navabi, and F. J. Hill, "An AHPL compiler/simulator
system," in Proc. 6th Texas Conf. on Comput. Syst., Austin, TX, Nov.
14-15, 1977.
[3] Z. Navabi, R. E. Swanson, and F. J. Hill, "User manual for AHPL
simulator, AHPL compiler," Eng. Experiment Station, Univ. of Arizona,
Sept. 1977.
[4] F. J. Hill, "Updating AHPL," in Proc. 1975 Int. Symp. on Hardware
Description Lang, and Their Applications, New York, Sept. 3-5,1975,
pp. 22-29.
[5] M. R. Barbacci et al., "An architecture research facility: ISP descriptions,
simulation data collection," in Proc. AFIPS, vol. 46, NCC 1977, pp.
161-173.
[6] Y. Chu, "An ALGOL-like computer design language," Commun. Ass.
Comput. Mach., pp. 607-615, Oct. 1965.
[7] J. R. Duley and D. L. Dietmeyer, "A digital system design language
(DDL)," IEEE Trans. Comput., vol. C-17, pp. 850-561, Sept. 1968.
[8] R. Piloty, "Segmentation constructs for RTS III," in Proc. 1975 Int.
Symp. on Hardware Description Lang., New York, Sept. 1975, pp.
115-124.
Synchronization and Voting
STEPHEN R. M c C O N N E L A N D DANIEL P. SIEWIOREK
AbstractThis is an elaboration of the paper "Synchronization
and Matching in Redundant Systems" by Davies and Wakerly [1]. The
design of voters for synchronization is strongly dependent on the sig
naling convention used. This correspondence presents voter designs
for three different signaling conventions (transition, level, and pulse).
The issue of improved voter performance is also addressed.
Index TermsAsynchronous signaling conventions, fault tolerant
computing, synchronization, triple modular redundancy (TMR),
voters.
Manuscript received August 14, 1978; revised July 18, 1980. This work was
supported by the Office of Naval Research under Contract NR-048-645. The
views and conclusions contained in this correspondence are those of the authors
and should not be interpreted as representing the official policies, either ex
pressed or implied, of the Office of Naval Research or the U.S. Govern
ment.
S. R. McConnel is with the Department of Electrical Engineering, Car
negie-Mellon University, Pittsburgh, PA 15213.
D. P. Siewiorek is with the Departments of Computer Science and Electrical
Engineering, Carnegie-Mellon University, Pittsburgh, PA 15213.
Fig. 2. Triplicated system model.
R E Q U E S T
R E P L Y
D A T A >C3C (VALID)
Fig. 3. Transition signaling convention.
INTRODUCTION
Different signaling conventions are used in the implementation of
control signals. When the control signals are used to mark the validity
of other (data) signals, such as on a bus, special care is required to
ensure that voter operation meets the specification of the signaling
convention. The purpose of the sample circuits discussed here is to
illustrate the types of modifications to the basic voting synchronizer
necessary to properly handle the three basic signaling conven
tionstransition, level, and pulseunder the assumption of the
stuck-at fault model. It should not be assumed that these circuits
totally solve all problems associated with synchronizing multiple
systems to allow majority voting.
The system model used as the basis of discussion is shown in Fig.
1. The master issues a "request" to the slave, which responds by
placing data on the bus and activating the "reply" signal. It is assumed
that the master will not post a second request until after the reply has
been received for the first one. The expansion of this simple system
into a triple modular redundancy (TMR) system is shown in Fig. 2.
Note the placement of synchronizing voters (marked "SV") on both
the "request" and "reply" lines. This allows asynchronous operation
of all (master and slave) modules. This model differs somewhat from
that used by Davies and Wakerly and more closely corresponds to the
typical bus structure of minicomputers and microcomputers. The
extension to multiple control lines and a bidirectional data bus is
straightforward, although unnecessary for the analysis developed
here.
REVIEW OF SYNCHRONIZATION VOTING
The basic synchronizing voter concept (as shown in Fig. 4) consists
of a simple majority voter followed by a delay element. The value of
the delay time is greater than the maximum allowable differences in
arrival times for the signals from the different modules. This delay
allows the system as a whole to wait for all modules to activate the

0018-9340/81/0200-0161$00.75 1981 IEEE

162
A REQUEST_
C
Fig. 4. Synchronizing voter.
A
C D
Fig. 5. Data voter.
Fig. 6. Synchronizing voter for transition signaling convention with
relaxed constraints.
signal on each cycle. If one of the modules operates so slowly that it
exceeds the a priori time limit imposed by the synchronizing voter,
it is considered to have failed. The synchronizing delay should be
considerably shorter than the total time required for a single "re
quest-reply" cycle.
TRANSITION SIGNALING CONVENTION
Each signaling convention requires a different voter circuit. The
transition convention, illustrated in Fig. 3, signals "requests" and
"replies" on all transitions, either from low to high or from high to
low. Assuming the associated data is kept valid from the time of a
"reply" until the next "request," the circuits shown in Figs. 4 and 5
will correctly handle synchronization of control and data signals. As
either transition requires synchronization, a simple delay is needed
in the voters for the control signals. (Note that this is the only con
vention for which the simple model of a synchronizer will work
properly.) Assuming that the voted data will be valid at least as soon
as two of the "replies" are returned and will stay valid until the next
voted "request," only a simple majority voter is required for the
data.
The voter shown in Fig. 4 requires that all three systems be ini
tialized to the same state, such that each system's control signal
undergoes the same transition (low-to-high or high-to-low) on each
transaction. If this restriction is lifted, a much more complicated
circuit such as the one shown in Fig. 6 is needed. The latter circuit
works by internally changing each transition to a "level" signal for
voting upon.
LEVEL SIGNALING CONVENTION
The level convention (commonly found in computers such as the
LSI-11) signals "requests" and "replies" by a transition from low to
high, with the request (or reply) being maintained until the level falls
low again. This convention is illustrated in Fig. 7. As the data are
specified to be valid until after the "reply" is removed, the voted
control signal cannot be maintained at the high level after the data
have been removed. Thus, the delay following the voter for synchro
nization purposes must occur only on the rising edge (low-to-high)
of the signals. If the falling edge were delayed, the voted "reply"
IEEE T R A N S A C T I O N S ON COMPUTERS, V O L . C-30, N O . 2, F E B R U A R Y 1981
REPLY
DATA
Fig. 7. Level signaling convention.
Q
A- I ONE
? 1
7474
V
SHOT ^CLR
C
Y
Fig. 8. Synchronizing voter for level signal convention.
A
V
C
Fig. 9. Data voter for level signaling convention.
REQUEST
REPLY
DATA (VALID) ^' (VALID) )<f~
Fig. 10. Pulse signaling convention.
would be signaling that the data were valid after such was no longer
the case. The resulting voters for synchronization and data signals
are shown in Figs. 8 and 9. The one-shot and D flip-flop serve for the
synchronizing delay. The rising edge of the signal coming from the
majority voter triggers a low-going pulse on the one-shot. When that
pulse terminates, the D flip-flop triggers, clocking in the value of the
voted signal. As the voted signal is attached to the "reset" line of the
flip-flop when a majority of the voter input signals go low, so does the
output of the circuit. Once again, a simple majority voter suffices for
the data.
PULSE SIGNALING CONVENTION
The pulse convention, illustrated in Fig. 10, provides only a mo
mentary signal for "request" and "reply." An alternative design to
that presented in Davies and Wakerly for the synchronizing voter is
shown in Fig. 11. The special problem posed by the pulse convention
is highlighted by the complexity of the data voter, shown in Fig. 12.
As the data is valid on each bus for only a short time and the three
independent "replies" may not be valid simultaneously, giving valid
voted data, care must be taken to properly latch the data to ensure
valid setup and hold times for the voted "reply." This is done by
latching the data on each bus when their own "reply" is received and
holding the data until the specified time after the voted "reply" is
released.
MINIMIZING DELAYS IN T H E SYNCHRONIZING VOTERS
As alluded to in Davies and Wakerly and earlier in Siewiorek et
al. [2], a bypass of the delay in a synchronizing voter can be desirable
when all three signals have arrived. The circuits of Figs. 13-16 show
such an addition for each of the three signal conventions. In each case
the voters for data signals given in the previous examples are still valid.
For this new scheme to function properly, additional restrictions may
have to be placed on the specified operation of the system, depending
on the particular signaling convention used. Using the level signaling
convention as an example, it is necessary that a slow slave be able to
remove its "reply" signal in response to the removal of the voted
"request" signal before the next voted "request" appears. Otherwise,
its lagging "reply" may be misinterpreted.
The circuit shown in Fig. 13 for transition-based signals produces
LOW (and HI) output whenever all three inputs are LOW (or HI).
Thus, when all three input signals have arrived, the voted result is send
immediately. The delay occurs only when one of the three inputs lags
behind the other two, allowing it to catch up. Fig. 14 shows a modified
voter for transition signaling convention systems which do not require
global initialization. Its operation is closely related to that of the
following circuits in Figs. 15 and 16.
IEEE T R A N S A C T I O N S O N C O M P U T E R S , V O L . C-30, N O . 2, F E B R U A R Y 1981 163

D Q
7474CLR

T
ONE y ONE
D Q S H O T l ISHOTI
7474CLR

Q
7474
Fig. 11. Synchronizing voter for pulse signaling convention.
DATA A
D Qh
REPLY A
7475
Fig. 14. Modified synchronizing voter for transition signaling convention
with relaxed constraints. .
DATA V O T E D
D Q D A T A
REPLY
7475
A-
DQ
DATA C B-
ONE
7474
CLR
ISHOTl
D Q c-
R E P L Y C
7475 Fig. 15. Modified synchronizing voter for level signaling convention.
V O T E D ONE TJ" I ONE THESE LATCHES PASS D TO

R E P L Y SHOT SHOT WHEN Q IS HIGH

D Q
Fig. 12. Data voter for pulse signaling convention.
7474
D Q
-A 744
/
D Q ONE b- >CLR
ISHOTI
7474
t><*R

Q
Fig. 13. Modified synchronizing voter for transition signaling
7474 SHOT|
convention. X
Fig. 16. Modified synchronizing voter for pulse signaling convention.
The circuit shown in Fig. 15 is in actual use in C.vmp [3]. The
change made to the circuit of Fig. 8 is "presetting" the D flip-flop and the delay is cut short using the "clear" input of the one-shot. Thus,
"clearing" the one-shot (e.g., type 74221 [4]) whenever all three voter the full delay is used only when necessary: a lagging signal is given
input signals have arrived. This bypasses the delay when it is unnec every chance to catch up with the other two. Note that this technique
essary, while still utilizing it when one of three inputs is slow or has of "clearing" the delay of the one-shot is very similar to that used in
failed. This circuit is used in C.vmp for the set of basic handshaking the previous circuit for level-based signals.
bus synchronization signals. It has proven capable of providing ade
quate synchronization of a three processor system. A similar circuit 1

has been implemented in a custom LSI voter chip designed to allow
easy implementation of triplicated microcomputer systems [5].
The circuit in Fig. 16 for pulse-based signals is based on the same
reasoning as the earlier ones: when all three input signals have arrived,
1
C.vmp uses a common clock to synchronize the three processors at the mi
crocode level in order to reduce bus disagreements as much as possible. This
is important in light of the goal of measuring transient errors. Normally, the
"bus reply" signal is latched by the voter one clock phase-100 ns-prior to its
sampling by the processors. This ensures totally synchronous reception of the
signal, minimizing the bus disagreements during normal operation. To test
the robustness of the synchronization provided by the special synchronizing
voter circuits used for the basic bus control lines, the system was rewired to
pass the voted "bus reply" directly to the processors without latching. During
this test it was observed that the three processors very frequently fell out of
step with each other at the microcode level, but were kept synchronized at the
bus transaction level, maintaining proper execution of programs.
REMAINING PROBLEMS
One problem of the last set of voters is that a failure in one of the
inputs can cause degradation of performance back to that of the
previous designs. This delay in the presence of an input fault is un
avoidable, as full allowance must be made for the variance in timing
between the two surviving modules.
A more serious problem is that posed by runt pulses or critical races
in inputs to bistable circuits [6], [7]. Fig. 17 shows a possible syn
chronizer to reduce this problem, based on a proposed circuit in [7].
Whenever the D flip-flops are in the meta-stable state, the Q and Q
outputs are equal, which is detected by the comparators, causing the
sampling clock to stop until a stable state is entered and the outputs
are again complementary. The signals A', B\ and C can be used as
the inputs to the circuits given previously, with greater assurance of
reliable synchronization.
 164 IEEE T R A N S A C T I O N S ON
FREE-RUNNING GENERATOR
Fig. 17. Sampling synchronizer for voter inputs.
It should be noted that the circuits given in Figs. 13-16 violate the
necessary conditions for solving the problem of "inconsistent failures"
by successive stages of synchronizers, as suggested by Davies and
Wakerly. It is a matter for engineering judgment whether greater
performance or greater fault survival is most important, depending
primarily on the relative frequency of that particular failure mode.
CONCLUSIONS
A significant amount of circuitry may be required to successfully
implement synchronizing voters. In the context of computer buses,
only a relatively few signals need this complexity (for example, less
than one-sixth for the LSI-11 bus [3]). This makes the application
of synchronizing voters quite practical, as it eliminates the overhead
of providing a fault tolerant common clock.
ACKNOWLEDGMENT
Dr. Davies provided several helpful comments concerning the op
eration of the voter circuits described in this paper.
REFERENCES
[ 1 ] D. Davies and J. F. Wakerly, "Synchronization and matching in redun
dant systems," IEEE Trans. Comput., vol. C-27, pp. 531-539, June
1978.
[2] D. P. Siewiorek, M. Canepa, and S. Clark, "C.vmp: The architecture and
implementation of a fault-tolerant multiprocessor," in Proc. 7th Annu.
Int. Symp. on Fault-Tolerant Comput., 1977, pp. 37-43.
[3] D. P. Siewiorek, V. Kini, H. Mashburn, S. R. McConnel, and M. Tsao,
"A case study of C.mmp, Cm*, and C.vmp: Part IExperiences with
fault tolerance in multiprocessor systems," Proc. IEEE, vol. 66, pp.
1178-1199, Oct. 1978.
[4] Texas Instruments Incorporated, The TTL Data Book for Design En
gineers, 2nd ed., 1976.
[5] S. R. McConnel and D. P. Siewiorek, "C-MU voter chip," Carnegie-
Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-80-107, Mar.
1980.
[6] T. J. Chaney and C. E. Molnar, "Anomalous behavior of synchronizer
and arbiter circuits," IEEE Trans. Comput., vol. C-22, pp. 421-422, Apr.
1973.
[7] M. Pechoucek, "Anomalous response times of input synchronizers," IEEE
Trans. Comput., vol. C-25, pp. 133-139, Feb. 1976.
0018-9340/81/0200-0164$00.75
C O M P U T E R S , V O L . C-30, N O . 2, FEBRUARY 1981
The Topology of Cellular Partitioning Networks
JAN G E C S E I A N D J-P. B R A S S A R D
AbstractWe investigate generalizations of triangular permuting networks
in two directions: the connecting power of cells and the network topology. The
main result indicates that permuting, coupling, and partitioning capabilities
can be obtained by using 2-, 3-, and 4-state cells, respectively, in a large class
of network topologies.
Index TermsCellular arrays, interconnection networks, partitioning
networks.
I. INTRODUCTION AND BACKGROUND
Most work to date in cellular interconnection networks has been
done on networks with 4-terminal, 2-state cells and with particular
topologies to achieve specific network characteristics. Such efforts
include minimization and study of different measures of complexity
(such as cell count, path length, control algorithms) [ l ] - [ 3 ] achieving
nonblocking or rearrangeable behavior, or specific applications, such
as sorting [4]. A recent excellent review is found in [5].
This paper investigates another aspect of cellular networks: the
linkage between connecting properties of 2-, 3-, and 4-state cells and
a whole class of network topologies defined by so-called M-diagrams.
The principal new results show that permuters, coupling networks,
and partitioning networks with even sized partitions can be obtained
by just changing the functional capabilities of cells and not the to
pology in a large family of networks. These results generalize findings
of two earlier papers [6], [10] concerning triangular networks with
3-state cells and path building in permuting triangular networks.
II. TERMINOLOGY
An interconnection network is a system capable of interconnecting
different subsets of its terminals. Various types of such networks are
useful as architectural components permitting dynamically variable
interconnections of systems modules and resources such as commu
nication lines, processors, memories, etc.
Now we review some notions used in this paper. This review is by
no means exhaustive; more complete terminology and classification
of networks is found, e.g., in [5]. Here we classify networks according
to three critiera: network function, network topology, and permissible
connection states of elementary cells.
A. Function
Functionally, we distinguish the following types of networks (in
decreasing order of functional generality).
Partitioning Network: n-terminal network capable of realizing any
partition of the set of terminals in such a way that terminals belonging
to each equivalence class (subset) are all connected together [ 7 ] .
-partitioning network: n~ terminal network (n even), capable of
realizing any partition with even-sized equivalence classes.
Coupling network (unspecialized network [8] or full switch [6]).
Partitioning network capable of realizing any partition with all
equivalence classes of size 2 (i.e., n/2 pairwise connections of termi
nals).
Permutation Network: Partitioning network where the terminals
are a priori divided in two subsets of size n/2 each, called input and
output terminals. The network is capable of realizing any partition
such that each equivalence class contains only one input and one
output terminal [9].
Manuscript received September 9, 1979; revised April 3, 1980 and August
1, 1980.
The authors are with the University of Montreal, Montreal, P.Q.,
Canada.
1981 IEEE