Models of Language Recognition:

Automata

1
 6. Variations of Automata
The four types of automata, TM, LBA, PDA, and FA (either deterministic or nondeterministic) are typical models for
computation that have been extensively investigated in the area of theory of computation. Many variations of these automata have
been introduced, including automata with variations on the number of tapes or stacks, tape dimensions, number of heads, head
mobility, etc. This chapter introduces some of these interesting variations and discusses their computational capability.
By computational capability we are concerned only with a computational model’s capacity to show us a final result, with no
regard to the amount of space or time used. If two models have the same computational capability, we say they are equivalent.
Simulation is a popular technique for showing that two computational models are equivalent. If we can show that one model can
simulate the other and vice versa, then they are equivalent. We will see a couple of application examples of this technique.

6.1 Transducers: automata with output 161

Mealy automata, Moore automata
6.2 Variations of TM and LBA 162
Multi-track TM, Multi-tape TM, 2D-tape TM
6.3 Variations of PDA 168
2-way PDA, Multi-stack PDA
PDA accepting with empty stack
6.4 Variations of FA 176
2-way FA, 2D-tape FA, Multi-head FA,
FA array: cellular automata, Hidden Markov models (HMM)
6.5 Church’s hypothesis 181
Exercises 182

2
 Variations of automata
6.1 Transducers: Automata with output
The automata that we have introduced produce (implicitly) a simple output “yes”
by entering an accepting state if the input string belongs to its language. Attaching
an output port to the finite state control we can extend this limited output capacity so
that for the given input string they produce an output string of finite length (see
Figure (a) below for an FA). We call such automata transducers.
Depending on the time of output, there are two variations. The one produces an
output while taking a transition, and the other after the transition (i.e., on the next
state). In particular for FA, the former models are called Mealy automata, and the
latter Moore automata. (See the figure (b) and (c), where b is an output symbol.)

b
--a-- a
a/b
output b
port

(a) Transducer (b) Mealy automaton (c ) Moore automaton

3
 Variations of Automata
6.2 Variations of TM
The TM with one-dimensional tape and one read/write head can be extended
to multi-track, multi-tape, or multi-dimensional tape TM. We will see that
these variations are equivalent to the standard TM. In other words, if one of
these variations recognizes a language L, we can build a standard TM that
recognizes L, and vice versa.

Multi-track TM
A multi-track TM has a read/write tape organized into multiple (i.e., some
constant k ≥ 2) tracks. The input string is written on one of the tracks and during
the computation, all the cells in one column are read and then written
simultaneously.

a b a a b a b a a b
b c

p r

4
 Variations of Automata
Variations of TM
The figures below illustrate that by encoding k symbols that can appear in a
column of k-track tape, we can transform a k-track tape TM M to the standard
1-track TM M’ that recognizes the same language.
Conversely, for a given 1-track TM, it is trivial to construct a k-track TM that
recognizes the same language. Hence, 1-track TM’s and multi-track TM’s are
equivalent.

a b a a b a a b
α β
b c Encoding
[a, B] = a
M r M’
r [b, B] = b
[a, b] = α
([a, B], [a, b], R) q (a, α , R) q
[b, c] = β
.
p . p

5
 Variations of TM Variations of Automata

Multi-tape TM
A multi-tape TM has some constant k ≥ 1 auxiliary tapes in addition to the input
tape. Each tape head can move independent of others. Given a (k+1)-tape TM M, we
can build a (k+1)-track TM M’ which simulates M as follows. Put the k+1 tracks of
M’ in correspondence with the k+1 tapes of M as illustrated below. Use a check mark
for each track to indicate the position of its corresponding tape head. To simulate the
next move of M, M’ scans its tape and collects the k+1 check marked symbols, applies
the transition function of M and then updates its tape accordingly.
Given a multi-track TM, it is trivial to construct a multi-tape TM that recognizes the
same language. It follows that both TM models are equivalent. Since multi-track TM’s
are equivalent to standard TM’s, multi-tape TM’s are equivalent to standard TM’s.

a b b a
Input tape a√ b b a
M
c√ d
auxiliary tapes
a b√
c d
M’
a b
6
 Variations of TM Variations of Automata

2D-tape TM
We can extend the tape dimension of the standard TM to a multi-dimension one. A
two-dimensional (2D) tape TM is illustrated below. The finite state control, with a
read/write head under it, can move up, down, to the right or to the left. This model is
widely used for pattern analysis and searching.
We can show that the tape’s dimension does not affect computational capability. For
example, given a standard TM M, it is trivial to build a 2D-tape TM M’, which, given
the input string on one of its tape rows, simply simulates M step by step.
Conversely, given a 2D-tape TM M, constructing a standard TM M’ that simulates
M is a little bit tricky. TM M’, with the 2D input mapped on its one-dimensional tape
raw by raw with a boundary mark, must keep track of the current position of M. Here
we present a rough sketch of an idea.

a b a
finite state control b b a
with 4-way read/ a
write head.

7
 Variations of TM Variations of Automata

Given a 2D pattern, we first transform it into the smallest rectangle by filling the
blank cells around the boundary with a special symbol (# in the figure below).
(We need this rectangular form for the convenience of keeping track of the current
position of M when it moves up or down.)
Then we make copies of the rectangular pattern row by row and concatenate
them into a one-dimensional string by inserting a boundary marker ($ in the
figure). Finally we put this one-dimensional string onto the standard TM tape with
a check mark on every symbol appearing in the column where M is positioned.
Now, we let M’ simulate M step by step. The simulation needs tedious house-
keeping operations, especially when M moves up, down, or off the pattern. We
leave these challenging details for the reader.

a b a
√ √ √
finite state b b # a b a $ b b # $ b a #
control b a #
with 4-way
read/write
head. M’
M
8
 Variations of Automata
Variations of LBA’s
We can extend the LBA model with multi-track, multi-tape, and multi-dimensional
tape LBA’s as we did for the TM. In these extensions, we restrict the additional tape
space available to the size of the input. Recall that the tape space of the standard LBA
is restricted to the length of the input. So, for the two-tape LBA, the machine cannot
use auxiliary tape longer than the input size. We can show that these LBA variations
are all equivalent to the standard LBA.

Classic Teaching Break Time

A ten year old boy was failing math. His parents tried everything from tutors to hypnosis, but to no avail. Finally,
at the insistence of a family friend, they decided to enroll their son in a private Catholic school. After the first day,
the boy's parents were surprised when he walked in after school with a stern, focused and very determined
expression on his face, and went right past them straight to his room, where he quietly closed the door.
For nearly two hours he toiled away in his room - with math books strewn about his desk and the surrounding
floor. He emerged long enough to eat, and after quickly cleaning his plate, went straight back to his room, closed
the door, and worked feverishly at his studies until bedtime. This pattern continued ceaselessly until it was time for
the first quarter report card. (To be continued on next slide)

9
 Variations of Automata
6.3 Variations of PDA
The PDA model also has many variations, such as two-way (input head), multi-
stack, finite-turn, and empty stack. A finite-turn PDA can have a constant number of
turns from pushing operations to popping and vice versa during the computation. An
empty stack PDA accepts the input string by empty stack (instead of being in an
accepting state) together with the condition of reading up to the last input symbol.
Contrary to the TM case, these variations of the standard PDA show different
computational capability, and in general it is not easy to prove the difference.

Classic Teaching Break Time

The boy walked in with his report card -- unopened -- laid it on the dinner table and went straight to his room.
Cautiously, his mother opened it, and to her amazement, she saw a bright red "A" under the subject of MATH.
Overjoyed, she and her husband rushed into their son's room, thrilled at his remarkable progress.
"Was it the nuns that did it?", the father asked. The boy only shook his head and said, "No."
"Was it the one-on-one tutoring? The peer-mentoring?"
"No."
"The textbooks? The teachers? The curriculum?"
"Nope," said the son. "On that first day, when I walked in the front
door and saw that guy they nailed to the 'plus sign,' I just knew they meant business!"
- Denise -

10
 Variations of PDA Variations of Automata

2-way PDA
The 1-way head of PDA can be extended to 2-way with the same two accepting
conditions. This two-way mobility gives more computational power. For example,
we can easily construct a 2-way PDA M that recognizes our familiar context-
sensitive language {aibici | i > 0 }.
To accept this language, M scans the input twice. During the first scan M reads
up to the first c and checks if the prefix before c is aibi, for some i > 0. If this part
is successful, M moves back to the left till it sees the last a. Then scanning right,
M makes sure that there is the postfix bici. (We leave the details for the reader.) In
Chapter 12 we will prove that this language is not a context-free and that no
standard PDA can recognize it.

aa . . . abb . . . bcc . . . c

2-way read only

Z0 stack

11
 Variations of PDA Variations of Automata

Multi-stack PDA
Given one extra stack to the standard PDA, the computational capability of the
machine jumps to that of TM. In other words, given a TM, we can construct a 2-
stack PDA which can simulate the TM. Here we will sketch how.
Before the simulation begins, the PDA reads the whole input and pushes it into
one of its stacks such that the leftmost symbol under the TM head appears at the
stack top as figure (b) below shows.
abbab . . . aab$

abbab . . . aab 1-way read only

2-way read/write
Stack 1 Z0

(a) TM
Stack 2 Z0baa . . .babba

(b) 2-stack PDA

12
 Variations of PDA Variations of Automata

The 2-stack PDA can do this by pushing the whole input into one of its stacks and
then transferring it to the other stack. Notice that since the input has no end marker, to
shift the whole input string into one of the stacks, the PDA needs a nondeterministic
move to guess the last symbol of the input.
During the simulation we can always let the symbol read by the TM appear at one
of the stack tops (stack 2 in figure (b) below) by popping a symbol from a stack and
pushing it onto the other stack. Using this trick, it is easy to simulate the TM step by
step. (The details are left to the reader.)
abbab . . . aab

abbab . . . aab
1-way read only
2-way read/write
Z0abba Stack 1
(a) TM
Z0baa . . .b Stack 2

(b) 2-stack PDA

13
 Variations of PDA Variations of Automata

PDA’s accepting by empty stack

Recall that for a PDA to accept the input string, it must satisfy two conditions:
(1) the whole input must be read and (2) the machine must be in an accepting state.
Substituting the empty stack for condition (2), we get a PDA accepting by empty stack.
Here, by empty stack we mean the stack completely emptied by popping out the bottom
of the stack symbol Z0. No computational move is possible with empty stack.
We can show that the standard PDA and the PDA accepting by empty stack are
computationally equivalent. Let Ma be a standard PDA and Me be a PDA accepting by
empty stack. Given an Me, we can construct an Ma which recognizes the language
L(Me). We simply change every transition of Me that pops Z0 to a transition that enters
an accepting state without empting the stack, as the following figure show.
(a, Z0/ε ) (a, Z0/Z0)

(ε , Z0/ε ) (ε , Z0/Z0)

(a) Me (b) Ma
14
 Variations of PDA Variations of Automata

Now suppose that we are given a standard PDA Ma, and assume that Ma does not
pop the bottom of stack symbol Z0. Otherwise, if Ma has a transition, for example
δ (p, a, Z0)= (q, ε ), we introduce a new stack symbol, say A0, and let the machine
rewrite Z0 with A0, instead of popping (see the figure below). Such modification
does not affect the language accepted by Ma. Notice that Ma does not empty the
stack, because no transition is defined with A0 at the stack top.

(a, Z0/ε ) (a, Z0 /A0 )

q0 (ε , Z0/ε ) q0
(ε , Z0/A0 )

start start

15
 Variations of PDA Variations of Automata

With Ma, which does not pop the bottom of the stack symbol Z0, we construct an Me
to recognize L(Ma) as follows. Introduce a new state, say qe, and add a transition to this
state from every accepting state of Ma without reading the input. Then let it dump the
stack contents in state qe. (In the figure below, X denotes any stack symbol including
Z0.)
(ε , X/ε )
(ε , X/ε ) q e

(ε , X/ε )

(a) Ma (b) Me

16
 Variations of PDA Variations of Automata

We learned that substituting the condition of empty-stack for the condition of being
in an accepting state does not affect the language accepted by the PDA, i.e., the classes
of languages accepted by both models are the same. In the proof, we paid no attention
to the types (deterministic or nondeterministic) of PDA Ma and Me. If the given Me is
DPDA (or NPDA), then obviously the PDA Ma that we have constructed is also DPDA
(or, respectively, NPDA). But it does not work the other way around. Here is an
example.
It is easy to construct a standard DPDA which recognizes the language is a+ (see
figure (a) below). If we transform this standard DPDA to a DPDA accepting by empty
stack, we get an NPDA Me as shown in figure (b) below. Actually, it is impossible to
construct a DPDA that recognizes this language with empty stack. So we can say that
the standard DPDA is more powerful than the DPDA accepting by empty stack.
( a, Z0/Z0 ) ( a, Z0/Z0 )

( a, Z0/Z0 ) ( a, Z0/Z0 ) ( a, Z0/ε )

(a) DPDA Ma (b) NPDA Me

17
 Variations of Automata
6.4 Variations of FA
The FA also has many variations. In this section we will introduce 2-way FA,
multi-head FA, 2-D tape FA and the hidden Markov model (HMM).
2-way FA
It is interesting to see that unlike the 2-way PDA, 2-way FA is computationally
equivalent to the standard 1-way FA. This implies that for FA, the mobility of the
head does not give any additional capability for recognizing the language. Actually
there is an algorithm that, given a 2-way FA, constructs 1-way FA recognizing the
same language. The algorithm is given in Appendix B with an example.

a b a b b

2-way read only

18
 Variations of FA Variations of Automata

2-dimensional (2D)-tape FA
A 2D-tape FA model is an FA with a read-only 2D-tape. As in the 2D-tape TM
model, the finite state control can move up, down, to the left or to the right.
However, it cannot write. This model is widely used in the area of pattern
recognition. Sometimes the tape size is restricted to some constant size.

2D tape
a b a (read-only)

b a
Finite state control b a

19
 Variations of FA Variations of Automata

Multi-head FA
We can add extra read-only heads to the standard FA model. Initially all the
heads are assumed to be reading the leftmost input symbol. The head mobility is
usually restricted to one-way, and we may (or may not) allow the heads to sense
the presence of the others on the same cell.
A multi-head FA model is more powerful than the standard FA. We can constrict
a 2-head FA that recognizes {aibi | i > 0 }, which is a non-regular context-free
language. (In Chapter 12 we will prove that this language is not regular.) (The
details for the construction are left for the reader.)

a a a b b b

2-head read only

We can construct a 1-way 2-head NFA that recognizes the context-sensitive

language {ww$ | w ∈ {a, b}+ }, which is not context-free. However, this fact does
not mean that multi-head FA outperforms PDA. For example, it is shown that no
1-way multi-head FA can recognize the context-free language {xxR$ | x ∈ {a, b}
+
}.
20
 Variations of FA Variations of Automata
FA array: cellular automata
A cellular automaton (CA) is a regular array of FA’s, called cells, connected by one-
way or two-way communication lines between neighboring cells. All the cells operate
in synchrony. The CA is a typical fine grained parallel computer model that has been
extensively investigated and implemented.
An input string is given to a dedicated cell to read in sequentially, or in parallel to all
the boundary cells. Interested readers are referred to “Theory and Applications of
Cellular Automata” (by Stephen Wolfram, 1986, World Scientific) or “Cellular
automata: Theory and Experiment” (Edited by Howard Gutowitz, 1991, MIT Press).

.....

..... .....

(a) 1D cellular automaton .....

. . . . . . . . .
.....

(b) 2D cellular automaton

21
 Variations of FA Variations of Automata

Hidden Markov Models (HMM)

A Markov model (MM) is a probabilistic model widely used in the areas of
pattern matching and image recognition. The MM is a variation of Moore
automata with a transition probability given on each edge (instead of an input
symbol) and an output probability given to each output symbol (see the figure
below). In general, no start state is defined and every state can be the start state
with a given probability.
An MM for a given system can be constructed with a probability that can be
computed with a sequence of state transitions and the outputs collected by
observing the system. HMM is the case when only the output sequence is available
and the state transitions are hidden and therefore not available.
a/0.1
0.3 b/0.9

a/0.5
b/0.5
22
 Variations of Automata

6.5 Church’s Hypothesis

In this chapter we learned that there are many variations of the four types of
automata. Some of them (e.g., multi-tape TM) are equivalent to the standard model,
and others (e.g., 2-way PDA and FA) show their computational capability upgraded.
With regards to these variations and other possible computational models, we may
ask the following questions:
[ Is there any upper limit on computational capability? In particular,
is there any computational model that is more powerful than TM? ]
To these questions Church proposed a thesis stating that all possible models of
computation, if they are sufficiently broad, must be equivalent to TM model. This is
called Church’s hypothesis or the Church-Turing thesis. This hypothesis implies that
there is an upper limit in the computational capability, which is TM’s computability.
Every computational model that has been introduced turned out having no more
power than the TM model, supporting the hypothesis that no such model exists.

23
 Variations of Automata
Exercise

6.1 Construct the state transition graph of a DPDA which recognizes each of the following languages by empty stack.
(a) L1 = {aibj; | i > j > 0 } (b) L2 = {aibj | i > j > 0 } (c) L3 = { aibkci | i, k > 0 }

6.2 Construct the transition graph of a 2-head 1-way DFA which recognizes the language L below and briefly describe how
it works. The two heads can move independently to the left, right or stay, after reading the input. When you construct a 2-
head 1-way DFA, use the notation shown in the figure below. (In Chapter 12 we will prove that it is impossible for any
standard FA to recognize L.)
L = { aibi | i > 0 }

In state p, if head 1 reads a and head 2 reads b, or

both heads read a, then the 2-head DFA enters state q.

( a, b ), (a, a )
q
p
t
In state t, head 2 does not read, and if head
s 1 reads a, then the 2-head DFA enters state s.
(a, ε )

24
 Exercises Variations of Automata

6.3 Construct the state transition graph of a 2-D DTM, which draws a figure 8 with character 8’s (see figure (b)) with as few
states as possible. When the work is complete your DTM should be in an accepting state.

Finite state control

8 8 8
8 8
8 8 8
8 8
8 8 8

(a) (b)

English Professor Break Time

An English professor wrote the words, “a woman without her man is nothing” on the blackboard and directed the
students to punctuate it correctly.

The men wrote: “A woman, without her man, is nothing.”

The women wrote: “ A woman: without her, man is nothing.”
- Mike & Corinne -

25