Automata, Grammars, and Languages

Automata -- abstract computing devices (machines)

Grammars formal rules for defining and modeling
(recursively) formal languages
Some History
Turing 1930s: what can and cannot be solved
computationally? Define and study abstract
(primitive) machine models that capture the essence
of computing. Investigate the boundary between the
solvable and unsolvable.




Finite Automata
Studied first in 1940s and 1950s.
Originally proposed to model brain function.

Formal grammars Chomsky 1950s, closely related
to finite automata.

Correspondence between grammars and finite
automata is the basis for compiler design.

Formal logic
Godel 1930s: what can and cannot be
proved?
Incompleteness Theorem: A formal
theory rich enough to include elementary
arithmetic always contains true
statements which are not provable (not
deducible from axioms).
Finite State Machine
state initial :
function output
n) transitio (state function state - next :
states of set finite :
symbols output of set finiete :
symbols input of set finite :
) , , , , , (
memory. finite and
states of number finite with machines of model Abstract
o
o
O I g:S
S I S f
S
O
I
g f S O I M


=
Example
M 001100 110011
0 ) 1 , ( , 1 ) 0 , (
) 1 , ( , ) 0 , (
} {
} 1 , 0 {
) , , , , , (
= =
= =
=
= =
=
o o
o o o o
o
o
g g
f f
S
O I
g f S O I M
0/1, 1/1
o
Example
M 001100 11bbbb
b g b g b g g
f f f f
S
b O I
g f S O I M
= = = =
= = = =
=
= =
=
) 1 , ( , ) 0 , ( , ) 1 , ( , 1 ) 0 , (
) 1 , ( , ) 0 , ( , ) 1 , ( , ) 0 , (
} , {
} , 1 { }, 1 , 0 {
) , , , , , (
t t o o
t t t t t o o o
t o
o
Replace prefix 0s by 1s, and each remaining symbol by b
0/1
0/b,
1/b
1/b
o t
Example
Output 1 if an even number of 1s have been input;
otherwise output 0.
States E,O
At E: has seen even number of 1s; output 1
At O: has seen odd number of 1s; output 0
0/1
0/0
1/0
E O
1/0
0/1
0/0
1/0
E O
1/0
Finite State Automata
state initial :
states accepting :
n) transitio (state function state - next :
states of set finite :
symbols output of set finiete :
symbols input of set finite :
) , , , , (
states. accepting and state initial designated with
functions, output without machines, state finite Similar to
o
o
S A
S I S f
S
O
I
A f S I M
c

=
Example
b
b
a
0
q
1
q
2
q
s
q
a
a,b
a
M: Accept strings over {a,b} with exactly two as.
FA as language recognizers
strings. of set
same accept the they if equivalent are FA Two
. by accepted strings of set the ) Ac(
. if by accepted is
) , (
....
) , (
where ... : n transitio State
... input On
) , , , , (
n
1
1 1
1
1
M M
A M
x f
x f
x x
A f S I M
n n n
n
n
=
e
=
=

=
=

o o
o o
o o
o o o
o
o
Example
Accept bit strings that contain 101 as a substring.
0
1
1
0
q
1
q
0
10
q f
q
0 1
0,1
A simple example of grammar
aabb bb aa b a
aabb
b a
n b a L
n n

> =
o o o
o
o o
follows as generated be can example, For
: grammar following by the generated be can } 0 | {
Definition of Grammar
. over string a is and symbol,
l nontermina one least at with over string a is where

form the of of set finite A
where symbols terminal of set finite A
of set finite A
of consists grammar A
T N B
T N A
B A
rules production P
T N T
symbols l nontermina N
G

C =
Definitionlanguage generated by a grammar
} | { ) (
then , , if y, Inductivel

of closure e transitiv and reflexive the :
. ) ( , all for

then , in rule production a is If
) , , , (
*
* *
*
*
*
o o o
o | | o
o o
| o
| o
o
=

=
G L
T N y x
y x y x
P
P T N G
Backus Normal Form (BNF)

| ... | | ::
as combined be may
:: ,..., :: , ::
:: as written production The
, by embraced symbols l Nontermina
2 1
2 1
n
n
| | | o
| o | o | o
| o | o
=
= = =
=
> <
ExampleA grammar for integers
<digit>::=0|1|2|3|4|5|6|7|8|9
<integer>::=<signed integer>|<unsigned integer>
<signed integer>::=+<unsigned integer>|-<unsigned
integer>
<unsigned integer>::=<digit>|<digit><unsigned integer>


-45
digit -4
digit digit -
integer unsigned digit -
integer unsigned -
integer signed integer

> <
> >< <
> >< <
> <
> >< <
Examplearithmetic expressions
< E >::=<integer>|(< E >)|< E >+< E >|< E >*< E >
67 345 * 90
integer integer * integer
E integer * integer
* integer
*
*
+
> < + > < > <
> < + > < > <
> < + > < > <
> < + > < > <
> < + > >< <
E E
E E E
E E E
Definitionsthree types of grammars
. ) ( , e wher

form the of is in rule every if is ) 2 (
} { ) ( , , ) ( , e wher

form the of is in rule every if is (1)
Then grammar. a be ) , , , ( Let
*
* *
T N u N A
u A
P ee context-f r G
T N u N A T N
u A
P nsitive context-se G
P T N G
e e

e e e

=
| o
| o | o
o
Definition continued
. if are and grammars Two
. such that grammar
regular) free, - context resp. sensitive( - context a is there if
) (resp. is language A
. , , e wher

or ,
or ,
form the of is in rule every if is (3)
L(G') L(G) equivalent G' G
L L(G) G
r ee, regula context-f r nsitive context-se L
T a N B A
A
aB A
a A
P regular G
=
=
e e

Examplefrom DFA to regular grammars

0
1
1
0
q
1
q
0
10
q f
q
0 1
0,1
f
f f
q
q q q q q q
q T N
xq p q x p f
010101
01010 0101 010 01 0
}; 1 , 0 { states; of set the
. by" simulated " ) , (
Ac(M) L(G) that so G regular a construct To
substring. a as 101 with strings accepts M
10 1 0 0
0

= = =
=
=
o
Theorem
For every DFA M, there is a regular grammar G such
that L(G)=Ac(M).
. under ... .....

on ...
). ( ) (
) (
}. | { } ) , ( | {
;
) , , , (
let Then ). , , , , ( Suppose : Proof
1 2 2 1 1 1
2
1
1
G q x x q x x q x
M q
x x
q
x
M Ac G L
q
x
p
A p p q x p f xq p P
I T S N
P T N G
A f S I M
n n
n
n



=

e = =
= =
=
=
o
o

o
o
Example -- From regular grammars to FA
b C bC C aC b
C N b a T

= =
, , ,
} , { } , {
o o o
o
a
o
C
b
F
b
b
M F
b
C
b
C
a b
babb babC baC b
b C bC C aC b
on

, , ,




o o
o o
o o o
b
) , (
) , (
) ( } , { ) , (
?

: possible are
sequences n transitio one than more input On
, , ,
C =
C =
e =



b F f
a F f
S P F C b C f
b
F
b
C
a b
F
b
C
b
C
a b
babb
b C bC C aC b
o o
o o
o o o
Definition: Nondeterministic Finite Automata
state initial :
states accepting :
n) transitio (state function state - next ) ( :
states of set finite :
symbols output of set finiete :
symbols input of set finite :
) , , , , (
o
o
S A
S P I S f
S
O
I
A f S I M
c

=
Language accepted by NFA
strings. of set
same accept the they if equivalent are NFA Two
. by accepted strings of set the ) Ac(
) , (
....
) , (
...
on n transitio state of
sequence a is there if by accepted is , ... For
iff by accepted is
) , , , , (
1
1 1
2
1
1
1
M M
x f
x f
A
x x x
M x x
A M
A f S I M
n n n
n
n
n
=
e
e
e
=
e
=

o o
o o
o o o
o
o o
o
o
Theorem
For every regular grammar G there is an NFA M such
that L(G)=Ac(M).
} | { } {
} | { } | { ) , (
} {
) , , , , (
) , , , ( : Pr
P q q F A
P x p F P xq p q x p f
F N S
T I
A f S I M
P T N G oof
e =
e e =
=
=
=
=

o
o
Example
NFA that accepts strings over {a,b} starting with ab or
aba.
b
a
0
q
1
q
2
q
b
a
a,b
4
q
3
q
a
Example
NFA that accepts strings over {a,b} starting with ab
but not ending with ba.

a
0
q
1
q
2
q
3
q
b
a,b
b
a
b
Equivalence between NFA and DFA
Simulate an NFA M by tracking all states M can
enter after reading an input symbol, staring from the
initial state.
Construct a DFA that carries out the simulation.
Example
a
o
C
b
F
b
b
b
a
o
C
b
} , { F C
b
b
b
C
a
a
a,b
Algorithm Constructing DFA that simulates NFA
). Ac( ) ' Ac(
A. in state some contain that ' in states of consists '
}) { , ' , ' , ' , ( '
state.] - super a is ) , ( ' [ . ) , ( ) , ( '
, every for Then ). from states reachable of subset (a
state - super a as formed is suppose y, Inductivel
.] reading after from enter can M states of set the is ) }, ({ ' [
)}). , ( | { )( , ( ) }, ({ '
let , symbol input every For
} { state - super form - - state initial the Start with
M M
S A
A f S I M
x Q f x p f x Q f
I x
S Q
x x f
x f q q x f x f
x
Q p
=
=
=
e
c
e = =
e
o
o
o o
o o o
o

). Ac( ) ' Ac(

A. in state some contain that ' in states of consists '
}) { , ' , ' , ' , ( '
} reading after in state some from enter can M states {
state - super a as formed is suppose y, Inductivel
} reading after from enter can M states { } {
, symbol input every For
} { state - super form - - state initial the Start with
manner. first - breadth a in Proceed
M M
S A
A f S I M
x Q
x
Q
S Q
x
x
x
=
=

c

o
o o
o
Outline of the algorithm converting NFA to DFA
Example
b
a
0
q
1
q
2
q
b
a
a,b
4
q
3
q
a
a
0
q
3 1
, q q
4 2
, q q
b
a
C
b
a,b
2
q
a,b
a,b
Theorems
1. For every DFA M, there is a regular grammar G
such that L(G)=Ac(M).
2. For every regular grammar G there is an NFA M
such that L(G)=Ac(M).
3. For every NFA M there is a DFA M such that
Ac(M)=Ac(M).
Cor.
A language is regular iff it is accepted by some
NFA iff it is accepted by some DFA.


Operations on languages
L L
L L
L x x x x x x L L L L L L
L y L x y x L L
i
i
i i
i i
E =
=
e = - = - =
e e = -
E

*
0
*
2 1 2 1
1 2
2 1 2 1
} ,... , | ... { ;
} , | {
alphabet an :

Theorem
,*,-. ,
under closed is languages regular of class the is, That
well. as languages regular are , , ' , ' Then
languages. regular two are ' and Suppose
*
-
- L L L L L L
L L
L=Ac(M)
a
0
q
1
q
2
q
b
a,b
4
q
3
q
b
L=Ac(M)

b
a,b
a
0
p
1
p
2
p
3
p
b
a
b
a
b
a
b
b
a a
LL=Ac(M) where M is obtained by gluing the accepting state of
M with the initial state of M. Note: M is an NFA.
a
0
p
1
p
2
p
3
p
b
a
a
b
a
b
b
a
0
q
1
q
2
q
b
a,b
4
q
3
q
b
a,b
a a
a
1
p
2
p
3
p
b
a
a
b
a
b
a
0
q
1
q
b
a,b
4
q
3
q
b
a,b
a a
Alternative construction for concatenation
a
0
p
1
p
2
p
3
p
b
a
a
b
a
b
b
a
0
q
1
q
2
q
b
a,b
4
q
3
q
b
a,b
a a
||
||
||
b a,
b a,
b
a
0
p
1
p
2
p
3
p
b
a
a
b
a
b
b
a
0
q
1
q
2
q
b
a,b
4
q
3
q
b
a,b
a a
Closed under concatenation
. , ' )}, , ( ' { ) , (
. ) , ( , , }, ), , ( { ) , g(
. ) , ( , , )}, , ( { ) , g(
NFA an ) , , ' , ' , ( "
'
DFA : ' ,
) , ' , ' , ' , ( ' ), ' ( '
) , , , , ( ), (
I x S p x p f x p g
A x p f I x S p x p f x p
A x p f I x S p x p f x p
g A S S I M
S S
M M
f A S I M M Ac L
f A S I M M Ac L
e e =
e e e =
e e e =
=
C =
= =
= =
t
o
t
o
NFA with s transition -
>
>

M M
L=Ac(M), L=AC(M).
The combined machine (NFA) accepts LL.
Example
0
1
1 , 0
>
) (M Ac L =
0
1 , 0
0
) ' ( ' M Ac L =
0
1
1 , 0
0
1 , 0
0

>
>
>
M
>

) " ( ' ) ' ( ' ), ( M L L L M L L M L L = = =

' M
Example
>
) (M Ac L = ) ' ( ' M Ac L =
0
1
0
1 , 0
0
>
' L L
0
1
0
1 , 0
0

>
M
>
M

) ' ( ) (
*
M Ac L M Ac L = =
Removing
s transition
p q
x

s
p q
x
x
s
ExampleDesign an NFA that accepts
10. begin with that strings of set the is where
*
L L
0
1
1 , 0
>
) (M Ac L =
0
1
1 , 0
>

0
1
1 , 0
>
) ' (
*
M Ac L =
S
A B
0
1 , 0
0
1
1 , 0
> S
A B
0
1 , 0 S
C
0
A
1
B
0
1
1 , 0
S B,
0
1 , 0
A S B , ,
1
1 , 0
S >
C
0
A
1
F
0
1
1 , 0
1 , 0
Closed under compliment
. by rejected but by accepted - in state last The
. by rejected but by accepted in state last The
input given a for
machines both on ns transitio of seq. same fn n transitio Same
. ) (
Then
) , , , , (
DFA ) , , , , ( ), (
M M A S
M M A
f
L M Ac
f A S S I M
f A S I M M Ac L

=
=
= =
o
o
Suppose M is a DFA with a unique accepting state.
Reverse all arrows of M.
Make the accepting state the initial state.
Make the initial state the accepting state.

The resulting machine is an NFA which accepts

. of reversal the denotes where
} | {
R
o o
o o L L
R R
e =
S
D
0
A
1
B
0
1
1 , 0
C
0
1
1
>
0
cycle. a contain must 5 length of path every states 5 has
: Cycle
S
: 101001 input on path Transition
principle hole pigeon
0 1 0
1 0 0 1 0 1
>

M
A C B A
C B A C B A
Example: a language which is not regular
ion. contradict a , ) ( 1
.. .. ...
is 1 for path n transitio
.. .. .. ...
. some for
cycle. a contains ... path the
.
1
...
1
...
. for path n transitio the and , 1 0 consider and , Let
. in states # Let
. DFA some for Otherwise,
regular. not is } 0 : 1 0 {
) (
1
) (
1
2 1
1 2 1
L M Ac o
A p q q
o
A p q q q q
i j q q
q
o
q
o
q
o
m n
A q p q
o
q
o
q
o
w L w m n
M m
M Ac(M) L
n L
n i j n
n n
o
i
o
n i j n
n n
o
i j
o
i
o
j i
n
n n
n n
n n
n j n i
n j n i j i
= e
e

e =
> =
>
e
e = >
=
=
> =

o
o
o
o