Principles of Programming Languages - ASU 2014

CSE340 - Principles of
Programming Languages
Lecture 01:
Course Presentation
Javier Gonzalez-Sanchez
javiergs@asu.edu
BYENG M1-21
Office Hours: By appointment
Definitions
Tell a computer what to do
Method of communication consisting of the use of

signs or words in a structured and conventional way
Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 2
Language Levels
C++
Java
Fortran
High-Level Language
Assembly Language
Machine Language
Hardware
Machine Language
Assembly Language
High-Level Languages
compilation
execution
// sorce code
int x;
Lexer
int foo () {
read (x);
print (5);
}
Parser
main () {
foo ();
}
Semantic Analyzer
Virtual Machine
(interpreter)
Code Generation
X,E,G,O,O
#e1,I,I,0,7
@
OPR 19, AX
STO x, AX
LIT 5, AX
OPR 21, AX
LOD #e1,AX
CAL 1, AX
OPR 0, AX
01001010101000010
01010100101010010
10100100000011011
11010010110101111
00010010101010010
10101001010101011
Assembler
Language Paradigms
Procedural
program = algorithms + data
Object-Oriented
program = objects + messages
Functional
program = functions functions
Logic Programming
program = facts + rules
Calendar
Calendar
Grading
Exams (2)
20% + 20%
40%
20%
20%
10% + 10% + 10% + 10%
40%
Final Comprehensive
Exam
Programming
Assignments (4)
100%
97
A+
86
B+
74
C+
93
82
70
89
A-
78
B-
Text book
Chapter 1. Introduction
Chapter 6. Syntax
Chapter 7. Basic Semantics
Chapter 8. Data Types
Chapter 9. Expressions and Statements
Chapter 10. Procedures
Chapter 3. Functional Programming

Chapter 4. Logic Programming
Chapter 5. OO Programming
Chapter 12. Formal Semantics
Homework
Read the Syllabus of the course
CSE340 - Principles of Programming Languages

javiergs@asu.edu
Fall 2014
Disclaimer. These slides can only be used as study material for the class CSE340 at ASU. They cannot be distributed or used for another purpose.
Lecture 02:
Introduction
javiergs@asu.edu
BYENG M1-21
Teaching Assistants
Bolun Li
bolunli@asu.edu
Graduate Student / Computer Science MS
Steven Lombardi
steven.lombardi@asu.edu
Undergraduate Student / Computer Science BS
Instructor
javiergs@asu.edu
GraduateTeaching Associate
www.javiergs.com
Keywords
Lexical
Level
Input:
Symbols
Output:
Words
Paradigm
Input:
Analysis
Language
Syntax
Words
Output:
Sentences
Translate or
Execute
Semantic
Input:
Sentences
Analysis
int x = 5;
float y = "hello;
String @z = "9.5";
int x = cse340;
if ( x > 14) while (5 == 5) if (int a) a = 1;
x = x; for ( ; ; );
y = 13.45.0;
int me = 99999000001111222000000111111222223443483045830948;
while { x != 9} ();
int {x} = 10;
compilation
execution
// sorce code
int x;
Lexer
int foo () {
read (x);
print (5);
}
Parser
main () {
foo ();
}
Semantic Analyzer
Virtual Machine
(interpreter)
Code Generation
X,E,G,O,O
#e1,I,I,0,7
@
OPR 19, AX
STO x, AX
LIT 5, AX
OPR 21, AX
LOD #e1,AX
CAL 1, AX
OPR 0, AX
01001010101000010
01010100101010010
10100100000011011
11010010110101111
00010010101010010
10101001010101011
Assembler
Keywords
Lexical
Alphabet
Symbol
String
Word
Token
Lexical Analysis
int x = 5;
float y = "hello;
String @z = "9.5";
int x = cse340;
if ( x > 14) while (5 == 5) if (int a) a = 1;
x = x; for ( ; ; );
y = 13.45.0;
int me = 99999000001111222000000111111222223443483045830948;
while { x != 9} ();
int {x} = 10;
Lexical Analysis | Steps

a) Read a text FILE line by line
b) For each LINE:
Read character by character.
Create sets of consecutive characters (STRING). Try to
group the bigger amount of characters as possible.
Start a new set each time that you need. Take care
of: Whitespace, Delimiter, Operator, End of Line and
others special characters.
c) For each STRING: verify if it is a valid WORD.
d) Create a VECTOR and store the STRINGs and WORDs.
Lexical Analysis
int x = 5; float y = "hello;
String@z="9.5;intx=cse340;if(x>
14) while
(5 == 5) if (int a) a = 1; x = x;
for ( ; ; );y = 13.45.0;int me
=99999000001111222000000111111222
223443483045830948;while { x !=
9} ();int {x} = 10;
Lexical Analysis
# of STRINGS
int x = 5; float y = "hello;

String@z="9.5;intx=cse340;if(x>
14) while
(5 == 5) if (int a) a = 1; x = x;
for ( ; ; );y = 13.45.0;int me
=99999000001111222000000111111222
223443483045830948;while { x !=
9} ();int {x} = 10;
hello "world" bye"
9
12
3
18
12
2
6
12
3
Keywords
Lexical
Alphabet
Symbol
String
Word
Regular
Expression
Token
Rules
Deterministic
Finite
Automata
Homework
Review the following topics:

Regular Expressions (Text Book: Chapter 6)
and Deterministic Finite Automata

javiergs@asu.edu
Fall 2014
Lecture 03:
Lexical Analysis
javiergs@asu.edu
BYENG M1-21
Keywords
Lexical
Alphabet
Symbol
String
Word
Regular
Expression
Token
Rules
Deterministic
Finite
Automata
Regular expression
A rule to describe finite combination of symbols
(sequences) that are considered well-formed.
Regular expression has symbols and operators
Symbols are defined in the alphabet
The operators used in regular expressions are: * (0 or
more), + (1 or more), ? (0 or 1), | (or). Besides those
we can use [ ] to enclose sets of symbols without
enumerating all of them, such as [0-9] or [A-Z]. Also,
we can use parenthesis.
Regular expression | Examples

Token
Regular Expression (rule)
Examples (words)
foobarOne
(a | b)
{a, b}
foobarTwo
(a | b)(a | b)
{aa, bb, ba, ab}
foobarThree
a*
, a, aa, aaa, aaaa, ... }
foobarFour
(a | b)*
, a, b, aa, bb, ...abba ...}
foobarFive
a+
{ a, aa, aaa, aaaa, ... }
foobarSix
[a-z]+
{hello, world, etc, }
number
[0-9]+
{1934, 0101, 33, 12321}
Regular expression | Examples

Token1
Example (word)
digit
0 | 1 | 2 | 3 | ... | 9
integer
digit+
1945
fraction
.digit+
.55
exponent
e(+|-)?digit+
e+210
floatDraftOne
integer(fraction?) (exponent?)
340.08e-14
floatDraftTwo
{[-+]?([0-9]+\.?[0-9]*|\.[0-9]+)([eE][-+]?[0-9]+)?}
binary
0b(0|1)+
0b1010
1. These definitions are NOT fully complete or correct. They purpose is only to exemplify RE. For
instance 07 match as an integer, which will NOT be the case for our language.
Regular expression | Operators
Deterministic Finite Automata

It is a finite state machine that accepts/rejects finite
strings of symbols and produces a unique result for
each input string.
In the automaton, there are three states (denoted
graphically by circles) and transition arrows
connecting one state with other.
Upon reading a symbol, a DFA jumps
deterministically from a state to another by
following the transition arrow.
DFA | Examples
binary
0b(0|1)+
string
([a-z] | [A-Z] | [0-9])*
DFA | Examples
{}
Char
{.}
{}
Operator
{.}
{+,-,*,/,%,
<,>,=,!,}
Start
{(, ), {, }, [, ]}
{}
{}
String
{a-z}
Delimiter
{0-9}
{_}
{\.}
{0-9}
ID
{$}
Integer
Float
{\.}
{0-9}
{$, _, 0-9, a-z}

{0-9}
Additional Examples
Regular Expressions and Deterministic Finite Automata
Handwritten notes
Regular expression
Regular expression
Regular expression
Handwritten notes
Deterministic
Finite Automa
Handwritten notes
Regular expression
Regular expression
-9
Handwritten notes
Handwritten notes
Is this correct?
Is this correct?
Is this correct?
Handwritten notes
Error
Correct
Regular expressions - Examples

Token
Examples (words)
foobar4
{"a", "b", "c"}*
{, "a", "b", "c", "aa", "ab", "ac", "ba", "bb", "bc",

"ca", "cb", "cc", ...}
foobar5
{"ab", "c"}*
{, "ab", "c", "abab", "abc", "cab", "cc", "ababab",

"ababc", "abcab, ...}
Regular expressions - Examples

Define a regular expression for each case
a) URLs
b) Email addresses
c) ZIP codes
DFA- Examples
Define a DFA for each case
a) URLs
b) Email addresses
c) ZIP codes
Ours Tokens
Which tokens are needed for a programming language?
a) Reserved words
b) Special Symbols: Operators and delimiters
c) Identifiers
d) Literals or constants
Drafting a Lexer
Keywords =
Operator =
Delimiter =
ID =
Drafting a Lexer
Float =
Integer =
Hexadecimal =
Octal =
Binary =
String =
Char =
Homework
Define the necessary lexical rules for a programming language

Express these rules using a DFA and Regular Expressions
Share them on Blackboard and discuss their correctness with your classmates.

javiergs@asu.edu
Fall 2014
Lecture 04:
Lexer Implementation 1
javiergs@asu.edu
BYENG M1-21
Review
Given the following token definitions (using regular expressions)
!
t1 = aabb!
t2 = aab!
t3 = (a | b) *!
1. Are the following strings correct?

aaba!
a!
aab!
!
2. Which are the token for each of them?

4. Which symbols are in the alphabet ?
3. Create a DFA that represents the previous rules.
Review
1. How many words:
-5
-5.5e-5
5-5
2. Which is the difference between these regular expressions?

[0-9]+.[0-9]+
[0-9]+\.[0-9]+
Programming a Lexer
Regular
Expresion
Programming a Lexer
Regular
Expresion
Using IF-ELSE
It is not a good idea!
February 13th, 2008 by Rich Sharpe. Posted in Software Quality, Software Quality Metrics
Using a State Machine

1. Put the DFA in a Table
Delimiter, operator,
b 0 1 ... whitespace, quotation
mark

S0
S1
S2
S3
S0 SE S1 SE SE

Stop
S1 S2 SE SE SE

Stop

S2 SE S3 S3 SE

Stop

S3 SE S3 S3 SE
Stop
SE SE SE SE SE
Stop

2. Put the Table in Java
b
// constants
private
private
private
private
private
private
private
static
static
static
static
static
static
static
final
final
final
final
final
final
final
int
int
int
int
int
int
int
ZERO
ONE
B
OTHER
DELIMITER
ERROR
STOP
= 1;
= 2;
= 0;
= 3;
= 4;
= 4;
= -2;
// table as a 2D array
...
whitespace, quotation
mark

S0 SE
S1 SE SE

Stop
S1 S2
SE SE SE

Stop

S2 SE
S3 S3 SE

Stop

S3
SE
S3
S3 SE
Stop
SE
SE
SE SE SE
Stop
private static int[][] stateTable = {

{ERROR,
1, ERROR, ERROR, STOP},
{
2, ERROR, ERROR, ERROR, STOP},
{ERROR,
3,
3, ERROR, STOP},
{ERROR,
3,
3, ERROR, STOP},
{ERROR, ERROR, ERROR, ERROR, STOP}
};

STEP 3. Algorithm
b
...
S0 SE S1 SE SE Stop

S1 S2 SE SE

void splitLine (line) {

state = S0;
String string ="";
do {
l = line.readNextLetter();
go = calculateNextState(state, l);
if( go != STOP ) {
string = string + l;
state = go;
}
} while (line.hasLetters() && go != STOP);
SE Stop

if (state == S3)
print (It is a BINARY number);
else
print (error);
S2 SE S3 S3 SE Stop

S3 SE S3
SE SE SE

S3
SE

if( isDelimiter(currentChar))
print (Also, there is a DELIMITER);
else if (isOperator(currentChar) )
print (Also, there is an OPERATOR);
SE Stop
// loop
if (line.hasLetters() ))
splitLine( line string );
SE Stop

Programming Assignment #1
1. Read a File; Split the lines using the
System.lineSeparator
2. For each line read character by character and use the
character as an input for the state machine
3. Concatenate the character, creating the largest
STRING possible. Stop when a delimiter, white space,
operator, or quotation mark and the current state
allowed. If there are more characters in the line, create
a new line with those characters and go to step 2.
4. For each STRING and WORD report its TOKEN or ERROR
as correspond.
Homework
Define the necessary lexical rules for a programming language

Express these rules using a DFA and Regular Expressions
Share them on Blackboard and discuss their correctness with your classmates.
Remember: Using a
DETERMINISTIC Finite Automata


javiergs@asu.edu
Fall 2014
Lecture 05:
Lexer Implementation 2
javiergs@asu.edu
BYENG M1-21
Review
1. how to exclude keywords from Identifiers?
2. The dot:
foobar1
foobar1
foobar1
foobar1
=
=
=
=
.
.*
.+
\.
3. What is the problem here?

string = " \.* "
Review
4. Rules and Sub-rules
URL definition:
https?:// ([a z]+ | [A Z]+ | [0 9]+ | - | . | _ | ~ | %21
| %23 | %24 | %26 | %27 | %28 | %29 | %2A | %2B | %2C | %3A |
%3B | %3D | %3F | %40 | %5B | %5D )* %2F ([a z]+ | [A Z]+ |
[0 9]+ | - | . | _ | ~ | ! | # | $ | & | | ( | ) | * | +
| , | / | : | ; | = | ? | @ | [ | ] )+
Review
5. Making things shorter
ZIP definition:
[1-9][1-9][1-9][1-9][1-9](-[1-9][1-9][1-9][1-9])?
[0-9]{5}(-[0-9]{4})?
Only BINARY, DELIMITER, and OPERATOR are implemented. You will implement the rest of the
required tokens (rules).
* Lexer.java is the only file that you are allowed to modify
Code | Token.java
Code | Gui.java
Code | Lexer.java
Code | Lexer.java
Code | Lexer.java
Code | Lexer.java
Code | Lexer.java
Code | Lexer.java
Code | input.txt
hello;world cse340 asu 2013/05/31 // end
boolean $xx= ((((((((23WE + 44 - 3 / 2 % 45 <=17) > 0xfffff.34.45;
Code | output.txt
IDENTIFIER
DELIMITER
IDENTIFIER
IDENTIFIER
IDENTIFIER
INTEGER
OPERATOR
OCTAL
OPERATOR
INTEGER
OPERATOR
OPERATOR
IDENTIFIER
KEYWORD
hello
;
world
cse340
asu
2013
/
05
/
31
/
/
end
boolean
IDENTIFIER
OPERATOR
DELIMITER
DELIMITER
DELIMITER
DELIMITER
DELIMITER
DELIMITER
DELIMITER
DELIMITER
$xx
=
(
(
(
(
(
(
(
(
ERROR
OPERATOR
INTEGER
OPERATOR
INTEGER
OPERATOR
INTEGER
OPERATOR
INTEGER
OPERATOR
OPERATOR
INTEGER
DELIMITER
OPERATOR
23WE
+
44
3
/
2
%
45
<
=
17
)
>
ERROR
DELIMITER
0xfffff.34.45
;
Homework
Develop a Lexical Analyzer by coding a DFA

javiergs@asu.edu
Fall 2014
Lecture 06:
Closing with Lexical Analysis
javiergs@asu.edu
BYENG M1-21
Errata #1
Errata #2
Errata #3
?
70%
?
100%
?
100%
?
70%
Firstname_Lastname_P1.zip
Compile and Run
Recognize BINARY
Recognize DELIMITER and OPERATOR
approx. 20%
Recognize INTEGER
Recognize OCTAL
Recognize HEXADECIMAL
Recognize IDENTIFIER
approx. 40%
Recognize STRING
Recognize CHAR
Recognize KEYWORD
Recognize FLOAT
approx. 40%
Review | Lexical Analysis

Are the following STRINGS correct or not? Why?
000000005
000000009
000000009.1
000000005
000000005.1
0x0000002
0123456789

1.2e---3++
$50
float ________________ = 5;
double x = 000000.1;
'''a'
'\''b'
'\'b'

" \\\\\\\\\\a"
"Hello""world"
abc"Hello"
''
'\x
\a'
\w
"\\\"
State Transition Table

for our Lexer
(step by step)
BINARY
INTEGER
OCTAL
HEXADECIMAL
IDENTIFIER
Lexer Step by Step
Lexer Step by Step
Lexer Step by Step
states
1
2
3
5
7
8
9
10
=
=
=
=
=
=
=
=
INTEGER
INTEGER
IDENTIFIER
OCTAL
INTEGER
IDENTIFIER
BINARY
HEXADECIMAL
columns
0
$
_
[1]
[2-7]
[8-9]
[A]
B
[C-F]
[G-W]
X
[Y-Z]
[a-z] = [A] B [C-F] [G-W] X [Y-Z]

[a-f] = [A] B [C-F]
Lexer Step by Step

0
[1]
[2-7]
[8-9]
S2
S2
[C-F]
[G-W]
S3
S3
S3
S3
SE
Stop
SE S4 SE SE
[Y-Z]
...
whitespace, quotation
mark
S1
S3
S3
S2
S1
S5
SE
SE
S5 S5 SE
S6
SE
SE
Stop
S2
S7
SE
SE
S7 S7 S7 SE SE SE SE SE
SE
SE
Stop
S3
S8
S8
S8 S8 S8 S8 S8 S8 S8 S8 S8
S8
SE
Stop
S4
S9
SE
SE
S9
SE
SE SE SE SE SE
SE
SE
Stop
S5
S5
SE
SE
S5 S5 SE
SE SE SE SE SE
SE
SE
Stop
S6
S10
SE
SE
S10 S10 S10 S10 S10 S10 SE SE
SE
SE
Stop
S7
S7
SE
SE
S7 S7 S7 SE SE SE SE SE
SE
SE
Stop
S8
S8
S8
S8 S8 S8 S8 S8 S8 S8 S8 S8
S8
SE
Stop
S9
S9
SE
SE
S9
SE SE SE SE SE
SE
SE
Stop
S10
S10
SE
SE
S10 S10 S10 S10 S10 S10 SE SE
SE
SE
Stop
SE
SE
SE
SE
SE
SE
SE
Stop
SE
SE
SE
SE
S3
S0
SE
S3
SE SE SE SE SE
Homework
Review Recursion
Solve the Problem Set #1 in preparation for your exam

javiergs@asu.edu
Fall 2014
Lecture 07:
Syntactic Analysis 1
javiergs@asu.edu
BYENG M1-21
Next Step
Lexical Analysis
Syntactic Analysis
Question
For each cases indicate whether it is possible or not to
generate a regular expression or a DFA.
i.
Detect the balance of N parenthesis in a string

that has N parenthesis nested and any characters
in between the parenthesis.
ii. Is it possible to detect binary strings with the same

quantity of 0s and 1s (it does not matter the order
or sequence).
Where are we now?

After lexical analysis, we have a series of tokens.
But we can not:
I.
define a regular expression matching all

expressions with properly balanced parentheses.
II. i.e., define a regular expression matching all

functions with properly nested block structure.
void a () { b (c); for (;;) {a=(-(1+2)+5); } }
Where are we now?

Now, we want to:
I.
Review the structure described by that series of

tokens
II. Report errors if those tokens do not properly

encode a structure
compilation
execution
// sorce code
int x;
Lexer
int foo () {
read (x);
print (5);
}
Parser
main () {
foo ();
}
Semantic Analyzer
Virtual Machine
(interpreter)
Code Generation
X,E,G,O,O
#e1,I,I,0,7
@
OPR 19, AX
STO x, AX
LIT 5, AX
OPR 21, AX
LOD #e1,AX
CAL 1, AX
OPR 0, AX
01001010101000010
01010100101010010
10100100000011011
11010010110101111
00010010101010010
10101001010101011
Assembler
Outline
Symbols
Rules
Token
Lexical Analysis
(Lexer)
Regular Expression
Tools
DFA
Language
Grammar
(Rules)
Syntactic
Analysis
(Parser)
Terminal
Non-terminal
BNF
(Backus-Naur Form)
Tools
Syntax Diagrams
Grammar | Example
Describe all legal arithmetic expressions using
addition, subtraction, multiplication, and division with
integer values
E E OP E
E integer
OP + | - | * | /
E ( E )
Grammar | Definition
A Grammar is a collection of
four elements:
E E OP E
E integer
OP + | - | * | /
E ( E )
Set of nonterminal symbols

(uppercase)
Set of terminal symbols
(lowercase). Terminals can be
tokens or specific words
Set of production rules saying
how each nonterminal can
be converted by a string of
terminals and nonterminals,
A start symbol
Javier Gonzalez-Sanchez | CSE340 | Summer 2013 | 9
Grammar | Derivation
5
/
20
integer operator integer
E
E OP E
integer OP E
integer / E
integer / integer
E E OP E
E integer
OP + | - | * | /
E ( E )
5
*
(
7
+
20
Integer operator delimiter integer operator integer
E
E OP E
integer OP E
integer * E
integer * (E)
integer * (E OP E)
integer * (integer OP E)
integer * (integer + E)
integer * (integer + integer)
)
delimiter
E E OP E
E integer
OP + | - | * | /
E ( E )
5
*
(
7
+
20
Integer operator delimiter integer operator integer
E
E OP E
E OP (E)
E OP (E OP E)
E OP (E OP integer)
E OP (E + integer)
E OP (integer + integer)
E * (integer + integer)
)
delimiter
E E OP E
E integer
OP + | - | * | /
E ( E )
Derivations
A leftmost derivation is a derivation in which
each step expands the leftmost
nonterminal.
A rightmost derivation is a derivation in
which each step expands the rightmost
nonterminal.
Derivation will be very important when we
talk about parsing.
Grammar | Example
H2 O
C O2 (S O4)3
Na Cl
S O3
Notation 1:
Comp
Mix
Elem
Num
Mix | Mix Num | Comp Comp

Elem | ( Comp )
H|O|C|S|Na|Cl| ...
1|2|3|4| ...
Notation 2:
<Comp>
<Mix>
<Elem>
<Num>
<Mix>|<Mix><Num> | <Comp><Comp>
<Elem> | ( <Comp> )
H|O|C|S|Na|Cl| ...
1|2|3|4| ...
C
Comp
Comp Comp
Term Comp
Elem Comp
C Comp
C Term Num
C Elem Num
CO Num
CO2
Comp Term | Term Num | Comp Comp

Term Elem | ( Comp )
Elem H|O|C|S|Na|Cl| ...
Num 1|2|3|4|
What about this?

BLOCK STMT | { STMTS } | { }
STMTS STMT | STMT STMTS
STMT
EXPR; |
if (EXPR) BLOCK |
while (EXPR) BLOCK |
BLOCK |
. . .
EXPR
EXPR + EXPR |
EXPR EXPR |
EXPR * EXPR |
identifier |
integer |
...
Homework
Using the rules in the previous slide, apply derivation to show that the following
expression is syntactically correct
while ( 5 ) { if ( 6 ) { } }
Homework
Review Recursion
Solve the Problem Set #1 in preparation for your exam

javiergs@asu.edu
Fall 2014
Lecture 08:
Syntactic Analysis II
javiergs@asu.edu
BYENG M1-21
Outline
Non-terminal
Grammar
(Rules)
Terminal
Language
Syntactic
Analysis
(Parser)
Derivation
Parse
Tree
BNF
Tools
Syntax Diagrams
5
Integer
*
operator
(
delimiter
7
integer
E
E OP E
integer OP E
integer * E
integer * (E)
integer * (E OP E)
integer * (integer OP E)
integer * (integer + E)
+
operator
20
integer
)
delimiter
E E OP E
E integer
OP + | - | * | /
E ( E )
5
Integer
*
operator
(
delimiter
7
integer
E
E OP E
E OP (E)
E OP (E OP E)
E OP (E OP integer)
E OP (E + integer)
E OP (integer + integer)
E * (integer + integer)
+
operator
20
integer
)
delimiter
E E OP E
E integer
OP + | - | * | /
E ( E )
Parse Tree
A parse tree is a tree encoding the steps in a
derivation.
Internal nodes represent nonterminal symbols used
in the production.
Inorder walk of the leaves contains the generated
string.
Encodes what productions are used, not the order
in which those productions are applied.
Parse Tree
Parse Tree
Goal
Goal of syntax analysis:
Recover the structure described by a series of
tokens.
Recover a parse tree for the given input.
The problem
5
Integer
*
operator
7
integer
+
operator
20
integer
E E OP E
E integer
OP + | - | * | /
E ( E )
A serious problem
Ambiguity
A grammar is said to be ambiguous if there is at
least one string with two or more parse trees.
Note that ambiguity is a property of grammars, not
languages.
We will review this topic in the next lecture

Our Tools
Backus-Naur Form (BNF)
Extended Backus-Naur Form (EBNF)
Syntax Diagrams
BNF (Backus-Naur Form)

Formal, mathematical way to specify grammars
All the previous examples, where we use:
or ::= is defined as
| or operator
<nonterminal> or use uppercases
terminal (lowercases)
* John Backus and Peter Naur

EBNF
Extended BNF include notation to indicate:
0 or more occurrences {}
1 or more occurrences +
0 or 1 occurrences []
Use of parentheses for grouping ( )
* Niklaus Wirth
Example
Grammar rule for calling a method:
draw(x, y, z);
print (a, b, c, d);
done();
foobar(one, two, three, four, five);
sqrt(x);
BNF vs EBNF
BNF
<call_method> identifier(<identifiers>); | identifier();
<identifiers> identifier | identifier,<identifiers>
EBNF
<call_method>
<identifiers>
identifier ('('<identifiers>')' | '('')' ) ';'

identifier | identifier,<identifiers>
EBNF
<call_method> identifier'('[<identifiers>]')' ';'
<identifiers> identifier | identifier,<identifiers>
EBNF
<call_method> identifier'('[<identifiers>]')' ';'
<identifiers> identifier { ,identifier }
Syntax Diagrams
Call_method
Identifiers
Is it BNF or EBNF?
BLOCK STMT | { STMTS } | { }
STMTS STMT | STMT STMTS
STMT
EXPR; |
if (EXPR) BLOCK |
while (EXPR) BLOCK |
BLOCK |
. . .
EXPR
EXPR + EXPR |
EXPR EXPR |
EXPR * EXPR |
identifier |
integer |
...
Syntax Diagrams
BLOCK
STMTS
STMT
EXPR
Is it BNF or EBNF?
BLOCK STMT | '{' { STMT } '}'
STMT
EXPR; |
if '(' EXPR ')' BLOCK |
while '(' EXPR ')' BLOCK |
BLOCK |
. . .
EXPR
EXPR '+' EXPR |

EXPR '' EXPR |
EXPR '*' EXPR |
identifier |
integer |
...
Syntax Diagrams
BLOCK
STMT
EXPR
Homework
Create a Parse Tree for the following expression.

Use the rules stated in the previous lecture
while ( 5 ) { if ( 6 ) { } }

javiergs@asu.edu
Fall 2014
Lecture 09:
Grammars 1
javiergs@asu.edu
BYENG M1-21
Ambiguity
A grammar is said to be ambiguous if there is at
least one string with two or more parse trees.
Note that ambiguity is a property of grammars, not
languages.
Ambiguity
Solution
If a grammar can be made unambiguous at all, it is
usually made unambiguous through layering.
Have exactly one way to build each piece of the
string.
Have exactly one way of combining pieces back
together.
Recursive constructions
Layering
Root
Start symbol
Rule 1
Rule 2
Rule 3
Layers
Rule 4
Rule 5
.....
Leaf (Terminals, i.e., Tokens)
Layering
inputs:
1+2+3<4*5
1*2+3+4<5
1<2+3+4*5
1+2<3*4+5
1+2*3<4+5
Exercise
Provide a Grammar that is Not ambiguous
for arithmetic expressions
10 * 20 + 15
Precedence of operators and Associativity
(10 * 20) + 15
10 * (20 + 15)
Exercise | Precedence
New Grammar:
Original Grammar:
<E> <__>
<__>
E E OP E
< > <__>
<__>
E integer
< > <__>
<__>
OP + | - | * | /
< > <__>
<__>
< > - <__>
Exercise | Hand written notes
Exercise | Hand written notes
<E> <A> | <A> {'+' <A>} | <A> {'-' <A>}

<A> | {'*' } | {'/' }
 '-'<C> | <C>
<C> integer
<E> <A> {(+'|'-) <A>}

<A> {('*'|'/') }
 ['-'] <C>
<C> integer
Syntax Diagrams
B
Exercise 1
Include rules for handling parenthesis into the previous
grammar. The grammar should accept as correct the
following expressions:
10 * 20 + 15
(10 * 20) + 15
10 * (20 + 15)
(10) * (20) + (15)
(10 * 20 + 15)
10 * (20) + 15
Exercise 1 | Hand written notes
Syntax Diagrams
B
Exercise 2
Include rules to accept variables names (identifiers) in
expressions. The grammar should accept as correct the
following expressions:
A * 20 + time
(x * y) + 15
10 * (ASU + cse340)
(10) * (20) + (15)
(hello * world + Arizona)
10 * (counter) + 15
Syntax Diagrams
B
Exercise 3
Provide a Grammar that is
Not ambiguous
with Precedence of operators and Associativity for this:
10 + 20 > 15 & -10 != 1 | 20 / ( 10 + 1) < 5
Exercise 3 | Note
Precedence of operators
|
&
!
< > == != <= >=
+*/
()
...

{
{
}
}
{
{
}
}
}...
Exercise 3
EXPRESSION
Exercise 3
B

javiergs@asu.edu
Fall 2014
Lecture 10:
Grammars II
javiergs@asu.edu
BYENG M1-21
Office Hours by appointment
Exam 1 | Review
Draw a DFA equivalent to the regular expression
email2 = character+ \. character+
@ character+ \. domain
Note:
a*
a+ = a a*
Exam 1 | Review
Which are the tokens that are been defined?
Exam 1 | Review
Create a grammar to validate well-written references:
Byron Lahey, Audrey Girouard, Winslow Burleson, and Roel

Vertegaal. 2011. Understanding the use of bend gestures in
mobile devices with flexible electronic paper displays. In
Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems (CHI '11). ACM, New York, NY, USA,
1303-1312.
Exam 1 | Review
Create a grammar to validate well-written references:
Byron Lahey, Audrey Girouard, Winslow Burleson, and Roel Vertegaal.
2011.
Understanding the use of bend gestures in mobile devices with flexible electronic
paper displays.
In Proceedings of the SIGCHI Conference on Human Factors in

Computing Systems (CHI '11).
ACM,
New York, NY, USA,
1303-1312.
Exam 1 | Review
Terminals
Non-terminals
string
number
.
,
-
<AUTORS>
<YEAR>
<TITLE>
<CONFERENCE>
<PUBLISHER>
<ADDRRESS>
<PAGES>
Exam 1 | Review
Rules
<REFERENCE>
<AUTORS>
<YEAR>
<TITLE>
<CONFERENCE>
<PUBLISHER>
<ADDRRESS>
<PAGES>
Review | Hand written notes
Syntax diagram

Parse tree
(it is incomplete)
Exam 1 | Review
Exam 1 | Review

javiergs@asu.edu
Fall 2014
Lecture 11:
Parser Implementation I
javiergs@asu.edu
BYENG M1-21
Parser
A
Grammar
AA
BNF
EBNF
Parser
Parser | Grammar 1
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<C>
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
<A> {(+'|'-) <A>}
 {('*'|'/') }
['-'] <C>
integer|
identifier|'(' <EXPRESSION> ')'
Parser | Grammar 2
<PROGRAM>
'{' <BODY> '}'
<BODY>
{<EXPRESSION>';'}
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<A> {(+'|'-) <A>}

 {('*'|'/') }
['-'] <C>
<C>
integer|
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
Parser | Input and Output

{
0;
1 + 2;
3 * (4 + hello);
}
Parser | Step by Step

For each rule in the grammar {
Step 1. left-hand side (new method)
Step 2. right-hand side (loops, ifs, call methods)
Step 3. identify errors (terminals)
Step 4. synchronize errors (first and follow sets)
}
Parser
public class Parser {
private static Vector<Token> tokens;
private static int currentToken;
public static void RULE_PROGRAM () {}
public static void RULE_BODY () {}
public static void RULE_EXPRESSION () {}
public static void RULE_X () {}
public static void RULE_Y () {}
public static void RULE_R () {}
public static void RULE_E () {}
public static void RULE_A () {}
public static void RULE_B () {}
public static void RULE_C () {}
Parser
PROGRAM
public static void RULE_PROGRAM() {

if (tokens.get(currentToken).getWord().equals({)) {
currentToken++;
else
error(1);
RULE_BODY();
if (tokens.get(currentToken).getWord().equals(}))
currentToken++;
else
error(2);
}
Parser
BODY
public static void RULE_BODY() {

while (!tokens.get(currentToken).getWord().equals(})) {
RULE_EXPRESSION();
if (tokens.get(currentToken).getWord().equals(;))
currentToken++;
else
error(3);
}
}
Parser
EXPRESSION
public static void RULE_EXPRESSION() {

RULE_X();
while (tokens.get(currentToken).getWord().equals(|)) {
currentToken++;
RULE_X();
}
}
Parser
X
public static void RULE_X() {

RULE_Y();
while (tokens.get(currentToken).getWord().equals(&)) {
currentToken++;
RULE_Y();
}
}
Parser
Y
public static void RULE_Y() {

if (tokens.get(currentToken).getWord().equals(!)) {
currentToken++;
}
RULE_R();
}
Parser
R
public static void RULE_R() {

RULE_E();
while ( tokens.get(currentToken).getWord().equals(<)
|tokens.get(currentToken).getWord().equals(>)
|tokens.get(currentToken).getWord().equals(==)
|tokens.get(currentToken).getWord().equals(!=)
) {
currentToken++;
RULE_E();
}
}
Parser
E
public static void RULE_E() {

RULE_A();
while (tokens.get(currentToken).getWord().equals(-)
| tokens.get(currentToken).getWord().equals(+)
) {
currentToken++;
RULE_A();
}
}
Parser
A
public static void RULE_A() {

RULE_B();
while (tokens.get(currentToken).getWord().equals(/)
| tokens.get(currentToken).getWord().equals(*)
) {
currentToken++;
RULE_B();
}
}
Parser
B
public static void RULE_B() {

if (tokens.get(currentToken).getWord().equals(-)) {
currentToken++;
}
RULE_C();
}
Parser
C
public static void RULE_C() {

if (tokens.get(currentToken).getToken().equals(integer)) {
currentToken++;
} else if (tokens.get(currentToken).getToken().equals(identifier)) {
currentToken++;
} else if (tokens.get(currentToken).getWord().equals(()) {
currentToken++;
RULE_EXPRESSION();
if (tokens.get(currentToken).getWord().equals())) {
currentToken++;
} else error(4);
}
} else { error (5); }
}
Homework
Programming Assignment 2
Level 1
Review and Understand the Source Code

posted in Blackboard. Specially, particularly the use of
DefaultMutableTreeNode)
Homework
Homework
Homework
Homework
Level 2
Modify the Source Code

to include the rules PROGRAM and BODY, EXPRESSION, X, Y, R
(from Grammar 2)

javiergs@asu.edu
Fall 2014
Lecture 12:
Parser Implementation II
javiergs@asu.edu
BYENG M1-21
Review
Level 1
Review and Understand the Source Code

posted in Blackboard.
Particularly the use of DefaultMutableTreeNode)
Review
* Parser.java is the only file that you are allowed to modify

Review
Level 2
Modify the Source Code

to include the rules PROGRAM and BODY, EXPRESSION, X, Y, R
(from Grammar 2)
Review
<PROGRAM>
'{' <BODY> '}'
<BODY>
{<EXPRESSION>';'}
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<A> {(+'|'-) <A>}

 {('*'|'/') }
['-'] <C>
<C>
integer|
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
Level 3
The complete grammar for our language
Assignment 2 | Language
ARITHMETIC OPERATORS { +,
Expressions
(operators)
Actions
LOGIC OPERATORS { &,
|, ! }
RELATIONAL OPERATORS {<,

Instructions
KEYWORD {return,
Control
Structures
KEYWORD { if,
-, *, /, =}
>, ==, !=}
print}
else, while, switch, case }
KEYWORD { void, int, char, string, float, boolean }
Language
KEYWORD { true,
false }
BINARY
Data
INTEGER
FLOAT
HEXADECIMAL
OCTAL
CHAR
STRING
IDENTIFIER
Delimiter
: ; , () {} []
Assignment 2 | Grammar
<PROGRAM>
'{' <BODY> '}
<BODY> {<PRINT>';'|<ASSIGNMENT>';'|<VARIABLE>';|<WHILE>|<IF>|<RETURN>';'}
<ASSIGNMENT> identifier '=' <EXPRESSION>
<VARIABLE>
('int'|'float'|'boolean'|'char|'string'|'void')identifier
<WHILE>
'while' '(' <EXPRESSION> ')' <PROGRAM>
<IF>
'if' '(' <EXPRESSION> ')' <PROGRAM> ['else' <PROGRAM>]
<RETURN>
'return'
<PRINT>
print ( <EXPRESSION> )
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<C>
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
<A> {(+'|'-) <A>}
 {('*'|'/') }
['-'] <C>
integer | octal | hexadecimal | binary | true | false |
string | char | float | identifier|'(' <EXPRESSION> ')'
Assignment 2 | Input
Are there syntactical errors?
{
float a;
x = 0;
int x;
y = 1 + 1;
x = (0b11) +(05 0xFF34);
while (2 == "hi") {
a = 2 > (4 + Y);
if (true) { if( 2 + 2 ) {} else {} }
}
print ("hello" + "world");
}
{
int
x =
x =
x =
x =
x =
x =
x =
x;
5;
05;
0x5ff;
5.55;
"five";
5';
false;
{
int
float
string
char
void
boolean
x;
x;
x;
x;
x;
x;
{
x
x
x
x
x
=
=
=
=
=
"hello" + "world" 'w' * 5 / 3.4;

y hello & 0xffff | 05;
-7;
!y;
(cse340 + cse310) / cse101 ;
Assignment 2 | Code
------------ program(root);
Assignment 2 | Code
PROGRAM
public static void RULE_PROGRAM() {

if (tokens.get(currentToken).getWord().equals({)) {
currentToken++;
else
error(1);
RULE_BODY();
if (tokens.get(currentToken).getWord().equals(}))
currentToken++;
else
error(2);
}
Assignment 2 | Code
BODY
public static void RULE_BODY() {

while (!tokens.get(currentToken).getWord().equals(})) {
if (tokens.get(currentToken).getToken().equals(identifier)) {
RULE_ASSIGNMENT();
if (tokens.get(currentToken).getWord().equals(;)) {
currentToken++;
else error(3);
} else if (tokens.get(currentToken).getToken().equals(int) | ...) {
RULE_VARIABLE();
if (tokens.get(currentToken).getWord().equals(";")) {
currentToken++;
else error(3);
} else if (tokens.get(currentToken).getWord().equals(while)) {
RULE_WHILE();
} else if (tokens.get(currentToken).getWord().equals(if)) {
RULE_IF();
} else if (tokens.get(currentToken).getWord().equals(return)) {
RULE_RETURN();
if (tokens.get(currentToken).getWord().equals(;)) {
currentToken++;
else
error(3);
} else error(4);
}
}
Assignment 2 | Code
ASSIGNMENT
public static void RULE_ASSIGNMENT() {
}
Assignment 2 | Code
VARIABLE
public static void RULE_VARIABLE() {
}
Assignment 2 | Code
WHILE
public static void RULE_WHILE() {
}
Assignment 2 | Code
IF
public static void RULE_IF() {
}
Assignment 2 | Code
RETURN
public static void RULE_RETURN() {
}
Assignment 2 | Code
PRINT
public static void RULE_PRINT () {
}
Assignment 2 | Code
C
public static void RULE_C() {
}
PREDICTIVE
DESCENDENT
RECURSIVE
PARSER
Concepts
{
int a;
a = 0xFF + 0b111;
while (a != 05) {
if (true) {
a = 2.5e-1 / 7;
} else {
a = 'A;
while(true) {
}
}
}
print ("hello");
}
PREDICTIVE
DESCENDENT
RECURSIVE
PARSER
Homework
(Complete Levels 1 to 3)

javiergs@asu.edu
Fall 2014
Lecture 13:
Parsing Techniques I
javiergs@asu.edu
BYENG M1-38
Assignment 2
1
Understand de provided source code

(3 rules)
Program the rules PROGRAM, BODY, EXPRESSION, X, Y, R, E, C

(11 rules)
Program the full set of rules in the grammar

(16 rules)
Report syntactical errors (one error and stop)

PANIC MODE
Implement error synchronization

ERROR RECOVERY
Level 4
Handling Syntactical Errors (part 1):

Error messages
Assignment 2 | Grammar
<PROGRAM>
'{' <BODY> '}
<VARIABLE>
<WHILE>
<IF>
<RETURN>
'return'
<PRINT>
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<C>
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
<A> {(+'|'-) <A>}
 {('*'|'/') }
['-'] <C>
Error Synchronization
PROGRAM
RETURN
PRINT
BODY
EXPRESSION
ASSIGNMENT
VARIABLE
IF
WHILE
Assignment 2
Input:
{}
Output:
Build successful
Assignment 2
Input:
{
hello word
}
Output:
Line 2:
Line 2:
expected =
expected ;
Assignment 2
Input:
{
int x
int
int int x;
}
Output:
Line
Line
Line
Line
2:
3:
3:
4:
expected
expected
expected
expected
;
identifier
;
identifier
Assignment 2
Input:
{
x = a;
x = 0x36AW;
x = ((((((((((y))))))))));
x = (5+(4-(3+(5+5/(2+(3+(1+(77+(1-(y)))))))))) + hello + q;
if (a < b) {} else {}
if (a < b) {
if (a < b) {
} else {
}
}
}
Output:
Line 3: expected value, identifier or (
Parser | Error Points
Line N: expected {
PROGRAM
Line N: expected }

Line N: expected ;
BODY
Line N: expected identifier or keyword

ASSIGNMENT
Line N: expected =
VARIABLE
Line N: expected identifier


Line N: expected )
WHILE
Line N: expected (

Line N: expected (
Line N: expected )
IF
RETURN

Line N: expected (
Line N: expected )
PRINT
EXPRESSION

C
Line N: expected value, identifier or (
Line N: expected )
Assignment 2
public static void error(int err) {
int n = tokens.get(currentToken).getLine();
switch (err) {
case 1: gui.writeConsole("Line + n + ": expected {); break;
case 2: gui.writeConsole("Line + n + ": expected }); break;
case 3: gui.writeConsole("Line + n + ": expected ;); break;
case 4:
gui.writeConsole("Line +n+": expected identifier or keyword);
break;
case 5:
gui.writeConsole("Line +n+": expected =); break;
case 6:
gui.writeConsole("Line +n+": expected identifier); break;
case 7:
gui.writeConsole("Line +n+": expected )); break;
case 8:
gui.writeConsole("Line +n+": expected (); break;
case 9:
gui.writeConsole("Line +n+": expected value, identifier, ();
break;
}
}
Updates
1. In Gui.java: make the method writeConsole public
2. In Gui.java: add this as a second parameter for the method Parser.run()
3. In Parser.java:
add the attribute gui (line 15)
add a second parameter to the method run (line 17)
initialize gui (line 18)
Level 5
Handling Syntactical Errors (part 2):

Error Recovery
Error Recovery
Input:
{
x = a;
x = 1 + (
x = (y;)
if (a < b + ) {} else {}
if (a < b) {
if (a < b) {
} else {
We will not allow multiline expressions; line 3

and 4 should not be
considered as follow:
x= 1 + ( x = ( y;)
}
}
Error Synchronization
PROGRAM
RETURN
PRINT
BODY
EXPRESSION
ASSIGNMENT
VARIABLE
IF
WHILE
Error Recovery
Output:
Line 3:
Line 3:
Line 3:
// move
expected value, identifier, (

expected )
expected ;
to the next line
Line 4: expected )
Line 4: expected identifier or keyword
// infinite loop or end
Line 5: expected value, identifier, (
// simple
Line 12: expected }
// reported by program
Error Recovery
At this point:
Errors do not increment currentToken.
currentToken increase when the token is used
(added to the tree).
Error recovery is about ignoring tokens
How to know which tokens should be ignored?
Error Recovery
to be continued...
Homework

javiergs@asu.edu
Fall 2014
Lecture 14:
Parsing Techniques II
javiergs@asu.edu
BYENG M1-38
Error Recovery
Input:
{
x = a;
x = 1 + (
x = (y;)
if (a < b + ) {} else {}
if (a < b) {
if (a < b) {
} else {
We will not allow multiline expressions; line 3

and 4 should not be
considered as follow:
x= 1 + ( x = ( y;)
}
}
Error Recovery
PROGRAM
RETURN
PRINT
BODY
EXPRESSION
ASSIGNMENT
VARIABLE
IF
WHILE
Error Recovery
Output:
Line 3:
Line 3:
Line 3:
// move
expected value, identifier, (

expected )
expected ;
to the next line
Line 4: expected )
Line 4: expected identifier or keyword
// infinite loop or end
Line 5: expected value, identifier, (
// simple
Line 12: expected }
// reported by program
Error Recovery
At this point:
Errors do not increment currentToken.
currentToken increase when the token is used
(added to the tree).
Error recovery is about ignoring tokens
How to know which tokens should be ignored?
Parser | Error Recovery

Line N: expected {
currentToken++;
Searching for
FIRST(BODY) or }
PROGRAM
Line N: expected }
Line N: expected ;
BODY
Line N: expected identifier or keyword
currentToken++;
Searching for
FIRST(BODY) or FOLLOW(BODY)
ASSIGNMENT
Line N: expected =
currentToken++;
Searching for
FIRST(EXPRESSION) or FOLLOW(EXPRESSION)
VARIABLE
Line N: expected identifier

currentToken++;
Searching for
FIRST(PROGRAM) or FOLLOW(PROGRAM)
Line N: expected )
WHILE
Line N: expected (
currentToken++;
Searching for
FIRST(EXPRESSION) or )

currentToken++;
Searching for
Line N: expected (
currentToken++;
Searching for
FIRST(PROGRAM) or
FOLLOW(PROGRAM)
Line N: expected )
IF
RETURN

currentToken++;
Searching for
Line N: expected (
Line N: expected )
PRINT
EXPRESSION

C
Line N: expected value, identifier or (
Line N: expected )

Rule
FIRST set
FOLLOW set
PROGRAM
EOF
BODY
FIRST (PRINT) U FIRST (ASIGNMENT) U FIRST(VARIABLE) U FIRST

(WHILE) U FIRST(IF) U FIRST (RETURN)
PRINT
print
ASSIGNMENT
identifier
VARIABLE
int, float, boolean, void, char, string
WHILE
while
} U FIRST(BODY)
IF
if
} U FIRST(BODY)
RETURN
return
EXPRESSION
FIRST(X)
), ;
FIRST(Y)
| U FOLLOW(EXPRESSION)
! U FIRST(R)
& U FOLLOW(X)
FIRST(E)
FOLLOW(Y)
FIRST (A)
!=, ==, >, < U FOLLOW(R)
FIRST (B)
-, + U FOLLOW(E)
- U FIRST (C)
*, /, U FOLLOW(A)
integer, octal, hexadecimal, binary, true, false, string, char, float, identifier, (
FOLLOW(B)

If ( tokens.get(currentToken).getLine() <
tokens.get(currentToken+1).getLine() ) {
// go back until reaching RULE_BODY()
Homework

javiergs@asu.edu
Fall 2014
Lecture 15:
Parsing Techniques III
javiergs@asu.edu
BYENG M1-38

Rule
FIRST set
FOLLOW set
PROGRAM
EOF
BODY
FIRST (PRINT) U FIRST (ASIGNMENT) U FIRST(VARIABLE) U FIRST

(WHILE) U FIRST(IF) U FIRST (RETURN)
PRINT
print
ASSIGNMENT
identifier
VARIABLE
int, float, boolean, void, char, string
WHILE
while
} U FIRST(BODY)
IF
if
} U FIRST(BODY)
RETURN
return
EXPRESSION
FIRST(X)
), ;
FIRST(Y)
| U FOLLOW(EXPRESSION)
! U FIRST(R)
& U FOLLOW(X)
FIRST(E)
FOLLOW(Y)
FIRST (A)
!=, ==, >, < U FOLLOW(R)
FIRST (B)
-, + U FOLLOW(E)
- U FIRST (C)
*, /, U FOLLOW(A)
integer, octal, hexadecimal, binary, true, false, string, char, float, identifier, (
FOLLOW(B)
Calculating the First
Set
FIRST set
Definition
FIRST (a) is the set of tokens that can begin the construction a.
Example
<E>
<A>

<C>
<A> {+ <A>}
 {* }
-<C> | <C>
integer
FIRST(E)
FIRST(A)
FIRST(B)
FIRST(C)
=
=
=
=
{-, integer}
{-, integer}
{-, integer}
{integer}
FIRST set
Define FIRST (BODY)
FIRST(BODY) =
FIRST (PRINT) U FIRST (ASSIGNMENT) U FIRST(VARIABLE) U FIRST(WHILE) U
FIRST(IF) U FIRST(RETURN)
FIRST set
Define FIRST (C)
FIRST set
Define FIRST (A)
Define FIRST (B)
FIRST set
<S>
<S>
<A>
<A>


<C>
<C>
<E>
<E>
<F>
<F>
<A><C>
<F>
<E><F>d
a
ab
c<C>
d
e<E>
<F>
<F>f
Calculate the FIRST set

1. FIRST(X) = {X}
if X is a terminal
2. FIRST() = {}.
note that this is not covered by the first rule because
is not a terminal.
3. If A X, add FIRST(X) {} to FIRST(A)
4. If A A1A2A3 ...AiAi+1 ... Ak and
F IRST (A1) and FIRST (A2) and . . . and FIRST (Ai),
then add FIRST (Ai+1) {} to FIRST (A).
5. If A A1A2A3 ...Ak and
FIRST(A1) and FIRST(A2) and... and FIRST(Ak),
then add to FIRST(A).
Calculate the FIRST set
loop
FIRST set
S
S
A
A
B
B
C
C
E
E
F
F
ABC
F
EFd
a
aBb
cC
d
eE
F
Ff
FIRST set - evolution
rule
{a}
{a, }
{c, d}
{a, }
{a, , e, f}
{a, e}
{a, e, f, d}
{e}
{e, }
{e, , f}
{}
{, f}
{a, , e, f, d}
FIRST set | Exercise

<X>
<A>

<C>
<D>
<E>
<A> | <A> a
 | b
<C><D><E> | c d e | <C> c <D> d <E> e
one
two
three

<X>
<X>
<A>
<A>



<C>
<D>
<E>
<A>
<A> a

 b
<C><D><E>
c d e
<C> c <D> d <E> e
one
two
three
OPTION 1:
FIRST(X)
FIRST(A)
FIRST(B)
FIRST(C)
FIRST(D)
FIRST(E)
=
=
=
=
=
=
{c, one}
{c, one}
{c, one}
{one}
{two}
{three}
OPTION 2:
FIRST(X)
FIRST(A)
FIRST(B)
FIRST(C)
FIRST(D)
FIRST(E)
=
=
=
=
=
=
{c, one, }
{b, c, one}
{c, one}
{one}
{two}
{three}

<X>
<X>
<A>
<A>



<C>
<C>
<D>
<D>
<E>
<A>
<A> a

 b
<C><D><E>
c d e
<C> c <D> d <E> e
one
two
three
FIRST(X)
FIRST(A)
FIRST(B)
FIRST(C)
FIRST(D)
FIRST(E)
=
=
=
=
=
=
{c, one, three, two}

{one, }
{two, }
{three}
Calculating the Follow
Set
FOLLOW set
Definition
FOLLOW (a) is the set of tokens that can follow the construction a.
Example
<E>
<A>

<C>
<A> {+ <A>}
 {* }
-<C> | <C>
integer
5 + 4 + -7 * 12 + 75
5 + 4 + ((-7) * 12) + 75
What follows <C> ?
What follows <A> ?
What follows <E> ?
FOLLOW set
Definition
FOLLOW (a) is the set of tokens that can follow the construction a.
Example
<E>
<A>

<C>
<A> {+ <A>}
 {* }
<C> | <C>
integer
FOLLOW(E)
FOLLOW(A)
FOLLOW(B)
FOLLOW(C)
=
=
=
=
{$}
// $ represents end of input, i.e., EOF
{+, $}
{*, +, $}
{*, +, $}
FOLLOW set
Define FOLLOW (BODY)
FIRST(BODY) =
FOLLOW set
Define FOLLOW (C)
FOLLOW set
<S>
<S>
<A>
<A>


<C>
<C>
<E>
<E>
<F>
<F>
<A><C>
<F>
<E><F>d
a
ab
c<C>
d
e<E>
<F>
<F>f
Calculate the FOLLOW set

1. First put $ (the end of input marker) in Follow(S) (S is the start
symbol)
2. If there is a production A aBb,
(where a can be a whole string)
then everything in FIRST(b) except for is placed in FOLLOW(B).
(apply the rule 4 in calculate FIRST set)
3. If there is a production A aB,
then add FOLLOW(A) to FOLLOW(B)
4. If there is a production A aBb,
where FIRST(b) contains ,
then add FOLLOW(A) to FOLLOW(B)
Calculate the FOLLOW set
FOLLOW set
S
S
A
A
B
B
C
C
E
E
F
F
ABC
F
EFd
a
aBb
cC
d
eE
F
Ff
FOLLOW set - evolution
rule
{eof}
{a}
{a, c, d}
{c, d}
{c, d, b}
{eof}
{f}
{f, d}
{eof}
{eof, d}
FIRST sets:
S={a,,e,f,d}
A={a, e, f, d}
B={a, }
C= {c, d}
E={e, , f}
F={,f}
{eof, d, f}
Another Example
<E> <T> {+<T>}
<T> <F> {*<F>}
<F> (<E>) | integer
FIRST (E) = {(, integer}

FIRST (T) = {(, integer}
FIRST (F) = {(, integer}
FOLLOW(E) = {$, )}
FOLLOW(T) = {$, ),+ }
FOLLOW(F) = {$, ),+, *
Prediction Rules
Prediction Rules
Rule 1.
It should always be possible to choose among several
alternatives in a grammar rule.
FIRST(R1)
FIRST(R2)
FIRST(R3)...
FIRST(Rn) =
BODY
Prediction Rules
Rule 1.1
The FIRST sets of any two choices in one rule must not
have tokens in common in order to implement a singlesymbol look ahead predictive parser.
Prediction Rules
Rule 2.
For any optional part, no token beginning the optional part
can also come after the optional part.
FIRST(RULE) != FOLLOW(RULE)
BODY
PROGRAM
Homework

javiergs@asu.edu
Fall 2014
Lecture 16:
Grammars III
javiergs@asu.edu
BYENG M1-38
Chomsky Hierarchy
*Noam Chomsky
Chomsky Hierarchy
Chomsky Hierarchy
Type
Name
Example Use
Recognizing
Automata
Parsing
Complexity
Recursively
Enumerated
Turing Machine
Undecidable
Context
Sensitive
Linear Bounded
Automata
NP Complete
Context Free
Pushdown
Automata
O(n3)
Finite
Automata
O(n)
Arithmetic
Expression
x = a + b * 75
Regular
Identifier
a110
Chomsky Hierarchy
Type 0 - Recursively Enumerated
structure:
where and are any string of terminals and nonterminals
Type 1 - Context-sensitive
structure: X
where X is a non-terminal, and ,, are any string of terminals and nonterminals, (
must be non-empty).
Type 2 - Context-free
structure: X |
where X is a nonterminal and is any string of terminals and nonterminals (may be
empty). It is discouraged to use only one nonterminal as .
Type 3 Regular
structure: X Y | |
where X,Y are single nonterminals, and is a string of terminals;
Regular Grammar
Type 3 Regular
structure: X Y | |
<DIGIT> integer| | integer<DIGIT>
<Q0> a<Q1> | b<Q0>
<Q1> a<Q1> | b<Q0> |
<S> a<S> |b<A>
<A> c<A> |
<A>
<A> a
<A> a
Regular Grammar
The Regular Grammars are either left of right:
Right Regular Grammars:
Left Regular Grammars:
Rules of the forms

<A>
<A> a
<A> a
Rules of the forms

<A>
<A> a
<A> a
A,B: nonterminals and

a: terminal
A,B: nonterminals and

a: terminal
Regular Grammar
Key point: The derivation process is linear
Example of Derivation process:
<S> a<S> | b<A>
<A> c<A> |

The grammar is equivalent to the regular expression a*bc*
S aS aaS
aaS aabA aabcA
aabccA
aabcc
Context-Free Grammar
structure: X |
where X is a nonterminal and is any string of terminals and
nonterminals (may be empty). It is discouraged to use only one
nonterminal as .
<S>
<A>
<A><S> |
0 <A> 1| <A>1 | 0 1
<S>
<NP>
<VP>
<V>
<N>
<NP><VP>
the <N>
<V><NP>
sings | eats
cat | song | canary
Context-Sensitive Grammar
structure: X
where X is a non-terminal, and ,, are any string of terminals
and nonterminals, ( must be non-empty).
<S> a<S><C>
<S> a<C>
<C> <H>
<H> <H><C>
<H><C> <C>
a a b
b b b
b<C> b c
c<C> c c
*The language generated by this grammar is {anbncn|n 1} .
Chomsky Hierarchy
structure:
structure: X
must be non-empty).
structure: X |
Type 3 Regular
structure: X Y | |
Exercise
G3 {
G1 {
<s>
<s>
<A>
<A>a
<A>
b

<A>
<A><A>
aa
<A>a
<A>
<A>b
b
<A>
<A><A>
<A>
a<A>
ab
<A>
b
<s>
<A>
<A>


<A>
<A>
a<A>
a
b
b
<A>
G4 {
G2 {
<s>
<s>
<s>
<A>
<A>
<A>
}
<s>
<s>
<A>
<A>
<A>


b<s>
a<A>
b
a<s>
b<A>
a
}
Examples
Types of Grammars
<E>
<E>
<F>
<F>
e<E>
<F>
<F>f
Is this a regular grammar (X Y | |) ?

Is this a context-free grammar (X | ) ?
Is this a context-sensitive grammar (X ) ?
Types of Grammars
if <condition> then
<condition>
if ( <condition> ) then
<A> = 

Types of Grammars
if ( <condition> ) then
<condition>
if <condition> then
<A> = 

Types of Grammars
<A>

Chomsky Hierarchy
structure:
structure: X
must be non-empty).
structure: X |
Type 3 Regular
structure: X Y | |

javiergs@asu.edu
Fall 2014
Lecture 17:
Semantic Analysis I
BYENG M1-38
Semantic Analysis
Understanding the meaning

i.e.,
Interpreting expressions in their context
Semantic Analysis
1.
Declaration and Unicity. Review for uniqueness and that the variable
has been declared before its usage.
2.
Types. Review that the types of variables match the values assigned
to them.
3.
Arrays indexes. Review that the indexes are integers.
4.
Conditions. Review that all expressions on the conditons return a

boolean value.
5.
Return type. Review that the value returned by a method match the
type of the method.
6.
Parameters. Review that the parameters in a method match in type

and number with the declaration of the method.
Study Cases
Case 1:
int i;
char j; int m;
void method(int n, char c) {
int n; short l;
i = j;
i = m;
}
Case 2:
int i, j;
void method() {
int i = 5;
int j = i + i;
int i = i + i;
}
Case 3:
int i, m, k; boolean j;
void main() {
if (i>5) { ++i; }
while (i + 1) { ++i; }
do {++i; } while (i);
for (i = 0; m; ++i) {
k++;
}
}
Case 4:
int a;
int b;
int c, d;
char c1, c2;
int test1(int x, int y) {
return x+y;
}
void main() {
int i; i = a++;
i = test1(a, b);
i = test1(c1, c2);
i = test1(a, c1);
} }
Case 5:
int i,
public
int
a =
}
m;
boolean j;
void main() {
m; int a[];
new int[j];
Case 6:
int i;
void main(int m) {
i++; return i;
}
1. Variable Declaration and Unicity
Symbol Table
int i;
char j; int m;
int n; short l;
i = j;
i = m;
}
void method() {
int i = 5;
int j = i + i;
}
name
type
scope
int
global
char
global
int
global
method-int-char void
n
int k;
int method(int i) {
if (i>5) { ++i; }
while (i + 1) { ++i; }
for (i = 0; m; ++i) {
k++;
}
}
int
value
function
method-int-char

short method-int-char

method
void
function
int
method
int
method
int
global
method-int
int
function
Int
method-int
Symbol Table
int i;
char j; int m;
int n; short l;
i = j;
i = m;
}
void method() {
int i = 5;
int j = i + i;
}
name
type
scope
int
global
char
global
int
global
method-int-char void
n
int k;
int method(int i) {
if (i>5) { ++i; }
while (i + 1) { ++i; }
for (i = 0; m; ++i) {
k++;
}
}
int
value
function
method-int-char

short method-int-char

method
void
function
int
method
int
method
int
global
method-int
int
function
Int
method-int
Symbol Table
names
bindings
int
Method-int
method
int
global
int
method
char
global
int
int
global
method-int-char
void
function
int
method-int-char
short
method-int-char
method
void
function
int
global
method-int
int
function
Exercise
int a, b; char c, d; float e,f;
void foo1(int a) {
// float a = 1;
float w = a;
}
void foo2(char b) {
int a = c + d;
}
int foo3() {
Create the symbol table
int i = a + b;
}
Level 1
Reviewing Declaration and Unicity
Symbol Table
Symbol Table
Grammar
<PROGRAM>
'{' <BODY> '}
<VARIABLE>
<WHILE>
<IF>
<RETURN>
'return'
<PRINT>
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<C>
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
<A> {(+'|'-) <A>}
 {('*'|'/') }
['-'] <C>
A3 :: Parser Update
VARIABLE
void rule_variable( ) {
. . .
if (tokens.get(currentToken1).getType().equals(identifier)) {
SemanticAnalizer.CheckVariable(
tokens.get(currentToken-1).getWord(),
tokens.get(currentToken).getWord()
);
currentToken++;
} else {
error (6);
}
. . .
A3 :: SemanticAnalyzer.java
public class SemanticAnalizer {
private Hashtable<String, Vector<SymbolTableItem>> symbolTable;
public static void CheckVariable(string type, string id) {
// A. search the id in the symbol table
// B. if !exist then insert:
type, scope=global, value={0, false, "", }
// C. else error: variable id is already defined

}
A3 :: Review
Homework

javiergs@asu.edu
Fall 2014
Lecture 18:
Semantic Analysis II
BYENG M1-38
Semantic Analysis
1.
2.
Type Matching. Review that the types of variables match the values
assigned to them.
3.
4.

boolean value.
5.
type of the method.
6.

2. Type Matching
Type matching | Example 1

int x, y, z;
char p, q, r;
float a, b, c;
boolean foo;
void method() {
x = a * c + p;
}
+
*
a
Type matching | Cube
fill one sheet for

each operator in
the language
cube
OPERATOR
int
oat
char string boolean void
int
oat
char
string
boolean
void
cube
int
oat
int
int
oat
string
oat
oat
oat
string
char
string
string
OPERATOR
string
string string string string
boolean
string
void
cube
int
oat
int
oat
char
string
boolean
boolean
void
OPERATOR
&
cube
OPERATOR
<
int
oat
int
boolean boolean
oat
boolean boolean

char
string
boolean
void
cube
int
oat
int
OK
oat
OK
OK
char
OK
string
OK
boolean
OK
void
OK
OPERATOR
cube
OPERATOR
int
int
oat
oat

X
int
oat
int
int
oat
string
oat
oat
oat
string
char
string
string
OPERATOR
string
cube (- unary)
string string string string
boolean
string
void
cube (- binary)

int x, y, z;
char p, q, r;
float a, b, c;
boolean foo;
void method() {
x = a * c + p;
}
+
*
a
p

symbol table
int a;
int c (int b) {
return b * 3 * 2 * 1 ;
}
void main () {
a = 1;
boolean a= c(14)/2 > 1;
}
cube for
matching types
stack
Level 2
Reviewing Type Matching
Grammar
<PROGRAM>
'{' <BODY> '}
<VARIABLE>
<WHILE>
<IF>
<RETURN>
'return'
<PRINT>
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<C>
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
<A> {(+'|'-) <A>}
 {('*'|'/') }
['-'] <C>
Assignment 2 | Code
C
1.
// except for the open parenthesis
SemanticAnalizer.pushStack(
tokens.get(currentToken).getToken()
);
Parser
1. Store in a flag (operatorWasUSed):
Did the operator - exist?
2.
if (operatorWasUsed)
String x = SemanticAnalizer.popStack();
String result = SemanticAnalizer.checkCube (x, - );
SemanticAnalizer.pushStack(result);
}
Parser
1. Store in a flag (twiceHere):
Did we pass this point twice?
2. Store the operator that

creates the loop?
3.
if (twiceHere)
String y = SemanticAnalizer.popStack();
String result = SemanticAnalizer.checkCube (x, y, operator );
}
twiceHere = false; // reset the flag

Parser

creates the loop?
3.
if (twiceHere)
}

Parser

creates the loop?
3.
if (twiceHere)
}

Parser
1. Store in a flag (operatorWasUSed):
Did the operator - exist?
2.
if (operatorWasUsed)
String result = SemanticAnalizer.checkCube (x, ! );
}
Parser
2.
if (twiceHere)
String result = SemanticAnalizer.checkCube (x, y, & );
}

Parser
2.
if (twiceHere)
String result = SemanticAnalizer.checkCube (x, y, & );
}

Assignment 2 | Code
String result = SemanticAnalizer.checkCube (x, y, = );
if (!result.equals(OK) {
error(2);
}
Homework

javiergs@asu.edu
Fall 2014
Lecture 19:
Semantic Analysis III
BYENG M1-38
Semantic Analysis
1.
2.
Types. Review that the types of variables match the values assigned
to them.
3.
4.

boolean value.
5.
type of the method.
6.

3. Conditions
Example
symbol table
int a, b;
boolean c;
{
a = 4;
b = a + 1;
IF (a > b) {
print (a);
}
}
cube for
matching types
stack
Level 3
Reviewing Conditions
Grammar
<PROGRAM>
'{' <BODY> '}
<VARIABLE>
<WHILE>
<IF>
<RETURN>
'return'
<PRINT>
<EXPRESSION>
<X>
<Y>
<R>
<E>
<A>

<C>
<X> {'|' <X>}

<Y> {'&' <Y>}
['!'] <R>
<E> {('>'|'<'|'=='|'!=') <E>}
<A> {(+'|'-) <A>}
 {('*'|'/') }
['-'] <C>
Conditions
WHILE
if (!x.equals(boolean) {
error(3);
}
Conditions
IF
if (!x.equals(boolean) {
error(3);
}
Semantic Analysis
Type Matching. Review that the types of variables match the values
assigned to them.
boolean value.
type of the method.
Workshop
SemanticAnalyzer.java
Homework

javiergs@asu.edu
Fall 2014
Lecture 20:
Intermediate Code I
BYENG M1-38
A Compiler in Action
Analysis
Lexer
Words
Tokens
Parser
Sentences
Semantic
Analyzer
Compilation
Assembly
Code
Generation
Symbol table
Uniqueness
Type matching
Translation
Source Code Intermediate Code
Intermediate Code Machine or Binary Code
Source Code
{
int a;
int b;
int c;
int d;
if (a != 5) {
b = c + d;
}
print (a);
}
Intermediate (Object) Code

a,int,global 0
b,int,global 0
c,int,global 0
d,int,global 0
#E1,int,label,9
#P,int,label,1
@
lod a, 0
lit 5, 0
opr 14, 0
jmc #e1, false
lod c, 0
lod d, 0
opr 2, 0
sto b, 0
lod a, 0
opr 21, 0
opr 1, 0
Symbol table
Instructions
Translate Source to Object

{
int
int
int
int
a;
b;
c;
d;
if (a != 5) {
b = c + d;
}
print (a);
}
a,int,global 0
b,int,global 0
c,int,global 0
d,int,global 0
#E1,int,label,9
#P,int,label,1
@
lod , 0
lit 5, 0
opr 14, 0
jmc #e1, false
lod c, 0
lod d, 0
opr 2, 0
sto b, 0
lod a, 0
opr 21, 0
opr 1, 0
compilation
execution
// sorce code
int x;
Lexer
int foo () {
read (x);
print (5);
}
Parser
main () {
foo ();
}
Semantic Analyzer
Virtual Machine
(interpreter)
Code Generation
X,E,G,O,O
#e1,I,I,0,7
@
OPR 19, AX
STO x, AX
LIT 5, AX
OPR 21, AX
LOD #e1,AX
CAL 1, AX
OPR 0, AX
01001010101000010
01010100101010010
10100100000011011
11010010110101111
00010010101010010
10101001010101011
Assembler
compilation
execution
// sorce code
int x;
Lexer
int foo () {
read (x);
print (5);
}
Parser
main () {
foo ();
}
Semantic Analyzer
Virtual Machine
(interpreter)
Code Generation
X,E,G,O,O
#e1,I,I,0,7
@
OPR 19, AX
STO x, AX
LIT 5, AX
OPR 21, AX
LOD #e1,AX
CAL 1, AX
OPR 0, AX
01001010101000010
01010100101010010
10100100000011011
11010010110101111
00010010101010010
10101001010101011
Assembler
A Simple Virtual Machine

Memory

ALU

Register
CPU


Code
sto
sto
sto
sto
lod
lit
opr
jmc
lod
0, s
0, d
0, c
0, d
s, 0
s, 0
14, 0
#a1, false
b, 0
ALU

program

Symbol
Table

Register
Memory

CPU


Code
sto
sto
sto
sto
lod
lit
opr
jmc
lod
0, s
0, d
0, c
0, d
s, 0
s, 0
14, 0
#a1, false
b, 0
ALU

program

Symbol
Table

PC

Register

Memory

CPU

Instructions
instruction
first
parameter

second
parameter
Instructions
LIT <value>, <register_id>
Put a constant value in a CPU register
Examples:
LIT
LIT
LIT
LIT
LIT
5, 0
5.5, 0
'a', 0
hello, 0
true, 0
Instructions
LOD <variable>, <register_id>
Search for <variable> in the symbol table
Read its value
Put the value of <variable> in the CPU register
Examples:
LOD
a, 0
LOD hello, 0
LOD cse340, 0
Instructions
STO <variable>, <register_id>
Read a value from the CPU register
Search for <variable> in the symbol table
Store the value into the variable
Examples:
STO
a, 0
STO hello, 0
STO cse340, 0
Example
Source Code:
{
int a; int b;
a = 10;
b = a;
Symbol Table:
Type
Name
Scope
Value
int
int
a
b
global
global
0
0
Intermediate (Object) Code:

LIT 10,0
STO a,0
LOD a,0
STO b,0
Instructions
OPR <operation>, <register_id>
Read one or two values from the CPU register
Do the operation <operation>
Put the result into the CPU register
Examples:
OPR
OPR
OPR
1, 0 ; return
2, 0 ; add
3, 0 ; subtract
Operations
Number
Action
Exit program
Return
ADD: POP A, POP B, PUSH B+A
SUBTRACT: POP A, POP B, PUSH B-A
MULTIPLY: POP A, POP B, PUSH B*A
DIVIDE: POP A, POP B, PUSH B/A
MOD: POP A, POP B, PUSH (B mod A)
POW: POP A, POP B, PUSH (A to the power B)
OR: POP A, POP B, PUSH (B or A)
AND: POP A, POP B, PUSH (B and A)
10
NOT: POP A, PUSH (not A)
11
TEST GREATER THAN: POP A, POP B, PUSH (B>A)
12
TEST LESS THAN: POP A, POP B, PUSH (B<A)
13
TEST GREATER THAN OR EQUAL: POP A, POP B, PUSH (B>=A)

Operations
Number
Action
14
TEST LESS THAN OR EQUAL: POP A, POP B, PUSH (B<=A)
15
TEST EQUAL: POP A, POP B, PUSH (B=A)
16
TEST NOT EQUAL: POP A, POP B, PUSH (B<>A)
17
18
clear screen
19
read a value from the standard input
20
print a value to the standard output
21
print a value to the standard output and a newline character
Example
{
int
int
a =
b =
a =
b =
}
a;
b;
10;
a;
a * 10;
2 + 3 + 4;
Type
int
int
Name
a
b
Scope
global
global
Value
0
0
LIT 10, 0
STO a, 0
LOD a, 0
STO b, 0
LOD a, 0
LIT 10, 0
OPR 4, 0 ; multiply
STO a, 0
LIT 2, 0
LIT 3, 0
OPR 2, 0 ; add
LIT 4, 0
OPR 2, 0 ; add
STO b, 0
OPR 1,0
OPR 0,0
to be continued...

javiergs@asu.edu
Fall 2014
Lecture 21:
Intermediate Code II
BYENG M1-38
Review
8
hello
{
int a;
int b;
a = 5;
b = 9;
a = a + b / 3;
print (a);
print ("hello");
}
Lexer
Parser
Semantic Analyzer
Code Generation
Virtual Machine
(interpreter)
a, int, global, 0
b, int, global, 0
@
LIT 5, 0
STO a, 0
LIT 9, 0
STO b, 0
LOD a, 0
LOD b, 0
LIT 3, 0
OPR 4, 0
OPR 2, 0
STO a, 0
LOD a, 0
OPR 21,0
LOD "hello", 0
OPR 21,0
OPR 0, 0
Review
Lexer
int a;
int b;
boolean foo;
a = 10 + 20 + 30 + 40;
print (a);
Parser
Semantic Analyzer
Code Generation
foo = 340 > 126;

print (foo);
a = a / 2;
print ("total:" + a);
return;
}
Review
100
true
total: 50
Virtual Machine
(interpreter)
Assignment #4
Assignment #4
To show
proficiency
building a
descendent
parser
assignment #1
or
Lexer.jar
Programming
workshops
Lexer
Parser
Semantic Analyzer
Code Generation
Do not required.
Bonus points include
Semantic Analysis from
assignment #3
Following
lectures
Deadline: December 4
Assignment #4 | Grammar
'{' <BODY> '}
{<PRINT>';'|<ASSIGNMENT>';'|<VARIABLE>';|
<WHILE>|<IF>|<SWITCH>|<RETURN>';'}
<VARIABLE>
<WHILE>
<IF>
<RETURN>
'return'
<PRINT>
<EXPRESSION> <X> {'|' <X>}
<X>
<Y> {'&' <Y>}
<Y>
['!'] <R>
<R>
<E> {('>'|'<'|'=='|'!=') <E>}
<E>
<A> {(+'|'-) <A>}
<A>
 {('*'|'/') }

['-'] <C>
<C>
<PROGRAM>
<BODY>
<SWITCH>
<CASES>
<DEFAULT>
'switch' '(' id ')' '{' <CASES> [<DEFAULT>] '}'

('case' (integer|octal|hexadecimal|binary) ':' <PROGRAM>)+
'default' ':' <PROGRAM>
Assignment #4 | Compiler
Assignment #4 | Compiler
Bonus Points
Assignment #4 | VM
Use it to test your compiler.
No changes required
Assignment #4 | Code
Add this lines to your Parser.run()
Assignment #4 | Code
The CodeGenerator.java file
Assignment #4 | Grading
Bonus 10 %
Implement SWITCH statement: (a) parser; and (b)
code generation.
Bonus 10 %
Integration: (a) graphical user interface including
token table, parse tree, and symbol table; (b)
syntactic errors handling and recovery; and (c)
semantic analysis.
Assignment #4 | Grading
assignment
bonus
0100%
020%
Implementing
Intermediate Code Generation
Code Generation
1 * 2 > 3 + 4 * 5
LIT 1,
LIT 2,
OPR 4,
LIT 3,
LIT 4,
LIT 5,
OPR 4,
OPR 2,
OPR 11,
>
+
*
1
*
2
0
0
0
0
0
0
0
0
0
Code
PROGRAM
Code
BODY
Code
PRINT
Code
ASSIGNMENT
Code
VARIABLE
Code
WHILE
Code
IF
Code
RETURN
Code
EXPRESSION
Code
Code
Code
Code
Code
Code
Code
C
Homework
Assignment #4
(LIT, LOD, STO, OPR)

javiergs@asu.edu
Fall 2014
Lecture 22:
Intermediate Code III
BYENG M1-38
Review
Level 1
LIT, LOD, STO, OPR
Code
OPR 0, 0
PROGRAM
Code
BODY
Code
OPR 21, 0
PRINT
Code
STO identifier, 0
ASSIGNMENT
Code
VARIABLE
identifier, <type>
Code
OPR 1, 0
RETURN
Parser
EXPRESSION
if (twice) {
OPR 8, 0
}
Parser
if (twice) {
OPR 9, 0
}
Parser
if (operatorUsed) {
OPR 10, 0
}
Parser
R
if (twice) {
OPR <operatorNumber>, 0
}
Parser
if (twice) {
}
Parser
if (twice) {
}
Parser
LIT 0, 0
if (operatorUsed) {
OPR 3, 0
}
Assignment 2 | Code
C
LIT <value>, 0
LOD identifier, 0
Review
Level 2
JMP, JMC
Instructions
JMP <line>, 0
Put the value <line> in the program counter;
thus, the next line to be executed will be <line>
Examples:
JMP
JMP
JMP
1, 0
14, 0
75, 0
Instructions
JMC <line>, <condition>
Read one value from the CPU register
If the value is equal to <condition>, put the value
<line> in the program counter; thus, the next line to
be executed will be <line>
Examples:
JMC
JMC
JMC
1, true
14, false
75, true
Example
{
int a; int b;
a = 10;
while (a>1){
print (a);
}
return;
}
int
int
@
a
b
LIT 10, 0
STO
a, 0
LOD
a, 0
LIT
1, 0
OPR 11, 0
JMC #e1, false
LOD
a, 0
OPR 20, 0
JMP #e2, 0
OPR
1, 0
OPR
0, 0
global
global
0
0
; >
; print
Labels
Compiler creates variables and adds them to the
symbol table to remember positions in the code.
This is useful for loops and conditions.
Labels | how while works
while (a < b) {
label#1
if (a < b) {
//code
//code
}
} else
goto label#2
}
goto label#1
label #2
Labels | how if works
if (a < b) {
//code1
} else {
// code2
}
if (a < b) {
if (a>b) goto label #1
//code1
} else
goto label#2
label #1
// code2
}
label #2
Example | while
{
int a; int b;
a = 10;
while (a>1){
print (a);
}
return;
}
int
int
a
b
global
global
0
0
int
#e1
global
int
#e2
global
10
3
LIT 10, 0
STO
a, 0
LOD
a, 0
LIT
1, 0
OPR 11, 0 ; >
JMC #e1, false
LOD
a, 0
OPR 20, 0 ;print
JMP #e2, 0
OPR
1, 0
OPR
0, 0
Example | if
{
int a; int b;
a = 10;
if (a>1) {
print (a);
} else {
print (b);
}
return;
}
int
int
a
b
global
global
0
0
int
#e1
global
int
#e2
global
10
12
LIT
10, 0
STO
a, 0
LOD
a, 0
LIT
1, 0
OPR
11, 0 ; >
JMC #e1, false
LOD
a, 0
OPR
20, 0 ;print
JMP #e2, 0
LOD
b, 0
OPR
20, 0 ;print
OPR
1, 0
OPR
0, 0
Exercise
{
int a; int b;
a = 10;
while (a>1) {
if (a != 0) {
print (a);
} else {
print (b);
}
a = a -1;
}
}
Handwritten notes
Code
WHILE
Code
IF
What about a switch-case
Handwritten notes
Add the symbol table here
Homework
Translate this source code to intermediate code:
int a, b;
switch (a) {
case 1: {
case 2: {
case 3: {
default:{
b
b
b
b
=
=
=
=
11;
22;
33;
99;
break;}
break;}
break;}
break;}
Homework
Assignment #4

javiergs@asu.edu
Fall 2014

Principles of Programming Languages - ASU 2014

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Principles of Programming Languages - ASU 2014

Hochgeladen von

Copyright:

CSE340 - Principles of

Tell a computer what to do

Method of communication consisting of the use of

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 2

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 4

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 5

program = algorithms + data

program = objects + messages

program = functions functions

program = facts + rules

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 7

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 8

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 9

10% + 10% + 10% + 10%

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 10

Chapter 3. Functional Programming

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 11

Read the Syllabus of the course

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 12

CSE340 - Principles of Programming Languages

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 2

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 3

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 4

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 5

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 7

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 8

Lexical Analysis | Steps

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 10

int x = 5; float y = "hello;

Review the following topics:

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 13

CSE340 - Principles of Programming Languages

Regular expression | Examples

Regular Expression (rule)

{aa, bb, ba, ab}

, a, aa, aaa, aaaa, ... }

, a, b, aa, bb, ...abba ...}

{ a, aa, aaa, aaaa, ... }

{hello, world, etc, }

{1934, 0101, 33, 12321}

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 4

Regular expression | Examples

Regular Expression (rule)

Regular expression | Operators

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 6

Deterministic Finite Automata

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 7

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 8

{$, _, 0-9, a-z}

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 9

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 10

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 11

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 12

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 14

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 16

Regular expressions - Examples

Regular Expression (rule)

{"a", "b", "c"}*

{, "a", "b", "c", "aa", "ab", "ac", "ba", "bb", "bc",

{, "ab", "c", "abab", "abc", "cab", "cc", "ababab",

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 17

Regular expressions - Examples

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 18

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 19

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 20

Javier Gonzalez-Sanchez | CSE340 | Fall 2014 | 21